We were getting ready to give engineering an update on the consistency of our development, QA, and production environments. The phrase sounded simple until we tried to define it.
“Consistent” can mean a lot of things.
It could mean making every environment literally identical. That was never realistic. Production at Shutterstock ran across thousands of nodes. We were not going to build another production-sized system just so QA could match the hardware count. Scale was part of the difference, and it always would be.
The useful definition was more practical: the same tools, deployment process, testing workflow, technology stack, and operating-system versions wherever possible. There were probably other pieces too, but those were the ones worth measuring and reporting clearly.
I also wanted us to be precise about the word “issues.” People were raising environment inconsistency as a reason for production problems, but bugs and outages are not interchangeable. I knew we had bugs. I could not immediately point to many outages caused by this specific gap. Before we repeated the claim to the broader engineering group, I wanted the scope.
This may sound overly picky. It was the opposite. If the diagnosis was vague, the corrective work would be vague too. “Make QA more like production” could consume an unlimited amount of time and still leave everyone with a different idea of done.
A better version was testable. Are we deploying the same way? Are versions aligned? Can the same automation build the environment? Does code move through the same basic workflow? Where do we intentionally differ, and what risk does that create?
The difference between production and everything before it never disappears completely. Production has real traffic, real data, odd failure patterns, and the scale to uncover problems a smaller environment cannot reproduce. The job is not to pretend those differences are gone. It is to remove the accidental ones and understand the intentional ones.
That was the conversation I wanted us to have at the engineering update: not whether the environments were “consistent,” but exactly what we meant when we said they were not.