Goodbye Microservices: From 100s of Problem Children to 1 Superstar
Twilio Segment collapsed 140+ destination microservices into a single monolith after the architecture that once solved head-of-line blocking became an operational weight three engineers couldn't keep running.
Microservices are supposed to let small teams move independently. Segment reached a point where three engineers spent most of their time keeping 140 services alive — which is not the same thing as building.
Source
Goodbye Microservices: From 100s of problem children to 1 superstar
By: Alexandra Noonan
Published: Jul 10 2018
Type: Engineering article
One of the clearest accounts of microservice complexity compounding past the point of manageability, written by someone who lived through both the splitting and the merging. The specifics around shared library drift and the Traffic Recorder fix are the parts worth keeping.
The hidden tax of distribution
The architecture started with a legitimate problem. Segment’s pipeline ingests events from customer SDKs and forwards each one to whichever third-party destinations the customer has enabled — Google Analytics, Optimizely, custom webhooks, and over a hundred others. Early on, all destinations consumed from a single shared queue. When one destination started failing and retrying, those retries flooded the queue. Head-of-line blocking meant every other destination experienced delivery delays even though nothing was wrong on their end. The fix was sound: give each destination its own queue and its own service. Destination X’s queue backs up independently; everything else runs unaffected.
That was the right call for the problem at hand. It also created 140+ services and 140+ repositories, and that’s where the compounding began.
Shared code was the first casualty. All 140 destinations do fundamentally similar work: normalize field names, transform event schemas, make HTTP requests. Segment built common utilities to avoid duplicating that logic across every destination — event.name() would check multiple casing variants, combine first and last names from the event payload, and return a consistent result regardless of what the SDK sent. When a utility needed to change, the update had to roll out to each service individually. At some point the rollout cost exceeded the perceived benefit. Engineers updated the library in one destination but not others. Over time, the shared code that was supposed to create uniformity was running at a different version in every repo. The codebases meant to stay consistent had silently diverged.
The test suite became the second casualty, and this is the part that made everything grind to a halt. Each destination’s tests made real HTTP requests to live endpoint APIs. Expired credentials, endpoint downtime, and rate limits caused failures that had nothing to do with any code change under review. With 140+ separate repos, there was no cross-cutting pressure to fix flaky tests — a broken test in a destination you hadn’t touched was someone else’s problem. A change that should have taken an hour could stall for a week while an engineer tracked down a failing test in an unrelated destination that the deploy pipeline required to pass.
The fix was Traffic Recorder, built on top of yakbak. The first time a test runs against a destination’s live endpoint, all HTTP traffic is recorded to a file and checked into the repo. Every subsequent run replays from that file. No live network calls, no credential expiry, no endpoint downtime. Total test time for all 140+ destinations dropped from potentially an hour to milliseconds. Once tests could be run quickly and trusted, combining every destination into a single repo became practical.
Service consolidation needed one more piece: Centrifuge, a new routing layer that replaced the 140 per-destination queues. Centrifuge received inbound events, fanned them out to the appropriate destinations, and handled retry logic — all the coordination that had been distributed across 140 services. With Centrifuge owning the queue layer, there was no longer any reason to maintain one service per destination. Everything could live in one service without needing its own queue.
The outcomes were concrete. Rolling a shared library improvement previously meant testing and deploying 140 services — enough operational exposure that engineers deferred changes. Improvements to shared utilities went from 32 in one year to 46 in the following year after the monolith shipped. The mixed CPU and memory profile of 140 different destination types running together in one service made autoscaling dramatically easier to configure than 140 individually-tuned services, each described in the post as “more art than science.” On-call paging from load spikes on low-traffic destinations stopped.
The tradeoffs are real and worth naming. Fault isolation is weaker: a bug that crashes the service crashes it for all destinations simultaneously. In-memory caches spread thin across thousands of processes lose hit rate compared to the per-destination setup, where a low-traffic destination’s small process pool kept its cache warm. A dependency upgrade can break multiple destinations rather than just one. Segment accepted each of these explicitly — they built the test suite first, documented the tradeoffs, and moved knowing what they were giving up.
My take
The microservice boundary earns its coordination cost when the services on either side have genuinely different failure profiles, scaling requirements, or owners. Segment’s destinations had none of these differences — they were all doing the same job, owned by the same team, and scaling together. The boundary added distribution overhead (separate deployments, diverging dependencies, isolated test suites) without providing the isolation it promised. The fix wasn’t just “go monolith” — it was building the infrastructure that made the monolith safe: deterministic tests first, then merge. The lesson isn’t that microservices are wrong. It’s that a service boundary only pays for itself when the isolation on either side is real. Splitting for modularity without genuine operational independence between the pieces creates coordination costs that compound with every shared dependency and every rollout.
Where this is useful
- When a system has many services doing the same class of work with heavy shared code — plugin systems, adapter pipelines, integration frameworks
- When shared library updates are being deferred because rolling them out across many services feels too risky to be worth it
- Not applicable when services have genuinely different owners, failure profiles, and scaling requirements — the isolation is real in those cases and the coordination cost is justified
Apply this: Before splitting a service, ask whether the resulting pieces will have different deployment schedules, failure modes, and owners. If the answer is no to all three, the boundary is adding overhead without providing isolation — and every shared dependency you add later will make that overhead compound.
What I want to remember
Remember: A service boundary only pays off when the isolation on either side is real — different failure modes, different owners, different scaling profiles. When services share code, scale together, and are maintained by the same team, each additional service boundary adds compounding coordination cost that eventually overwhelms any agility gain.