Imagine a data platform team of six people, which we will call the Ledger Lane crew. Every detail here is made up for the telling, so please do not read it as news about an actual firm. They own roughly forty small services: ingestion workers, schema checkers, transformation jobs, a handful of API shims. Every one of them ships as a container. For two years nobody thought of that as a discipline. Then a Tuesday morning arrived that made them name it. The name they settled on was ContainerOps, borrowed from a short reference entry they happened to read.
The morning the nightly job did not run
The first symptom was boring: a nightly load job failed to start. Its image could not be pulled. Nothing had changed in the team’s code. What had changed, they discovered after an hour of digging, was that the free base image they inherited in a dozen Dockerfiles had been relocated by its publisher. Their pipelines still pointed at the old location, and the old location no longer received fixes.
That sounds like a one-off, but the Ops Atlas lists a real event of this shape. According to the Atlas page describing ContainerOps, a well-known publisher announced in July 2025 that, from the end of August that year, its free images would leave the original registry namespace, with the legacy catalogue receiving no more updates and free images limited to the latest tag. Users were told to revisit their pipelines, mirrors and clusters. The Ledger Lane crew had not been through that exact episode, but their Tuesday rhymed with it.
What the team did that week
Nobody proposed buying a platform. The crew’s lead, a pragmatic data engineer, drew a three-column page on a whiteboard: what we build, where it is stored, what runs. Then the team filled it in honestly.
- What we build: thirty-odd Dockerfiles, copied from each other, with base images referenced by moving tags.
- Where it is stored: one internal registry that had never had a cleanup policy.
- What runs: an unknown number of containers started from images no one could trace back to a commit.
The third column hurt most. In data work, a transformation that produces a different answer after a redeploy is not an annoyance, it is a reconciliation incident. Being unable to say which image produced last Thursday’s output meant being unable to say which version of the cleansing logic touched the numbers.
The registry that quietly ate a disk
A few weeks later the registry host raised a storage alarm. Someone ran the arithmetic the Atlas uses as a teaching example: forty services, twenty-five builds per week each, two platform variants, around fifteen mebibytes of new layer data per image. Multiply it out and you get roughly twenty-nine gibibytes of growth every week. Left alone, that never shrinks.
The remedy was dull and effective. They agreed on retention rules, such as keeping every released tag and only the latest few from feature branches, and scheduled garbage collection after each pruning run. The Atlas describes the steady state under a sensible policy as being in the region of a couple of hundred gibibytes, a number you can plan a disk around. They also reminded themselves of a warning from the same page: a mirror that caches downloads does not count as a copy you can restore from, so released images still needed a durable home of their own.
Pinning by digest, and the fear that came with it
The next decision was to deploy by digest, the content hash of an image, with release tags that are never moved. One teammate voiced a fear that pinning would leave them stuck on old software. It is a fair worry and the Atlas addresses it directly: pinning only freezes you if nothing is moving the pin. They enabled an update bot that opened a pull request whenever a base image changed, so a human reviewed every refresh. The pins turned from a cage into a ledger of what had been approved.
They gave themselves a simple scoreboard. Digest coverage began at zero and crept toward all production workloads over two months. They also noted the age of each base image, which exposed three services running on foundations nobody had rebuilt in a year.
The log file nobody rotated
Month three brought a smaller scare. A chatty ingestion worker filled a host’s disk, and the cause was not data at all. The default json-file logging driver keeps everything it is given unless someone configures rotation, and nobody had. One service had written a single log file of several gigabytes. The fix took ten minutes: a size cap and a file count in the daemon defaults, applied everywhere, plus a reminder that real log pipelines are a separate topic with their own owners.
It was also the moment the crew stopped debating whether this was “real engineering”. Disks, signals, hashes and limits are all very concrete things. They have exact failure modes and exact repairs. What had been missing was a place on the calendar for them.
A rough ladder for the next quarter
To keep momentum, the lead sketched a ladder of five levels in the Atlas’s style, running from improvised laptop builds up to automatic rebuilds, and placed the team on it. They were honestly at the lowest rung, with builds from laptops and floating tags. A month of work would reach the repeatable stage, where continuous integration builds every image and a linter checks each Dockerfile. After that came the managed stage: digests, unprivileged users and limits for the busiest services. They deliberately did not promise the measured stage, where fixes reach production inside an agreed window, until they had two quarters of data.
Where the story touches the rest of their work
Ledger Lane already cared about pipelines that break silently. Their earlier reading on keeping real-time analytics pipelines from snapping covered the defences on the data side, the contracts and the monitoring. ContainerOps turned out to be the unglamorous layer underneath, the part that decided whether a job starts at all. Another team article, the playbook that recovered hours from a tire manufacturer’s analysts, made them think in terms of time saved per person, and they started to measure the hours lost to mystery image problems. For teams spanning several providers, the notes on running data operations across multiple clouds raise the same question the registry did: where does each artifact live, and who owns it?
What they would tell another team
When the crew presented their changes to the wider engineering group, they kept to five points. They did not pretend these were exhaustive, and the Atlas is clear that tools such as image scanners and signing belong to adjacent disciplines rather than here.
- Write down every image, its owner and its base before touching anything.
- Set retention and cleanup on the registry on day one, not after the alarm.
- Deploy by digest and let a bot, not a person’s memory, move the pins.
- Give every container memory and process limits, and treat CPU limits more cautiously.
- Choose two or three numbers to chart, such as digest coverage and base-image age, and review them monthly.
The nightly job eventually ran again, a day late and with a lesson attached. Months afterwards, when someone asked why the team spent a sprint on what looked like housekeeping, the answer was in the number of incidents that no longer happened. That is the odd thing about this discipline: when it works, the story is that nothing happened, which makes it hard to celebrate and very easy to postpone. The Ledger Lane crew learned to schedule it anyway, in small pieces, before a Tuesday morning schedules it for them.