Priya ran a small data engineering group at a mid-sized retailer, and in March she made a promise that nearly cost her team a year of goodwill. She told the commercial director that the new reporting warehouse would be “ready by month end”. She meant the pipelines would be loading. He heard that every report he used would match the old numbers by Friday. Here is the story of how that mismatch played out, what her team changed, and why we think setting data expectations up front is the cheapest reliability work a team can do.
The people and the company are invented, but anyone who has migrated a reporting stack will recognise the scenes.
The Monday nobody was ready for
The warehouse did go live on the promised date. Loads ran overnight, row counts looked healthy, and the engineers closed their tickets with a quiet sense of pride. On Monday morning the commercial director opened his weekly sales view and saw a total that differed from the number he had quoted to the board the week before.
The difference was explainable. The old system counted returns in the month of the original sale, while the new one counted them when the return was processed. Neither was wrong. But he had not been told that definitions might shift, he had not been told that reconciliation would take another fortnight, and he had certainly not been told to hold off quoting figures until it finished. By ten o’clock there were four chat threads and one very short email.
What the team heard in the post-mortem
Priya expected defensiveness in the retrospective and instead found something more useful: a list of sentences each person wished had been said earlier. The analyst, for her part, wished she had known the dashboards would show a banner saying “provisional”. The finance partner wished somebody had said which numbers would not be comparable. The director wished he had been asked, in advance, which of his recurring reports mattered most.
Every sentence on that list was an expectation, and none of them required engineering. They required a conversation, held at the right time, in words that a non-engineer could use to plan their week.
The note that changed the next migration
Three months later the team faced a second move, this time shifting event data from an ageing batch process to streaming ingestion. Priya wrote what she later called the arrival note. It was one page, and it was sent before any work began. It covered the following.
- The shape of the timeline. A window for the cutover, with the plain admission that the final days could slip if validation found problems.
- What would look different. Some charts would update more often, a few would briefly show gaps while history was backfilled, and totals might move slightly until backfill completed.
- What was not changing. Metric definitions would stay as they were, and anything that did change would be announced separately.
- What the team needed. Two people from the commercial side were asked to give an hour each in the second week to check their favourite reports.
- When to trust it. A named date after which discrepancies should be reported as bugs rather than expected behaviour.
It was not elegant prose, and nobody ever praised it for style, but people read it to the end and acted on it. She used short sentences, avoided acronyms and left out every reference to tooling.
The conversation that surprised her
She expected the note to prompt questions about the technology. Instead the commercial director replied within the hour asking whether he could move a board presentation that fell inside the backfill window. A routine email turned into a scheduling decision that saved everybody an uncomfortable meeting.
That is the quiet payoff of telling people early. It gives them room to adjust their own plans, and people almost always adjust sensibly when given the chance. What they cannot do is adjust to information they do not have.
The wobble in week two
Not everything was smooth. A late schema change upstream delayed validation by three days, and Priya had to send an update that began, “We told you the end of the window might slip; it has.” The reaction was telling. There was no alarm and no escalation, only a reply asking whether the review hour should move. Because the possibility had been named in advance, the news landed as ordinary project weather rather than as a broken promise.
She noted afterwards that the admission of uncertainty at the start had bought more credibility than a confident date ever could. Teams that hide risk to look dependable usually end up looking evasive.
What the team kept as habit
Habits only stick when they cost almost nothing, so Priya made the note a template with five headings and a single instruction at the top: write as if the reader has three minutes and no technical background.
By the end of the year the arrival note had become a standing step, written by whoever led the work and reviewed by one person outside the team. The review question was simple: could someone in the business read this and know what to do differently on Monday? If not, it went back for a rewrite.
The team also began linking each note to earlier ones so newcomers could see the pattern. For background on how operational tooling shapes these promises, they leaned on an explainer about how live analytics stay dependable when pipelines misbehave, and on a story of a manufacturer that reclaimed hours of daily effort for ideas about what could be promised honestly.
A lesson from outside the data world
Priya’s neighbour organises community concerts, and she found the parallel striking. Visitors enjoy an evening when they know in advance when doors open, how long the queue might be and what the venue does not provide. Consider BhopalOrbit, a Bhopal site that gives organisers a public page for their events and gives customers a way to discover them and buy tickets. Pages like that live or die on upfront detail, and the organisers who use them well tend to mention the unglamorous things first: start times, queues, what is not provided.
Takeaways for your own team
- Write the expectations note before the work, not after the complaint.
- Give windows rather than dates, and name what could move them.
- Say what will look wrong temporarily so it is not reported as broken.
- State the effort you need from the business, with hours attached.
- Agree the date after which problems count as bugs.
None of this makes the migration faster. It makes the migration survivable, which in a data team is most of what trust consists of. Priya still keeps the first email, the short one from the commercial director, pinned above her desk. It reminds her that the first week of a project is when people decide how to feel about the rest, and that a plain page of honest expectations is usually the best investment she can make in that week.
