The complaint always arrives the same way. A finance analyst opens a report on Monday morning, sees last Thursday’s numbers, and sends a short, sharp message to the data team. The pipeline did not break. The job ran exactly as designed. The analyst simply expected fresh data by nine, and nobody had ever said otherwise. If this sounds familiar, you are dealing with a gap in data delivery expectations, not a gap in engineering.

This post treats the problem like a diagnosis. The plan is to list the symptoms, trace them to their origin, and apply a remedy that costs an afternoon, not a quarter. No frameworks, no new tools. Just a short, plain note that tells data consumers what to expect and when.

The symptoms you can see today

Before prescribing anything, check whether your team shows these signs.

• Consumers ask “is the data updated yet?” in chat, several times a week.
• Engineers answer the same question differently depending on who is asked.
• Incidents are declared for things that were working as intended.
• Stakeholders stop trusting a table after a single surprise, and go back to exporting spreadsheets.
• Every new request starts with a vague “can you make it faster?”

None of these tell you the pipelines are bad. They tell you that the people who rely on the output are guessing about its behaviour. Guessing is expensive: it produces escalations, workarounds and a slow erosion of goodwill.

The underlying cause: promises that were never made explicit

Data teams tend to think in terms of schedules. A job runs at a given hour, takes a variable time, and lands a table. Consumers think in terms of moments: “when I sit down, it should be there.” Between those two mental models sits a set of unspoken assumptions about freshness, completeness, corrections and downtime.

Because the assumptions are unspoken, each side fills in the blanks generously. The consumer assumes the best case. The engineer assumes everyone understands the usual delays. Both are surprised when reality shows up.

The fix: a one-page delivery note per dataset

The treatment is a short document, written for the consumer, attached to each dataset that people actually depend on. It does not need to be fancy. It needs to answer these questions in plain language.

1. When is it normally ready? A window, not a single minute. For example, “usually ready between early and mid-morning on working days.”
2. What can delay it? Upstream source late, a large backfill, a maintenance period.
3. What does a late day look like? Where the consumer will see a notice and who sends it.
4. What is covered, and what is deliberately absent? Which sources feed it, which are deliberately left out.
5. How are errors handled? Whether corrected rows replace old ones, and how far back corrections may reach.
6. What do we need from you? Report oddities with the table name and time, and tell us before you build something that depends on a tight deadline.

Six short answers. Most teams can draft them for their top five datasets in a single sitting.

Give the window an honest width

The hardest part of the note is the delivery window. Engineers resist because it feels like a commitment. But the alternative is worse: an implied commitment that is stricter than anything you intended. Pick a window you can meet on a normal day with room to spare, then say what the exceptions are.

If you are unsure what the true spread looks like, look at the last stretch of run times and be honest about the slow days, not just the median. Then tell consumers: “most days it is ready by this time; occasionally later; here is how you will find out.”

Be explicit about what checks run

Consumers often assume that if a table exists, someone has verified it. Sometimes that is true and sometimes it is not. Say which checks run automatically and which do not. For a practical walk-through of automated rules on incoming data, our piece on writing validation rules with the Great Expectations library shows how a team can codify what “good enough” means. And if you want to catch bad inputs before anyone downstream sees them, the discussion of how a data observability platform flags trouble early covers the monitoring side.

Whichever route you choose, put the result into the note in ordinary words: “row counts and null checks run on every load; business-rule checks run weekly.” Consumers can then judge for themselves how far to lean on the data.

Where to publish the note

A note nobody can find does nothing. Place it where consumers already look: beside the table in the catalogue, in the header of the report, or pinned in the team channel. Aim for something a person can finish reading before their coffee cools. Review it when anything material changes, and say so when you update it.

A good test is to ask a new analyst, on their first day, to find the delivery note for a dataset without help. If they cannot, move it.

What this looks like outside data

The idea is not unique to pipelines. Anyone who has organised a public gathering knows how much goes wrong when visitors are left to guess. Doors, entry rules, what the ticket covers, how long it lasts: if those are not stated, the audience invents answers.

Event listing sites work well precisely because they push those facts to the front. On BhopalOrbit, a Bhopal portal that lets organisers post their events while customers browse and purchase tickets, the organiser’s description is the delivery note: it tells the visitor what to expect before they commit. Data teams can steal the habit gladly.

Handling the awkward conversation

Introducing delivery notes will surface disagreement. A consumer may say the window is too wide. An engineer may say a tighter window is impossible. That is not a failure of the note; it is the note doing its job. The conversation was always going to happen. Now it happens before the incident rather than during it.

When consumers ask for a tighter window, respond with the cost in plain terms. A faster delivery may mean fewer checks, a more expensive pipeline, or a dependency on a source that is not reliable. Let the stakeholder decide with the trade-off visible. Most will accept a slightly later arrival once they understand what it buys.

Measure whether it worked

You do not need a dashboard of metrics. Watch three simple signals over the next month or two.

• The number of “is it ready?” questions in your channels.
• The number of incidents opened for behaviour that was already described in a note.
• Whether stakeholders cite the note when they ask for a change.

If the questions drop and the conversations become about specific trade-offs rather than general frustration, the treatment is working.

Common mistakes when writing the note

• Writing for engineers. Use the words your consumers use, not the names of jobs.
• Promising a single minute. Windows survive real life; exact times do not.
• Hiding the exceptions. The exceptions are the most useful part.
• Letting the note go stale. An out-of-date note is worse than none.
• Treating it as a contract. It is a courtesy and a guide; it should invite correction.

Prognosis

Teams that adopt delivery notes rarely go back. The notes take little effort, they turn vague irritation into specific requests, and they give engineers something concrete to point at when a day does go wrong. Start with the dataset that generates the most questions. Draft the six answers, send them to two consumers, and ask what surprised them. Then publish, and move to the next one. By the end of the month the Monday-morning message will have become a rarity.