A pipeline went live, the backfill finished, and the dashboards agree with finance. Now what? Most data teams move straight to the next ticket. There is a low-cost alternative, the start-stop-continue retrospective: thirty minutes to sort the last stretch of work into three lists. Below are the questions newcomers raise most often, each answered with an example from ordinary data engineering.
What is a start-stop-continue retrospective for a data team?
Picture a brief review that happens once a defined chunk of work wraps up, such as a release, a migration or a sprint. The team writes three lists. Start: things we are not doing but should try. Stop: things we are doing that cost more than they return. Continue: things that work and need protecting. Each item must be specific enough that someone could tell, next time, whether it happened.
When should we hold one?
At a natural boundary. After a pipeline goes into production, after a quarter-end reporting crunch, after an incident has been resolved and the dust has settled. Do it while memory is fresh, ideally within a week. Avoid the middle of a crisis, when everyone is too tired to be fair to each other.
Who needs to be in the room?
Everyone who touched the work: engineers, analysts, whoever owns the source system, and a representative of the people who consume the data. The consumer view matters. Engineers often believe a release went smoothly while an analyst quietly spent two days reconciling a column that changed meaning.
What does a good “start” look like for data work?
Small, concrete, and tied to something that hurt. Examples:
- Start adding a row-count and null-rate check to every new table before it feeds a report.
- Start writing a short data contract note when a source team changes a column, so downstream users hear about it before the dashboard breaks.
- Start running the backfill on a sample first and comparing it to the old output.
- Start giving each scheduled job a named owner who gets the alert.
If the start would need a quarter of engineering time, break it down until the first step fits in a day.
What belongs on the “stop” list?
Habits that create rework or hide problems. Typical entries from data teams:
- Stop patching numbers by hand in the reporting layer. Fix the cause upstream or record the exception openly.
- Stop shipping transformations without tests because the deadline felt tight; the time comes back with interest on the next incident.
- Stop copying a notebook into a scheduled job unchanged. Move the logic into versioned code first.
- Stop answering the same ad hoc question in chat forty times; write it up once.
Phrase each stop as a habit of the team, not a person. And pair it with a replacement. “Stop patching by hand” is easier to honour if “open a ticket against the upstream model” sits next to it.
Why does the “continue” list matter?
Because teams under pressure drop their best habits first. If everyone agrees that the pre-release check against yesterday’s totals caught three problems, state it in the meeting and record it. Next time the schedule tightens, someone will remember it is on the continue list and will defend the half hour it takes.
Can you give a realistic example of all three lists?
Here is an invented scenario for illustration. A team migrated a set of nightly jobs to a new orchestrator. Looking back, they wrote:
- Start: tag every job with the business report it feeds, so an outage can be translated into “which numbers are stale.”
- Stop: stop letting failed jobs retry silently overnight; surface the first failure to a human.
- Continue: keep the morning check where one person compares headline totals with the previous day and posts a one-line status.
Three items. Within a month the tagging habit alone cut the time to answer “is the revenue report wrong?” from an hour of guessing to a few minutes of looking.
How do validation tools fit into this?
Often a start item is about adopting checks that already exist in the ecosystem. For instance, an article on validating data with the Great Expectations library describes the kind of habit that frequently lands on a start list. Likewise, teams that retire fragile SQL scripts may note on their stop list the practice of hand-built transformations and read the walkthrough of dbt in modern DataOps pipelines for a replacement. Neither tool is mandatory; the point is that the retro tells you which problem to solve first.
How do we avoid it turning into a complaint session?
Use a timer, a facilitator, and a rule that every complaint must be converted into a start, a stop or a continue. If someone says “the data is always late,” the facilitator asks: what is one action that would make it less late? That conversion is the whole skill. After a few sessions the team does it without prompting.
How many items should we keep?
Two or three per list, each with an owner. More than that and nothing gets finished. Park the rest in a backlog and look at it next time. A lean list that gets completed beats a long list that gets admired.
Where do outside examples come from?
Other professions hold similar reviews, and their framing can lend a fresh angle. An aviation browsing site such as BestPilotsNow, which is organised around pilots, flying schools, courses, resources, a discussion board and articles, shows how one website can file very different material under clear headings. A data team reviewing its documentation could ask the same thing: which questions deserve their own labelled place, and which are clutter?
What happens after the meeting?
Put the three lists where the team will see them, such as the top of the team’s wiki or the board’s header. Create tickets for the starts and the stops. At the next retrospective, begin by reading the previous lists aloud and marking each item done, partly done or dropped. If a stop appears twice in a row, read it as a sign that the root of the problem lies elsewhere, and go and look there.
What if the team works remotely across time zones?
Run it asynchronously. Share a document with the three headings a day ahead, give everyone until a set hour to add items quietly, then hold a short call only to group duplicates and choose the final few. Written input also helps quieter people, who often hold the sharpest observations about handovers between regions. Keep a single owner for the final lists so that nothing sits in a comment thread unclaimed.
How do we know the retrospective actually worked?
Look for changes in the work, not in the mood of the meeting. Did the number of manual fixes go down? Did the morning check get shorter or more reliable? Did a new analyst find the answer in the wiki instead of asking in chat? Pick one observable sign per item when you write it, such as “no unowned jobs left on the schedule,” and check it at the next session. Items that cannot be checked tend to drift, so rewrite them until they can.
Is it worth doing for a team of two?
Yes, and it takes ten minutes. Small teams benefit most because every habit is visible and one person leaving can erase an unwritten routine. The lists become the team’s memory.
The short version
Look back at the last stretch of work. Write three lists. Keep two or three items each, name an owner for each, and read the lists again next time. That is what a data team needs from a start-stop-continue retrospective, and it costs less than one bad reconciliation.