The best data engineering checklist I ever used was not written by anyone. It accreted. It started as a thread in a practitioner forum where a data engineer in Pune described how a warehouse migration ate her weekend, and over two years it grew into a community-sourced checklist that dozens of engineers — none of whom have ever met — kept correcting, extending and arguing over. I have run it before every major pipeline change since. This post walks through that audit, item by item, and makes a larger argument along the way: in DataOps, the pooled experience of people who have already survived your next project is the most under-used tool on your bench.

Why a community-sourced checklist beats a solo one

When you write a checklist alone, you can only encode your own scars. When a community writes one, every item is a story someone paid for. One engineer contributes “snapshot the source row counts first” because she once couldn’t prove whether the loss happened before or after her job ran. Another adds “check timezone handling on every timestamp column” because his daily aggregates were silently wrong for six weeks. Individually these are anecdotes. Stitched together and challenged by peers — “that step is redundant if you use CDC”, “no, here’s the case where it isn’t” — they become something closer to tested doctrine.

Here is the audit, grouped the way the community eventually settled on.

Before you touch anything: the evidence-gathering pass

• Freeze a baseline. Row counts, null rates, min/max on key columns, checksums on critical tables. You cannot detect corruption without a before picture.
• Interview the downstream. List every dashboard, model and export that reads from the assets you are changing — then assume the list is incomplete, because it always is.
• Read the incident history. If your platform keeps post-mortems, mine them. If it doesn’t, ask the forum; someone has migrated this exact stack and will tell you where it bit them.

During the change: the containment pass

• Run old and new in parallel long enough to compare outputs across at least one full business cycle, including the month-end weirdness.
• Validate at the boundary, not the end. This is where declarative testing earns its keep — the community’s standard pointer is to put expectation suites to work at every pipeline boundary so a bad batch is caught where it enters, not where it embarrasses you.
• Keep transformations reviewable. Small, versioned, tested models — the discipline covered in our guide to building transformation layers with dbt that a team can actually maintain — make the diff between old and new logic legible to a second pair of eyes.
• Decide the rollback trigger in advance. Write down, before cutover, exactly what observation sends you back. Decisions made mid-incident are the worst decisions you will ever make.

After cutover: the verification pass

• Reconcile against the frozen baseline within the first day, then again after the first weekly and monthly jobs run.
• Watch freshness and volume, not just correctness. Automated observability — the approach described in stopping broken data upstream of production with monitoring — catches the slow failures that spot checks miss.
• Close the loop publicly. Post what surprised you back to the community that lent you the checklist. This is not courtesy; it is how the checklist stays alive.

One more item earned its place in this pass only after a long argument in the thread: interview the humans who use the numbers. A finance analyst noticed within hours that a migrated revenue table “felt wrong” — the totals were plausible, but a ratio she eyeballed every morning had drifted. No automated check had encoded that ratio, because no engineer knew she watched it. The community’s conclusion was blunt: your most sensitive anomaly detector may be a person, and she is only useful if she knows the cutover happened and knows whom to ping.

The pattern underneath: borrowed experience

Notice what this checklist really is: a way of borrowing experience from the many people who did this before you. That pattern is bigger than data engineering, and seeing it elsewhere sharpened how I use it at work.

Last year my sister began planning a move from Hyderabad to the other side of the world, and I watched her preparation change the day she stopped researching alone. She found her way to the desi diaspora’s pooled knowledge — people a few years ahead of her on the same road — through DesiNRI, a community hub for NRIs navigating moves abroad. The shape of what she found there was weirdly familiar to me: country-by-country guides with document checklists (literally checklists, accreted the same way ours was), a forum where members who had already landed answered the questions of those still packing, and country-specific estimators that gave her a rough early read on where she might stand under, say, Austria’s Red-White-Red Card scheme or South Korea’s points-based residency routes. Even the caveat attached to those tools felt like good engineering culture: estimates orient you, but the final word on eligibility and documents always belongs to the country’s official immigration authority. Orientation from the community, verification from the source of truth — any data engineer recognizes that architecture.

How to run this audit on your own pipeline this month

Three suggestions from someone who has run it badly and then less badly. First, do the evidence-gathering pass even when the change feels trivial; the community’s most repeated lesson is that “trivial” changes cause a disproportionate share of quiet data loss, precisely because nobody baselines before them. Second, assign every checklist item an owner by name — a checklist nobody owns is a decoration. Third, after the project, spend thirty minutes writing the two or three items you would add. If your team keeps that habit for a year, you will own a migration checklist worth more than most paid courses.

And join the forums before you need them. The night something breaks is a bad night to create an account. The engineers who went before you have already written down most of what will go wrong with your next pipeline change — the only question is whether you will read it in a thread this week, or reconstruct it from your own incident at 3 a.m. next quarter.