{"id":4451,"date":"2026-11-09T00:35:00","date_gmt":"2026-11-09T00:35:00","guid":{"rendered":"https:\/\/dataopsschool.com\/blog\/?p=4451"},"modified":"2026-11-09T00:35:00","modified_gmt":"2026-11-09T00:35:00","slug":"data-estate-health-audit","status":"publish","type":"post","link":"https:\/\/dataopsschool.com\/blog\/data-estate-health-audit\/","title":{"rendered":"A Checklist for Auditing the Data Pipelines You Already Run"},"content":{"rendered":"<p>Data teams measure plenty of things, yet few measure how much of their own estate is still alive. Pipelines get built for a project, the project ends, and the tables keep refreshing for years because nobody dared switch them off. This checklist treats your existing DataOps estate as something to inspect before you add another source or another model. Work through it in order and keep a note of the evidence for each answer, because a claim of &#8220;it seems fine&#8221; is exactly what these checks exist to replace.<\/p>\n<p>The approach is analytical on purpose. For each item you gather a small piece of evidence, compare it with what you expected, and decide to keep, repair or retire. That three-way decision is the whole method.<\/p>\n<h2>Part one: find out what exists<\/h2>\n<p>You cannot judge what you cannot list. Export every scheduled job, every orchestrated workflow and every table that receives writes. Then add the consumers: dashboards, notebooks, exports and downstream models. The list will usually reveal orphans on both sides, jobs that write data nobody reads and reports that read data nobody refreshes.<\/p>\n<ul>\n<li>Record the owner, the schedule and the last time the output was actually consumed.<\/li>\n<li>Mark anything with no named owner as &#8220;unclaimed&#8221; and give it a review date.<\/li>\n<li>Note which jobs have not failed in a long time. Silence can mean health, or it can mean nobody is watching.<\/li>\n<\/ul>\n<p>One practical trick is to ask the people who read the dashboards, not only the people who build the pipelines. A short conversation with an analyst often uncovers the table everyone avoids because it was wrong once and never fixed. That folk knowledge is evidence too, and it points you at the assets with the weakest reputations.<\/p>\n<h2>Part two: test the tests<\/h2>\n<p>Validation suites decay too. A rule written for last year&#8217;s schema may pass trivially because the column it guards is now always populated by a default value. Check that each expectation can still fail. One way is to feed it a deliberately broken sample and confirm that it complains. If you use a library for declarative checks, the guide to <a href=\"https:\/\/dataopsschool.com\/blog\/great-expectations-for-data-validation-in-dataops-a-practical-guide\/\">writing data validation rules in practice<\/a> is a useful companion while you review which expectations still earn their place.<\/p>\n<ul>\n<li>Look for checks that have never failed. They are either perfect or toothless.<\/li>\n<li>Look for checks that fail constantly and have been muted. They are noise that hides real signal.<\/li>\n<li>Confirm that thresholds still match current data volumes.<\/li>\n<\/ul>\n<p>When you review a suite, sort it by age. The oldest expectations were written under assumptions that may no longer hold, and the newest were written in a hurry after an incident. Both groups deserve a second look, for opposite reasons.<\/p>\n<h2>Part three: inspect the transformation layer<\/h2>\n<p>Transformation code accumulates sediment. Models are copied, renamed and left behind, and lineage grows tangled. Review which transformations still feed something. A walkthrough of <a href=\"https:\/\/dataopsschool.com\/blog\/master-data-transformations-with-dbt-in-modern-dataops-pipelines\/\">building reliable transformations with dbt<\/a> describes the structure you should be aiming for, and comparing your repository with that shape quickly shows where drift has set in.<\/p>\n<ul>\n<li>Identify models with no downstream dependants and no exposure to a report.<\/li>\n<li>Check for duplicated logic that now produces slightly different numbers in two places.<\/li>\n<li>Confirm that documentation strings still describe what the model does.<\/li>\n<\/ul>\n<p>Be wary of the temptation to rewrite everything at this stage. The audit is about knowing the state of things. Rewrites come after, and only for the assets that justify them. Small repairs, such as deleting a dead model or renaming a confusing one, deliver most of the value for a fraction of the effort.<\/p>\n<h2>Part four: verify orchestration and credentials<\/h2>\n<p>Workflow engines are happy to run stale graphs forever. Open the scheduler and review each workflow for outdated dependencies, retired source systems and tasks that always succeed in zero seconds, which usually means they do nothing. Then list every secret the estate relies on: service accounts, tokens, warehouse roles and API keys. Note the expiry of each and who rotates it. For a broader view of coordinating tasks, the article on <a href=\"https:\/\/dataopsschool.com\/blog\/how-dagster-brings-order-to-your-data-chaos\/\">bringing order to scattered data workflows<\/a> offers a good vocabulary for the review.<\/p>\n<ul>\n<li>Flag any credential that belongs to a person who has left.<\/li>\n<li>Check that backfills and retries still behave as documented.<\/li>\n<li>Look for tasks pointing at hosts or buckets that no longer exist.<\/li>\n<\/ul>\n<p>Plan for the unglamorous cases as well. A retired source system may still be queried by a legacy report that someone checks once a quarter. A rotated credential may break a monthly export that nobody remembers. Searching your logs for rare access patterns, not only frequent ones, catches these stragglers before the owner finds them the hard way.<\/p>\n<h2>Part five: verify the references around the data<\/h2>\n<p>Data work is surrounded by links. Catalog entries point to source documentation, dashboards link to definitions, tickets link to design notes, and onboarding pages link to dozens of tools. These references decay silently. A link-management platform like <a href=\"https:\/\/urlsnow.com\/\">URLSnow<\/a> performs automatic health checks and flags links that stopped responding, so the shared pages your analysts depend on can be checked without anyone clicking through them by hand. Feed it the references from your catalog and wiki, then repair what it surfaces.<\/p>\n<p>Treat the output as a living artefact. When a new hire asks which dashboards to trust, you will be able to answer with evidence instead of folklore, and that alone justifies the first round of work.<\/p>\n<h2>Part six: decide and record<\/h2>\n<p>The audit has value only when it ends in decisions. Build a simple table with one row per asset and three columns: keep as is, repair, or retire. For retirements, agree a notice period with the consumers, turn the job off first, wait, and only then delete data and remove access. Keep a snapshot where policy requires it.<\/p>\n<p>Record the findings in the same place as your runbooks, but resist turning the audit into a document nobody opens. The most useful output is a short list of repairs with owners and dates.<\/p>\n<h2>How to read the results<\/h2>\n<p>Expect to retire more than you think. Expect a handful of silent failures, mostly in places where nobody had looked for a long time. If you find nothing, check that you looked hard enough rather than celebrating, and rerun the exercise on a rotating basis so that every area receives attention in turn.<\/p>\n<p>A final analytical note: compare the effort of this audit with the cost of a single bad number reaching a leadership meeting. The inspection is cheap by comparison. When the estate is trustworthy, new pipelines can be added with the confidence that they sit on solid ground, and the people who rely on your data stop wondering which table to believe.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Before adding another source or model, inspect the pipelines, tests, transformations and credentials already running. A six-part evidence-led checklist.<\/p>\n","protected":false},"author":6,"featured_media":4452,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[814],"tags":[],"class_list":["post-4451","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dataops"],"_links":{"self":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/4451","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=4451"}],"version-history":[{"count":1,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/4451\/revisions"}],"predecessor-version":[{"id":4508,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/4451\/revisions\/4508"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media\/4452"}],"wp:attachment":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=4451"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=4451"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=4451"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}