Karuna runs a twelve-person analytics team at a fictional mid-size retailer we will call Meridian Goods, and her year is the clearest way to explain what DataSecOps looks like when it leaves the slide deck and enters an actual warehouse. Nothing in this story is exotic: one Snowflake account, dbt models, a BI tool, and a compliance letter that arrives in February. If you work in data engineering, every beat of it will feel familiar, including the parts that hurt.<\/p>\n
The letter asks a simple thing \u2014 list every place customer payment and health-adjacent data lives, who can read it, and under which rule. Karuna’s team ships excellent pipelines and their DataOps discipline is real: tests run before deploy, monitors catch freshness breaks, the sort of setup described in our piece on how Monte Carlo catches bad data before production<\/a>. But governance is a different muscle. The honest answer to the auditor is a shrug. Nobody knows which of the four hundred tables hold sensitive columns, because classification happened informally, in people’s heads. The team spends two demoralising weeks grepping schemas. That exercise is the moment most organisations start paying attention, and it is exactly the moment the practice known as DataSecOps is built for.<\/p>\n February turns into March and the team does the least glamorous thing in this entire field: they count. A discovery sweep runs across every schema and produces candidate tags \u2014 identifiers here, financial values there, free-text notes that might contain anything. Here is the part worth repeating to every eager stakeholder: the scanners are helpers, not oracles. Both Google’s classifier and Presidio miss odd formats and flag innocent ones, so the team institutes a rule that a human confirms every IDENTIFIER and SENSITIVE tag before any policy keys off it. They restrict the first pass to the fifty most-queried tables rather than the whole estate, which keeps the effort shippable. Ownership gets written down too \u2014 each tier-1 table now has a name attached, and that name is a person, not a department.<\/p>\n April brings the structural change: policies stop living in a console and start living in the repository, reviewed like any other code. A masking rule for email columns is a file. A row filter restricting regional sales tables to regional staff is a file with tests. Access stops being a favour \u2014 a request now names the dataset, the purpose and how long it is needed, the table’s owner approves it like a pull request, and the grant carries an expiry from day one. Standard requests clear within the day; anything odd requires a ticket and a recorded risk sign-off. The team also agrees on a quarterly revoke pass, on the theory that a grant nobody can justify is a liability with a timestamp. Readers who want the wider framing of governance inside fast pipelines will find it in our article on governing risk in fast-moving data pipelines<\/a>.<\/p>\n May is the month of engine quirks, and the logs are worth sharing because they generalise. Snowflake remains an excellent warehouse and simultaneously a place where assumptions go to die:<\/p>\n Every one of those surprises gets a workaround and a line in the runbook. Password-only sign-ins are also being phased out through 2026, which the team treats as a deadline rather than a suggestion, since keys and MFA are precisely what the 2024 credential attacks exploited.<\/p>\n By the summer review the dashboard finally has four figures worth defending. Column coverage: 1,085 of 1,240 columns sit behind a policy, which computes to 87.5%. Plain-text exposure: 17.1% of reads against sensitive columns still land on unmasked values, mostly from service accounts with legacy grants. Over-exposure: 212 of 940 principals can see more than their role needs, a rate of 22.6%. Dormancy: 310 of 1,900 grants \u2014 16.3% \u2014 have gone ninety days without use. Each number has an owner and a trend arrow. Maturity is now measured against explicit exits: past the halfway mark on tier-1 coverage earns level two, and level three wants over 90% coverage plus zero console-only policies. The practice has a name and a ladder \u2014 the Ops Atlas page cataloguing DataSecOps<\/a> is where the team sends curious stakeholders instead of explaining it over lunch for the tenth time.<\/p>\n The autumn review also fixes a gap the metrics exposed but the warehouse could not. PostgreSQL side systems feeding the lake turn out to have their own audit blind spot \u2014 pgAudit, solid as it is, cannot reliably capture what superusers do, so the team adds OpenLineage events from the transformation jobs as a second trail. Non-production environments get one more treatment: instead of cloning raw customer tables into staging, the pipeline rebuilds them from masked sources, which removes an entire category of “we only leaked it internally” incidents. Both changes came out of the scorecard conversation, which is the quiet argument for publishing your numbers somewhere people argue about them.<\/p>\n The story ends undramatically, which is the point. By autumn, a new sensitive table gets classified, tagged and covered before its first grant goes out \u2014 every time, without a memo. That habit is the top rung of the Atlas ladder, called Adaptive, and it arrives not through a tool purchase but through the unglamorous sequence the team installed: count, tag with human review, version the rules, enforce at query time, read the history, revoke on schedule. The auditor’s follow-up letter that October asks for evidence and receives a link instead of an apology. Teams wanting to push the same rigour into their release mechanics will find the natural next step in our introduction to DataOps testing automation<\/a>. Fictional Karuna would be the first to say nothing here required genius \u2014 only that somebody finally owned the question.<\/p>\n","protected":false},"excerpt":{"rendered":" A fictional analytics team gets an auditor’s letter, counts its columns, version-controls its rules, and learns where Snowflake quietly pushes back.<\/p>\n","protected":false},"author":6,"featured_media":4316,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[816],"tags":[],"class_list":["post-4315","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-security"],"_links":{"self":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/4315","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=4315"}],"version-history":[{"count":1,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/4315\/revisions"}],"predecessor-version":[{"id":4336,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/4315\/revisions\/4336"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media\/4316"}],"wp:attachment":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=4315"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=4315"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=4315"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Counting Before Claiming<\/h2>\n
Rules You Can Version<\/h2>\n
Where Snowflake Pushes Back on DataSecOps<\/h2>\n
\n
Numbers That Fit on One Slide<\/h2>\n
DataSecOps, One Grant at a Time<\/h2>\n