Early in a data career, a report number refuses to match the source, and you spend an evening staring at SQL that looks perfectly fine. Then you start muttering the query to a mug on your desk, and halfway through the second join you hear yourself say something that cannot be true. This habit has a name, rubber duck debugging for data pipelines, and it is the friendliest first skill a newcomer can learn. This primer shows how it works with one small story plus a small word list to keep nearby.
The story: the orders table that grew overnight
A junior analyst, new to a retail data team, builds a daily revenue table. Source orders: 4,000 rows for the day. The finished table says revenue is almost double what the shop’s own till reports. The analyst checks the formulas, then the filters, then the dates. Everything looks right, and the mood sinks.
A teammate passing by says only, “Tell the duck.” Embarrassed but out of ideas, the analyst turns to a small yellow figure by the monitor and begins: “I take orders. One row per order. I join order lines, because I need quantities. Then I join payments, because I need the amount paid. Then I add up the amount paid…”
The voice slows. “One row per order, and then I join order lines, which has one row per item. So after that join I have one row per item. And then I add the payment amount to each of those rows. An order with three items gets its payment counted three times.”
The analyst laughs. The listener never answered, yet the answer arrived. The bug was never in the arithmetic; it was in an unspoken belief that the table still had one row per order after a join.
A mini-glossary for the first week
These words come up every time someone narrates a pipeline. Learn them as you would learn the names of tools in a kitchen drawer.
- Grain: what one row of a table stands for, such as one order, one item or one customer per day. Saying the grain aloud at every step is the single most useful part of the routine.
- Join key: the column used to match rows across two tables. If the key repeats on one side, matches multiply.
- Fan-out: the multiplication that happens when one row matches several rows. Totals taken after a fan-out are inflated.
- Null: a missing value. It is not zero and not an empty string, and many comparisons quietly drop it.
- Idempotent: safe to run twice with the same result. A load job that appends duplicates on re-run is not idempotent.
- Lineage: the path a number travelled from source to report, step by step.
Talking a pipeline through: five moves
No equipment is required. A quiet corner, a few minutes and a rule that every sentence must name the grain will do.
- Start at the source. Say which system the data comes from and what a single row means there.
- Narrate each transformation. “Then I filter,” “then I join,” “then I group.” After each, restate the grain out loud.
- Say the expected count. “I expect about the same number of rows here.” Then check the actual count. A mismatch points straight at the step you just described.
- Listen for filler. Words like “somehow,” “just” and “basically” cover places where you are not sure what happens.
- Stop at the first doubt. Do not push on. Test that one step on a tiny sample.
Why rubber duck debugging beats staring
When you read your own SQL, your eyes confirm what your memory already believes. Speaking cannot skip: you have to produce each sentence, and a sentence needs a subject. “I join payments” invites the question “to what grain?”, which silent reading never asks. That is also why the technique helps beginners most. Experts hold the grain in their heads automatically; newcomers have to say it until it becomes instinct.
Being stuck also hurts less this way. Asking a colleague can feel like admitting weakness, and the exercise lets you reach for a fix before you have to ask anyone. It is solitary on purpose, and it is quick enough to try before your coffee goes cold.
Two more small examples to try
Dates that drift. A daily load reports yesterday’s figures under today’s label. Narrated aloud: “The job starts at midnight, reads the previous day, and stamps the run date.” Hearing “stamps the run date” instead of the data date reveals which timestamp the label was copied from.
Totals that dip after a clean-up. A new rule removes test accounts. Aloud: “I remove accounts whose email ends in the test domain… and I count customers by joining to accounts, so any customer with no account row vanishes.” The word “vanishes” is the clue: an inner join dropped them.
Where to practise and read next
Practice on pipelines you already know before you apply it to a mystery. If you want worked material on the tooling side, a guide to writing automated data validation checks describes the assertions you can place once your narration has located a trouble spot, and a walkthrough of building transformations with dbt shows how stepwise models make each stage easier to narrate, since every model has its own grain.
Curious how other subjects package their reading lists? One example is a gastroenterology-oriented directory site arranged around hospital and doctor listings plus a blog and forum. Whatever the subject, a new reader benefits from naming each section’s purpose aloud, exactly as you name each table’s grain.
Common beginner worries
“I feel silly.” Everybody does the first time. Whisper if you must. The value comes from forming sentences, not from volume.
“My pipeline has forty steps.” Narrate in sections and verify row counts at the boundaries. Split where the grain changes.
“What if I find nothing?” Then you have ruled out the described path, and the surprise lives in something you did not describe: a hidden default, a scheduled job, a view somebody edited. Write down what you skipped, and begin there.
Take-away
Your first week in data will bring plenty of numbers that look wrong. Before you open another tab, tell the story of the data to something that cannot answer back, and keep the grain in every sentence. When the story snags, you have found the place to look.