{"id":339,"date":"2025-08-06T01:02:59","date_gmt":"2025-08-06T01:02:59","guid":{"rendered":"https:\/\/dataopsschool.com\/blog\/?p=339"},"modified":"2025-08-06T01:03:00","modified_gmt":"2025-08-06T01:03:00","slug":"complete-data-glossary-terminology","status":"publish","type":"post","link":"https:\/\/dataopsschool.com\/blog\/complete-data-glossary-terminology\/","title":{"rendered":"Complete Data Glossary & Terminology"},"content":{"rendered":"\n

Here\u2019s a comprehensive glossary of all the key data platform, engineering, and analytics terms<\/strong> we discussed\u2014including everything from your earlier questions and the expanded list<\/strong>. Each keyword includes a simple explanation. This will give you a full \u201ccheat sheet\u201d of modern data terminology.<\/p>\n\n\n\n

\n\n\n\n

Complete Data Glossary & Terminology<\/strong><\/h2>\n\n\n\n
Keyword<\/strong><\/th>Meaning \/ Description<\/strong><\/th><\/tr><\/thead>
Raw Data Sources<\/strong><\/td>The origin of data (APIs, apps, logs, sensors, files, databases, etc.)<\/td><\/tr>
Data Lake<\/strong><\/td>A storage system for raw or semi-structured data, supporting all formats (CSV, JSON, images, logs, etc.)<\/td><\/tr>
Data Lakehouse<\/strong><\/td>Combines a data lake\u2019s flexibility with data warehouse reliability (e.g., Databricks Lakehouse)<\/td><\/tr>
Data Warehouse<\/strong><\/td>Centralized, structured storage for cleaned and processed data, optimized for analytics and reporting<\/td><\/tr>
Data Mart<\/strong><\/td>A subset of a data warehouse, focused on a specific business unit or subject area<\/td><\/tr>
Delta Lake<\/strong><\/td>Databricks\u2019 open-source storage layer that brings ACID transactions to data lakes<\/td><\/tr>
ETL (Extract, Transform, Load)<\/strong><\/td>Process of extracting data from sources, transforming it, and loading it into a warehouse or lake<\/td><\/tr>
ELT (Extract, Load, Transform)<\/strong><\/td>Similar to ETL, but data is loaded first, then transformed inside the data warehouse\/lakehouse<\/td><\/tr>
Data Pipeline<\/strong><\/td>A workflow or series of steps that move and process data automatically<\/td><\/tr>
Data Engineering<\/strong><\/td>The discipline of building systems\/pipelines to collect, clean, transform, and store data for further use<\/td><\/tr>
Data Analytics<\/strong><\/td>The practice of analyzing and visualizing data to find insights and support business decisions<\/td><\/tr>
Business Intelligence (BI)<\/strong><\/td>Tools and processes for building dashboards, visualizations, and reports from data (e.g., Tableau, Power BI)<\/td><\/tr>
Data Science<\/strong><\/td>The field of applying statistics and machine learning to data for predictions, classification, and modeling<\/td><\/tr>
Machine Learning (ML)<\/strong><\/td>Techniques and models that enable computers to learn from data and make predictions or decisions<\/td><\/tr>
Streaming Data<\/strong><\/td>Data that is generated and processed in real-time (continuous flow), e.g., IoT, logs, transactions<\/td><\/tr>
Batch Processing<\/strong><\/td>Processing data in discrete chunks or batches, often on a schedule<\/td><\/tr>
Structured Streaming<\/strong><\/td>Apache Spark\u2019s method for processing streaming data with the same API as batch data<\/td><\/tr>
Auto Loader<\/strong><\/td>Databricks\u2019 tool to automatically ingest new files as they land in a data lake<\/td><\/tr>
Delta Live Tables (DLT)<\/strong><\/td>Databricks\u2019 framework for declarative, automated data pipeline development and management<\/td><\/tr>
Orchestration<\/strong><\/td>Managing and scheduling sequences of data processing tasks or pipelines (e.g., Airflow, Databricks Workflows)<\/td><\/tr>
Data Platform<\/strong><\/td>An environment that combines data storage, processing, analytics, ML, and governance (e.g., Databricks, Snowflake)<\/td><\/tr>
Data Analytics Platform<\/strong><\/td>Synonym for \u201cdata platform\u201d\u2014emphasizes analytics and ML as part of the solution<\/td><\/tr>
Data Governance<\/strong><\/td>Policies and controls for data access, privacy, quality, security, cataloging, and compliance<\/td><\/tr>
Data Quality<\/strong><\/td>Ensuring data is accurate, complete, consistent, and reliable<\/td><\/tr>
Data Catalog<\/strong><\/td>An organized inventory of all data assets (tables, files, fields), enabling data discovery and management<\/td><\/tr>
Metadata<\/strong><\/td>\u201cData about data,\u201d such as schema, owner, creation date, usage, etc.<\/td><\/tr>
Schema<\/strong><\/td>The structure\/definition of a dataset (field names, types, constraints)<\/td><\/tr>
Data Lineage<\/strong><\/td>The record of where data comes from, how it moves, and how it\u2019s transformed over time<\/td><\/tr>
Data Stewardship<\/strong><\/td>Responsibility for managing and overseeing the proper use of data assets<\/td><\/tr>
Data Masking<\/strong><\/td>Obscuring or hiding sensitive data values to protect privacy<\/td><\/tr>
Data Encryption<\/strong><\/td>Securing data (at rest\/in transit) by encoding it so only authorized parties can read it<\/td><\/tr>
Role-Based Access Control (RBAC)<\/strong><\/td>Security system that restricts data access based on user roles\/permissions<\/td><\/tr>
Table Access Control<\/strong><\/td>Permissions system for controlling who can read\/write specific tables\/views<\/td><\/tr>
Unity Catalog<\/strong><\/td>Databricks\u2019 unified data governance solution for managing access, auditing, and cataloging<\/td><\/tr>
Data Sharing<\/strong><\/td>The ability to share datasets securely between teams, organizations, or platforms (e.g., Delta Sharing)<\/td><\/tr>
Data Mesh<\/strong><\/td>A decentralized data architecture that assigns ownership and responsibility to domain teams<\/td><\/tr>
Data Fabric<\/strong><\/td>An architecture and set of data services providing integrated, consistent data management across the enterprise<\/td><\/tr>
DataOps<\/strong><\/td>Application of DevOps practices (automation, CI\/CD, monitoring) to data engineering and pipelines<\/td><\/tr>
MLOps<\/strong><\/td>DevOps for machine learning: automating deployment, monitoring, and lifecycle management of models<\/td><\/tr>
Master Data Management (MDM)<\/strong><\/td>Ensuring a consistent, accurate \u201csingle source of truth\u201d for core business data<\/td><\/tr>
Semantic Layer<\/strong><\/td>An abstracted data model providing consistent business logic and definitions to analytics users<\/td><\/tr>
Data Steward<\/strong><\/td>A person responsible for the quality and governance of specific data assets<\/td><\/tr>
Data Orchestration<\/strong><\/td>See Orchestration above<\/td><\/tr>
Data Integration<\/strong><\/td>Combining data from different sources and formats into a unified view<\/td><\/tr>
Data Transformation<\/strong><\/td>Changing, cleaning, or converting data from one format to another<\/td><\/tr>
Data Ingestion<\/strong><\/td>The process of importing or bringing new data into your platform\/lake\/warehouse<\/td><\/tr>
Data Cleansing<\/strong><\/td>Identifying and correcting errors, duplicates, or inconsistencies in data<\/td><\/tr>
Data Modeling<\/strong><\/td>Defining and structuring how data is stored, connected, and related<\/td><\/tr>
Data Partitioning<\/strong><\/td>Dividing large datasets into segments (partitions) for faster processing and querying<\/td><\/tr>
Z-Ordering<\/strong><\/td>A technique (in Delta Lake) to co-locate related data in storage for performance optimization<\/td><\/tr>
Time Travel<\/strong><\/td>The ability (in Delta Lake) to query or restore data as it existed at a previous point in time<\/td><\/tr>
Versioning<\/strong><\/td>Keeping track of changes to data or tables so you can audit or roll back<\/td><\/tr>
Compaction<\/strong><\/td>Merging many small files into fewer, larger files for performance (common in Delta Lake, Hudi, Iceberg)<\/td><\/tr>
Partition Pruning<\/strong><\/td>Query optimization by skipping partitions of data that don\u2019t match filter criteria<\/td><\/tr>
Data Skipping<\/strong><\/td>Optimization by skipping irrelevant data blocks during query<\/td><\/tr>
Broadcast Join<\/strong><\/td>A Spark optimization where a small table is sent to all nodes for faster joins<\/td><\/tr>
Checkpointing<\/strong><\/td>Periodically saving the state of a stream or computation for fault tolerance<\/td><\/tr>
Pipeline DAG<\/strong><\/td>Directed Acyclic Graph; visualizes the dependencies and order of tasks in a data pipeline<\/td><\/tr>
Job Cluster<\/strong><\/td>A cluster in Databricks that\u2019s spun up for a specific job and then terminated<\/td><\/tr>
All-Purpose Cluster<\/strong><\/td>A cluster for interactive work, like notebooks and ad-hoc queries<\/td><\/tr>
Serverless SQL Warehouse<\/strong><\/td>Databricks\u2019 serverless endpoint for SQL analytics, scalable and managed<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n

<\/h2>\n\n\n\n
\n\n\n\n

<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Here\u2019s a comprehensive glossary of all the key data platform, engineering, and analytics terms we discussed\u2014including everything from your earlier questions and the expanded list. Each keyword… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-339","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/339","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=339"}],"version-history":[{"count":1,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/339\/revisions"}],"predecessor-version":[{"id":340,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/339\/revisions\/340"}],"wp:attachment":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=339"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=339"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=339"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}