1. Introduction & Overview What is Dagster? Dagster is an open-source data orchestrator for machine learning, analytics, and ETL (Extract, Transform, Load) workflows. It focuses on writing, deploying, and monitoring data pipelines in a structured, modular, and testable way. Unlike traditional orchestrators (e.g., Airflow), Dagster promotes a software engineering mindset—which aligns closely with DevSecOps principles … Read more
1. Introduction & Overview What is Prefect? Prefect is an open-source data workflow orchestration tool designed to build, run, and monitor complex pipelines with ease. It enables teams to automate and manage workflows in a scalable, observable, and fault-tolerant manner. Prefect supports dynamic scheduling, retries, caching, and parameterization—all critical features for maintaining robust pipelines in … Read more
1. Introduction & Overview What is dbt (Data Build Tool)? dbt (data build tool) is an open-source command-line tool that enables data analysts and engineers to transform data in their data warehouse more effectively. It allows teams to write modular SQL queries, version-control their analytics code, and automate data transformations using software engineering best practices. … Read more
1. Introduction & Overview Apache Airflow has become a staple in modern data orchestration and workflow automation. In the DevSecOps paradigm, where automation, traceability, and security are critical, Airflow offers capabilities that make it a natural fit for orchestrating security controls, CI/CD tasks, compliance workflows, and more. What is Apache Airflow? Apache Airflow is an … Read more
1. Introduction & Overview What is Schema Evolution? Schema Evolution refers to the process of managing changes to the structure of data (schemas) in a way that maintains compatibility, data integrity, and system performance. In the context of databases or data pipelines, this often means evolving table structures, message formats (e.g., Avro, JSON), or APIs … Read more
📘 Introduction & Overview What is Data Masking? Data Masking is the process of hiding original sensitive data with modified content (characters or other data) that retains the functional format. The goal is to protect data while ensuring that masked datasets remain useful for development, testing, or analytics. Masked data may look real but is … Read more
1. Introduction & Overview What is Tokenization? Tokenization is the process of substituting sensitive data elements with a non-sensitive equivalent—called a token—that has no exploitable value. Unlike encryption, tokenization doesn’t use reversible cryptographic functions but maps sensitive values to tokens through a secure token vault. History or Background Why is it Relevant in DevSecOps? 2. … Read more
📌 Introduction & Overview What is Anonymization? Anonymization is the process of transforming personal or sensitive data in a way that prevents the identification of individuals, even indirectly. Unlike pseudonymization (which replaces identifiers with pseudonyms but still allows re-identification with additional data), anonymization removes or masks all identifiable information irreversibly. In DevSecOps—where security is a … Read more
1. Introduction & Overview What is Normalization? Normalization in the context of DevSecOps refers to the process of transforming data, configurations, logs, or system inputs into a standardized and consistent format. This enables better comparison, automation, validation, security analysis, and decision-making across environments and toolchains. It is applied in areas such as: History or Background … Read more
1. Introduction & Overview What is Cleansing? In DevSecOps, cleansing refers to the practice of removing, sanitizing, or redacting sensitive data, metadata, or malicious inputs from systems, codebases, logs, and configurations to reduce security risks and maintain compliance. It ensures that secrets, personally identifiable information (PII), or vulnerabilities are not propagated across the software development … Read more