Metadata-Version: 2.4
Name: airflow-pydantic
Version: 1.5.9
Summary: Pydantic models for Apache Airflow
Project-URL: Repository, https://github.com/airflow-laminar/airflow-pydantic
Project-URL: Homepage, https://github.com/airflow-laminar/airflow-pydantic
Author-email: the airflow-pydantic authors <t.paine154@gmail.com>
License: Apache-2.0
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Requires-Python: >=3.10
Requires-Dist: pendulum
Requires-Dist: pkn!=0.4.0,!=0.4.1,>=0.1.3
Requires-Dist: pydantic-extra-types
Requires-Dist: pydantic>=2
Requires-Dist: pytz
Provides-Extra: airflow
Requires-Dist: apache-airflow-providers-ssh; extra == 'airflow'
Requires-Dist: apache-airflow-providers-standard; extra == 'airflow'
Requires-Dist: apache-airflow<3,>=2.8; extra == 'airflow'
Requires-Dist: paramiko<4; extra == 'airflow'
Requires-Dist: structlog<25.5.0; extra == 'airflow'
Provides-Extra: airflow3
Requires-Dist: apache-airflow-providers-ssh; extra == 'airflow3'
Requires-Dist: apache-airflow-providers-standard; extra == 'airflow3'
Requires-Dist: apache-airflow<3.2,>=3; extra == 'airflow3'
Requires-Dist: paramiko<4; extra == 'airflow3'
Requires-Dist: structlog<25.5.0; extra == 'airflow3'
Provides-Extra: develop
Requires-Dist: build; extra == 'develop'
Requires-Dist: bump-my-version; extra == 'develop'
Requires-Dist: ccflow; extra == 'develop'
Requires-Dist: check-dist; extra == 'develop'
Requires-Dist: codespell; extra == 'develop'
Requires-Dist: hatchling; extra == 'develop'
Requires-Dist: mdformat; extra == 'develop'
Requires-Dist: mdformat-tables>=1; extra == 'develop'
Requires-Dist: pytest; extra == 'develop'
Requires-Dist: pytest-cov; extra == 'develop'
Requires-Dist: ruff; extra == 'develop'
Requires-Dist: twine; extra == 'develop'
Requires-Dist: ty; extra == 'develop'
Requires-Dist: uv; extra == 'develop'
Requires-Dist: wheel; extra == 'develop'
Provides-Extra: laminar
Requires-Dist: airflow-balancer; extra == 'laminar'
Requires-Dist: airflow-common-operators; extra == 'laminar'
Requires-Dist: airflow-config; extra == 'laminar'
Requires-Dist: airflow-ha; extra == 'laminar'
Requires-Dist: airflow-supervisor>=1.9; extra == 'laminar'
Requires-Dist: apache-airflow-providers-ssh; extra == 'laminar'
Requires-Dist: apache-airflow-providers-standard; extra == 'laminar'
Requires-Dist: apache-airflow<3.2,>=2.8; extra == 'laminar'
Requires-Dist: paramiko<4; extra == 'laminar'
Requires-Dist: structlog<25.5.0; extra == 'laminar'
Requires-Dist: supervisor-pydantic; extra == 'laminar'
Description-Content-Type: text/markdown

# airflow-pydantic

Pydantic models for Apache Airflow

[![Build Status](https://github.com/airflow-laminar/airflow-pydantic/actions/workflows/build.yaml/badge.svg?branch=main&event=push)](https://github.com/airflow-laminar/airflow-pydantic/actions/workflows/build.yaml)
[![codecov](https://codecov.io/gh/airflow-laminar/airflow-pydantic/branch/main/graph/badge.svg)](https://codecov.io/gh/airflow-laminar/airflow-pydantic)
[![License](https://img.shields.io/github/license/airflow-laminar/airflow-pydantic)](https://github.com/airflow-laminar/airflow-pydantic)
[![PyPI](https://img.shields.io/pypi/v/airflow-pydantic.svg)](https://pypi.python.org/pypi/airflow-pydantic)

## Overview

[Pydantic](https://docs.pydantic.dev/latest/) models of Apache Airflow data structures.

**Primary Use Case:** This library is designed to enable **declarative DAG definitions** using [airflow-config](https://github.com/airflow-laminar/airflow-config) or other YAML/JSON-based configuration frameworks. By representing Airflow constructs as Pydantic models, DAGs can be defined in configuration files rather than Python code, enabling better separation of concerns, easier testing, and configuration-driven workflows.

## Core

- [DAG / DAG Arguments](https://airflow.apache.org/docs/apache-airflow/2.10.4/_api/airflow/models/dag/index.html#airflow.models.dag.DAG)
- [Task / Task Arguments](https://airflow.apache.org/docs/apache-airflow/2.10.4/_api/airflow/models/baseoperator/index.html#airflow.models.baseoperator.BaseOperator)

### Operators

- [ApprovalOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/hitl/index.html#airflow.providers.standard.operators.hitl.HITLOperator)
- [BashOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/bash/index.html#airflow.providers.standard.operators.bash.BashOperator)
- [BranchDayOfWeekOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/weekday/index.html#airflow.providers.standard.operators.weekday.BranchDayOfWeekOperator)
- [BranchExternalPythonOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.BranchExternalPythonOperator)
- [BranchPythonOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.BranchPythonOperator)
- [BranchPythonVirtualenvOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.PythonVirtualenvOperator)
- [EmptyOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/empty/index.html#airflow.providers.standard.operators.empty.EmptyOperator)
- [ExternalPythonOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.ExternalPythonOperator)
- [ExternalTaskMarker](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/external_task/index.html#airflow.providers.standard.sensors.external_task.ExternalTaskMarker)
- [HITLEntryOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/hitl/index.html#airflow.providers.standard.operators.hitl.HITLEntryOperator)
- [HITLOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/hitl/index.html#airflow.providers.standard.operators.hitl.HITLOperator)
- [HITLBranchOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/hitl/index.html#airflow.providers.standard.operators.hitl.HITLOperator)
- [PythonOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.PythonOperator)
- [PythonVirtualenvOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.PythonVirtualenvOperator)
- [ShortCircuitOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/operators/python/index.html#airflow.providers.standard.operators.python.ShortCircuitOperator)
- [SSHOperator](https://airflow.apache.org/docs/apache-airflow-providers-ssh/stable/_api/airflow/providers/ssh/operators/ssh/index.html#airflow.providers.ssh.operators.ssh.SSHOperator)
- [TriggerDagRunOperator](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/operators/trigger_dag_run.html)

### Sensors

- [BashSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/bash/index.html#airflow.providers.standard.sensors.bash.BashSensor)
- [DateTimeSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/date_time/index.html#airflow.providers.standard.sensors.date_time.DateTimeSensor)
- [DateTimeSensorAsync](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/date_time/index.html#airflow.providers.standard.sensors.date_time.DateTimeSensorAsync)
- [DayOfWeekSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/weekday/index.html#airflow.providers.standard.sensors.weekday.DayOfWeekSensor)
- [ExternalTaskSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/external_task/index.html#airflow.providers.standard.sensors.external_task.ExternalTaskSensor)
- [FileSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/filesystem/index.html#airflow.providers.standard.sensors.filesystem.FileSensor)
- [PythonSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/python/index.html#airflow.providers.standard.sensors.python.PythonSensor)
- [TimeSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/time/index.html#airflow.providers.standard.sensors.time.TimeSensor)
- [WaitSensor](https://airflow.apache.org/docs/apache-airflow-providers-standard/stable/_api/airflow/providers/standard/sensors/time_delta/index.html#airflow.providers.standard.sensors.time_delta.WaitSensor)

### Other

- [Param](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/params.html)
- [Pool](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/pools.html)
- [SSHHook](https://airflow.apache.org/docs/apache-airflow-providers-ssh/stable/_api/airflow/providers/ssh/hooks/ssh/index.html#airflow.providers.ssh.hooks.ssh.SSHHook)
- [Variable](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/variables.html)
- [CronDataIntervalTimetable](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/timetables/interval/index.html#airflow.timetables.interval.CronDataIntervalTimetable)
- [CronTriggerTimetable](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/timetables/trigger/index.html#airflow.timetables.trigger.CronTriggerTimetable)
- [DeltaDataIntervalTimetable](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/timetables/interval/index.html#airflow.timetables.interval.DeltaDataIntervalTimetable)
- [DeltaTriggerTimetable](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/timetables/trigger/index.html#airflow.timetables.trigger.DeltaTriggerTimetable)
- [EventsTimetable](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/timetables/events/index.html#airflow.timetables.events.EventsTimetable)
- [MultipleCronTriggerTimetable](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/timetables/trigger/index.html#airflow.timetables.trigger.MultipleCronTriggerTimetable)

## Usage

### Declarative DAGs with airflow-config (Recommended)

The primary use of `airflow-pydantic` is to build **declarative, configuration-driven DAGs** using [airflow-config](https://github.com/airflow-laminar/airflow-config) or similar YAML/JSON-based frameworks:

```yaml
# config/my_dag.yaml
default_args:
  _target_: airflow_pydantic.TaskArgs
  owner: data-team
  retries: 3

default_dag_args:
  _target_: airflow_pydantic.DagArgs
  schedule: "@daily"
  start_date: "2024-01-01"
  catchup: false
```

This approach allows you to:

- Define DAGs in YAML/JSON instead of Python
- Separate configuration from code
- Easily manage environment-specific settings
- Version control your DAG configurations
- Generate and validate DAGs programmatically

### Programmatic Usage

All operators and sensors support two methods:

- `instantiate()`: Create a concrete Airflow instance at runtime
- `render()`: Generate Python code as a string for the Airflow construct

#### Code Generation with render()

The `render()` method generates valid Python code from your Pydantic models, enabling code generation workflows:

```python
from airflow_pydantic import Dag, BashTask
from datetime import datetime

dag = Dag(
    dag_id="generated-dag",
    schedule="@daily",
    start_date=datetime(2024, 1, 1),
    tasks={
        "hello": BashTask(
            task_id="hello",
            bash_command="echo 'Hello World'",
        ),
    },
)

# Generate Python code
python_code = dag.render()

# Save to a DAG file
with open("dags/generated_dag.py", "w") as f:
    f.write(python_code)
```

**Generated File:**

```python
from datetime import datetime

from airflow.models import DAG
from airflow.providers.standard.operators.bash import BashOperator

with DAG(schedule="@daily", start_date=datetime.fromisoformat("2024-01-01T00:00:00"), dag_id="generated-dag") as dag:
    hello = BashOperator(bash_command="echo 'Hello World'", task_id="hello", dag=dag)
```

This is useful for:

- Generating DAG files from configuration during CI/CD
- Creating DAG templates programmatically
- Migrating from configuration-driven to static DAG files
- Debugging and inspecting generated DAG code

> [!NOTE]
> This library was generated using [copier](https://copier.readthedocs.io/en/stable/) from the [Base Python Project Template repository](https://github.com/python-project-templates/base).
