Base Configuration

Define shared defaults for all DAGs using a base.yaml file. This eliminates duplication and ensures consistent behavior across your workflows.

Overview

The base configuration file provides default values that are automatically inherited by every DAG in your Dagu installation. Use it for:

Shared environment variables - Common settings like timezone, API keys
Global handlers - Failure notifications, cleanup routines
Default timeouts - Consistent execution limits
Email/alerting setup - Team-wide notification configuration
External service defaults - SSH, S3, database connections

Individual DAG files can override any setting from the base configuration.

File Location

Dagu looks for the base configuration file based on your directory structure:

Mode	Base Config Path
XDG (default)	`~/.config/dagu/base.yaml`
Unified (`DAGU_HOME` set)	`{DAGU_HOME}/base.yaml`
Legacy (`~/.dagu` exists)	`~/.dagu/base.yaml`

Custom Location

Override the default path in your server configuration:

yaml

# ~/.config/dagu/config.yaml
paths:
  baseConfig: "/path/to/custom/base.yaml"

Or via environment variable:

bash

export DAGU_BASE_CONFIG=/path/to/custom/base.yaml

How Inheritance Works

When Dagu loads a DAG, it follows this process:

1. Load base.yaml (if exists)
2. Load the DAG file
3. Merge: DAG values override base values
4. Apply runtime parameters (highest priority)

Merging Behavior

Standard fields - DAG values override base values:

yaml

# base.yaml
timeoutSec: 3600

# my-dag.yaml
timeoutSec: 7200  # Result: 7200

Array fields (env) - Values are appended, not replaced:

yaml

# base.yaml
env:
  - TZ=UTC
  - LOG_LEVEL=info

# my-dag.yaml
env:
  - API_KEY=secret

# Result: [TZ=UTC, LOG_LEVEL=info, API_KEY=secret]

Full override fields (mailOn) - Entire structure is replaced:

yaml

# base.yaml
mailOn:
  failure: true
  success: true

# my-dag.yaml
mailOn:
  failure: false
# Result: failure=false, success=false (not inherited)

Precedence Order

From lowest to highest priority:

OS environment variables
base.yaml configuration
Individual DAG file
Runtime parameters (CLI or API)

Configuration Fields

Environment Variables

Set environment variables available to all DAG steps:

yaml

# base.yaml
env:
  - TZ=UTC
  - ENVIRONMENT=production
  - LOG_LEVEL=info
  - SLACK_WEBHOOK=${SLACK_WEBHOOK_URL}  # Expand from OS env

Environment variables from base config are appended to those defined in individual DAGs, allowing you to set common defaults while DAGs add their specific variables.

Lifecycle Handlers

Define handlers that run on specific events for all DAGs:

yaml

# base.yaml
handlerOn:
  init:
    command: echo "Starting DAG ${DAG_NAME}"

  success:
    command: |
      curl -X POST ${SLACK_WEBHOOK} \
        -d '{"text":"✅ ${DAG_NAME} completed successfully"}'

  failure:
    command: |
      curl -X POST ${SLACK_WEBHOOK} \
        -d '{"text":"❌ ${DAG_NAME} failed! Run ID: ${DAG_RUN_ID}"}'

  exit:
    command: echo "Cleanup complete"

Available handler types:

Handler	Trigger
`init`	Before steps execute (after preconditions pass)
`success`	All steps completed successfully
`failure`	Any step failed
`abort`	DAG was cancelled
`exit`	Always runs on completion (cleanup)
`wait`	DAG enters HITL wait status

Special environment variables available in handlers:

Variable	Description
`DAG_NAME`	Name of the executing DAG
`DAG_RUN_ID`	Unique identifier for this run
`DAG_RUN_LOG_FILE`	Path to the log file
`DAG_RUN_STATUS`	Current status (running, success, failed)
`DAG_RUN_STEP_NAME`	Name of the handler step

Email Notifications

Configure SMTP and email alerts for all DAGs:

yaml

# base.yaml
smtp:
  host: smtp.sendgrid.net
  port: 587
  username: apikey
  password: ${SENDGRID_API_KEY}

mailOn:
  failure: true
  success: false
  wait: false

errorMail:
  from: dagu@mycompany.com
  to:
    - ops-team@mycompany.com
    - oncall@mycompany.com
  prefix: "[DAGU ALERT]"
  attachLogs: true

infoMail:
  from: dagu@mycompany.com
  to:
    - team@mycompany.com

waitMail:
  from: dagu@mycompany.com
  to:
    - approvers@mycompany.com
  prefix: "[APPROVAL REQUIRED]"

Execution Defaults

Set default execution parameters:

yaml

# base.yaml
# Default shell for all steps
shell: "bash"
# Or with arguments:
# shell: ["bash", "-e", "-o", "pipefail"]

# Working directory (default: DAG file location)
workingDir: "/app/workflows"

# Maximum execution time (seconds)
timeoutSec: 3600

# Delay before starting (seconds)
delaySec: 0

# Wait time before restart (seconds)
restartWaitSec: 60

# Max concurrent DAG runs (per DAG)
maxActiveRuns: 1

# Max concurrent steps within a run
maxActiveSteps: 4

# History retention (days)
histRetentionDays: 30

# Cleanup timeout when stopped (seconds, default 5)
maxCleanUpTimeSec: 60

# Max step output capture size (bytes, default 1MB)
maxOutputSize: 1048576

Logging

Configure log output behavior:

yaml

# base.yaml
# Custom log directory
logDir: /var/log/dagu

# Log output mode:
# - "separate": stdout/stderr in separate files (.out, .err)
# - "merged": combined into single file (.log)
logOutput: merged

External Service Defaults

SSH Configuration

yaml

# base.yaml
ssh:
  user: deploy
  host: ""  # Set per-DAG or per-step
  port: 22
  key: ~/.ssh/deploy_key
  strictHostKey: true
  knownHostFile: ~/.ssh/known_hosts
  shell: bash

S3 Configuration

yaml

# base.yaml
s3:
  region: us-east-1
  bucket: my-data-bucket
  # For S3-compatible services (MinIO, etc.)
  endpoint: ""
  forcePathStyle: false
  # Credentials (prefer IAM roles in production)
  accessKeyId: ${AWS_ACCESS_KEY_ID}
  secretAccessKey: ${AWS_SECRET_ACCESS_KEY}

LLM Configuration

yaml

# base.yaml
llm:
  provider: openai
  model: gpt-4
  temperature: 0.7
  maxTokens: 4096
  # API key from environment
  apiKeyName: OPENAI_API_KEY

Redis Configuration

yaml

# base.yaml
redis:
  host: localhost
  port: 6379
  password: ${REDIS_PASSWORD}
  db: 0
  tls: false

Container Defaults

Set default Docker container configuration:

yaml

# base.yaml
container:
  image: python:3.11-slim
  pullPolicy: IfNotPresent
  workingDir: /app
  env:
    - PYTHONUNBUFFERED=1
  volumes:
    - /data:/data:ro

Registry authentication:

yaml

# base.yaml
registryAuths:
  ghcr.io:
    username: ${GITHUB_USER}
    password: ${GITHUB_TOKEN}
  docker.io:
    auth: ${DOCKER_AUTH_BASE64}

Queue & Worker Settings

Configure distributed execution defaults:

yaml

# base.yaml
# Assign DAGs to a specific queue
queue: default

# Require specific worker labels
workerSelector:
  region: us-east
  capability: gpu

Preconditions

Set global preconditions that all DAGs must satisfy:

yaml

# base.yaml
preconditions:
  - condition: "test -f /data/system-ready"
    expected: "true"

Secrets

Reference external secrets:

yaml

# base.yaml
secrets:
  - name: DB_PASSWORD
    provider: env
    key: SECRET_DB_PASSWORD

  - name: API_KEY
    provider: file
    key: /run/secrets/api_key

OpenTelemetry

Configure tracing for all DAGs:

yaml

# base.yaml
otel:
  enabled: true
  endpoint: http://otel-collector:4317
  insecure: true
  headers:
    Authorization: Bearer ${OTEL_TOKEN}

Run Configuration

Control UI behavior:

yaml

# base.yaml
runConfig:
  disableParamEdit: false   # Prevent parameter editing in UI
  disableRunIdEdit: false   # Prevent custom run ID input

Complete Example

A production-ready base configuration:

yaml

# ~/.config/dagu/base.yaml

# Environment defaults
env:
  - TZ=UTC
  - ENVIRONMENT=production
  - LOG_LEVEL=info

# Execution limits
timeoutSec: 3600
histRetentionDays: 30
maxActiveRuns: 1
maxCleanUpTimeSec: 30

# Logging
logOutput: merged

# Shell with strict mode
shell: ["bash", "-e", "-o", "pipefail"]

# Lifecycle handlers for alerting
handlerOn:
  failure:
    command: |
      curl -s -X POST "${SLACK_WEBHOOK}" \
        -H "Content-Type: application/json" \
        -d '{
          "text": "❌ DAG Failed",
          "blocks": [
            {
              "type": "section",
              "text": {
                "type": "mrkdwn",
                "text": "*DAG Failed*\n• Name: `'"${DAG_NAME}"'`\n• Run ID: `'"${DAG_RUN_ID}"'`"
              }
            }
          ]
        }'

  exit:
    command: |
      # Cleanup temp files
      rm -rf /tmp/dagu-${DAG_RUN_ID}-* 2>/dev/null || true

# Email notifications
smtp:
  host: smtp.sendgrid.net
  port: 587
  username: apikey
  password: ${SENDGRID_API_KEY}

mailOn:
  failure: true

errorMail:
  from: dagu@mycompany.com
  to:
    - platform-team@mycompany.com
  prefix: "[DAGU]"
  attachLogs: true

# SSH defaults for deployment servers
ssh:
  user: deploy
  key: ~/.ssh/deploy_key
  strictHostKey: true

# S3 defaults for data pipelines
s3:
  region: us-east-1
  bucket: company-data-lake

Common Patterns

Team-wide Failure Alerting

yaml

# base.yaml
env:
  - PAGERDUTY_KEY=${PAGERDUTY_ROUTING_KEY}

handlerOn:
  failure:
    command: |
      curl -X POST https://events.pagerduty.com/v2/enqueue \
        -H "Content-Type: application/json" \
        -d '{
          "routing_key": "'"${PAGERDUTY_KEY}"'",
          "event_action": "trigger",
          "payload": {
            "summary": "DAG '"${DAG_NAME}"' failed",
            "severity": "error",
            "source": "dagu"
          }
        }'

Environment-specific Configuration

Use different base configs per environment:

bash

# Development
export DAGU_BASE_CONFIG=/etc/dagu/base-dev.yaml

# Production
export DAGU_BASE_CONFIG=/etc/dagu/base-prod.yaml

Shared Database Credentials

yaml

# base.yaml
env:
  - DB_HOST=postgres.internal
  - DB_PORT=5432
  - DB_NAME=analytics

secrets:
  - name: DB_PASSWORD
    provider: env
    key: POSTGRES_PASSWORD

Handler Inheritance in Sub-DAGs

When a DAG invokes another DAG (sub-DAG), handler inheritance is controlled automatically:

Main DAG execution: Inherits handlers from base config
Sub-DAG execution: Handlers from base config are skipped by default

This prevents duplicate notifications when a parent DAG's failure handler would trigger alongside a child's.

To define handlers for a sub-DAG, add them explicitly in the sub-DAG file:

yaml

# sub-dag.yaml
handlerOn:
  failure:
    command: echo "Sub-DAG specific failure handling"

steps:
  - name: process
    command: ./process.sh

Workers

Base Configuration

Overview

File Location

Custom Location

How Inheritance Works

Merging Behavior

Precedence Order

Configuration Fields

Environment Variables

Lifecycle Handlers

Email Notifications

Execution Defaults

Logging

External Service Defaults

SSH Configuration

S3 Configuration

LLM Configuration

Redis Configuration

Container Defaults

Queue & Worker Settings

Preconditions

Secrets

OpenTelemetry

Run Configuration

Complete Example

Common Patterns

Team-wide Failure Alerting

Environment-specific Configuration

Shared Database Credentials

Handler Inheritance in Sub-DAGs

See Also

Base Configuration ​

Overview ​

File Location ​

Custom Location ​

How Inheritance Works ​

Merging Behavior ​

Precedence Order ​

Configuration Fields ​

Environment Variables ​

Lifecycle Handlers ​

Email Notifications ​

Execution Defaults ​

Logging ​

External Service Defaults ​

SSH Configuration ​

S3 Configuration ​

LLM Configuration ​

Redis Configuration ​

Container Defaults ​

Queue & Worker Settings ​

Preconditions ​

Secrets ​

OpenTelemetry ​

Run Configuration ​

Complete Example ​

Common Patterns ​

Team-wide Failure Alerting ​

Environment-specific Configuration ​

Shared Database Credentials ​

Handler Inheritance in Sub-DAGs ​

See Also ​

Base Configuration

Overview

File Location

Custom Location

How Inheritance Works

Merging Behavior

Precedence Order

Configuration Fields

Environment Variables

Lifecycle Handlers

Email Notifications

Execution Defaults

Logging

External Service Defaults

SSH Configuration

S3 Configuration

LLM Configuration

Redis Configuration

Container Defaults

Queue & Worker Settings

Preconditions

Secrets

OpenTelemetry

Run Configuration

Complete Example

Common Patterns

Team-wide Failure Alerting

Environment-specific Configuration

Shared Database Credentials

Handler Inheritance in Sub-DAGs

See Also