Workflows

A workflow is a versioned JSON document, not a program. Because it is data, it can be hashed, diffed, approved and exported.

The intermediate representation

A DAG of nodes. Each node names an operator and version, its configuration, its inputs from upstream outputs, and its policy. Triggers, partitions and budgets belong to the workflow.

{
  "ir_version": 0,
  "workflow": "trips-backfill",
  "version": 1,
  "description": "Every station of the network, month by month since 2019, from the public open-trips archive, reduced to trips per station per day. A partitioned backfill: a crash resumes only unfinished months.",
  "triggers": [
    {
      "kind": "cron",
      "cron": "10 2 4 * *"
    }
  ],
  "partitions": {
    "kind": "time",
    "granularity": "month",
    "start": "2019-01"
  },
  "budgets": {
    "bytes": 2147483648,
    "wall_ms": 7200000
  },
  "nodes": [
    {
      "id": "stations",
      "op": "crawl@1",
      "config": {
        "start": "https://api.open-trips.example.org/v1/stations?limit=1000",
        "headers": {
          "X-API-Key": {
            "secret_ref": "archive_key"
          }
        },
        "select": {
          "json_path": "$.results[*].id"
        },
        "emit": "https://open-trips-archive.s3.amazonaws.com/trips/station={id}/year={partition.year}/month={partition.month}/"
      }
    },
    {
      "id": "fetch",
      "op": "acquire@1",
      "inputs": [
        "stations.urls"
      ],
      "config": {
        "allowed_domains": [
          "open-trips-archive.s3.amazonaws.com"
        ],
        "listing": "s3_prefix"
      },
      "policy": {
        "retries": 5,
        "concurrency": 4
      }
    },
    {
      "id": "daily",
      "op": "sql@1",
      "inputs": [
        "fetch.artifacts"
      ],
      "config": {
        "table": "trips"
      },
      "code": {
        "blake3": "7e92f3fe649e617e27e91a609efcd89955192c39f887c940c9e98610fc961448",
        "path": "examples/bikeshare/ops/trips_monthly.sql"
      }
    },
    {
      "id": "publish",
      "op": "duckdb@1",
      "inputs": [
        "daily.tables"
      ],
      "config": {
        "write": "trips_daily.parquet"
      }
    }
  ]
}

$ ogun validate examples/bikeshare/trips-backfill.workflow.json
trips-backfill v1: valid, 4 nodes
  workflow hash  c4699a1616aada04c07e9dfbeacb49195593fd5128ba8076a65d2ffa9ffbcf19
  models         not used; runs with no model access
  user code      daily (approval bound to this hash)
$ ogun validate examples/bikeshare/rebalance-plan.workflow.json
rebalance-plan v1: valid, 3 nodes
  workflow hash  087b2bb2dcd1be4b446320d0fd2a1d6c8eb80d0798b41420b02fcc34cce16cd0
  models         not used; runs with no model access
  user code      estimate (approval bound to this hash)
$ ogun hash examples/bikeshare/rebalance-plan.workflow.json
087b2bb2dcd1be4b446320d0fd2a1d6c8eb80d0798b41420b02fcc34cce16cd0

$ ogun diff examples/bikeshare/rebalance-plan.workflow.json examples/bikeshare/rebalance-plan.v2.workflow.json
v1 -> v2   087b…6cd0 -> b90b…2a18
  ~ estimate  (config, code)
  needs approval:
    {"reason":"code_changed","node":"estimate"}
[exit status 1]

$ ogun partitions examples/bikeshare/trips-backfill.workflow.json --from 2019-01 --to 2026-09
93 partitions complete: 2019-01 .. 2026-09
  to run      93 (2019-01 .. 2026-09)
  up to date  0
$ ogun next examples/bikeshare/district-report.workflow.json
2026-11-02T03:00:00.000Z

Rules of the IR

JSON, versioned, no floating-point numbers (rates are intervals in milliseconds; money is integer micro-dollars)
Workflow hash: BLAKE3 over the JCS form; approvals and runs bind to it
Expressions: a closed template set and a closed predicate set for routing
Code operators referenced by content hash; adding or changing one needs a human approval
The workflow hash is 64 lowercase hex characters, printed by ogun validate and ogun hash
Acquisition nodes name their sources and allowed domains in config; budgets are bytes, files, wall_ms and llm_usd_micro

Authoring

JSON, today

The IR is the authoring format in this release. The VS Code extension and the language server (ogun lsp) give diagnostics, hover, completion and the workflow hash as you type.

Your agent, over MCP

An agent drafts IR and calls ogun_validate and ogun_diff; the plan diff names what needs a person. Never prompt to shell.

YAML and a Python SDK, planned

Both are meant to compile to the same IR, so they add no new way to reach the runtime. Neither is built yet.

Dynamic discovery is allowed, silent self-modification is not: if an agent finds new work mid-run, it proposes a plan amendment, which is a new workflow version and a new approval.

Run states

Table-driven, like Zeus’s job state machine. Engine job states map onto node states, so a node waiting for a browser session says exactly that.

State	Meaning
`pending`	Declared, inputs not yet available
`ready`	Inputs available, waiting for a worker or a budget
`running`	An operator is executing
`waiting`	Blocked on an external condition or trigger
`waiting_for_user`	Needs a human approval (maps to the engine’s awaiting_approval)
`waiting_for_session`	Needs a browser session through the Zeus bridge (awaiting_session)
`retry_scheduled`	Failed transiently; backing off before the next attempt
`succeeded`	Outputs materialized and verified
`failed`	Failed permanently, with a typed error
`cancelled`	Stopped by a person or a policy
`skipped`	Not needed: routing, a denied approval or an up-to-date partition

Triggers

IR v0 has four trigger kinds. The rest are in the design and not built.

Kind	What it does	Status
`manual`	Starts only when a person or an agent asks for a run.	Built
`cron`	Five-field cron, evaluated in UTC. After downtime the scheduler starts one run, and its partitions make up for what was missed.	Built
`interval`	Fires `interval_ms` after the previous run started; one minute at least.	Built
`stream`	Fires when a window of a sensor stream is sealed (MQTT, HTTP push, LoRaWAN, Modbus, file tail).	Built; the ingest bridge has not run against real brokers or devices
Event, data, agent, condition	File, webhook, Git or S3 events; a dataset version or row count; another agent’s task; Wi-Fi or mains power.	Planned (VISION section 5); not in IR v0

Partitions and backfills

Declare a partition key: a day or a month, a static list of keys, or a stream window. Each partition succeeds, fails and retries on its own, and its status is part of the run record. A backfill plans first and starts only with --yes.

$ ogun backfill exit-backfill --from 2024-01-01 --to 2024-04-30
exit-backfill 2024-01-01 .. 2024-04-30: 121 to run, 0 up to date
  missing      121  2024-01-01 .. 2024-04-30
dry run; pass --yes to start it
$ ogun backfill exit-backfill --from 2024-01-01 --to 2024-04-30 --yes
exit-backfill 2024-01-01 .. 2024-04-30: 121 to run, 0 up to date
  missing      121  2024-01-01 .. 2024-04-30
orun_01M5V4KGACEC47NZK30319FE9W  started, 121 partition(s)
$ ogun partition-status exit-backfill
exit-backfill  b10f029c3d5e2a89a6cf7cd4ea530883283d3ad04f58fdf3b145246ea98c6fdb
  done          30  2024-01-01 .. 2024-01-30
  running       91  2024-01-31 .. 2024-04-30

Real output, captured 2026-10-26 with ogun 0.0.1-alpha on Windows 11. The full transcript is on the home page.

Partitions already complete for the same workflow hash are up to date and are skipped; --force runs them again
Range-compressed status: ogun partition-status prints done, running and missing ranges, not one line per key
Budgets on the workflow: bytes, files, wall_ms and llm_usd_micro. Only model spend is enforced today
Concurrency and retries per node in policy

Read the design

The decisions and first-draft specifications are public, and the runtime is being tested. Feedback is most useful now, before the contracts are frozen.

Read the architecture See the roadmap