Pre-alpha release

Forge raw sources into reproducible datasets.

Ogun is a local-first runtime for acquiring and transforming data on a schedule, built on the Zeus download engine. Workflows are versioned JSON documents. A run that crashes resumes with only the unfinished partitions.

Try the pre-alpha Read the architecture
ogun validate examples/bikeshare/trips-backfill.workflow.json

Runs with ogun 0.0.1a, a pre-alpha for Windows and Linux, on the sample workflows in the ogun repository. Download it or try the hosted hub.

Ogun Studio, Overview page: six workflows, two approvals pending, and the latest run of each workflow, with the daemon version 0.0.1-alpha.Ogun Studio, Overview page: six workflows, two approvals pending, and the latest run of each workflow, with the daemon version 0.0.1-alpha.
Pixel-art translation of Gustave Courbet, The Stonebreakers (1849): two labourers break stones by a roadside.Pixel-art translation of Gustave Courbet, The Stonebreakers (1849): two labourers break stones by a roadside.

Runs resume where they stopped

A backfill commits one partition at a time to a journal. Kill the daemon halfway and start it again: only the partitions that had not finished run.

exit-backfill · 2024-01-01 to 2024-04-30121 done · 0 running · 0 to run

121 of 121 partitions done. A second backfill over the range has 0 to run.

The O1 exit test, drawn. One cell per day of a 121-partition backfill: cold iron is waiting, ember is running, verdigris is done. ogund is killed with SIGKILL and restarted, and only the cold cells are struck again.Illustration of ROADMAP phase O1 and crates/ogun-daemon/tests/exit.rs (a partition is done under O-01 section 5.3 and skipped as up to date under O-04 section 4). Drawn, not recorded: the transcript below is.
$ ogund &
$ ogun daemon status
ogund 0.0.1-alpha  up since 2026-10-26T03:38:27.353Z
  workflows         0
  active runs       0
  pending approvals 0
  streams           0
  scheduler         on
  connection        human (not interactive)
$ ogun put exit-backfill.workflow.json
exit-backfill v1 stored  b10f029c3d5e2a89a6cf7cd4ea530883283d3ad04f58fdf3b145246ea98c6fdb
  warning ir.node_has_no_inputs at /nodes/0/inputs: manifest does not start a DAG and nothing feeds this node.
  runnable now
$ ogun backfill exit-backfill --from 2024-01-01 --to 2024-04-30
exit-backfill 2024-01-01 .. 2024-04-30: 121 to run, 0 up to date
  missing      121  2024-01-01 .. 2024-04-30
dry run; pass --yes to start it
$ ogun backfill exit-backfill --from 2024-01-01 --to 2024-04-30 --yes
exit-backfill 2024-01-01 .. 2024-04-30: 121 to run, 0 up to date
  missing      121  2024-01-01 .. 2024-04-30
orun_01M5V4KGACEC47NZK30319FE9W  started, 121 partition(s)
$ ogun partition-status exit-backfill
exit-backfill  b10f029c3d5e2a89a6cf7cd4ea530883283d3ad04f58fdf3b145246ea98c6fdb
  done          30  2024-01-01 .. 2024-01-30
  running       91  2024-01-31 .. 2024-04-30
# ogund killed here (SIGKILL; TerminateProcess on Windows)
$ ogund &
$ ogun partition-status exit-backfill
exit-backfill  b10f029c3d5e2a89a6cf7cd4ea530883283d3ad04f58fdf3b145246ea98c6fdb
  done          66  2024-01-01 .. 2024-03-06
  running       55  2024-03-07 .. 2024-04-30
$ ogun partition-status exit-backfill
exit-backfill  b10f029c3d5e2a89a6cf7cd4ea530883283d3ad04f58fdf3b145246ea98c6fdb
  done         121  2024-01-01 .. 2024-04-30
$ ogun show orun_01M5V4KGACEC47NZK30319FE9W | grep "attempt [2-9]"
  m [2024-02-01]  Succeeded (OutputsCommitted)  attempt 2
  m [2024-02-02]  Succeeded (OutputsCommitted)  attempt 2
  m [2024-02-03]  Succeeded (OutputsCommitted)  attempt 2
  m [2024-02-04]  Succeeded (OutputsCommitted)  attempt 2
# 4 of 242 node instances ran again; the other 238 were not touched
$ ogun backfill exit-backfill --from 2024-01-01 --to 2024-04-30
exit-backfill 2024-01-01 .. 2024-04-30: 0 to run, 121 up to date
  done         121  2024-01-01 .. 2024-04-30
dry run; pass --yes to start it
The same run on the real daemon, captured 2026-10-26 with ogun 0.0.1-alpha on Windows 11. ogund was killed after 30 partitions; four node instances that were in flight ran a second attempt, and the other 238 were not touched.

How resumption works

  • Transfers resume from the Zeus engine’s journal, byte-exact, after checking validators.
  • Node instances checkpoint; artifact edges are committed before a node is marked succeeded.
  • Each partition is its own unit of success, failure and retry.
  • The phase O1 exit test is this run. It passes on Linux; the transcript was captured on Windows.
Pixel-art translation of J. M. W. Turner, Rain, Steam and Speed: The Great Western Railway (1844): a steam locomotive crosses a railway bridge in rain and mist, its firebox glowing.Pixel-art translation of J. M. W. Turner, Rain, Steam and Speed: The Great Western Railway (1844): a steam locomotive crosses a railway bridge in rain and mist, its firebox glowing.

From acquisition to publication

A workflow moves data through eight stages. Each stage is a node or a group of nodes in the run, and all of them write to the same lineage graph.

Workflows are data, not code

A workflow is a JSON document: validated, hashed, diffed and approved before it runs. YAML and a Python SDK are planned; they will compile to the same IR.

{
  "ir_version": 0,
  "workflow": "trips-backfill",
  "version": 1,
  "description": "Every station of the network, month by month since 2019, from the public open-trips archive, reduced to trips per station per day. A partitioned backfill: a crash resumes only unfinished months.",
  "triggers": [
    {
      "kind": "cron",
      "cron": "10 2 4 * *"
    }
  ],
  "partitions": {
    "kind": "time",
    "granularity": "month",
    "start": "2019-01"
  },
  "budgets": {
    "bytes": 2147483648,
    "wall_ms": 7200000
  },
  "nodes": [
    {
      "id": "stations",
      "op": "crawl@1",
      "config": {
        "start": "https://api.open-trips.example.org/v1/stations?limit=1000",
        "headers": {
          "X-API-Key": {
            "secret_ref": "archive_key"
          }
        },
        "select": {
          "json_path": "$.results[*].id"
        },
        "emit": "https://open-trips-archive.s3.amazonaws.com/trips/station={id}/year={partition.year}/month={partition.month}/"
      }
    },
    {
      "id": "fetch",
      "op": "acquire@1",
      "inputs": [
        "stations.urls"
      ],
      "config": {
        "allowed_domains": [
          "open-trips-archive.s3.amazonaws.com"
        ],
        "listing": "s3_prefix"
      },
      "policy": {
        "retries": 5,
        "concurrency": 4
      }
    },
    {
      "id": "daily",
      "op": "sql@1",
      "inputs": [
        "fetch.artifacts"
      ],
      "config": {
        "table": "trips"
      },
      "code": {
        "blake3": "7e92f3fe649e617e27e91a609efcd89955192c39f887c940c9e98610fc961448",
        "path": "examples/bikeshare/ops/trips_monthly.sql"
      }
    },
    {
      "id": "publish",
      "op": "duckdb@1",
      "inputs": [
        "daily.tables"
      ],
      "config": {
        "write": "trips_daily.parquet"
      }
    }
  ]
}
The workflow is verbatim from examples/bikeshare in the ogun repository; the terminal tabs are real output, captured 2026-10-26 with ogun 0.0.1-alpha on Windows 11. IR v0 is a draft (O-02, Draft 2) and may change.

Why data, not code

  • No floating-point numbers, so the JCS + BLAKE3 hash is stable and can bind an approval.
  • Expressions are a closed template and predicate set: no loops, calls or I/O in the IR.
  • Code lives in operators, pinned by content hash; adding or changing one needs your approval.
  • Prompt to IR to validator to plan diff to approval to runtime. Never prompt to shell.
How workflows work
Pixel-art translation of Claude Monet, Arrival of the Normandy Train, Gare Saint-Lazare (1877): a steam train arrives under the iron-and-glass roof of the Gare Saint-Lazare in clouds of steam.Pixel-art translation of Claude Monet, Arrival of the Normandy Train, Gare Saint-Lazare (1877): a steam train arrives under the iron-and-glass roof of the Gare Saint-Lazare in clouds of steam.

Studio reads the real daemon

Ogun Studio ships in the release for Windows and Linux. On the hosted hub it runs on the real daemon with sample workflows: runs, partitions, approvals, artifacts and lineage.

Ogun Studio, Partitions page for the workflow trips-backfill: 45 keys in two ranges, 36 done from 2023-01 to 2025-12 and 9 missing from 2026-01 to 2026-09, with a row of green cells for the done keys.Ogun Studio, Partitions page for the workflow trips-backfill: 45 keys in two ranges, 36 done from 2023-01 to 2025-12 and 9 missing from 2026-01 to 2026-09, with a row of green cells for the done keys.
The Partitions page of Ogun Studio 0.0.1a on the hosted hub: the sample trips-backfill workflow, 36 monthly partitions done and 9 missing. Studio draws one cell per key, as the strip above does.

Acquisition runs on the Zeus engine

Pixel-art translation of Joseph Wright of Derby, The Blacksmith's Shop (1771): a smith and his helpers work a glowing piece of iron in a dark workshop lit by the forge.Pixel-art translation of Joseph Wright of Derby, The Blacksmith's Shop (1771): a smith and his helpers work a glowing piece of iron in a dark workshop lit by the forge.

Ogun embeds zeus-core, the Rust engine behind the Zeus app: segmented transfers, a crash-safe journal, validator checks before every resume, SHA-256 and BLAKE3 verification, and polite defaults.

Sources behind your browser login are the one exception: Ogun never holds cookies. It asks the Zeus daemon, as a service client, to acquire them, and imports the result with its provenance. That bridge is built and tested against a fake zeusd, not yet against a real one.

See the architecture
Pixel-art translation of J. M. W. Turner, The Burning of the Houses of Lords and Commons, 16 October 1834 (1834 to 1835): the Houses of Parliament burn at night, with the fire reflected in the Thames.Pixel-art translation of J. M. W. Turner, The Burning of the Houses of Lords and Commons, 16 October 1834 (1834 to 1835): the Houses of Parliament burn at night, with the fire reflected in the Thames.

Lineage for every artifact

Every artifact is content-addressed and recorded with the run, node and partition that produced it. Parents and operator versions are carried in the lineage events.

{
  "artifact_id": "art_01M5V4KHKAJP8EZQEE9BNRKEAP",
  "blake3": "826a435cc93bbcae8f876f5c8c036d1a7879c8b2b0f2b4d36e34565fddb9c419",
  "size_bytes": 98,
  "media_type": "application/json",
  "run_id": "orun_01M5V4KGACEC47NZK30319FE9W",
  "node": "m",
  "partition": "2024-01-01",
  "output": "manifest",
  "committed_at": "2026-10-26T03:38:28.852Z"
}
One artifact record from ogun artifacts --json, captured 2026-10-26 with ogun 0.0.1-alpha on Windows 11. Its BLAKE3 digest is its address in the store.
artifacts/store.json  {"format":"zeus-artifact-store","format_version":1,"object_algorithm":"blake3","fanout":[2,2],"index":"redb"}
objects/blake3/00/f4/00f4e791631167aabc7c951e4df7b4fe88bc5c253efef2ef37d1bc32825e33a2
objects/blake3/04/74/0474f23641f77e2f6054af901e6005485fd1e2480b1bd27f476a4e833c452711
objects/blake3/05/e9/05e9911c8ce2e14519b49569c9f6010b65738e9530b4c9eb125f084e660084e1
objects/blake3/07/06/0706905f576ea69a5ee852e1adebfa82fe99281d0684eca708e60ed6fc42c51b
... 121 objects in all, one per distinct content
The store ogund creates on first start: one object per distinct content, fanned out by the first four hex digits of its digest.

Questions it answers

  • Which run, node and partition produced this file?
  • Which operator version and code hash ran, and with which workflow hash?
  • Which input artifacts fed it?
  • Are two artifacts the same bytes?
Artifacts and lineage
Pixel-art translation of Joseph Wright of Derby, An Experiment on a Bird in the Air Pump (1768): a lecturer shows a family a bird in a glass air pump by candlelight.Pixel-art translation of Joseph Wright of Derby, An Experiment on a Bird in the Air Pump (1768): a lecturer shows a family a bird in a glass air pump by candlelight.

Benchmark runs

A benchmark is a controlled set of runs, each with a workflow, environment, agent, policy, inputs and evaluator. Ogun reads existing task formats and records cost, efficiency and recovery alongside the official score.

Adapters for Spider 2.0-DBT, ADE-bench and Terminal-Bench 2.0 tasks in the Harbor format are written. Their task formats come from public descriptions and are not yet verified against the official releases, so there are no results. The benchmark binary is not part of release 0.0.1a.

Cost per success

Cs = Σi costi / successful tasks

Compares agents at the same pass rate by what each success costs.

Intervention rate

interventions / run

How often a person or an outside recovery had to step in.

Tool efficiency

useful tool calls / all tool calls

Proxies: duplicate, failed and repeated commands; files re-read.

Harness gain

HGE(H) = (1/N) Σm [ Score(m, H) − Score(m, Hbase) ]

How much capability a harness unlocks across N models.

The benchmark profile

Pixel-art translation of Diego Velázquez, The Forge of Vulcan (1630): Apollo brings news to Vulcan and his workers at the forge.Pixel-art translation of Diego Velázquez, The Forge of Vulcan (1630): Apollo brings news to Vulcan and his workers at the forge.

Design principles

Each traces to an architecture decision record. ADRs 0002 to 0005 are accepted.

Workflows are data

A workflow is a versioned JSON document: a DAG of operator calls with no floating-point numbers, hashed with JCS and BLAKE3. It can be diffed, approved, replayed and exported, because it is not code.

Runs are durable

Every node instance, checkpoint and artifact edge is journaled. If the daemon is killed or the machine restarts, the run resumes and completed partitions are not fetched again.

Every byte has a lineage

Outputs are content-addressed artifacts that know their parents, the operator and version that made them, the model and prompt hash if any, and the run that produced them.

Built on the Zeus engine

Acquisition runs on zeus-core: segmented, resumable, verified transfers with a crash-safe journal, written in Rust and shared with the Zeus app.

Integrate, do not replace

DuckDB, dbt, Polars, ffmpeg, pandoc, OCR engines and Python run as out-of-process operators. Integrations for Airflow, Dagster, Prefect and Kestra start and watch Ogun runs; they are in the repository and not yet released as packages.

People approve

A new or changed code hash, a larger budget, a first model operator and any node whose policy asks every time need a human approval bound to the request hash. No agent channel can approve.

Pixel-art translation of Louis Le Nain, The Forge (La Forge) (about 1640 to 1642): a smith, his family and his workmen gather around the forge in a dim room.Pixel-art translation of Louis Le Nain, The Forge (La Forge) (about 1640 to 1642): a smith, his family and his workmen gather around the forge in a dim room.

Existing tools run as operators

Ogun orchestrates tools you already trust as out-of-process operators with pinned versions, and has integrations for the orchestrators you already run.

  • DuckDB
  • dbt
  • Polars
  • Python
  • SQL
  • ffmpeg
  • pandoc
  • Tesseract
  • PaddleOCR
  • yt-dlp
  • rclone
  • git
  • Hugging Face
  • OpenLineage
  • MCP
  • VS Code
  • Airflow
  • Dagster
  • Prefect
  • Kestra

Names belong to their owners and indicate compatibility, not endorsement. The orchestrator integrations are in the repository, not yet on PyPI, and were tested against fakes rather than the real services.

Pixel-art translation of Jan Brueghel the Elder, Fire (Allegory of Fire) (1606): an allegory of fire: a crowded scene of forges, furnaces and burning tools.Pixel-art translation of Jan Brueghel the Elder, Fire (Allegory of Fire) (1606): an allegory of fire: a crowded scene of forges, furnaces and burning tools.

Common questions

What is Ogun?

Ogun is a local-first developer platform for data acquisition, data engineering and AI-agent benchmarks, made by Verne. It turns authenticated, messy web and filesystem sources into verified, reproducible datasets on a schedule, and it can run benchmark tasks for data and terminal agents in isolation. It is built on the Zeus download engine.

Can I use Ogun today?

As a pre-alpha. Release 0.0.1a has Ogun Studio for Windows and Linux and the ogun and ogund binaries, and a hosted hub runs Studio on the real daemon with sample data (see Get started). The runtime passed its phase O1 exit test on Linux in October 2026: a partitioned backfill and a stream window survive kill -9 of the daemon and resume only the unfinished work. Code operators, remote approvals and most connectors are not ready; expect breaking changes. See the roadmap.

Is Ogun a replacement for Airflow, Dagster or Prefect?

No. Ogun contains enough durable orchestration to run acquisition and data workflows by itself, but its job is materializing trustworthy artifacts. It is designed to be called from orchestrators as a task, resource or operator, not to replace them.

What makes runs reproducible?

The workflow is hashed; operators are pinned by version and content hash; inputs are content-addressed artifacts; model calls record the model and a prompt hash; and every output records its parents. Re-running a version with the same inputs is a lookup, not a guess.

Which benchmarks will Ogun run?

Adapters for Spider 2.0-DBT, ADE-bench and Terminal-Bench 2.0 tasks in the Harbor format are written; their task formats are still being checked against the official releases. Official evaluators run unmodified and their versions are recorded. Ogun does not invent a competing task format.

All questions

Read the design

The decisions and first-draft specifications are public, and the runtime is being tested. Feedback is most useful now, before the contracts are frozen.

Read the architecture See the roadmap