ArkFlow documentation

ArkFlow is a high-performance stream processing engine written in Rust. It consumes from any source, processes with SQL / Python / VRL, and delivers to any sink — with write-ahead-log durability, checkpointed state, and an optional control plane for running a fleet.

🚀 Get Started

Install the binary, run your first pipeline in five minutes, and make it survive a crash.

🔧 Build

Streams, jobs, and DAGs; the component catalog; durability and delivery semantics; task-focused recipes.

🗄️ SQL

The full SQL language surface: data types, SELECT, aggregates, window functions, and user-defined functions.

🛠️ Operate

Kubernetes deployment, the control plane and web console, observability, and a recovery runbook.

📖 Reference

CLI, the complete HTTP API, configuration schema, and the generated component inventory.

⚙️ Develop

The unified execution kernel, plugin authoring, and performance notes from the source.

Pick your path

You are...	Start with
Evaluating ArkFlow	Quickstart → Architecture
Writing a pipeline	Streams → Components → Recipes
Migrating from Kafka Connect / Flink	Compatibility policy → Delivery semantics
Running in production	Kubernetes → Observability → Recovery
Running a fleet	Control plane → Web console
Extending the engine	Writing a plugin → The kernel