📄️ Debezium JSON
The debezium_json codec decodes Debezium CDC (Change Data Capture) Envelope JSON into a columnar Arrow MessageBatch. Attach it to a Kafka input that consumes a topic written by Debezium to turn database change events (c/u/d/r) into queryable rows. CDC offset is not managed here — it is the Kafka input's ack-gated offset.
📄️ JSON
The JSON codec converts between line-delimited JSON byte payloads and columnar Arrow RecordBatches. Decoding uses Arrow's schema inference to map JSON objects to columns; encoding writes each row as one JSON object separated by newlines. It is the most common codec for attaching to inputs that emit JSON (Kafka, Redis, HTTP, etc.).
📄️ Protobuf
The Protobuf codec converts between binary Protobuf messages and columnar Arrow RecordBatches using a descriptor compiled from .proto files at startup. Decoding parses each byte payload against the configured MessageDescriptor; encoding reverses the process. Use it when an input emits raw Protobuf (no Confluent schema-id prefix).
📄️ Schema Registry
The schema_registry codec decodes Confluent wire-format Protobuf messages by resolving the embedded schema id from a Confluent Schema Registry at runtime. Each schema version (id) is fetched at most once and cached per codec instance, so multi-version schema evolution is supported within the same stream.