📄️ Debezium JSON
The debezium_json codec decodes Debezium CDC (Change Data Capture) Envelope JSON into a columnar Arrow MessageBatch. Attach it to a Kafka input that consumes a topic written by Debezium to turn database change events (c/u/d/r) into queryable rows. CDC offset is not managed here — it is the Kafka input's ack-gated offset.
📄️ JSON
The JSON codec converts between line-delimited JSON byte payloads and columnar Arrow RecordBatches. Decoding uses Arrow's schema inference to map JSON objects to columns; encoding writes each row as one JSON object separated by newlines. It is the most common codec for attaching to inputs that emit JSON (Kafka, Redis, HTTP, etc.).
📄️ Protobuf
The Protobuf codec converts between binary Protobuf messages and columnar Arrow RecordBatches using a descriptor compiled from .proto files at startup. Decoding parses each byte payload against the configured MessageDescriptor; encoding reverses the process. Use it when an input emits raw Protobuf (no Confluent schema-id prefix).
📄️ Schema Registry
The schema_registry codec decodes Confluent wire-format messages by resolving the embedded schema id from a Confluent Schema Registry at runtime. Each schema version (id) is fetched at most once and cached per codec instance, so multi-version schema evolution is supported within the same stream. Both Protobuf and Avro subjects are supported, dispatched on the registry's schemaType response. An optional subject compatibility gate fails the stream fast when a subject's registered compatibility level drops below the configured minimum.