Kafka output connector
Feldera can output a stream of changes to a SQL table or view to a Kafka topic.
- The Kafka connector uses librdkafka in its implementation. Relevant options supported by it can be defined in the connector configuration.
Example usage
We will create a Kafka output connector named total-sales
.
Kafka broker is located at example.com:9092
and the topic is total-sales
.
CREATE VIEW V AS ...
WITH (
'connectors' = '[
{
"transport": {
"name": "kafka_output",
"config": {
"bootstrap.servers": "example.com:9092",
"auto.offset.reset": "earliest",
"topic": "total-sales"
}
},
"format": {
"name": "json",
"config": {
"update_format": "insert_delete",
"array": false
}
}
}
]'
)
Additional resources
For more information, see:
-
Kafka input connector for examples on how to write the configuration to connect to Kafka brokers (e.g., how to specify authentication and encryption).
-
Tutorial section which involves creating a Kafka output connector.
-
Overview of Kafka configuration options: librdkafka options