Skip to main content

Pipelines

Feldera expresses incremental computations via Pipelines. A Pipeline is defined using SQL tables and views. Pipelines receive input data through input connectors. Pipelines can send the results computed by views to destinations via output connectors.

A connector is a combination of two things: a transport and often a format. The transport describes where the input data comes from or output data is sent to (e.g., Kafka, S3, PubSub and so on). The format defines the shape or envelope of the data (e.g., JSON, CSV, Parquet and more). Typically, the format has to be compatible with the schema of the tables and views.

Read the references below to learn more about SQL pipeline features.