Insert Data

Required role: write or higher.

Push data to a SQL table.

The client sends data encoded using the format specified in the ?format= parameter as a body of the request. The contents of the data must match the SQL table schema specified in table_name

The pipeline ingests data as it arrives without waiting for the end of the request. Successful HTTP response indicates that all data has been ingested successfully.

On success, returns a completion token that can be passed to the '/completion_status' endpoint to check whether the pipeline has fully processed the data.

Path Parameters
pipeline_name string — REQUIRED Unique pipeline name
table_name string — REQUIRED SQL table name. Unquoted SQL names have to be capitalized. Quoted SQL names have to exactly match the case from the SQL program.

Query Parameters
force boolean — REQUIRED When true, push data to the pipeline even if the pipeline is paused. The default value is false
format string — REQUIRED Input data format, either `csv' or 'json'.
array boolean Set to true if updates in this stream are packaged into JSON arrays (used in conjunction with format=json). The default values is false.
update_format string Possible values: [insert_delete, weighted, debezium, snowflake, raw, redis] Supported JSON data change event formats. Each element in a JSON-formatted input stream specifies an update to one or more records in an input table. We support several different ways to represent such updates. InsertDelete Each element in the input stream consists of an "insert" or "delete" command and a record to be inserted to or deleted from the input table. {"insert": {"column1": "hello, world!", "column2": 100}} Weighted Each element in the input stream consists of a record and a weight which indicates how many times the row appears. {"weight": 2, "data": {"column1": "hello, world!", "column2": 100}} Note that the line above would be equivalent to the following input in the InsertDelete format: {"insert": {"column1": "hello, world!", "column2": 100}} {"insert": {"column1": "hello, world!", "column2": 100}} Similarly, negative weights are equivalent to deletions: {"weight": -1, "data": {"column1": "hello, world!", "column2": 100}} is equivalent to in the InsertDelete format: {"delete": {"column1": "hello, world!", "column2": 100}} Debezium Debezium CDC format. Refer to Debezium input connector documentation for details. Snowflake Uses flat structure so that fields can get parsed directly into SQL columns. Defines three metadata fields: __action - "insert" or "delete" __stream_id - unique 64-bit ID of the output stream (records within a stream are totally ordered) __seq_number - monotonically increasing sequence number relative to the start of the stream. {"PART":1,"VENDOR":2,"EFFECTIVE_SINCE":"2019-05-21","PRICE":"10000","__action":"insert","__stream_id":4523666124030717756,"__seq_number":1} Raw This format is suitable for insert-only streams (no deletions). Each element in the input stream contains a record without any additional envelope that gets inserted in the input table. JSON data change event format (used in conjunction with format=json). The default value is 'insert_delete'.

Request Body — REQUIRED Input data in the specified format
string

Responses
200 Data successfully delivered to the pipeline. The body of the response contains a completion token that can be passed to the '/completion_status' endpoint to check whether the pipeline has fully processed the data. Schema — OPTIONAL token string Completion token. An opaque string associated with the current position in the input stream generated by an input connector. Pass this string to the /completion_status endpoint to check whether all inputs associated with the token have been fully processed by the pipeline.
400 Schema — OPTIONAL details Detailed error metadata. The contents of this field is determined by error_code. error_code string Error code is a string that specifies this error type. message string Human-readable error message.
404 Pipeline and/or table with that name does not exist Schema — OPTIONAL details Detailed error metadata. The contents of this field is determined by error_code. error_code string Error code is a string that specifies this error type. message string Human-readable error message.
500 Schema — OPTIONAL details Detailed error metadata. The contents of this field is determined by error_code. error_code string Error code is a string that specifies this error type. message string Human-readable error message.
503 Schema — OPTIONAL details Detailed error metadata. The contents of this field is determined by error_code. error_code string Error code is a string that specifies this error type. message string Human-readable error message.