CSV

CSV pipeline enables a user to transfer data from CSV file attachments into a user determined data warehouse. For more details on reading and writing CSV files click here

Data Pipelines Details

Data Pipeline

Select CSV from the dropdown

email csv list
File Name

Enter a CSV file name.

Setting Parameters

Select the fields that are necessary as per the file or folder .

Parameter Description Values

Header Column Present

Required

Specify whether Header is present in the sheet. If "No", is selected, the entire range will be treated as data.

Yes, No

Default Value: Yes

Header Row

Required

Row number on which headers are present

Integer Value

Default Value: 1

Data Row

Required

Row number from which data starts

Integer Value

Default Value: 2

Delimiter

Required

Specify the one character string used to separate fields in the file.

comma, pipe, semicolon, tab, caret, custom

Default Value: comma

Insert Mode

Required

Specifies the manner in which data will get updated in the data warehouse : UPSERT will insert only new records or records with changes, APPEND will insert all fetched data at the end, and REPLACE will drop the existing table and recreate a fresh one on each run. Recommended to use "Upsert" option unless there is a specific requirement.

Upsert, Append, Replace

Default Value: Replace

Upsert Key

Dependant

Required

(If Upsert is chosen as the Insert Mode Type)

Enter the column name based on which data is to be upserted.

String value

Quote Character

Required

Specify the one character string used to quote fields.

Double Quotes, Tilda, Custom

Default Value: Double Quotes

Custom Quote Character

Dependant

Required

(If Custom is chosen as the Quote Character Type)

Specify the custom Quote Character string used to quote fields.

String value

Attempt Schema Inference

Required

If Yes then value types will be fetched as it is, eg: Float will be fetched as float. If No then everything will be fetched as string irrespective of its type.

Yes, No

Default Value: No

Compressed

Required

Choose Yes or No depending on the file compression

Yes, No

Default Value: No

Compression Type

Dependant

Required (If Compressed = Yes)

Specify the file compression type

Zip, Gzip

Footer

Optional

Specify the row number containing the footer, data after this row will not be extracted

Integer value (eg:10)

File Encoding

Required

Specify the encoding type of the file which will be used to decode the file

String value (eg:utf-8)

Double Quote

Optional

Controls how instances of Quote character appearing inside a field should themselves be quoted. When Yes, the character is doubled. When No, the Escape character is used as a prefix to the Quote character.

Yes, No

Default Value: No

Escape Character

Optional

A one-character string used by the writer to indicate that the character immediately following it should be interpreted differently than it normally would. On reading, the escapechar removes any special meaning from the following character. It defaults to 'None', which disables escaping.

String value (eg:None)

Default Value: None

Line Terminator

Optional

Specify the string used to terminate text

String value (eg:\r\n)

Default Value: \r\n

Quoting

Optional

Specify the type of Quoting : 'QUOTE_ALL' instructs writer objects to quote all fields, 'QUOTE_MINIMAL' instructs writer objects to only quote those fields which contain special characters such as delimiter, quote character or any of the characters in line terminator, 'QUOTE_NONNUMERIC' instructs writer objects to quote all non-numeric fields and reader to convert all non-quoted fields to type float, 'QUOTE_NONE' Instructs writer objects to never quote fields.

quote_minimal,quote_none, quote_all, quote_nonnumeric

Default Value: quote_minimal

Skip Initial Space

Optional

Select Yes, if the whitespace immediately following the delimiter is to be ignored, else No.

Yes, No

Default Value: No

Include Source File Name

Required

Set this parameter to 'YES' if you want to include source file name in the data warehouse. Default: 'NO'.

Yes, No

Default Value: No

email csv config
email csv config 2

Still have Questions?

We’ll be happy to help you with any questions you might have! Send us an email at info@datachannel.co.

Subscribe to our Newsletter for latest updates at DataChannel.