CSV
CSV report enables a user to transfer data from CSV files into a user determined data warehouse.
Data Pipelines Details
- Data Pipeline
-
Select CSV from the dropdown
- Choose File
-
Select a CSV file by clicking on the Choose File button.
Setting Parameters
Select the fields that are necessary as per the file or folder .
Parameter | Description | Values |
---|---|---|
Delimiter |
Required Specify the one character string used to separate fields in the file. |
{comma,pipe,dash,semicolon,space,tab_char} Default Value: comma |
Skip Initial Space |
Required Select True, if the whitespace immediately following the delimiter is to be ignored, else No. |
{Yes,No} Default Value: No |
Line Terminator |
Required Specify the string used to terminate text |
String value (eg:/r/n) Default Value: /r/n |
Quote Character |
Required Specify the one character string used to quote fields |
String value (eg:") Default Value: " |
Escape Character |
Required Removes any special meaning from the following character. The default value None disables escaping |
String value (eg:None) Default Value: None |
Double Quote |
Required Controls how instances of Quote character appearing inside a field should themselves be quoted. When Yes, the character is doubled. When No, the Escape character is used as a prefix to the Quote character. |
{Yes,No} Default Value: Yes |
Quoting |
Required Specify the type of Quoting : QUOTE_ALL instructs writer objects to quote all fields, QUOTE_MINIMAL instructs writer objects to only quote those fields which contain special characters such as delimiter, quote character or any of the characters in line terminator, QUOTE_NONNUMERIC instructs writer objects to quote all non-numeric fields and reader to convert all non-quoted fields to type float, QUOTE_NONE Instructs writer objects to never quote fields. |
{quote_minimal,quote_none,quote_all,quote_nonnumeric} Default Value: quote_minimal |
Attempt Schema Inference |
Required If Yes then value types will be fetched as it is, eg: Float will be fetched as float. If No then everything will be fetched as string irrespective of its type. |
{Yes,No} Default Value: No |
Insert Mode |
Required Specifies the manner in which data will get updated in the data warehouse : UPSERT will insert only new records or records with changes, APPEND will insert all fetched data at the end, REPLACE will drop the existing table and recreate a fresh one on each run. |
{Upsert,Append,Replace} Default Value: UPSERT |
DataSet Name |
Required Enter the DataSet Name |
String value |
DataSet Description |
Optional Enter a brief description about the DataSet here |
String value |
Datapipeline Scheduling
Scheduling specifies the frequency with which data will get updated in the data warehouse. You can choose between Manual Run, Normal Scheduling or Advance Scheduling.
- Manual Run
-
If scheduling is not required, you can use the toggle to run the pipeline manually.
- Normal Scheduling
-
Use the dropdown to select an interval-based hourly, monthly, weekly, or daily frequency.
- Advance Scheduling
-
Set schedules fine-grained at the level of Months, Days, Hours, and Minutes.
Detailed explanation on scheduling of pipelines can be found here
Dataset & Name
- Dataset Name
-
Key in the Dataset Name(also serves as the table name in your data destination).Keep in mind, that the name should be unique across the account and the data source. Special characters (except underscore _) and blank spaces are not allowed. It is best to follow a consistent naming scheme for future search to locate the tables.
- Dataset Description
-
Enter a short description (optional) describing the dataset being fetched by this particular pipeline.
- Notifications
-
Choose the events for which you’d like to be notified: whether "ERROR ONLY" or "ERROR AND SUCCESS".
Once you have finished click on Finish to save it. Read more about naming and saving your pipelines including the option to save them as templates here
Still have Questions?
We’ll be happy to help you with any questions you might have! Send us an email at info@datachannel.co.
Subscribe to our Newsletter for latest updates at DataChannel.