Export
This sync enables moving data from a user specified datawarehouse to the Amazon S3.
Configuring the Credentials
Select the account credentials which has access to relevant Amazon S3 account from the given list & Click Next
Data Sync Details
- Data Sync
-
Select Export & click Next
- How do you want to fetch data?
-
Select whether you want to fetch data from the Data Model or from Table/ View.
- Data Model
-
In case you want to fetch data using Data Model, select the data model that you would like to use for this sync. Checkout how to configure a model here.
- Data Warehouse
-
In case you want to fetch data using Table/ View, select the data warehouse that you would like to use for this sync.
- Table / View
-
Select the Table/ view in the data warehouse that you would like to use for this sync.
Setting Parameters
Parameter | Description | Values |
---|---|---|
Fields Selection |
Required Select the field(s) you would like to push in your Amazon S3 here. You can also rename one or more field(s). To push all the fields, simple select SELECT ALL. |
{Model Field Name, Destination Field Name} |
Fetch Mode |
Required This refers to the manner in which data will get updated : FULL will update the entire column(s) from the selected data, INCREMENTAL will update the fresh record(S) added since last fetch |
{Incremental,Full} Default Value: FULL |
Destination File Type |
Required Select the destination file type, that is whether to export data in CSV or newline delimited JSON or Apache Parquet or Apache Avro. |
{CSV, txt, parquet, avro} Default Value: CSV |
Include Headers Dependant |
Required (If Destination File Type = CSV) Choose whether to include headers in the exported CSV file. |
{Yes, No} Default Value: Yes |
File Path |
Required Enter the absolute path where the created file will be saved/uploaded. Supported datetime string formats such as tmp/dc_s3_csv_%Y-%m-%d %H:%M:%S. You can find further references at https://strftime.org/ |
{tmp/dc_s3_csv_%Y-%m-%d %H:%M:%S} |
Data Sync Scheduling
Set the schedule for the sync to run. Detailed explanation on scheduling of syncs can be found here
Dataset & Name
Give your sync a name and some description (optional) and click on Finish to save it. Read more about naming and saving your syncs including the option to save them as templates here
Still have Questions?
We’ll be happy to help you with any questions you might have! Send us an email at info@datachannel.co.
Subscribe to our Newsletter for latest updates at DataChannel.