Export

This sync enables moving data from a user specified datawarehouse to the Amazon S3.

SETUP

Configuring the Credentials

Select the account credentials which has access to relevant Amazon S3 account from the given list & Click Next

Credentials not listed ?

Click on + for adding new credentials and fill the form that pops-up.

amazons3 credentials

Data Sync Details

Data Sync

Select Export & click Next

amazons3 export list
How do you want to fetch data?

Select whether you want to fetch data from the Data Model or from Table/ View.

Data Model

In case you want to fetch data using Data Model, select the data model that you would like to use for this sync. Checkout how to configure a model here.

Data Warehouse

In case you want to fetch data using Table/ View, select the data warehouse that you would like to use for this sync.

Table / View

Select the Table/ view in the data warehouse that you would like to use for this sync.

amazons3 export config1
amazons3 export config2

Setting Parameters

Parameter Description Values

Fields Selection

Required

Select the field(s) you would like to push in your Amazon S3 here. You can also rename one or more field(s). To push all the fields, simple select SELECT ALL.

{Model Field Name, Destination Field Name}

Fetch Mode

Required

This refers to the manner in which data will get updated : FULL will update the entire column(s) from the selected data, INCREMENTAL will update the fresh record(S) added since last fetch

{Incremental,Full}

Default Value: FULL

Destination File Type

Required

Select the destination file type, that is whether to export data in CSV or newline delimited JSON or Apache Parquet or Apache Avro.

{CSV, txt, parquet, avro}

Default Value: CSV

Include Headers Dependant

Required (If Destination File Type = CSV)

Choose whether to include headers in the exported CSV file.

{Yes, No}

Default Value: Yes

File Path

Required

Enter the absolute path where the created file will be saved/uploaded. Supported datetime string formats such as tmp/dc_s3_csv_%Y-%m-%d %H:%M:%S. You can find further references at https://strftime.org/

{tmp/dc_s3_csv_%Y-%m-%d %H:%M:%S}

Data Sync Scheduling

Set the schedule for the sync to run. Detailed explanation on scheduling of syncs can be found here

Dataset & Name

Give your sync a name and some description (optional) and click on Finish to save it. Read more about naming and saving your syncs including the option to save them as templates here

Still have Questions?

We’ll be happy to help you with any questions you might have! Send us an email at info@datachannel.co.

Subscribe to our Newsletter for latest updates at DataChannel.