Custom Query

Custom Query report enables a user to transfer data from the configured Snowflake Data Warehouse into the DataChannel with the granularity of a Custom Query result.

Configuring the Credentials

Select the account credentials which has access to relevant Snowflake Data Warehouse data from the dropdown menu & Click Next

Credentials not listed in dropdown ?: Click on + Add New for adding new credentials

Data Pipelines Details

Data Pipeline: Select Custom Query from the dropdown

Setting Parameters

Select the fields that are necessary as per the file or folder .

Parameter	Description	Values
Fetch Mode	*Required* Specifies the manner in which data will get fetched from the data warehouse : FULL will fetch all the column(s) from the selected table/view, INCREMENTAL will bring the fresh record(S) added since last fetch. If Increment is selected, please make sure that the entered key is in the case you want. Multiple keys should be comma-seperated.	{Incremental,Full} *Default Value:* FULL
Insert Mode	*Required* Specifies the manner in which data will get updated in the data warehouse : UPSERT will insert only new records or records with changes, APPEND will insert all fetched data at the end, REPLACE will drop the existing table and recreate a fresh one on each run. If UPSERT is selected, please make sure that the entered key is in the case you want. Multiple keys should be comma-seperated.	{Upsert, Append, Replace} *Default Value:* UPSERT
Query	*Required* Write a Query in the given space. Please ensure that all the column and table names are enclosed in double quotes, to make sure that the case is preserved. By default Snowflake takes all column and table names are in upper-case.	Query
Attempt Schema Inference	*Required* If Yes then value types will be fetched as it is, eg: Float will be fetched as float. If No then everything will be fetched as string irrespective of its type.	{Yes,No} *Default Value:* No

Parameter

Description

Values

Fetch Mode

Required

Specifies the manner in which data will get fetched from the data warehouse : FULL will fetch all the column(s) from the selected table/view, INCREMENTAL will bring the fresh record(S) added since last fetch. If Increment is selected, please make sure that the entered key is in the case you want. Multiple keys should be comma-seperated.

{Incremental,Full}

Default Value: FULL

Insert Mode

Required

Specifies the manner in which data will get updated in the data warehouse : UPSERT will insert only new records or records with changes, APPEND will insert all fetched data at the end, REPLACE will drop the existing table and recreate a fresh one on each run. If UPSERT is selected, please make sure that the entered key is in the case you want. Multiple keys should be comma-seperated.

{Upsert, Append, Replace}

Default Value: UPSERT

Query

Required

Write a Query in the given space. Please ensure that all the column and table names are enclosed in double quotes, to make sure that the case is preserved. By default Snowflake takes all column and table names are in upper-case.

Query

Attempt Schema Inference

Required

If Yes then value types will be fetched as it is, eg: Float will be fetched as float. If No then everything will be fetched as string irrespective of its type.

{Yes,No}

Default Value: No

Datapipeline Scheduling

Scheduling specifies the frequency with which data will get updated in the data warehouse. You can choose between Manual Run, Normal Scheduling or Advance Scheduling.

Manual Run: If scheduling is not required, you can use the toggle to run the pipeline manually.
Normal Scheduling: Use the dropdown to select an interval-based hourly, monthly, weekly, or daily frequency.
Advance Scheduling: Set schedules fine-grained at the level of Months, Days, Hours, and Minutes.

Detailed explanation on scheduling of pipelines can be found here

Dataset & Name

Dataset Name: Key in the Dataset Name(also serves as the table name in your data warehouse).Keep in mind, that the name should be unique across the account and the data source. Special characters (except underscore _) and blank spaces are not allowed. It is best to follow a consistent naming scheme for future search to locate the tables.
Dataset Description: Enter a short description (optional) describing the dataset being fetched by this particular pipeline.
Notifications: Choose the events for which you’d like to be notified: whether "ERROR ONLY" or "ERROR AND SUCCESS".

Once you have finished click on Finish to save it. Read more about naming and saving your pipelines including the option to save them as templates here

Still have Questions?

We’ll be happy to help you with any questions you might have! Send us an email at info@datachannel.co.

Subscribe to our Newsletter for latest updates at DataChannel.