JSON

Process All Matching Files

Optional

If Yes then all the files matching the name will be processed. If No then the files modified or created within the specified period will be processed.

Yes, No

Default Value: Yes

File Selection Criteria

Dependant

Required

(If you do not want to process all files)

Enter the criteria based on which the file will be selected

Date Created , Date Modified

Default Value: Date Created

File Created/Modified Time Dependant

Required

(If you do not want to process all files)

Choose File’s creation or modification time from dropdown

Latest Created/Modified , Last 2 hours, Last 4 hours, Last 6 hours, Last 8 hours, Last 12 hours, Last 24 hours, Last 1 week

Default Value: Latest Created/Modified

Attempt Schema Inference

Required

If Yes then while processing it will be attempted to convert text to numbers and dates; WARNING: WILL DROP EXISTING WAREHOUSE TABLE. If No then everything will be fetched as it is irrespective of its type.

Yes, No

Default Value: No

Insert Mode

Optional

Specifies the manner in which data will get updated in the data warehouse : Upsert will insert only new records or records with changes, Append will insert all fetched data at the end, Replace will drop the existing table and recreate a fresh one on each run.

Upsert, Append, Replace

Default Value: Replace

Key

Dependant

Required

(If Upsert is chosen as the Insert Mode Type)

Enter the column name based on which data is to be upserted.

String value

Compressed

Optional

Indicate whether folder or file is compressed

Yes, No

Default Value: No

Compression Type

Dependant

Required

(If folder/file is compressed)

Enter the Compression type

Zip , Gzip

Default Value: Zip

Post Processing Action

Optional

Indicate the actions or processes carried out after the initial processing

No Action, Move files, Move and Compress files

Default Value: No Action

Move File Destination

Dependant

Required

(If folder/file is to be moved after processing)

Enter the Destination folder path

String Value

Target Compression Type

Dependant

Required

(If folder/file is to be compressed after processing)

Enter target compression type

Zip , Gzip

Include Source File Name

Required

Set this parameter to 'YES' if you want to include source file name in the data warehouse.

No, Yes

Default Value: No

Content Structure

Required

This refers to the manner in which your data in a file can be structured. In the case of Newline Delimited JSON, each line in the file represents a JSON object. On the other hand, for the Array of Objects structure, the data is wrapped within an array.

Newline Delimited JSON, Array of Objects

Default Value: Newline Delimited JSON