Upsert Data

This sync can be used to load data from a user specified data warehouse to the Pinecone vector database.

SETUP

Configuring the Credentials

Select the account credentials which has access to relevant Pinecone account from the given list & Click Next

Credentials not listed ?

Click on + for adding new credentials and fill the form that pops-up.

pinecone new creds 1
pinecone new creds 2
pinecone new creds 3

Data Sync Details

Data Sync

Select Upsert Data & click Next

pinecone upsert data list
Indexes

Select the indexes that you would like to use for this sync.

How do you want to fetch data?

Select whether you want to fetch data from the Data Model or from Table/ View.

Data Model

In case you want to fetch data using Data Model, select the data model that you would like to use for this sync. Checkout how to configure a model here.

Select Segment

Select the segment that you would like to use for this sync. Checkout how to configure a segment here.

Data Warehouse

In case you want to fetch data using Table/ View, select the data warehouse that you would like to use for this sync.

Table / View

Select the Table/ view in the data warehouse that you would like to use for this sync.

Setting Parameters

Parameter Description Values

Fields Selection

Dependent Required

Select the field(s) you would like to push in your Pinecone Vector database here. To push all the fields, simply select "Select All Fields". Using the checkboxes indicate whether you would like to push metadata/ embeddings.

{Model Field Name, Destination Field Name, Metadata, Embeddings}

Fetch Mode

Required

This refers to the manner in which data will get fetched from the warehouse: FULL will fetch all records available in the data model / segment and will send tham to the Pinecone Vector database,and INCREMENTAL will fetch only new records since the last sync and will send them to the Pinecone Vector database.

Incremental, Full

Default Value: FULL

Incremental Key Dependant

Required (If Fetch Mode = Incremental) This refers to the field using which the rows will be incrementally fetched for upload.

String value

Insert Mode

Required

This refers to the manner in which data will be updated in the Pinecone Vector database; with 'Upsert' selected, the data will be upserted (only new records or records with changes) and with 'Append' selected, all data fetched will be inserted.

Upsert, Append

Default Value: Upsert

Upsert Key Dependant

Required (If Insert Mode = Upsert)

Specify the upsert key based on which data is to be upserted.

String value
pinecone upsert data config 1
pinecone upsert data config 2

Data Sync Scheduling

Set the schedule for the sync to run. Detailed explanation on scheduling of syncs can be found here

Dataset & Name

Give your sync a name and some description (optional) and click on Finish to save it. Read more about naming and saving your syncs including the option to save them as templates here

Still have Questions?

We’ll be happy to help you with any questions you might have! Send us an email at info@datachannel.co.

Subscribe to our Newsletter for latest updates at DataChannel.