A pipeline only moves the data you tell it to. The Objects tab is where you choose which source objects or folders to ingest, decide how each one gets synced, and fine-tune per-object behavior like cursors and record filters. This page walks through each of those controls.
Overview
Every pipeline has an Objects tab that lists the tables, entities, files, or folders it pulls from your source. Nothing is ingested until you add at least one object here, and everything else on the tab — sync toggles, field mapping, cursors, and filters — is scoped to the objects you've selected.
Selecting source objects or folders
Open the Objects tab and click Add Objects. What you see next depends on the source you connected:
For most sources, you'll get Select Source Objects, letting you pick the tables, entities, or files to bring in.
For file-store sources, you'll instead get Select Source Folders.
When picking folders, keep in mind that all files inside a single folder are combined and treated as one object, while any subfolders are treated as separate objects in their own right. Plan your folder structure with that in mind before you select — splitting related files across subfolders will split them into separate objects downstream. You must select at least one object or folder before you can move on.
File formats for file-based sources
If your source is file-based, you'll also be asked to set the File Format to Parse. Only files matching the format you choose — JSON, CSV, AVRO, TSV, or XML — will actually be parsed and ingested; everything else in the folder is ignored.
Under advanced settings you can narrow things further with a file name pattern, and specify a compression method if your files are archived — bz2, gz, gzip, zip, and tar are all supported. If none of the files in your selected folders match the format you chose, you'll see a message letting you know no files were found, which is a good signal to double-check your format or pattern before troubleshooting anything else.
Managing objects in the table
Once objects are added, each one appears as a row in a table with a few key controls:
Sync — a toggle that turns ingestion for that object on or off. Turn it off to pause an object without removing it from the list.
Object — the name of the source object or folder.
Fields Mapped — a running count of how many fields are mapped to your destination, shown as a fraction like 8 of 12, so you can spot incomplete mappings at a glance.
Webhook Url — for sources that push data via webhooks, this column shows the URL events should be sent to. Click to copy it into whatever system is sending the events.
Setting a cursor
A cursor tells the pipeline which field to watch when checking for recent updates in a source object — commonly a "last modified" timestamp. To set one, open the row menu for an object and choose Set cursor. In the dialog that appears, select the field you want the pipeline to track for recent updates, then save. You'll see a confirmation once the cursor is updated.
Setting a cursor is what allows the pipeline to pick up only new or changed records on later syncs instead of re-processing the entire object every time.
Filtering which records get ingested
You don't have to ingest every record in an object. Switch an object from All Records to Filter Records to define a condition that limits what comes in — only records matching that condition are ingested. This is worth using whenever you know upfront that you'll never need certain records, since filtering at the source is more efficient than pulling everything and filtering it out later in your destination. Once a filter is saved, the object's row shows how many fields the filter is based on, so you can tell at a glance which objects are filtered.
Notes
If you change which objects are selected and some of them already have field mappings in place, you'll be asked to confirm the change — continuing discards those existing mappings. Only confirm if you're ready to remap those objects.
Folders and their subfolders are ingested as separate objects, so review your source folder structure before adding folders rather than after.
Turning off Sync for an object pauses ingestion but keeps its configuration, cursor, and filters intact for when you turn it back on.