Unify Logo Footer.svg
Unify Data
Logo
Schema Mapping & Transformations

Schema Mapping & Transformations

Logo

5 mins read READ

The Schema tab controls how objects and fields from your source land in your destination, and how their values change along the way. This page covers mapping methods, upsert keys, and transformations.

Overview

Every pipeline needs a way to turn source objects into destination objects. Start by picking a mapping method, then either let the platform replicate your source structure automatically or map each field yourself. Once fields are mapped, layer on transformations — functions that reshape, mask, encode, or otherwise change a value before it reaches the destination.

Choosing a mapping method

On the Schema tab, choose a mapping method for each source object:

  • Map Manually — choose an existing destination object per source object and wire up field mappings yourself.

  • Replicate Source — the pipeline automatically creates matching destination objects that mirror your source structure.

For file or object store destinations, manual mapping isn't available, so Replicate Source is selected automatically.

Configuring Replicate Source

With Replicate Source, you control how the destination stays in sync:

  • Schema Evolution — sync all future changes to existing objects and fields, or block them.

  • Schema Evolution for New Objects — on by default; replicates new source objects and keeps syncing their changes.

  • Populate Created and Updated Timestamps — adds two system fields tracking when each record was created and last updated.

  • Destination Object Prefix — a required prefix marking the objects this pipeline created, so you can tell them apart from other data.

For storage destinations, also set output file details: file format (JSON Lines or CSV), a field delimiter for CSV, and a folder path and file name, where you can insert pipeline metadata dynamically.

Mapping fields manually

With Map Manually, first choose a destination object for each source object, then match each source field to a destination field:

  • On destinations that let you define fields on the fly, click a destination cell to edit or add a field directly. Press Enter to save or Escape to discard, and use Unmap Destination to clear a mapping.

  • On destinations with a fixed schema, pick an existing destination field from a dropdown instead.

Working with upsert keys

Mark a field as the Upsert Key so incoming records update matching rows instead of creating duplicates. If several fields together identify a row uniquely, mark them as an Upsert Composite Key; fields from your source's primary key are flagged separately.

Renaming a destination field clears its upsert and primary key flags — re-mark a renamed key field as the Upsert Key, or new records may duplicate instead of update.

Using Auto Map

Click Auto Map Schema to have the platform create likely field mappings automatically instead of mapping each one by hand. While it runs you'll see a progress indicator, then a summary of fields mapped. Treat the result as a starting point — always review mappings before saving, since automatic matches aren't guaranteed correct.

Adding transformations

To change a value in transit, click Add Transformation on the field, pick a function from the catalog, and configure it. Before saving, test it: enter a sample input, or pull one from real data, and check the result.

A transformation can overwrite its field or save its output into a new field, in which case further transformations in that chain must be added there. Save each transformation before adding the next — deleting one that created a field also deletes that field and its mapping, so you'll see a warning first.

Callable Automation and Execute App and Action instead call another automation or a connected app's action, mapping the response back into your pipeline. Enable continue-on-failure to keep processing a record even when that call fails. Pick which response fields to keep and name each one; edited names persist on reopen.

Supported transformation types

The function catalog includes these transformation types:

TypeDescription
EncryptionEncrypts a value so only someone with the matching key can read it again.
EncodingConverts a value into an encoded form, such as Base64, for safe transport or storage.
MaskingHides part of a value, such as showing only the last digits of an account number, while keeping the rest visible.
HashingConverts a value into a fixed, irreversible hash, useful for comparing or de-identifying data.
Text Casing ModificationChanges text capitalization to upper case, lower case, or title case.
Spreadsheet FormulaApplies a spreadsheet-style formula to compute a new value from one or more fields.
Replace ValueFinds and replaces specific text or patterns within a value.
Duplicate FieldCopies a field's value into another field, keeping the original intact.
Extract TextPulls a specific portion of a value, such as a substring or pattern match, into a new field.
File TransferDownloads or uploads file content to and from cloud storage such as Amazon S3 or Azure Blob Storage.

Why can't I edit destination objects on a replicated pipeline?

When Replicate Source and schema evolution are both on, object selection locks so the destination stays in sync automatically — disable schema evolution in the replication settings first to edit objects directly. You can also disable replication for one object, but this discards its mappings and replicates it fresh next run, so keep auto mapping enabled if you go this route.

Notes

  • Mapping problems show as a count badge on the Schema tab and block deployment until resolved, though you can still save a schema with errors.

  • Auto Map is a convenience, not a guarantee — always review suggested mappings before saving.

  • Renaming a mapped destination field clears its Upsert Key and primary key flags, so double-check keys after any rename.