Unify Logo Footer.svg
Unify Data
Logo
Pipeline Logs & Monitoring

Pipeline Logs & Monitoring

Logo

3 mins read READ

Every running pipeline has a Logs tab that shows you exactly what's happening to your data — how fast it's moving, how many records each object has processed, and which ones failed. Pair it with the Activity tab to see who changed what, and set up an alert so you don't have to keep the tab open to catch problems.

Overview

Open a pipeline and go to its Logs tab to see four stat cards at the top: Latency, Throughput, Average Processing Duration, and Current Sync Phase. Current Sync Phase tells you whether the pipeline is still working through its initial snapshot load or has moved on to processing real-time changes as they happen.

Latency reflects recent real-time events only, so treat it as meaningful once the pipeline is past its historic load. While the initial load is still running, the latency figure won't represent the pipeline's ongoing, steady-state performance.

Per-Object Migration Counts

Below the stat cards, a table breaks activity down by Source Object and Destination Object. For each pairing you'll see:

  • Records Ingested — the total number of records the pipeline picked up for that object

  • Migration Successful — records that completed processing without issue

  • Migration Skipped — records the pipeline chose not to process

  • Migration Failed — records that hit an error

Every processed record lands in exactly one of those three outcome buckets. Use the date-range filter above the table to scope the counts to a specific window, and drag any column header left or right if you want to reorder how the counts are laid out.

Inspecting and Retrying Failed Records

When a record fails, open its event to see the details in two tabs:

  • Event Log — the event's attributes as key/value pairs, giving you the raw context around what happened

  • View Records — the record's fields side by side, with Value (Before) and Value (After), so you can see exactly what change the pipeline was attempting when it failed

Full record content is captured for every failed event, so you're never left guessing at what was in the record. Once you understand the cause, you can retry the migration directly from this view. The result comes back as either Retry Successful or Retry Failed, so you always know whether the fix worked without having to dig back through the table.

Tracking Changes with the Activity Tab

The Activity tab is an audit trail for the pipeline — who changed what, and when. It's most useful for correlating a change in behavior with a change in configuration. For example, if throughput drops on a particular day, check Activity to see whether someone edited the schema, a mapping, or another setting around the same time.

Setting Up Alerts

Rather than keeping the Logs tab open and watching for problems, create an alert rule from the Alerts tab. On an individual pipeline, Alerts scopes the rule to that pipeline; from the Pipelines manager, Alerts covers your pipelines more broadly. Once a rule is created, its notifications fire automatically whenever the condition is met, so you find out about a problem without having to go looking for it.

Notes

  • Latency only reflects real-time events — during the initial snapshot load, this stat won't yet represent steady-state performance.

  • Every record ends up in exactly one outcome — successful, skipped, or failed — so the three counts in the objects table always account for everything ingested.

  • Want metrics across every pipeline in your workspace instead of just this one? Use the Insights tab on the Pipelines manager — it aggregates numbers across all your pipelines, while per-pipeline detail stays on each pipeline's own Logs tab.

  • You can drag column headers in the objects table to reorder them to whatever layout is most useful for you.