Unify Logo Footer.svg
Unify Automations
Logo
Consume Events

Consume Events

Logo

2 min READ

Consume Events pulls a batch of events from a named stream inside a running automation. A consumer group tracks your position so polling resumes exactly where it left off. Choose where a brand-new group starts — Earliest (from the beginning of the stream) or Latest (only events published after the group is created) — and pick a commit strategy that matches how much processing control you need.

Overview

Consume Events is the polling consumer action. It is designed for use inside a loop where you repeatedly pull batches from the stream and process them. The commit strategy controls when your position (offset) advances, which directly determines what gets re-read if a run fails partway through.

Screenshot: Consume Events action configuration panel

Input

Field

Type

Required

Description

Stream

String

Yes

Name of the stream to consume from.

Consumer group

String

Yes

Name of the consumer group that tracks your offset. Groups with the same name share a position; use distinct names when each consumer must see every event.

Starting position

Enum

Yes

Where a brand-new consumer group begins reading: Earliest (from the start of the stream) or Latest (only new events). Has no effect once the group has an established offset.

Commit strategy

Enum

Yes

When the offset advances: After each readAfter loop, or Manual.

Max events

Number

No

Maximum number of events to return in one batch.

Enable DLQ tracking

Boolean

No

When enabled, holds failed events so they can be routed to the dead-letter queue via the Update DLQ action. Default: off.

Output

The action returns the events fetched in this batch.

Field

Type

Description

events

Array

List of events consumed in this batch. Each entry includes the event ID, key, payload, and timestamp.

offset

Number

The current stream offset after this read.

Notes

  • After each read — advances the offset after every individual event. Simplest to set up; a mid-batch failure does not re-read already-committed events but may leave a partially processed batch.

  • After loop — advances the offset once the containing loop iteration completes. A mid-loop failure re-reads the full batch on the next run; suitable for batch work where partial processing is unacceptable.

  • Manual — you call the Manage Offsets action to advance the offset yourself. Gives full control; a missed commit leaves the position stuck and those events are re-read on the next poll.

  • The starting position only applies the first time a consumer group reads a stream. After that, the stored offset is always used.

  • Enable DLQ tracking if you plan to route failed events to the dead-letter queue using the Update DLQ action.

Idempotent consumers: Delivery is at-least-once. An uncommitted offset or a retry can deliver the same event more than once. Design your processing logic to handle a repeat gracefully.

FAQs

When should I choose manual commit?

Choose manual commit when you need to advance the offset only after confirming the event was processed successfully — for example, after a downstream write or API call returns a success response. Use Manage Offsets to commit when ready.

What happens if the automation stops without committing?

The offset stays at its last committed position. On the next run, the consumer group re-reads all events from that point, including any already processed in the failed run. Make consumers idempotent to handle this safely.

Can two automations share the same consumer group?

Sharing a consumer group means each event goes to whichever automation reads it first — they compete for events. Use separate consumer groups if each automation must see every event independently.