Events

Events are the data records that flow between tasks. Each task receives events, processes them, and emits new events to the next task.

Structure

FieldTypeDescription
dataEventDataPayload — JSON, Arrow RecordBatch, Avro, or raw bytes.
subjectstringTask name that produced the event.
idstringOptional identifier set by the source (e.g., Salesforce record ID, NATS message ID).
timestampintCreation time in microseconds since Unix epoch.
task_idintTask index in the flow.
task_typestringTask type (e.g., script, http_request).
metamapKey-value metadata that travels with the event.
errorstringError message if a task failed after retries.

Data formats

  • JSON — default for REST APIs, webhooks, scripts. Accessed as event.data.field in templates and scripts.
  • Arrow RecordBatch — columnar format for BigQuery, Parquet, CSV. Stays in Arrow through the flow without serialization overhead.
  • Avro — binary format for Salesforce Pub/Sub and gRPC streams.
  • Bytes — opaque binary payload for content that cannot be safely coerced to UTF-8 (archive contents, binary OCI layers, http_request responses with response_type: bytes). Handlebars templates against event.data see a base64-encoded string so they do not crash on binary; downstream tasks that need the raw bytes pattern-match EventData::Bytes directly. Only tasks that opt in accept this variant — text- or JSON-shaped processors reject it with an UnsupportedEventData error.

Metadata

event.meta is a key-value map that travels through the entire event chain. Each task preserves meta from the upstream event automatically.

Use meta for:

  • Passing context between tasks (e.g., source file path, request headers).
  • Correlation — event.meta.correlation_id is auto-generated (UUIDv7) for every event chain.
  • User context — event.meta.auth contains resolved user identity for authenticated webhooks.

Set meta in scripts:

event.meta.source = "salesforce";
event.meta.batch_id = uuid();
event

Access meta in templates:

endpoint: "https://api.example.com/{{event.meta.region}}/orders"

Correlation ID

Every event chain gets a correlation_id in meta, generated automatically when the first event is created. It stays the same through all downstream tasks. Use it to trace a single request through the entire flow in logs.

Webhook, AI gateway, and MCP tasks set their own correlation_id for response streaming. These override the auto-generated one.

Event chain

Tasks are connected by channels. Each task receives from the previous task and sends to the next:

[source] → channel → [transform] → channel → [sink]

If a task returns no data (e.g., a filter script returns ()), the event chain stops — no downstream event is emitted.