Git Sync

Clones or pulls a Git repository and emits one event per file. Downstream tasks decide what to do with the content — parse it, store it, transform it.

Each event contains {path, content, commit} where path is relative to the scanned directory.

Configuration

- git_sync:
    name: sync_flows
    repository_url: "git@github.com:org/configs.git"
    branch: main
    path: "flows/"
    auth:
      type: ssh
      ssh_key_path: /etc/git/deploy-key

Fields

FieldTypeDefaultDescription
namestringrequiredTask name.
repository_urlstringrequiredGit repository URL (SSH or HTTPS).
branchstringmainBranch to track.
pathstringDirectory within the repo to scan. All files under this path are emitted.
clone_pathstring<temp>/<flow_name>/<task_name>Local path to clone into. Defaults to a per-task subdirectory of the system temp directory so multiple git_sync tasks in one worker do not collide. Override only when you need a stable path on a persistent volume. Paths containing .. are rejected.
authobjectnoneAuthentication configuration (see below).
depends_onlistUpstream task names.
retryobjectRetry configuration.

Authentication

FieldTypeDescription
typestringnone, ssh, or token.
ssh_key_pathstringPath to SSH private key (for ssh type).
ssh_known_hosts_pathstringPath to known_hosts file (for ssh type).
tokenstringToken for HTTPS auth (for token type).

Example: Sync flows from Git to NATS KV

flow:
  name: git_sync_flows
  tasks:
    - generate:
        name: trigger
        interval: "5m"

    - git_sync:
        name: pull_repo
        repository_url: "git@github.com:org/configs.git"
        path: "flows/"
        auth:
          type: ssh
          ssh_key_path: /etc/git/deploy-key

    - script:
        name: normalize_key
        code: |
          let key = "flows." + event.data.path.replace("/", ".");
          event.data.key = key;
          event

    - nats_kv_store:
        name: save_to_kv
        operation: put
        bucket: flowgen_system
        key: "{{event.data.key}}"
        credentials_path: /etc/nats/credentials.json