Skip to main content

Conduit CLI

Conduit CLI is how you configure, run, and operate Conduit — pipelines, connectors, processors, and the engine itself. It's built on Ecdysis, Conduit's own Go CLI framework.

This page is the exhaustive command reference. For narrower how-tos, see:

Conventions that apply to every command​

  • --json — every command supports it. Output is a single JSON object to stdout: {"command": "...", "ok": true|false, "summary": {...}, "result": {...}, "error": null}. ok/error are always present; error is only set on a hard command failure (bad input, unreachable dependency), never for domain findings (a validation run that finds problems is ok:false, error:null, with the problems under result). See Structured errors & JSON output.
  • Exit codes are deterministic: 0 success · 1 runtime/internal error · 2 validation (bad input, not found, failed precondition) · 3 environment (server/database/index unreachable). Some commands (doctor, connectors audit) reduce multiple findings to one exit code by taking the worst. See Exit codes.
  • --quiet / -q — suppress passing/OK lines and progress chrome; print only warnings, failures, and the summary. Doesn't change the exit code.
  • --no-color — disable colored/glyph output even on a color-capable terminal (glyphs also fall back to ASCII automatically when not a TTY, or when NO_COLOR is set).
  • --yes / -y — skip an interactive confirmation prompt on a mutating command.
  • Global persistent flags on every command: --api.grpc.address (address where a running Conduit is reachable) and --config.path (path to conduit.yaml).

Quick reference​

CommandAliasesNeeds a running server?Summary
conduit run—Start the Conduit server and run configured pipelines
conduit initofflineScaffold a workspace (conduit.yaml + directories)
conduit quickstart—Run an ephemeral demo pipeline, no config needed
conduit doctoroffline (optional)Preflight checks: would conduit run succeed here?
conduit mcp—Run Conduit's MCP server for AI agents
conduit generateoffline (calls an LLM provider)Generate a pipeline config file from a plain-language description
conduit migrate kafka-connectofflineTranslate Kafka Connect connector configs into Conduit pipelines plus a compatibility report
conduit open docsdocsofflineOpen the Conduit documentation in a browser
conduit configofflineShow the resolved configuration
conduit versionofflinePrint the Conduit version
conduit pipelines listlsyesList registered pipelines
conduit pipelines describedescyesFull static config of one pipeline
conduit pipelines initofflineScaffold a pipeline config file
conduit pipelines validateofflineParse/validate one or more pipeline configs
conduit pipelines lintofflineValidate + advisory warnings
conduit pipelines dry-runofflineValidate + show the enriched graph run would load
conduit pipelines inspectyesLive status + per-stage state of one pipeline
conduit pipelines deployprefers live, falls back offlinePreview (and optionally apply) a pipeline's changes
conduit pipelines applyprefers live, falls back offlineApply a previously-previewed plan
conduit pipelines repairofflinePreview/apply machine-fixable config fixes
conduit pipelines startyesStart a stopped pipeline
conduit pipelines stopyesStop a running pipeline
conduit pipelines dev—run --dev sugar: hot-reload watcher
conduit connectors listlsyes (see --installed)List pipeline connector instances, or installed plugin artifacts
conduit connectors describedescyesDetail of one connector instance
conduit connectors newofflineScaffold a new Go connector repository
conduit connectors installofflineInstall a standalone connector from the registry
conduit connectors uninstallofflineRemove an installed standalone connector
conduit connectors auditofflineRe-verify installed connectors against the registry
conduit connectors bundleofflinePrepare an offline install bundle
conduit connector-plugins listlsyesList connector plugins available to pipelines
conduit connector-plugins describedescyesDetail of one connector plugin
conduit processors listlsyesList registered processor instances
conduit processors describedescyesDetail of one processor instance
conduit processors newofflineScaffold a new Go processor repository
conduit processor-plugins listlsyesList processor plugins available to pipelines
conduit processor-plugins describedescyesDetail of one processor plugin
conduit processor-plugins installofflineInstall a standalone WASM processor from the registry
conduit processor-plugins uninstallofflineRemove an installed standalone WASM processor

"Needs a running server" means the command dials the API of a conduit run process. "offline" commands never do. deploy/apply are hybrid: they dial a live server if one is reachable at --api.grpc.address and fall back to opening the local store directly (see Deploy and apply).


Top-level commands​

conduit run​

Starts the Conduit server and runs the pipelines under --pipelines.path. Blocks until shutdown. See Configuration for the full flag/env/config-file reference — highlights relevant here:

FlagDescription
--devAlias for --dev.enabled: watch --pipelines.path and hot-reload changes into the running engine as you save. A processor-only edit applies in place (no restart); a source/destination/topology edit applies via a labeled graceful restart; a bad edit is reported and never touches the running pipeline.
--pipelines <path>Alias for --pipelines.path.
--api.allow-live-restart-applyAuthorize pipelines apply/MCP apply (from any client) to drain-and-restart a running pipeline. Off by default. See Deploy and apply.
--config.pathPath to conduit.yaml.

--dev is self-authorizing (running --dev and watching the output is the authorization) — it does not set --api.allow-live-restart-apply, which stays independently gated for the gRPC/HTTP/MCP surface.

conduit run
conduit run --dev
conduit run --dev --pipelines.path ./pipelines --dev.json

Exit codes: 0 on a graceful single SIGINT/SIGTERM; 130/143 on a forced second signal (128 + signum). See Exit codes.

conduit init​

Sets up a Conduit workspace in the current directory: writes conduit.yaml and creates the pipelines, connectors, and processors directories. Does not create a pipeline — that's conduit pipelines init.

FlagDescription
--path <dir>Where to initialize the workspace. Default: current directory.
conduit init
conduit init --path ./my-workspace

conduit quickstart​

The fastest way to see Conduit working: scaffolds an ephemeral demo pipeline (built-in generator → built-in log) and runs it immediately, in a temp directory removed on exit, with in-memory state. Press Ctrl+C to stop.

FlagDescription
--jsonEmit the demo's logs and records as JSON instead of human-readable text.
conduit quickstart

conduit doctor​

Offline, non-destructive preflight: "would conduit run succeed here?" Checks config resolution/validation, database reachability, API address availability, plugin directories, and (optionally) whether a running engine is reachable. It does not start a Runtime — distinct from the running server's /readyz and /healthz endpoints.

FlagDescription
--deepAlso run plugins.standalone_compat, which dispenses standalone connector plugin binaries in an isolated subprocess.
--require-serverFail (instead of warn) if the API server isn't reachable.
--check <name>Run only the named check(s); repeatable. --help lists all check names.
--quiet / -qSuppress passing checks.
--no-colorDisable colored output.
conduit doctor
conduit doctor --deep --json
conduit doctor --check db --check plugins.builtin

Exit codes: 0 every check passed (warnings never fail) · 2 invalid configuration · 3 a required dependency (database, network address, or — with --require-server — the API server) is unreachable.

conduit mcp​

Runs Conduit's MCP server, registering pipeline operations as MCP tools that are 1:1 with the CLI verbs and call the exact same engines: validate/lint/ dry_run/deploy/doctor/inspect are always available (deploy only computes a diff + hash, never mutates); apply/start/stop/ scaffold_connector/scaffold_processor are additionally registered only with --allow-mutations.

FlagDescription
--allow-mutationsRegister the write tools. An operator/process-level switch — never agent-settable.
--http <addr>Serve the streamable-HTTP transport on this address instead of stdio (EXPERIMENTAL). Requires --token-file and --tls-cert/--tls-key.
--token-file <path>Bearer token file for --http (compared constant-time).
--tls-cert <path> / --tls-key <path>TLS cert/key for --http.
--api-address <addr>gRPC address of a running Conduit, dialed by the inspect/start/stop tools.

Serves stdio by default (no auth needed — the agent owns the process). --http is a network-daemon mode (systemd/container) and serves HTTP only (never stdio simultaneously); it refuses to start unless both a bearer token and TLS are configured — no plaintext, no unauthenticated HTTP.

conduit mcp
conduit mcp --allow-mutations --api-address localhost:8084
conduit mcp --http :8443 --token-file token.txt --tls-cert cert.pem --tls-key key.pem

See docs/operations/mcp-server.md in the conduit repo for the full transport and hardening reference.

conduit generate​

Turns a plain-language description into a pipeline configuration file. Every candidate is checked with the offline validate engine (including that each referenced plugin exists) and with a deterministic intent check before it is shown or written; failures are retried with the findings, up to --max-retries. It never deploys: it writes a file, and deploying it is the usual conduit pipelines deploy step. Preview in v0.20: it picks connectors reliably but often leaves out requested processor steps, so review what it writes.

Needs a provider: ANTHROPIC_API_KEY, OPENAI_API_KEY, or a local Ollama server (OLLAMA_HOST, default http://localhost:11434). With more than one available, choose with --provider or CONDUIT_GENERATE_PROVIDER.

FlagDescription
--provider <name>anthropic, openai or ollama. Auto-detected when exactly one is configured.
--model <id>Provider-specific model. The provider's default when unset.
--out <path>Output file. Defaults to a name derived from the pipeline id, in the working directory.
--max-retries <n>Provider calls allowed per run, including the first. Default 3.
--forceOverwrite an existing output file.
--jsonEmit the result as JSON.
--no-colorDisable colored output.
conduit generate "read from postgres and write new rows to s3 as json"
conduit generate "stream orders from postgres into kafka, only orders over 100" --out orders.yaml
conduit generate "sync kafka into postgres" --provider anthropic --json

Error codes include generate.no_provider_configured, generate.validate_failed, generate.semantic_mismatch and generate.destination_exists. See Generate a pipeline from a description for the full list and how provider selection works.

conduit migrate kafka-connect​

Offline: reads Kafka Connect worker and connector configs and writes, per connector, a Conduit pipeline plus a compatibility report. Never drops a key it can't translate. See the Migrating from Kafka Connect guide for the concept mapping, covered connectors, the report contract, and every flag.

conduit migrate kafka-connect connectors/orders.json
conduit migrate kafka-connect --worker connect-distributed.properties ./connectors --dry-run

Exit codes: 0 no entry blocks · 2 an entry blocks or input is invalid · 3 an output file cannot be written · 1 internal error.

conduit open docs (alias: conduit docs)​

Opens the Conduit documentation (conduitdata.io) in your default web browser.

conduit open docs
conduit docs
note

conduit docs at the top level is a hidden alias (it works, but doesn't show in conduit --help) for conduit open docs.

conduit config​

Prints the fully-resolved configuration Conduit would run with — flags, env vars, and config file merged — using the same flags as conduit run.

conduit config

conduit version​

Prints the current Conduit version.

conduit version

conduit pipelines (alias: conduit pipeline)​

Initialize and manage pipelines.

list (alias ls)​

Lists pipelines registered by a running Conduit. Requires a running server.

conduit pipelines list
conduit pipelines ls --json

describe PIPELINE_ID (alias desc)​

Full static configuration detail of one pipeline: config, connectors (with their processors), pipeline-level processors, and DLQ config. Requires a running server.

conduit pipelines describe orders
conduit pipeline desc orders --json

init [PIPELINE_NAME]​

Scaffolds a pipeline configuration file (offline — no running server needed). With no flags, produces a runnable demo pipeline (generator → log). --source/--destination select any built-in connector instead, or --template <name> scaffolds a complete, ready-to-run pipeline from the templates gallery.

FlagDescription
--source <name>Source connector (any built-in). Default: generator (demo).
--destination <name>Destination connector (any built-in). Default: log (demo).
--template <name>Scaffold from a named vendored template. --template list enumerates the gallery. Mutually exclusive with --source/--destination.
--pipelines.path <dir>Where to save the file. Default: ./pipelines.
--forceOverwrite an existing pipeline file at the destination path.
--dry-runPrint the configuration that would be written, without touching the filesystem.
--jsonMachine-readable output (the rendered config, resolved path, and — with --template list — the template list).

Refuses to overwrite an existing pipeline file at the destination path, returning a structured error with a suggested fix unless --force is set. --dry-run never writes, so it is exempt from that check.

conduit pipelines init
conduit pipelines init --source generator --destination s3
conduit pipelines init file-to-pg --source file --destination postgres --pipelines.path ./my-pipelines
conduit pipelines init --template list --json
conduit pipelines init --template postgres-cdc-kafka
conduit pipelines init --template generator-log --dry-run --json

validate <path>​

Offline, errors-only check: runs the same parse, enrich, and validate steps conduit run uses to load a pipeline config, without starting Conduit or dialing its API. <path> is a single .yml/.yaml file, or a directory of them (not recursed). Every problem in every file is reported — one bad file never stops the rest from being checked.

FlagDescription
--quiet / -qSuppress passing/OK lines.
--no-colorDisable colored/glyph output.
conduit pipelines validate pipelines/orders.yaml
conduit pipelines validate ./pipelines --json

Exit 0 if every pipeline is valid, 2 otherwise. See also lint (adds advisory warnings) and dry-run (adds the enriched graph).

lint <path>​

Everything validate checks, plus the parser's advisory warnings (deprecated/renamed fields, unrecognized fields, config version fallback), located by line and column. Offline, like validate.

FlagDescription
--strictEscalate warnings to failures (exit 2). Validation errors always fail regardless.
--quiet / -qSuppress passing/OK lines.
--no-colorDisable colored/glyph output.
conduit pipelines lint pipelines/orders.yaml
conduit pipelines lint ./pipelines --strict --json

dry-run <path>​

Everything validate checks, then reports the fully-enriched pipeline graph conduit run would actually load: final connector/processor IDs, injected DLQ defaults, and worker counts. Offline, no side effects.

FlagDescription
--resolve-pluginsCheck that referenced builtin plugins exist (default: true). An unknown builtin fails (exit 2). Standalone plugin references stay advisory — not statically verifiable offline.
--quiet / -qSuppress passing/OK lines.
--no-colorDisable colored/glyph output.
conduit pipelines dry-run pipelines/orders.yaml
conduit pipelines dry-run ./pipelines --resolve-plugins=false --json

inspect PIPELINE_ID​

The online, status-forward operational view of a running pipeline: live status (running/stopped/degraded/recovering), any error, and a per-stage summary of sources, destinations, and the DLQ. Unlike validate/lint/ dry-run, this dials the API. Use list to find pipeline IDs, describe for full static config.

conduit pipelines inspect orders
conduit pipelines inspect orders --json

deploy <file>​

Computes the diff between <file> and the pipeline's current state — one create/update/delete per pipeline/connector/processor, each classified in_place (safe on a running pipeline) or restart — without applying anything. Emits a plan hash. Prefers a live server (if reachable at --api.grpc.address); falls back to opening the local store directly if not. See Deploy and apply for the full workflow and the in-place/restart distinction.

FlagDescription
--applyCompute the plan and apply it in one call (prompts for confirmation unless --yes).
--yes / -yWith --apply, skip the confirmation prompt.
--quiet / -qSuppress per-change lines; print only the summary.
--no-colorDisable colored/glyph output.
conduit pipelines deploy orders.yaml
conduit pipelines deploy orders.yaml --json
conduit pipelines deploy orders.yaml --apply --yes

deploy never mutates on its own — only --apply (or apply itself) does.

apply <file>​

Recomputes the plan for <file> and executes it, but only if --plan-hash matches the freshly recomputed plan's hash exactly — a mismatch (the file or the live pipeline changed since deploy computed that hash) refuses with provisioning.plan_stale (exit 2), never partially applied.

FlagDescription
--plan-hash <hash>The hash from conduit pipelines deploy. Required unless --yes.
--yes / -yApply the freshly recomputed plan directly, without a hash.
--no-colorDisable colored/glyph output.
conduit pipelines apply orders.yaml --plan-hash 9f3a2c...
conduit pipelines apply orders.yaml --yes --json

Idempotent — applying an already-applied config computes an empty plan and does nothing (exit 0). Against a running pipeline reached via a live server, applying requires operator authorization (--api.allow-live-restart-apply on the server) unless the change is entirely live-swappable and taken via the in-place path — see Deploy and apply.

repair <file>​

Collects every finding validate/lint can find a structured, machine-appliable fix for (a deprecated/renamed field, an unambiguous invalid /status value, a negative processor /workers, an over-long /description) and shows the proposed edit as a diff with a plan hash. Mutates nothing by default — edits the config file only, never the pipeline store or a running pipeline.

FlagDescription
--applyApply the plan instead of only previewing it (safe fixes only, unless --fix selects others).
--plan-hash <hash>The hash from a prior repair read. Required with --apply unless --yes.
--yes / -yWith --apply, apply the freshly recomputed plan directly.
--fix <configPath>Apply only the fix(es) at this configPath; repeatable.
--escalatePermit applying an explicitly --fix-selected data-path-adjacent fix (ack/position/checkpoint-adjacent config). Human-only — never available via MCP.
--no-colorDisable colored/glyph output.
conduit pipelines repair orders.yaml
conduit pipelines repair orders.yaml --apply --plan-hash 9f3a2c...
conduit pipelines repair orders.yaml --apply --yes --fix /processors/0/workers

Restart-class and data-path-adjacent fixes are never applied by default — select one explicitly with --fix; a data-path-adjacent fix additionally requires --escalate. Getting a repaired file into a running engine is still deploy/apply.

start PIPELINE_ID​

Transitions a stopped pipeline to Running against a live Conduit server. Requires a reachable server — there is deliberately no offline fallback (a started pipeline's goroutines only stay alive inside a long-running process).

conduit pipelines start orders
conduit pipelines start orders --json

Starting an already-running pipeline is refused with pipeline.running (exit 2); nothing is mutated. Once the server has begun shutting down, a start is refused with pipeline.shutting_down (gRPC Unavailable, exit 3); start it again after the server restarts.

stop PIPELINE_ID​

Transitions a running pipeline to UserStopped against a live server: gracefully by default (drains in-flight records before returning), or immediately with --force.

FlagDescription
--forceSkip the graceful drain and stop immediately. Positions are crash-safe and delivery is at-least-once, so this behaves like a crash on restart, not data loss.
conduit pipelines stop orders
conduit pipelines stop orders --force --json

This command does not wait for the drain to fully settle before returning — it reports the pipeline's status immediately after the transition RPC returns, which may still be transitional. Stopping an already-stopped pipeline is refused with pipeline.not_running (exit 2).

A stopped pipeline stays UserStopped and is never restarted by error recovery. If an error interrupts the drain, or the stop is forced (recorded as force stop), the error is kept on the pipeline and shown by conduit pipelines inspect; it does not make the pipeline Degraded and does not trip pipelines.exit-on-degraded. See Statuses.

dev [dir]​

Sugar for conduit run --dev [--pipelines.path dir]: starts a full Conduit server with the hot-reload watcher enabled and pipelines.exit-on-degraded forced off (so one intentionally-broken pipeline mid-edit doesn't take the whole dev server down). Carries no watcher logic of its own — it's a thin alias over run --dev.

conduit pipelines dev
conduit pipelines dev ./pipelines
conduit pipelines dev --dev.json

conduit connectors (alias: conduit connector)​

list (alias ls)​

Lists connector instances in pipelines registered by a running Conduit.

FlagDescription
--pipeline-id <id>Filter connectors by pipeline ID.
--installedList installed connector plugin artifacts from the local install manifest instead — a completely different thing from a pipeline connector instance. Mutually exclusive with --pipeline-id.
--index-url / --index-fileRegistry index source, only consulted with --installed (for a best-effort "latest available" column).
conduit connectors list
conduit connectors list --installed

describe CONNECTOR_ID (alias desc)​

Detail of one connector instance and its attached processors.

conduit connectors describe connector:source

new [name]​

Scaffolds a full connector repository from ConduitIO/conduit-connector-template — SDK wiring, tests, CI, release workflow, acceptance test harness — ready to build with no edits. --lang go (default) and --lang python are available; the generated Python project's conduitio dependency does not install from PyPI yet.

FlagDescription
--module <path>Go module path, e.g. github.com/you/conduit-connector-<name> (required).
--path <dir>Destination directory. Default: ./conduit-connector-<name>.
--git / --no-gitInitialize a git repository and first commit (default: on).
--forceOverwrite the destination directory if it already exists.
conduit connectors new s3 --module github.com/you/conduit-connector-s3
conduit connector new s3 --yes --json

install <name>[@version], uninstall <name>[@version], audit, bundle <name>[@version]​

Registry connector-lifecycle commands — install/remove standalone connector plugin artifacts, re-verify installed connectors against the registry index, and prepare an offline install bundle. All are offline (they never dial the running Conduit API; they operate directly on --connectors.path and the registry index).

CommandOne-line summary
installResolve, download, verify (signature + SLSA provenance), and install a connector artifact for this host's platform.
uninstallRemove an installed artifact and its manifest entry; refuses if a pipeline still references it, unless --force.
auditRe-verify every already-installed connector against the current signed registry index (catches a version yanked or publisher revoked after install).
bundlePackage a fully-verified artifact + its signatures/provenance + the signed index snapshot into a tarball for fully offline installation elsewhere.
conduit connectors install postgres@0.14.1
conduit connectors uninstall postgres --force
conduit connectors audit --json
conduit connectors bundle postgres@0.14.1 --os linux --arch amd64

See the connector registry guide (/docs/using/connectors/installing) for the full flag reference, trust model, and offline-bundle workflow for these four commands.


conduit connector-plugins​

Aliases: connector-plugin, connectorplugins, connectorplugin, connectorsplugins.

list (alias ls)​

Lists connector plugins available to add to a pipeline — built-in and standalone (installed under --connectors.path). Requires a running server.

FlagDescription
--name <substr>Filter by name (substring match).
conduit connector-plugins list

describe CONNECTOR_PLUGIN_ID (alias desc)​

Full spec of one connector plugin: summary, author, version, and every source and destination config parameter (type, default, validations).

conduit connector-plugins describe builtin:postgres@v0.10.1

conduit processors (alias: conduit processor)​

list (alias ls)​

Lists processor instances registered by a running Conduit.

conduit processors list

describe PROCESSOR_ID (alias desc)​

Detail of one processor instance: plugin, parent (pipeline or connector), condition, config settings, worker count.

conduit processors describe pipeline-processor

new [name]​

Scaffolds a full processor repository from ConduitIO/conduit-processor-template — SDK wiring, tests, CI, release workflow — ready to build with no edits. Same flags as connectors new (--module, --path, --git/--no-git, --force). Processors are Go only; --lang python is rejected.

conduit processors new uppercase --module github.com/you/conduit-processor-uppercase

conduit processor-plugins​

Aliases: processor-plugin, processorplugins, processorplugin, processorsplugins.

list (alias ls)​

Lists processor plugins available to add to a pipeline. Requires a running server.

FlagDescription
--name <substr>Filter by name (substring match).
conduit processor-plugins list

describe PROCESSOR_PLUGIN_ID (alias desc)​

Full spec of one processor plugin: summary, author, version, and every config parameter.

conduit processor-plugins describe builtin:base64.decode@v0.1.0

install <name>[@version], uninstall <name>[@version]​

Offline commands that install and remove standalone WASM processors under --processors.path. They never dial a running Conduit; restart conduit run to pick up a change.

CommandOne-line summary
installResolve the processor in the signed registry index, download the .wasm, verify its signature and SLSA provenance, check it loads, and place it under --processors.path. --dry-run, --index-file, --bundle, --allow-unsigned, --json.
uninstallRemove an installed processor and its manifest entry. Refuses if a pipeline under --pipelines.path still references it, unless --force; needs @version when several versions are installed.
conduit processor-plugins install ai.embed
conduit processor-plugins install ai.embed@0.1.0 --dry-run
conduit processor-plugins uninstall ai.embed --force

See Installing standalone processors for every flag, the offline paths, and the error codes.


JSON output​

Every command in this reference accepts --json. Read commands that talk to a running Conduit (pipelines list/describe/inspect, connectors list/ describe, processors list/describe, connector-plugins list/describe, processor-plugins list/describe) emit the same shape as the HTTP API. Newer (v0.17+) commands (validate, lint, dry-run, doctor, deploy, apply, repair, install/uninstall/audit/bundle, processor-plugins install/uninstall, connectors new/processors new, generate) emit the shared {command, ok, summary, result, error} envelope described above.

conduit pipelines list --json

--json makes the CLI scriptable and agent-legible: the same structured data, without parsing table output. Errors are structured too — see Structured errors & JSON output. Exit codes are deterministic, so a script can branch on the kind of failure without reading the error text — see Exit codes.

scarf pixel conduit-site-docs-using