CLI
Use the Pipelines CLI for guided terminal workflows, artifacts, config swaps, and experiments.
Use the CLI when you want a terminal-first workflow: inspect libraries, create configs from JSON, import artifact bundles, launch experiments, wait for completion, and copy returned workflow links into the UI.
Environment Setup
Set these once per terminal:
export PIPELINES_API_KEY="pk_live_..."
export PROJECT_ID="22"
export WORKFLOW_ID="10"
export ORG_ID="2"The CLI autoloads a .env file from the current directory if present, so you can
keep PIPELINES_API_KEY and friends in .env instead of exporting them. Pass
--no-dotenv to skip that autoload (explicit environment variables and CLI flags
always win regardless).
Auth And API Keys
Already have an API key from the platform UI?
Just set it and you're done — no auth login required:
export PIPELINES_API_KEY="pk_live_..."
pipelines whoamiWant the CLI to remember your API key?
pipelines auth login --api-key "pk_live_..."Create the API key in the platform UI, then run pipelines auth login. If --api-key is omitted, the CLI prompts securely. From that point on, every subsequent pipelines … call uses the saved key automatically.
The saved config contains a usable API key, so don't commit it. In CI, set PIPELINES_API_KEY directly instead.
Platform admins can create, update, or revoke API keys in the Pipelines web app.
Finding IDs
The current CLI uses database IDs for orgs, projects, workflows, node rows, fields, and artifacts. Slug-based selection is planned as a follow-up, but today you can find IDs with list/detail commands:
pipelines organizations list
pipelines projects list
pipelines workflow list --project-id "$PROJECT_ID"
pipelines workflow configurable-fields --project-id "$PROJECT_ID" --workflow-id "$WORKFLOW_ID"nodeId is the workflow-builder node identifier used in config JSON. node_db_id / --node-db-id is the integer database row ID for a node created on a task after a workflow run. Use task details or tool-call history outputs after creating tasks to find node_db_id; it is not the same as the workflow config nodeId.
Five-Minute Prompt-Only Quickstart
This path works even when model or evaluation access is restricted.
- Confirm the CLI can authenticate:
pipelines prompts list- Check whether the workflow supports direct task creation:
pipelines workflow creation-mode \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID"If requires_csv is true, choose a simpler workflow for this quickstart or use CSV/dataset seeding.
- Find configurable fields:
pipelines workflow configurable-fields \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID"Copy one nodeId and fieldId from the output. nodeId is the workflow node. fieldId is the field inside that node.
- Create a prompt:
pipelines prompts template > prompt.jsonEdit prompt.json, then push it:
pipelines prompts create --file prompt.jsonCopy the returned id; this is your promptId.
- Create
experiment.json:
{
"count": 1,
"variants": [
{
"name": "baseline-copy",
"configOverrides": {}
},
{
"name": "prompt-swap",
"configOverrides": {
"prompts": [
{
"nodeId": "paste-node-id-here",
"fieldId": "paste-field-id-here",
"promptId": 12,
"version": 1
}
]
}
}
]
}- Run and wait:
pipelines workflow experiment create \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID" \
--file experiment.json \
--idempotency-key "prompt-demo-$(date +%s)" \
--wait \
--summaryThe summary includes copied workflow IDs and UI links when PIPELINES_APP_URL is set.
Full Config Swap Walkthrough
List available configs:
pipelines prompts list
pipelines criteria list
pipelines evaluations list
pipelines models list --org-id "$ORG_ID"If evaluations list or models list returns 403, your key is valid but lacks access. Use the prompt-only quickstart or ask an admin for access.
Dataset Export
Use dataset export when you want to download an imported dataset from Pipelines for local inspection, notebooks, or snapshots.
export DATASET_ID="paste-dataset-id"
pipelines datasets export \
--dataset-id "$DATASET_ID" \
--format csv \
--output dataset-export.csvSupported formats are csv, json, zip, and zip-json. Use ZIP formats when the dataset includes file columns.
If export returns 400, the dataset is probably workflow-backed rather than imported. Export that data from the workflow task export path instead.
Generate starter files:
pipelines prompts template > prompt.json
pipelines criteria template > criterion.json
pipelines evaluations template > evaluation.jsonpipelines criteria template gives one simple exact-match example. For real criteria, edit the JSON based on the criterion type you want:
Keyword match:
{
"name": "Mentions Rome Highlights",
"display_label": "Mentions Rome Highlights",
"type": "programmatic",
"config": {
"subtype": "contains_keywords",
"keywords": ["Colosseum", "Vatican", "Trastevere"],
"mode": "any",
"case_sensitive": false
},
"output_schema": {"type": "boolean"},
"scope_type": "org",
"org_id": 2
}Regex match:
{
"name": "Has Day Number",
"display_label": "Has Day Number",
"type": "programmatic",
"config": {
"subtype": "regex_match",
"pattern": "Day\\s+[1-5]",
"flags": ["IGNORECASE"]
},
"output_schema": {"type": "boolean"},
"scope_type": "org",
"org_id": 2
}LLM judge:
{
"name": "Itinerary Quality Judge",
"display_label": "Itinerary Quality",
"type": "llm_judge",
"config": {
"model": "gemini-3.5-flash",
"temperature": 0,
"prompt_template": "Score the itinerary from 1 to 5 for specificity, feasibility, and usefulness. Return only the number."
},
"output_schema": {"type": "rating", "range": [1, 5]},
"scope_type": "org",
"org_id": 2
}Human rating:
{
"name": "Human Quality Rating",
"display_label": "Rate itinerary quality",
"type": "human_rating",
"config": {
"rubric": "1 = poor, 3 = acceptable, 5 = excellent"
},
"output_schema": {
"type": "rating",
"range": [1, 5],
"labels": {"1": "Poor", "3": "Acceptable", "5": "Excellent"}
},
"scope_type": "org",
"org_id": 2
}JSON validity:
{
"name": "Valid JSON Itinerary",
"display_label": "Valid JSON Itinerary",
"type": "programmatic",
"config": {
"subtype": "json_validity",
"schema": {
"type": "object",
"required": ["days"],
"properties": {"days": {"type": "array"}}
}
},
"output_schema": {"type": "boolean"},
"scope_type": "org",
"org_id": 2
}Built-in LLM metric:
{
"name": "Answer Relevancy",
"display_label": "Answer Relevancy",
"type": "llm_metric",
"config": {
"subtype": "answer_relevancy",
"threshold": 0.7
},
"output_schema": {"type": "numeric", "range": [0, 1]},
"scope_type": "org",
"org_id": 2
}Create configs:
pipelines prompts create --file prompt.json
pipelines criteria create --file criterion.json
pipelines evaluations create --file evaluation.jsonAfter creating criteria, put returned IDs into evaluation.json:
{
"name": "Itinerary quality",
"description": "Scores itinerary quality",
"criteria": [
{"criterion_id": 7},
{"criterion_id": 8}
],
"scope_type": "org",
"org_id": 2
}If pipelines criteria create returns 422, check that the type matches the config: contains_keywords needs keywords, regex_match needs pattern, json_validity can include a JSON schema, and llm_judge needs prompt_template.
Prompt swap variant:
{
"name": "prompt-swap",
"configOverrides": {
"prompts": [
{
"nodeId": "node-id-from-configurable-fields",
"fieldId": "field-id-from-configurable-fields",
"promptId": 12,
"version": 1
}
]
}
}Model swap variant:
{
"name": "model-swap",
"configOverrides": {
"models": [
{
"nodeId": "node-id-from-configurable-fields",
"modelSlug": "gemini-3.5-flash"
}
]
}
}Evaluation swap variant:
{
"name": "eval-swap",
"configOverrides": {
"evaluations": [
{
"nodeId": "review-node-id",
"fieldId": "quality-field-id",
"evaluationId": 9,
"version": 1
}
]
}
}Run all variants together:
pipelines workflow experiment create \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID" \
--file experiment.json \
--idempotency-key "full-config-swap-$(date +%s)" \
--wait \
--summaryAn experiment means multiple copied workflows with different configs. The source workflow stays unchanged.
Tool Endpoint Walkthrough
Use this when a workflow already has, or should have, tool-calling fields.
- Create a tool endpoint JSON file:
pipelines tool-endpoints template > tool-endpoint.jsonEdit tool-endpoint.json. For a public MCP server, the minimum shape is:
{
"name": "Docs MCP endpoint",
"description": "Search internal docs from LLM fields",
"endpoint_type": "mcp_server",
"url": "https://tools.example.com/mcp",
"auth_type": "none",
"auth_config": {}
}Supported auth_type values:
none: no credentials.api_key: API key credentials inauth_config.bearer_token: bearer token credentials inauth_config.oauth2: OAuth-managed endpoint; create/connect through the platform flow.
Platform endpoints such as Code Tools and File Tools are created by the backend with auth_type: "none".
- Create and inspect the endpoint:
pipelines tool-endpoints create \
--org-id "$ORG_ID" \
--file tool-endpoint.json
export TOOL_ENDPOINT_ID="paste-created-endpoint-id"
pipelines tool-endpoints tools \
--org-id "$ORG_ID" \
--endpoint-id "$TOOL_ENDPOINT_ID"
pipelines tool-endpoints test \
--org-id "$ORG_ID" \
--endpoint-id "$TOOL_ENDPOINT_ID"If tools are missing or stale, refresh discovery:
pipelines tool-endpoints refresh-tools \
--org-id "$ORG_ID" \
--endpoint-id "$TOOL_ENDPOINT_ID"tool_count: 0 or an empty tool_names list means the endpoint has no discovered tools yet. Tool recommendations only rank the available_tools you provide in recommend-tools.json; they do not discover tools from the org automatically. Build that list from pipelines tool-endpoints tools or the tool_names field returned by pipelines tool-endpoints list.
- Find the workflow field to bind:
pipelines workflow configurable-fields \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID"Copy the nodeId and fieldId for the LLM field that should use tools.
- Create a binding:
pipelines workflow tool-bindings template > tool-binding.jsonEdit tool-binding.json:
{
"node_id": "paste-node-id",
"field_id": "paste-field-id",
"endpoint_id": "paste-tool-endpoint-id",
"selected_tools": ["search_docs"],
"max_tool_rounds": 3
}Push it:
pipelines workflow tool-bindings create \
--workflow-id "$WORKFLOW_ID" \
--file tool-binding.json- Add tool ground truth for later inspection:
pipelines workflow tool-ground-truths template > tool-ground-truth.jsonEdit tool-ground-truth.json:
{
"node_id": "paste-node-id",
"field_id": "paste-field-id",
"expected_tools": [{"name": "search_docs"}],
"excluded_tools": [{"name": "delete_record"}],
"ordering_matters": false,
"source": "manual",
"confidence": 1.0
}Push it:
pipelines workflow tool-ground-truths create \
--workflow-id "$WORKFLOW_ID" \
--file tool-ground-truth.json- After a run creates nodes, inspect and evaluate tool calls:
export NODE_DB_ID="paste-node-db-id-from-task-details"
pipelines workflow tool-call-history \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID" \
--node-db-id "$NODE_DB_ID"
pipelines workflow tool-ground-truths evaluate \
--workflow-id "$WORKFLOW_ID" \
--node-db-id "$NODE_DB_ID" \
--file tool-evaluation.jsonUse pipelines tools recommend --file recommend-tools.json when you want the API to rank candidate tools for a prompt before writing ground truth. The available_tools[].name values must be real tool names from an endpoint that exists in the org.
Auth note: some tool binding and ground-truth operations follow the backend's current user/session permission checks. If prompt commands work but these return 401 or 403, confirm the same user can manage the workflow/tool endpoint in the UI.
Artifact Bundles
Artifacts are portable JSON files you can version in git and import into Pipelines. Keep each bundle focused on one artifact family in normal use: a prompt bundle, a criteria bundle, an evaluation bundle, a tool endpoint bundle, a tool binding bundle, or a tool ground-truth bundle. Mixed bundles are supported for advanced migrations, but they are harder to review and are not the recommended default.
Create a bundle for the artifact family you want to import:
pipelines artifacts template --kind prompts > prompts-bundle.json
pipelines artifacts template --kind criteria > criteria-bundle.json
pipelines artifacts template --kind evaluations > evaluations-bundle.json
pipelines artifacts template --kind tool-endpoints > tool-endpoints-bundle.json
pipelines artifacts template --kind tool-bindings > tool-bindings-bundle.json
pipelines artifacts template --kind tool-ground-truths > tool-ground-truths-bundle.jsonEdit the bundle you need. A prompt bundle can contain multiple prompts and leaves every other section empty:
{
"artifact_type": "pipelines.artifact.bundle.v1",
"name": "rome-prompt-bundle",
"description": "Prompt bundle for Rome itinerary demos",
"prompts": [
{
"name": "Concise Rome Prompt",
"prompt_text": "Create a concise 5 day Rome itinerary.",
"role": "user",
"scope_type": "org",
"org_id": 2
},
{
"name": "Detailed Rome Prompt",
"prompt_text": "Create a detailed 5 day Rome itinerary with morning, afternoon, and evening plans.",
"role": "user",
"scope_type": "org",
"org_id": 2
}
],
"criteria": [],
"evaluations": [],
"experiments": [],
"tool_endpoints": [],
"tool_bindings": [],
"tool_ground_truths": []
}Import the bundle:
pipelines artifacts import --file prompts-bundle.json
pipelines artifacts import --file criteria-bundle.json
pipelines artifacts import --file evaluations-bundle.json
pipelines artifacts import --file tool-endpoints-bundle.jsonPush one artifact at a time:
pipelines artifacts push --kind prompt --file prompt.json
pipelines artifacts push --kind criterion --file criterion.json
pipelines artifacts push --kind evaluation --file evaluation.json
pipelines artifacts push --kind tool-endpoint --file tool-endpoint.json
pipelines artifacts push --kind tool-binding --file tool-binding-artifact.json
pipelines artifacts push --kind tool-ground-truth --file tool-ground-truth-artifact.jsonUse returned IDs in configOverrides. For example, a created prompt ID becomes promptId, and a created evaluation ID becomes evaluationId.
For criteria, put multiple criterion objects in the criteria array and leave prompts, evaluations, and experiments empty. For evaluations, put multiple evaluation objects in the evaluations array and reference existing criterion IDs. For tools, put multiple endpoint objects in tool_endpoints, multiple binding objects in tool_bindings, or multiple ground-truth objects in tool_ground_truths. Binding and ground-truth artifact items include workflow_id; endpoint artifact items include org_id.
Current limitation: artifact bundles create existing Pipelines objects. They do not persist arbitrary run-output files as first-class experiment artifacts yet.
Inspect Results
Check experiment status:
pipelines workflow experiment status \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID" \
--experiment-id "exp-123" \
--summaryOpen copied workflows:
https://platform.pipelines.tech/projects/22/pipelines/build/3087
https://platform.pipelines.tech/projects/22/pipelines/build/3088Use copied workflow_id values returned by the experiment. Do not inspect only the source workflow; variants run in copied workflows.
Cancellation
Cancel an experiment:
pipelines workflow experiment cancel \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID" \
--experiment-id "exp-123"Cancel async task creation:
pipelines workflow create-tasks-cancel \
--project-id "$PROJECT_ID" \
--workflow-id "$WORKFLOW_ID" \
--job-id "job-123"Cancellation is best effort. Jobs already completed remain completed, and tasks already created are not deleted.
Troubleshooting
pipelines: command not found
Why it happens:
pipelines-sdkis not installed in the active environment.- The install landed in a different virtualenv than the one on your
PATH.
How to fix:
pip install pipelines-sdk
pipelines prompts listMissing PIPELINES_API_KEY
Why it happens:
- The CLI defaults to hosted Pipelines, but it still needs an API key.
How to fix:
export PIPELINES_API_KEY="pk_live_..."401 Unauthorized
Why it happens:
- The key is missing, placeholder text, expired, revoked, or for the wrong environment.
How to fix:
- Use a real
pk_live_...key. - Use production keys for public SDK usage.
403 Forbidden
Why it happens:
- The key is valid, but the creating user cannot access the requested project, model, evaluation, or org library.
- Tool binding and ground-truth commands may require the backend's current user/session permissions.
How to fix:
- Use a workflow the user can open in the UI.
- Use prompt-only swaps if models/evaluations are restricted.
- Ask an admin to grant access.
- For tool commands, confirm the user can manage the workflow and org tool endpoint in the UI.
requires_csv: true
Why it happens:
- The workflow depends on CSV/dataset-provided fields, prompt/model columns, labels, or task grouping.
How to fix:
- Choose a workflow that supports direct task creation for SDK/CLI demos.
- Or use the CSV/dataset seeding flow instead of direct create-tasks.
Stuck processing
Why it happens:
- The job is queued.
- Model/tool execution is slow.
How to fix:
- Check status with
pipelines workflow experiment status --summary. - Increase
--timeout.
No Configurable Fields Found
Why it happens:
- The workflow may not have prompt/model/evaluation-backed fields.
- The workflow may not be the one you intended to test.
How to fix:
- Open the workflow in the UI and confirm it has an LLM or evaluation field.
- Try another workflow.
- Re-run
pipelines workflow configurable-fields.
409 Conflict
Why it happens:
- You reused an
--idempotency-keywith a different JSON payload.
How to fix:
- Use the exact same payload with the same key, or use a fresh key:
--idempotency-key "demo-$(date +%s)"Saved Config
The CLI persists base_url, api_key, email, and org_id at ~/.config/pipelines/config.json (mode 0600). pipelines auth login writes it for you; pipelines config lets you inspect or mutate it after the fact.
pipelines config show # api_key is masked
pipelines config path # prints the resolved config file path
pipelines config set base_url https://pipelines.example.com
pipelines config set email demo@example.com
pipelines config unset api_key
pipelines config clear # delete the config fileFor tests or alternate profiles, set PIPELINES_CONFIG_FILE to a different path before running the CLI.
Environments
Installed CLIs default to https://api.pipelines.tech. Pass --env prod to set it explicitly, or point at a self-hosted deployment with PIPELINES_BASE_URL (or a saved base_url):
pipelines --env prod projects listThe resolution order stays CLI flag > env var > saved config > production default, so saved configs and PIPELINES_BASE_URL continue to work.
Output Formats
Every command accepts --output {json,yaml,table} (alias -o):
pipelines projects list # JSON (default, indent=2)
pipelines projects list --output yaml # requires pyyaml; falls back to JSON
pipelines projects list -o table # ASCII table — handy for terminal demosyaml output requires the optional extra:
pip install 'pipelines-sdk[yaml]'Verbose / Debug Logging
--verbose enables the SDK's pipelines debug logger and prints request/response info to stderr:
pipelines --verbose projects list
PIPELINES_DEBUG=1 pipelines projects list # equivalentEvery request also sends a User-Agent: pipelines-sdk/<version> ... header that backend logs can filter on.
Identity, Logout, Version
pipelines whoami # GET /api/auth/me — prints the user behind the current API key
pipelines health # GET /api/health — API liveness probe
pipelines version # prints the SDK version
pipelines logout # clears saved CLI config
pipelines logout --keep-base-url # keep base_url/email/org_id, only remove api_keylogout clears the on-disk config entry. If you used PIPELINES_API_KEY directly in your shell or CI environment, unset that variable separately.
Shell Completion
# Bash
pipelines completion bash >> ~/.bashrc
# Zsh
pipelines completion zsh > ~/.zsh/completions/_pipelines # then ensure that dir is in fpath
# Fish
pipelines completion fish > ~/.config/fish/completions/pipelines.fishAfter re-sourcing your shell config, pipelines <TAB> lists subcommands and pipelines projects <TAB> lists actions.
Tasks Runtime
The CLI mirrors the human-task SDK surface via the existing pipelines workflow subcommand for node-level work, and adds top-level reads where it makes sense:
# Inspect a task or node (uses the per-node workflow path)
pipelines workflow tool-call-history --project-id 22 --workflow-id 10 --node-db-id 456For most of the tasks surface (submit_subtask, submit_review, admin overrides, get_my_work, etc.), use the Python SDK directly — those flows are more useful from a script than from a one-shot CLI invocation.
Notebooks
pipelines notebooks list --dataset-id 42
pipelines notebooks get --notebook-id 7
pipelines notebooks update --notebook-id 7 --file notebook.json
pipelines notebooks delete --notebook-id 7Notifications
pipelines notifications list
pipelines notifications unread-count
pipelines notifications mark-read --all
pipelines notifications mark-read --ids 101,102,103Use notifications unread-count for the badge/count endpoint; there is no
notifications count alias.
Organizations And Users
pipelines organizations list
pipelines organizations get --org-id 2
pipelines organizations users --org-id 2
pipelines users me # alias for whoami
pipelines users list
pipelines users get --user-id 42Per-Node LLM Generation
LLM generation in Pipelines is per-node, not free-form. The CLI matches:
pipelines generate fields --project-id 22 --workflow-id 10 --node-db-id 456
pipelines generate fields --project-id 22 --workflow-id 10 --node-db-id 456 --file overrides.json
pipelines generate regenerate --project-id 22 --workflow-id 10 --node-db-id 456
pipelines generate progress --project-id 22 --workflow-id 10 --node-db-id 456
pipelines generate error --project-id 22 --workflow-id 10 --node-db-id 456
pipelines generate history --project-id 22 --workflow-id 10 --node-db-id 456Agent Commands
Agent authoring and registration live under the pipelines odyssey group.
Install once:
pip install 'pipelines-sdk[odyssey]'Core commands:
pipelines odyssey initpipelines odyssey doctorpipelines odyssey devpipelines odyssey pushpipelines odyssey syncpipelines odyssey mcp introspectpipelines odyssey mcp doctor
CLI/SDK implentation docs:
- SDK wrapper + dispatch route contract: Agent SDK (Python)
- Local scaffold / preflight / tunnel workflow: Local development
- Registration decision and mode-specific setup: Runtime Setup Overview
Typed Error Exit Codes
The CLI exits non-zero on any PipelinesError:
| Error | When |
|---|---|
AuthenticationError (HTTP 401) | Key missing / invalid / wrong environment |
ForbiddenError (HTTP 403) | Key valid but lacks permission |
NotFoundError (HTTP 404) | Resource ID is wrong |
ConflictError (HTTP 409) | Idempotency-key reused with different payload |
ValidationError (HTTP 422) | Payload shape / nodeId / fieldId mismatch |
RateLimitError (HTTP 429) | Slow down; the SDK auto-retries with the Retry-After header on safe methods |
ServerError (HTTP 5xx) | Backend hiccup |
Use --verbose to see the failing request + response in the log stream.
What Else Is Available
Run pipelines --help to see every top-level subcommand, and
pipelines odyssey --help for the agent group. The full method-to-route catalog
is published with the SDK reference at platform.pipelines.tech/docs.
Every CLI path is validated against a real backend route by the SDK's OpenAPI
contract test before release.