PipelineStepConfig

Configuration for a single pipeline step.

Each step references a deployed WorkflowTemplate component and defines its parameters, dependencies, and artifact consumption.

nameName (string)required

Unique name for this step within the pipeline (e.g., 'ocr', 'chunk', 'embed')

Examples:

Example 1

Example: ocr

Example 2

Example: chunk

Example 3

Example: embed

Example 4

Example: store-qdrant

componentComponent (string)required

Component identifier (WorkflowTemplate name in Argo)

Examples:

Example 1

Example: mistral-ocr-processor-v1

Example 2

Example: markdown-chunker-v1

Example 3

Example: data-cluster-embedding-v1

templateTemplate (string)required

Template name to reference (usually same as component without version suffix)

Examples:

Example 1

Example: mistral-ocr-processor

Example 2

Example: markdown-chunker

Example 3

Example: data-cluster-embedding

parameters object

Component-specific parameters

property name*any

Component-specific parameters

Examples:

Example 1

Example: {"overlap":128,"size":1024}

Example 2

Example: {"batchsize":32,"concurrency":10}

dependsstring[]

List of step names this step depends on (for ordering)

Examples:

Example 1

Example: ["ocr"]

Example 2

Example: ["chunk","embed"]

workflow_parametersstring[]

Workflow-level parameters to inject (e.g., entry_id, dataset_id)

Examples:

Example 1

Example: ["entry_id"]

Example 2

Example: ["entry_id","dataset_id"]

Example 3

Example: []

inputs object[]

Explicit artifact inputs with source step and artifact name

Array [

nameName (string)required

Input artifact name expected by this step's template

Examples:

Example 1

Example: input-text

Example 2

Example: input-pdf

Example 3

Example: chunks

from_stepFrom Step (string)required

Name of the step that produces this artifact

Examples:

Example 1

Example: ocr

Example 2

Example: fetch-entry

Example 3

Example: chunk

from_artifactFrom Artifact (string)required

Output artifact name from the source step

Examples:

Example 1

Example: output-markdown

Example 2

Example: entry-file

Example 3

Example: chunks

]

outputsstring[]

Declared output artifact names (for documentation only, not validated)

Examples:

Example 1

Example: ["output-markdown","output-figures"]

Example 2

Example: ["chunks"]

Example 3

Example: ["embeddings"]

PipelineStepConfig

{
  "component": "markdown-chunker-v1",
  "depends": [
    "ocr"
  ],
  "inputs": [
    {
      "from_artifact": "output-markdown",
      "from_step": "ocr",
      "name": "input-text"
    }
  ],
  "name": "chunk",
  "outputs": [
    "chunks"
  ],
  "parameters": {
    "overlap": 128,
    "size": 1024
  },
  "template": "markdown-chunker",
  "workflow_parameters": []
}