Chapter 4: Argo Workflows Fundamentals

Knowledge Points

What is Argo Workflows

Argo Workflows is a Kubernetes-native workflow engine for orchestrating parallel tasks. Each step runs in an independent container, providing powerful dependency management, parallel execution, and error handling capabilities.

flowchart TB
    subgraph Traditional["Traditional CI/CD"]
        Agent[Build Agent]
        Agent --> Step1[Step 1]
        Step1 --> Step2[Step 2]
        Step2 --> Step3[Step 3]
        Note1[Shared environment<br/>Resource contention<br/>Poor isolation]
    end

    subgraph ArgoWorkflows["Argo Workflows"]
        Controller[Workflow Controller]
        Controller --> Pod1[Pod: Step 1]
        Controller --> Pod2[Pod: Step 2]
        Controller --> Pod3[Pod: Step 3]
        Note2[Container isolation<br/>Parallelizable<br/>Scalable]
    end

Architecture Components

flowchart TB
    subgraph ArgoWorkflows["Argo Workflows Components"]
        Controller[Workflow Controller<br/>Workflow Scheduling]
        Server[Argo Server<br/>API/UI]
        Executor[Executor<br/>Container Execution]
    end

    subgraph Storage["Storage"]
        PG[(PostgreSQL/MySQL)]
        S3[S3/MinIO/GCS<br/>Artifact Storage]
    end

    subgraph K8s["Kubernetes"]
        Pods[Workflow Pods]
        CRD[Workflow CRDs]
    end

    User[User] --> Server
    Server --> Controller
    Controller --> Pods
    Controller --> CRD
    Pods --> Executor
    Executor --> S3
    Server --> PG

Core Concepts

Installing Argo Workflows

Quick Installation

# Create namespace
kubectl create namespace argo

# Install Argo Workflows
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/latest/download/install.yaml

# Wait for Pods to be ready
kubectl wait --for=condition=Ready pods --all -n argo --timeout=300s

# Check installation status
kubectl get pods -n argo

Installation with Storage

# Install using Helm
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update

helm install argo-workflows argo/argo-workflows \
  --namespace argo \
  --create-namespace \
  --set server.extraArgs[0]="--auth-mode=server" \
  --set controller.workflowDefaults.spec.serviceAccountName=argo-workflow

# Advanced configuration values.yaml
controller:
  replicas: 1
  workflowDefaults:
    spec:
      serviceAccountName: argo-workflow
      ttlStrategy:
        secondsAfterCompletion: 3600
      podGC:
        strategy: OnPodCompletion

server:
  replicas: 1
  extraArgs:
    - --auth-mode=server
  ingress:
    enabled: true
    hosts:
      - argo.example.com

# Artifact storage configuration
artifactRepository:
  s3:
    endpoint: minio.argo:9000
    bucket: argo-artifacts
    insecure: true
    accessKeySecret:
      name: argo-artifacts
      key: accesskey
    secretKeySecret:
      name: argo-artifacts
      key: secretkey

Installing CLI

# macOS
brew install argo

# Linux
curl -sLO https://github.com/argoproj/argo-workflows/releases/latest/download/argo-linux-amd64.gz
gunzip argo-linux-amd64.gz
chmod +x argo-linux-amd64
sudo mv argo-linux-amd64 /usr/local/bin/argo

# Verify installation
argo version

# Configure access
# Port forwarding
kubectl -n argo port-forward svc/argo-server 2746:2746

# CLI configuration
argo config set server localhost:2746
argo config set insecure true

First Workflow

Hello World

# hello-world.yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
  namespace: argo
spec:
  entrypoint: hello
  templates:
  - name: hello
    container:
      image: busybox
      command: [echo]
      args: ["Hello, Argo Workflows!"]

# Submit workflow
argo submit -n argo hello-world.yaml

# List workflows
argo list -n argo

# Get workflow status
argo get -n argo @latest

# View logs
argo logs -n argo @latest

# Wait for workflow completion
argo wait -n argo @latest

# Delete workflow
argo delete -n argo @latest

Using UI

# Port forwarding
kubectl -n argo port-forward svc/argo-server 2746:2746

# Access in browser
# https://localhost:2746

Template Types

Container Template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: container-example-
spec:
  entrypoint: main
  templates:
  - name: main
    container:
      image: python:3.9
      command: [python, -c]
      args:
      - |
        import os
        print("Hello from Python!")
        print(f"Working directory: {os.getcwd()}")
      resources:
        requests:
          memory: "128Mi"
          cpu: "100m"
        limits:
          memory: "256Mi"
          cpu: "200m"
      env:
      - name: MY_VAR
        value: "my-value"

Script Template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: script-example-
spec:
  entrypoint: main
  templates:
  - name: main
    script:
      image: python:3.9
      command: [python]
      source: |
        import json
        import random

        result = {
            "random_number": random.randint(1, 100),
            "message": "Hello from script template"
        }
        print(json.dumps(result))

Resource Template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: resource-example-
spec:
  entrypoint: main
  templates:
  - name: main
    resource:
      action: create
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: my-configmap-{{workflow.name}}
        data:
          key: value

Suspend Template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: suspend-example-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: build
        template: build
    - - name: approve
        template: wait-for-approval
    - - name: deploy
        template: deploy

  - name: build
    container:
      image: busybox
      command: [echo, "Building..."]

  - name: wait-for-approval
    suspend:
      duration: "0"  # Wait indefinitely until manually resumed

  - name: deploy
    container:
      image: busybox
      command: [echo, "Deploying..."]

# Resume suspended workflow
argo resume -n argo <workflow-name>

Steps Type Workflows

Sequential Execution

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: steps-sequential-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    # First step
    - - name: step1
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Step 1"
    # Second step
    - - name: step2
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Step 2"
    # Third step
    - - name: step3
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Step 3"

  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: busybox
      command: [echo]
      args: ["{{inputs.parameters.message}}"]

flowchart LR
    S1[Step 1] --> S2[Step 2] --> S3[Step 3]

Parallel Execution

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: steps-parallel-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    # First step: sequential execution
    - - name: init
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Initializing"
    # Second step: parallel execution (in same - list)
    - - name: parallel-a
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Parallel A"
      - name: parallel-b
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Parallel B"
      - name: parallel-c
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Parallel C"
    # Third step: sequential execution
    - - name: finish
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Finished"

  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: busybox
      command: [echo]
      args: ["{{inputs.parameters.message}}"]

flowchart LR
    Init[Init] --> PA[Parallel A]
    Init --> PB[Parallel B]
    Init --> PC[Parallel C]
    PA --> Finish[Finish]
    PB --> Finish
    PC --> Finish

DAG Type Workflows

Basic DAG

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-example-
spec:
  entrypoint: main
  templates:
  - name: main
    dag:
      tasks:
      - name: task-a
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Task A"

      - name: task-b
        dependencies: [task-a]
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Task B"

      - name: task-c
        dependencies: [task-a]
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Task C"

      - name: task-d
        dependencies: [task-b, task-c]
        template: echo
        arguments:
          parameters:
          - name: message
            value: "Task D"

  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: busybox
      command: [echo]
      args: ["{{inputs.parameters.message}}"]

flowchart TB
    A[Task A] --> B[Task B]
    A --> C[Task C]
    B --> D[Task D]
    C --> D

Complex DAG Example

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: ci-pipeline-
spec:
  entrypoint: ci-pipeline
  templates:
  - name: ci-pipeline
    dag:
      tasks:
      # Checkout code
      - name: checkout
        template: git-clone

      # Run in parallel: linting and unit tests
      - name: lint
        dependencies: [checkout]
        template: run-lint

      - name: unit-test
        dependencies: [checkout]
        template: run-tests

      - name: security-scan
        dependencies: [checkout]
        template: run-security-scan

      # Build (depends on all checks passing)
      - name: build
        dependencies: [lint, unit-test, security-scan]
        template: docker-build

      # Push image
      - name: push
        dependencies: [build]
        template: docker-push

      # Integration tests
      - name: integration-test
        dependencies: [push]
        template: run-integration-tests

  - name: git-clone
    container:
      image: alpine/git
      command: [sh, -c]
      args: ["echo 'Cloning repository...'"]

  - name: run-lint
    container:
      image: node:16
      command: [sh, -c]
      args: ["echo 'Running linter...' && sleep 2"]

  - name: run-tests
    container:
      image: node:16
      command: [sh, -c]
      args: ["echo 'Running unit tests...' && sleep 3"]

  - name: run-security-scan
    container:
      image: aquasec/trivy
      command: [sh, -c]
      args: ["echo 'Running security scan...' && sleep 2"]

  - name: docker-build
    container:
      image: docker:dind
      command: [sh, -c]
      args: ["echo 'Building Docker image...' && sleep 5"]

  - name: docker-push
    container:
      image: docker:dind
      command: [sh, -c]
      args: ["echo 'Pushing to registry...' && sleep 2"]

  - name: run-integration-tests
    container:
      image: curlimages/curl
      command: [sh, -c]
      args: ["echo 'Running integration tests...' && sleep 3"]

flowchart TB
    Checkout[Checkout] --> Lint[Lint]
    Checkout --> Test[Unit Test]
    Checkout --> Security[Security Scan]
    Lint --> Build[Build]
    Test --> Build
    Security --> Build
    Build --> Push[Push Image]
    Push --> Integration[Integration Test]

Parameter Passing

Input Parameters

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: params-example-
spec:
  entrypoint: main
  # Workflow-level parameters
  arguments:
    parameters:
    - name: message
      value: "Hello from parameter"
    - name: count
      value: "3"

  templates:
  - name: main
    steps:
    - - name: print
        template: echo
        arguments:
          parameters:
          - name: msg
            value: "{{workflow.parameters.message}}"

  - name: echo
    inputs:
      parameters:
      - name: msg
    container:
      image: busybox
      command: [echo]
      args: ["{{inputs.parameters.msg}}"]

# Override parameters on submit
argo submit params-example.yaml -p message="Custom message" -p count="5"

Output Parameters

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: output-params-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: generate
        template: generate-output
    - - name: consume
        template: print-input
        arguments:
          parameters:
          - name: message
            value: "{{steps.generate.outputs.parameters.result}}"

  - name: generate-output
    container:
      image: busybox
      command: [sh, -c]
      args: ["echo 'Generated value: 42' > /tmp/result.txt"]
    outputs:
      parameters:
      - name: result
        valueFrom:
          path: /tmp/result.txt

  - name: print-input
    inputs:
      parameters:
      - name: message
    container:
      image: busybox
      command: [echo]
      args: ["Received: {{inputs.parameters.message}}"]

Parameter Passing in DAG

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-params-
spec:
  entrypoint: main
  templates:
  - name: main
    dag:
      tasks:
      - name: generate-a
        template: generate
        arguments:
          parameters:
          - name: prefix
            value: "A"

      - name: generate-b
        template: generate
        arguments:
          parameters:
          - name: prefix
            value: "B"

      - name: combine
        dependencies: [generate-a, generate-b]
        template: combine
        arguments:
          parameters:
          - name: value-a
            value: "{{tasks.generate-a.outputs.parameters.result}}"
          - name: value-b
            value: "{{tasks.generate-b.outputs.parameters.result}}"

  - name: generate
    inputs:
      parameters:
      - name: prefix
    container:
      image: busybox
      command: [sh, -c]
      args: ["echo '{{inputs.parameters.prefix}}-value' > /tmp/output"]
    outputs:
      parameters:
      - name: result
        valueFrom:
          path: /tmp/output

  - name: combine
    inputs:
      parameters:
      - name: value-a
      - name: value-b
    container:
      image: busybox
      command: [echo]
      args: ["Combined: {{inputs.parameters.value-a}} + {{inputs.parameters.value-b}}"]

Conditional Execution

When Conditions

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: conditional-
spec:
  entrypoint: main
  arguments:
    parameters:
    - name: environment
      value: "production"

  templates:
  - name: main
    steps:
    - - name: build
        template: build

    # Conditional execution: only in production environment
    - - name: deploy-prod
        template: deploy
        when: "{{workflow.parameters.environment}} == 'production'"
        arguments:
          parameters:
          - name: env
            value: "production"

      - name: deploy-staging
        template: deploy
        when: "{{workflow.parameters.environment}} == 'staging'"
        arguments:
          parameters:
          - name: env
            value: "staging"

  - name: build
    container:
      image: busybox
      command: [echo, "Building..."]

  - name: deploy
    inputs:
      parameters:
      - name: env
    container:
      image: busybox
      command: [echo]
      args: ["Deploying to {{inputs.parameters.env}}"]

Conditions in DAG

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-conditional-
spec:
  entrypoint: main
  arguments:
    parameters:
    - name: run-tests
      value: "true"

  templates:
  - name: main
    dag:
      tasks:
      - name: build
        template: build

      - name: test
        dependencies: [build]
        when: "{{workflow.parameters.run-tests}} == 'true'"
        template: test

      - name: deploy
        dependencies: [build, test]
        template: deploy

  - name: build
    container:
      image: busybox
      command: [echo, "Building..."]

  - name: test
    container:
      image: busybox
      command: [echo, "Testing..."]

  - name: deploy
    container:
      image: busybox
      command: [echo, "Deploying..."]

Conditional Expressions

# Supported conditional expressions
when: "{{steps.step1.outputs.result}} == 'success'"
when: "{{tasks.task1.outputs.result}} != 'failed'"
when: "{{workflow.parameters.enabled}} == 'true'"
when: "{{inputs.parameters.count}} > 10"
when: "'{{inputs.parameters.env}}' =~ 'prod.*'"  # Regex match

Loops

withItems Loop

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loop-items-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: process-items
        template: echo
        arguments:
          parameters:
          - name: message
            value: "{{item}}"
        withItems:
        - "item-1"
        - "item-2"
        - "item-3"

  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: busybox
      command: [echo]
      args: ["Processing: {{inputs.parameters.message}}"]

withParam Loop

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loop-param-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: generate-list
        template: generate

    - - name: process
        template: process-item
        arguments:
          parameters:
          - name: item
            value: "{{item}}"
        withParam: "{{steps.generate-list.outputs.result}}"

  - name: generate
    script:
      image: python:3.9
      command: [python]
      source: |
        import json
        items = ["server-1", "server-2", "server-3"]
        print(json.dumps(items))

  - name: process-item
    inputs:
      parameters:
      - name: item
    container:
      image: busybox
      command: [echo]
      args: ["Processing: {{inputs.parameters.item}}"]

Object Array Loop

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loop-objects-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: deploy
        template: deploy-app
        arguments:
          parameters:
          - name: name
            value: "{{item.name}}"
          - name: replicas
            value: "{{item.replicas}}"
        withItems:
        - { name: "frontend", replicas: "3" }
        - { name: "backend", replicas: "5" }
        - { name: "database", replicas: "1" }

  - name: deploy-app
    inputs:
      parameters:
      - name: name
      - name: replicas
    container:
      image: busybox
      command: [echo]
      args: ["Deploying {{inputs.parameters.name}} with {{inputs.parameters.replicas}} replicas"]

Practical Exercise

Complete CI Workflow

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: ci-workflow-
spec:
  entrypoint: ci-pipeline
  arguments:
    parameters:
    - name: repo
      value: "https://github.com/myorg/myapp.git"
    - name: branch
      value: "main"
    - name: image
      value: "myregistry.com/myapp"

  templates:
  - name: ci-pipeline
    dag:
      tasks:
      - name: clone
        template: git-clone
        arguments:
          parameters:
          - name: repo
            value: "{{workflow.parameters.repo}}"
          - name: branch
            value: "{{workflow.parameters.branch}}"

      - name: lint
        dependencies: [clone]
        template: run-command
        arguments:
          parameters:
          - name: cmd
            value: "npm run lint"

      - name: test
        dependencies: [clone]
        template: run-command
        arguments:
          parameters:
          - name: cmd
            value: "npm test"

      - name: build-image
        dependencies: [lint, test]
        template: build-docker
        arguments:
          parameters:
          - name: image
            value: "{{workflow.parameters.image}}"

      - name: push-image
        dependencies: [build-image]
        template: push-docker
        arguments:
          parameters:
          - name: image
            value: "{{workflow.parameters.image}}"

  - name: git-clone
    inputs:
      parameters:
      - name: repo
      - name: branch
    container:
      image: alpine/git
      command: [sh, -c]
      args:
      - |
        git clone --branch {{inputs.parameters.branch}} {{inputs.parameters.repo}} /workspace
        echo "Cloned successfully"
      volumeMounts:
      - name: workspace
        mountPath: /workspace

  - name: run-command
    inputs:
      parameters:
      - name: cmd
    container:
      image: node:18
      command: [sh, -c]
      args:
      - |
        cd /workspace
        {{inputs.parameters.cmd}}
      volumeMounts:
      - name: workspace
        mountPath: /workspace

  - name: build-docker
    inputs:
      parameters:
      - name: image
    container:
      image: docker:dind
      command: [sh, -c]
      args:
      - |
        cd /workspace
        docker build -t {{inputs.parameters.image}}:{{workflow.name}} .
      volumeMounts:
      - name: workspace
        mountPath: /workspace
      securityContext:
        privileged: true

  - name: push-docker
    inputs:
      parameters:
      - name: image
    container:
      image: docker:dind
      command: [sh, -c]
      args:
      - |
        docker push {{inputs.parameters.image}}:{{workflow.name}}
      securityContext:
        privileged: true

  volumes:
  - name: workspace
    emptyDir: {}

Summary

Through this chapter, you should have mastered:

• Core Concepts: Workflow, Template, DAG, Steps
• Template Types: Container, Script, Resource, Suspend
• Execution Modes: Sequential execution, parallel execution, DAG dependencies
• Parameter Passing: Input parameters, output parameters, inter-step passing
• Control Flow: Conditional execution, loop execution

In the next chapter, we will dive deeper into advanced features of Argo Workflows, including Artifact management, workflow templates, and error handling.