In December 2020, security researchers discovered that the SolarWinds Orion build system had been compromised nine months earlier. Attackers injected malicious code into the legitimate Orion software build process — not into the source code repository, not through a vulnerability in the deployed product, but directly into the CI/CD pipeline that compiled, signed, and distributed the software.
The result: a cryptographically signed, legitimate-looking update was shipped to 18,000 customers including the US Treasury, State Department, and Pentagon. The build pipeline — trusted by everyone — was weaponised against everyone who trusted it.
The SolarWinds attack made supply chain security and CI/CD pipeline security unavoidable topics for every organisation that ships software. But the attack surface is broader than just the build system.
Every stage of the software delivery pipeline introduces security risk: source code with insecure patterns (SAST), open-source dependencies with known CVEs (SCA), deployed applications with exploitable endpoints (DAST), hardcoded credentials committed to repositories (secrets scanning), infrastructure defined as misconfigured Terraform (IaC scanning), and container images with unpatched base image vulnerabilities.
A mature DevSecOps programme addresses all of these gates, automates the checks, and integrates them into the developer workflow so security feedback arrives in seconds — not in a quarterly penetration test report.
This tutorial covers the complete DevSecOps pipeline from pre-commit hooks through production monitoring: every security gate, the tools that implement it, complete pipeline configurations for GitHub Actions and GitLab CI, metrics for measuring DevSecOps maturity, and a phased implementation roadmap that lets any team start generating value on day one.
- What is DevSecOps — shift-left security explained
- The secure CI/CD pipeline — all seven security gates
- SAST — Static Application Security Testing
- SCA — Software Composition Analysis (open-source vulnerabilities)
- Secrets scanning — blocking hardcoded credentials
- Container image scanning and signing
- Infrastructure as Code (IaC) security scanning
- DAST — Dynamic Application Security Testing
- Complete GitHub Actions security pipeline
- SBOM — Software Bill of Materials
- DevSecOps maturity model and metrics
- Frequently asked questions
DevSecOps extends the DevOps philosophy — breaking down silos between development and operations — to include security as a first-class concern at every stage of software delivery. The core principle is "shift left": move security checks as early as possible in the development lifecycle, where defects are cheapest to find and fix.
| Stage found | Relative cost to fix | Who finds it | Time to fix |
|---|---|---|---|
| Design / threat model | 1× (baseline) | Security architect + developer | Hours — fix the design |
| Pre-commit (IDE / hook) | 5× | Developer | Minutes — fix before commit |
| CI pipeline (SAST/SCA/secrets) | 10× | Automated gate | Hours — fix to unblock the PR |
| QA / staging (DAST) | 15× | Security team + DAST tool | Days — triage and fix |
| Pre-production pentest | 60× | Penetration tester | Weeks — fix + retest + release delay |
| Production (breach / bug bounty) | 100–300× | Attacker or researcher | Incident response + remediation + breach costs |
DevSecOps does not eliminate penetration testing or security architecture reviews — it adds automated gates that catch the vast majority of common vulnerabilities at the earliest possible stage, so that human security review focuses on logic flaws and business context that automation cannot assess.
Pre-commit hooks run on the developer's machine before each commit is accepted. They catch the lowest-effort, highest-value issues: hardcoded credentials, obvious insecure patterns, and syntax errors. Feedback is immediate — the developer fixes the issue in their IDE without opening a browser or leaving their workflow.
Tools: pre-commit framework + detect-secrets + gitleaks + Semgrep (custom rules). Lightweight checks only — hook runtime should be under 10 seconds or developers disable them.
SAST scans source code for patterns that indicate security vulnerabilities — SQL injection via string concatenation, XSS via unsanitised output, hardcoded credentials, use of deprecated cryptographic functions, insecure deserialization, path traversal, and hundreds of other vulnerability classes. It does not run the code; it reasons about its structure.
Tools: Semgrep (fast, rule-based, language-agnostic), Bandit (Python), ESLint security plugins (JavaScript/TypeScript), SpotBugs + FindSecBugs (Java), CodeQL (GitHub native, deep semantic analysis). Block on HIGH/CRITICAL; warn on MEDIUM.
Modern applications are 80–90% open-source code. SCA checks every direct and transitive dependency against vulnerability databases (NVD, GitHub Advisory, OSV) and flags packages with known CVEs. It also checks for licence compliance — GPL dependencies in a commercial product can create legal exposure. Unlike SAST, SCA does not read your code; it reads your dependency manifest (package.json, requirements.txt, pom.xml, go.sum).
Tools: Trivy (also does containers and IaC), Snyk (commercial, excellent developer UX with fix PRs), OWASP Dependency-Check, Grype, npm audit / pip-audit / Bundler-Audit (language-native, fast in CI). Block on CRITICAL CVEs with a fix available.
Hardcoded secrets in source code are one of the most consistently exploited misconfigurations — automated scanners crawl public GitHub for patterns like AWS access key prefixes (AKIA) and have bots that can compromise an account within four minutes of a key being pushed. Secrets scanning must run at both pre-commit (to catch before the commit) and in CI (to catch what pre-commit misses, and to scan git history for historical commits).
Tools: gitleaks (open-source, fast, comprehensive rule set), detect-secrets (Yelp, widely used), truffleHog (verifies if found credentials are active), GitHub Secret Scanning (native, free for public repos, push protection for private repos on Advanced Security). Block on any verified secret; warn on probable secrets.
Infrastructure as Code defines cloud resources, network configuration, and security controls in version-controlled files. IaC scanning finds misconfigurations before they are deployed — a public S3 bucket defined in Terraform, a security group with 0.0.0.0/0 ingress on port 22, or a Kubernetes deployment without resource limits. Finding and fixing these in a PR takes minutes; finding them after deployment via a cloud pentest takes weeks.
Tools: Checkov (Bridgecrew, 1000+ checks, supports Terraform/CF/Helm/K8s/ARM), tfsec (Terraform-focused, fast), KICS (multi-platform, Checkmarx), Terrascan, Trivy (config mode). Block on HIGH/CRITICAL; warn on MEDIUM.
After the container image is built, scan it for vulnerabilities in the OS packages and language dependencies installed in the image — separate from the SCA scan of the application's manifest, this catches vulnerabilities introduced by the base image itself. Then sign the image with Cosign so admission controllers can verify the image came from a trusted pipeline before allowing it into the cluster.
Tools: Trivy (image scan), Grype (Anchore), Docker Scout, Cosign (signing, Sigstore). Block deployment if CRITICAL CVEs exist with no active exception. Attach SBOM to the image signature for provenance tracking.
DAST deploys the application into a test environment and actively probes it — sending SQL injection payloads, XSS vectors, authentication bypass attempts, and OWASP Top 10 attack patterns. Unlike SAST (which reads code), DAST can find vulnerabilities that only manifest at runtime: authentication issues, business logic flaws, CORS misconfigurations, and insecure API endpoints. Typically runs in staging; a lightweight "baseline" scan can run per-PR.
Tools: OWASP ZAP (free, scriptable, Docker-friendly CI integration), Nuclei (template-based, very fast), Burp Suite Enterprise (commercial), StackHawk (API-focused DAST, CI-native). Block on confirmed HIGH/CRITICAL findings; triage MEDIUM manually.
| Language | Primary tool | Secondary | Key findings |
|---|---|---|---|
| Python | Bandit | Semgrep p/python | Use of exec/eval, subprocess shell=True, use of MD5/SHA1, pickle deserialization, hardcoded passwords |
| JavaScript / TypeScript | ESLint + eslint-plugin-security | Semgrep p/javascript | eval(), innerHTML assignment, prototype pollution, RegExp DoS, dangerous crypto |
| Java | SpotBugs + FindSecBugs | Semgrep p/java | SQL injection, XXE, SSRF, insecure deserialization (ObjectInputStream), weak crypto |
| Go | gosec | Semgrep p/go | Hardcoded credentials, SQL injection, file path traversal, unsafe use of math/rand |
| Ruby | Brakeman | Semgrep p/ruby | Rails-specific: mass assignment, unsafe redirects, XSS, SQL injection, command injection |
| C / C++ | Flawfinder | CodeQL | Buffer overflows, use of dangerous functions (gets, strcpy, sprintf), format string vulns |
| Any (deep) | CodeQL (GitHub) | — | Deep semantic analysis — tracks data flow across functions, finds complex multi-step vulns |
SCA scanning tells you what is vulnerable. Dependency update automation fixes it automatically. Both GitHub Dependabot and Renovate Bot open pull requests to update vulnerable or outdated dependencies — the PR includes the changelog, the CVE details, and passes your CI pipeline before it reaches a reviewer. This is the most effective way to keep open-source dependencies current: the update arrives as a PR with all context, approved and merged in minutes rather than tracked in a backlog for months.
A Software Bill of Materials (SBOM) is a complete inventory of every component in a software product — every library, package, and dependency, with its version and licence. SBOMs became mandatory for US federal software vendors under the 2021 Executive Order on Cybersecurity and are increasingly required by enterprise procurement contracts. In DevSecOps, SBOMs enable continuous vulnerability monitoring: when a new CVE is published, you can immediately query your SBOM inventory to determine which deployed applications are affected — rather than manually checking each application's dependencies.
- Security tested only at year-end pentest
- No automated security gates
- Secrets committed to git regularly
- CVEs discovered by attackers
- No SBOM or dependency tracking
- Secrets scanning in CI (non-blocking)
- npm audit / pip-audit on merge
- Basic SAST on main branch
- Manual CVE triage monthly
- Quarterly pentest
- Blocking SAST + SCA + secrets on every PR
- Container image scanning before push
- IaC scanning in pipeline
- Dependabot for auto-update PRs
- DAST on staging pre-release
- All gates blocking on HIGH+
- Image signing + admission policy
- SBOM generated and stored per release
- Falco runtime detection in prod
- Security metrics tracked (MTTR, escape rate)
- SBOM-driven CVE alerting across all services
- Custom SAST rules per business logic
- Automated MTTR SLA enforcement
- Threat modelling integrated in design
- Security champions in every team
| Metric | Definition | Target | Anti-pattern to avoid |
|---|---|---|---|
| Mean Time to Remediate (MTTR) | Average time from vulnerability discovery to fix deployed to production | CRITICAL: <24h · HIGH: <7d · MEDIUM: <30d | Closing tickets without fixing — "won't fix" abuse |
| Security debt ratio | Open HIGH/CRITICAL findings / total codebase size (findings per KLOC) | Trending toward zero; reduce 10% per quarter | Measuring absolute count (growing codebase inflates) |
| Pipeline gate escape rate | % of security findings discovered in production that were not caught by CI gates | <5% — most findings should be caught before prod | Measuring only CI gate findings (misses prod escapes) |
| Time to detect (TTD) | Time from vulnerability introduction to detection in CI | <1h for committed code (next pipeline run) | Using pentest as the detection mechanism (months) |
| SAST false positive rate | % of SAST findings that are confirmed false positives | <30% — high FP rates cause developers to ignore findings | Measuring suppressed findings (teams suppress to meet quota) |
| Dependency freshness | % of dependencies within N versions of current release | >85% within 2 major versions | Ignoring transitive (indirect) dependencies |
⚡ Start your DevSecOps pipeline — in priority order
- Add secrets scanning today — it takes 15 minutes and stops the most embarrassing breach type. Add gitleaks to your CI pipeline with three lines of GitHub Actions YAML (see Job 1 in Section 9). Enable GitHub Secret Scanning push protection on all repositories (Settings → Code security → Secret scanning → Push protection). Run a one-time historical scan with gitleaks detect --source . --log-opts=--all on all repositories — you may find previously committed credentials that need immediate rotation.
- Add Trivy SCA to every PR build — blocks CVEs with 4 lines of YAML. The GitHub Actions snippet in Section 4 adds Trivy filesystem scanning with SARIF upload to your GitHub Security tab. Start with --exit-code 0 (warn only) for the first two weeks while you triage existing findings and add .trivyignore suppressions for accepted risks, then flip to --exit-code 1 to make it blocking.
- Enable Semgrep SAST on pull requests — free for open-source, $0 for up to 1 developer. The Semgrep GitHub Action in Section 9 runs p/owasp-top-ten rules on every PR diff and posts findings directly in the PR as review comments. Start with warning mode, baseline the existing findings, then enable blocking on new findings only. This does not slow the developer — it runs in parallel with the build, and most PRs will have zero findings.
- Add Checkov IaC scanning to your Terraform PRs. If your team writes Terraform or Helm charts, Checkov catches public S3 buckets, open security groups, missing encryption, and 1000+ other misconfigurations before they reach a terraform apply. The GitHub Action in Section 7 uploads findings to GitHub Security. A typical Terraform directory passes its first Checkov scan with 5–20 findings that take an afternoon to review and suppress or fix.
- Extend to the full pipeline over 90 days. Secrets + SCA + SAST + IaC scanning is Level 2 maturity — covering the most common, most impactful findings automatically. From there, add container image scanning and Cosign signing (Section 6), DAST for your staging environment (Section 8), SBOM generation for every release (Section 10), and Falco runtime detection in your Kubernetes cluster. Kubernetes security → | Supply chain attacks → | Vulnerability management →
DevSecOps integrates security controls directly into the software development and delivery pipeline — at the IDE, commit, pull request, build, test, and deployment stages — rather than treating security as a separate gate at the end of the release cycle. Traditional security typically involves a penetration test or security review shortly before production release. DevSecOps catches the same vulnerabilities (and more) at the earliest possible stage, where fixes take minutes instead of weeks, and remediation does not require a release delay. The principle is "shift left" — move security checks earlier in the lifecycle. IBM research shows bugs fixed at the design stage cost 1× to remediate; the same bug found in production costs 30–300×. DevSecOps does not replace penetration testing — it complements it by ensuring automated tools handle the high-volume common vulnerabilities, leaving human security review to focus on logic flaws and business context that automation cannot assess.
SAST (Static Application Security Testing) analyses source code without executing it, looking for insecure patterns — SQL injection via string concatenation, hardcoded credentials, use of deprecated crypto libraries, path traversal. It runs at the PR stage, takes seconds to minutes, and catches issues before the code is built or deployed. It cannot find runtime issues. DAST (Dynamic Application Security Testing) runs against a deployed, running application — sending actual attack payloads and observing responses. It finds runtime vulnerabilities that SAST cannot: authentication bypass, CORS misconfigurations, insecure API endpoints, business logic flaws, and issues that only manifest when the full application stack is running. DAST runs in staging and is slower (minutes to hours for a full scan). A complete DevSecOps pipeline uses both: SAST gates at the PR level for fast feedback, DAST gates in staging before production deployment.
SCA audits every open-source dependency — direct and transitive — against known vulnerability databases to find packages with published CVEs. Modern applications are 80–90% open-source code, and 77% of production applications have at least one open-source vulnerability. SCA does not read your application code; it reads the dependency manifest (package.json, requirements.txt, pom.xml, go.sum) and checks each package against NVD, GitHub Advisory Database, and OSV. Tools like Trivy, Snyk, and Grype produce a list of vulnerable packages with CVE IDs, CVSS scores, affected version ranges, and fixed versions. SCA findings are generally easier to fix than SAST findings — update the dependency version — making automated fix PRs via Dependabot or Renovate highly effective. Block on CRITICAL CVEs that have a fix available; track the rest in a security backlog with SLA-based remediation deadlines.
False positives in SAST are inevitable — even the best tools produce 20–40% false positive rates on first deployment. The key is building a suppression workflow from day one rather than trying to achieve zero false positives. First, use inline suppression annotations (Semgrep: # nosemgrep: rule-id) with a mandatory comment explaining why the suppression is legitimate — this creates an auditable record and discourages blanket suppression. Second, establish a triage process where security team reviews suppression PRs, not developers self-approving. Third, set the blocking threshold high (ERROR/CRITICAL only) and put WARNING findings in a non-blocking advisory report — developers see them but are not blocked by them. Fourth, regularly review the suppressed findings list for drift — a suppression that was valid six months ago may no longer apply after a code refactor. Finally, choose a SAST tool that allows team-level configuration files (.semgrepignore, .bandit) so suppressions are version-controlled and auditable.
An SBOM (Software Bill of Materials) is a complete inventory of every software component in a product — every library, package, dependency, and operating system package — with version numbers, licences, and provenance information. It is the software equivalent of an ingredient list. SBOMs are required for US federal government software vendors (Executive Order 14028, 2021) and increasingly mandated by enterprise security procurement requirements (PCI DSS, NIST SP 800-218). In DevSecOps, SBOMs enable continuous vulnerability monitoring: when a new CVE is published (like Log4Shell), teams with SBOMs can query their entire inventory in seconds to identify every affected service and version — rather than spending days manually checking application dependencies. Tools: Syft (generates SBOMs for containers and filesystems), Cosign (attaches SBOMs to container image signatures), Dependency-Track (open-source SBOM management platform with CVE alerting). Generate SBOMs in SPDX or CycloneDX format — both are open standards with broad tooling support.
Developer resistance to DevSecOps is almost always driven by one of three things: slow pipelines, high false positive rates, or lack of actionable context in findings. Address each directly. For speed: run security scans in parallel with the build (not sequentially), use incremental scanning on PR diffs rather than full-repo scans, and set a target of under 3 minutes for all security gates combined. For false positives: start all new tools in warning (non-blocking) mode for 2–4 weeks, triage aggressively, configure suppressions, then enable blocking only when the signal-to-noise ratio is acceptable. For actionable context: configure tools to post PR review comments with the finding, the CWE, and a concrete remediation example — a developer who receives "SQL injection in user.py line 42: use parameterised queries, example: cursor.execute('%s', (user_input,))" fixes it immediately; a developer who receives "SEMGREP: python.lang.security.audit.sqli" files a ticket and moves on. Security champions — senior developers in each team who own security culture — are the most effective long-term solution to adoption resistance.