Static Application Security Testing (SAST) is a cornerstone of modern DevSecOps, but out-of-the-box rulesets rarely cover organisation-specific business logic, proprietary frameworks, or unique architectural patterns. This is where a semgrep custom rules tutorial becomes invaluable. Semgrep (Semantic Grep) is a fast, open-source static analysis engine that finds bugs and detects vulnerabilities using lightweight, readable pattern-matching rules.
Unlike AST-based SAST tools that require complex compiler knowledge to write rules, Semgrep allows developers and security engineers to write rules that look like the code they are trying to find. This tutorial will guide you through the anatomy of a Semgrep rule, basic and advanced pattern matching, taint analysis, testing methodologies, and CI/CD integration.
Every Semgrep rule is written in YAML. A rule file can contain a single rule or a list of rules under the rules: key. Understanding the core components is the first step in any semgrep custom rules tutorial.
| Key | Required | Description |
|---|---|---|
| id | Yes | Unique, snake_case identifier for the rule (e.g., no-unsafe-yaml-load). |
| languages | Yes | List of languages the rule applies to (e.g., python, java, go, javascript). |
| severity | Yes | Impact level: INFO, WARNING, ERROR, or INVENTORY. |
| message | Yes | Human-readable explanation shown to the developer when the rule triggers. Include remediation steps. |
| patterns | Yes | The core matching logic. Can use pattern, pattern-either, pattern-not, etc. |
| metadata | No | Optional tags for categorisation, OWASP mapping, CWE references, and confidence scoring. |
Semgrep’s superpower is that patterns look like code. If you want to find a specific function call, you write that function call in the pattern key.
The ellipsis (...) is a wildcard that matches zero or more arguments, statements, or function arguments. In the example above, eval(...) will match eval(user_input), eval("1+1"), or eval().
As your codebase grows, simple literal matching isn't enough. Semgrep provides powerful constructs for complex logic.
Metavariables match any expression and can be reused within the same rule to ensure consistency. A metavariable must start with an uppercase letter.
Here, $LOCK ensures that the object calling .release() is the exact same object that called .acquire().
Sometimes the code you want to find is nested deeply within other statements (e.g., inside an if block or a loop). The deep expression operator <... ...> tells Semgrep to ignore nesting levels.
Use pattern-either to match multiple variations of a vulnerability, and pattern-not to exclude safe patterns (reducing false positives).
Pattern matching finds code that looks dangerous. Taint analysis finds code that is dangerous by tracking untrusted data (sources) as it flows through the application to a dangerous function (sinks), unless it is cleaned (sanitised).
To enable this, set mode: taint in your rule.
In this example, Semgrep will only flag the rule if data from request.GET or request.POST flows into cursor.execute() without first passing through int() or escape().
Let’s build a production-ready rule to prevent unsafe YAML deserialization in Python, a common vector for Remote Code Execution (RCE).
A critical step in any semgrep custom rules tutorial is validation. Writing a rule is only half the battle; proving it works without generating false positives is the other half. Semgrep provides a built-in testing framework.
Create a directory structure with your rule and a target file:
In your test file (test_unsafe_yaml.py), use special comments to tell Semgrep what should and shouldn't trigger:
Run the test command:
Custom rules should be enforced automatically in your CI/CD pipeline. Below is a GitHub Actions workflow that runs your custom rules alongside the official Semgrep Registry rules.
- Start specific, then generalise: Write a rule that catches one specific instance of a bug in your codebase first. Then, expand it using metavariables and pattern-either to catch similar patterns.
- Aggressively use pattern-not: The difference between a useful rule and a noisy, ignored rule is the pattern-not clause. Actively hunt for false positives in your codebase and add exceptions for safe patterns.
- Write actionable messages: The message field should not just say "Bad code". It must explain why it is bad and how to fix it, ideally with a code snippet of the safe alternative.
- Namespace your IDs: Use a consistent naming convention like company-name.language.category.rule-name (e.g., hoc.python.security.unsafe-yaml-load) to avoid collisions with the public Semgrep Registry.
- Test against real code: Always run semgrep --test against a diverse set of real-world examples from your own repositories before deploying to CI/CD.
"Better" depends on the use case. Traditional SAST tools excel at deep, cross-file data flow analysis out of the box but are notoriously slow and generate high false-positive rates. Semgrep is significantly faster (often 10-30x), easier to write custom rules for, and integrates seamlessly into developer workflows (like pre-commit hooks). Many mature organisations use Semgrep for fast, custom, framework-specific checks in CI, while reserving heavier SAST tools for nightly, comprehensive scans.
Yes, but with caveats. Basic pattern matching is single-file. However, Semgrep's taint analysis (mode: taint) can track data flow across function boundaries within a single file. For true cross-file, interprocedural data flow analysis, Semgrep Pro (the paid tier) offers advanced reachability analysis that tracks sources to sinks across multiple files and repositories.
The simplest method is to store your .yaml rule files in a dedicated directory within your organisation's monorepo or a shared internal Git repository. You can then reference this directory in your CI/CD pipeline (e.g., semgrep --config ./internal-rules/). Alternatively, you can publish a private Semgrep App account and share rulesets via the Semgrep Registry.
Semgrep supports over 30 languages, including Python, JavaScript, TypeScript, Java, Go, Ruby, C++, C#, PHP, Rust, and Kubernetes YAML/Helm. The pattern matching syntax adapts to the AST of each specific language, meaning a metavariable like $X will correctly match a Java object, a Python dictionary, or a Go struct depending on the languages key defined in the rule.
