Semgrep rules, hands-on

A grep that reads code the way a compiler does

Semgrep is a static application security testing (SAST) tool: it reads source code without running it and reports the places that match rules you write. A rule is a few lines of YAML, and its patterns look like the code they find, so a pattern for a Python call is written as a Python call. It parses many languages, including the three on this page: Python, Terraform's HCL (HashiCorp Configuration Language) and the YAML that GitHub Actions workflows are written in. Application security and platform teams use it to turn "never write this in our code again" into a check that a pipeline enforces.

The rule language is shared by two engines, and the pieces around it carry different licences:

• The Semgrep engine, now called Semgrep Community Edition (it was Semgrep OSS until December 2024), is open source under the GNU Lesser General Public License version 2.1 (LGPL 2.1). It runs locally and needs no account.
• The rules Semgrep, Inc. maintains in its registry and its semgrep-rules repository are under the Semgrep Rules License v1.0, which since December 2024 allows use "only for your own internal business purposes" and does not allow distributing the rules or offering them to others as a service. Semgrep's paid products add analysis the Community Edition does not do, such as following tainted data from one function into another.
• Opengrep is a fork of the Semgrep 1.100.0 engine, created in December 2024 and backed by a group of application security companies. It reached 1.0 in March 2025, is also LGPL 2.1, and reads the same rule format.

Rules you write yourself are yours, under neither licence. Every rule on this page was written for it, so you can copy them into your own repositories.

The one idea that makes it click

A pattern is code with holes in it, and Semgrep compares it with the code's structure, never its text. For every language it has a parser for, Semgrep parses both the pattern and each file into a syntax tree and looks for places where the trees line up. (Step 7 meets the exception, a mode for plain text.) This pattern has two kinds of hole: ... stands for any arguments, and "..." for any string.

boto3.client(..., aws_secret_access_key="...", ...)

Run against this file, it matches the first two calls and neither of the last two:

import boto3

boto3.client("s3", aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")
boto3.client(
    'sqs',
    region_name="eu-west-1",
    aws_secret_access_key='csohFAKEsecretKEYforTHEsemgrepLAB0000000',
)
boto3.Session(aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")
note = 'boto3.client("s3", aws_secret_access_key="...")'

Quotes, line breaks and the order of keyword arguments are not structure, so the second call matches although a text search for the pattern would miss it. The last line holds the pattern's exact text and does not match, because it is a string, where a text search would report it. The Session call does the same job as the others and does not match either: it is a different call, so it is a different tree.

Every failure on this page comes from that one idea. Code that does the same job in a different shape is invisible to a pattern written for one shape, and a file Semgrep never parses is never compared with anything.

Five failures that print no error

Each of these ends with a clean exit and, in Semgrep's own summary, a green tick. Each is reproduced in the walk-through, at the step linked.

1. The pattern misses a variant. The code does the same thing in a shape the pattern does not describe: an extra argument, a different call, another way to write the same expression. (steps 2 and 3)
2. The exclusion excludes nothing. pattern-not removes a finding only when it matches that finding's exact code, so using it where pattern-not-inside was needed leaves every finding in place. (step 5)
3. The rule's language is not the file's language. A rule only reads files of the languages in its languages list. A shell-script rule pointed at workflow files scans zero files and reports zero findings. (step 7)
4. The taint source never matches. A taint rule reports nothing until its source pattern matches something, so a source written for code shaped differently from yours leaves a rule that can never fire. (step 8)
5. The directory is ignored. Without a .semgrepignore file, Semgrep skips tests/, build/, vendor/ and several other directories, so a key pasted into an integration test is never read. (step 9)

The defence is the same for all five: a test file holding at least one case the rule must fire on and one it must stay quiet on, and a look at how many files a scan read before believing its count of findings.

Walk-through

Every step adds a file to ~/semgrep-lab: rules and their test files in rules/, and a small project to scan in code/. Paste the blocks in order into one terminal window. The secret values are fake: each is 40 characters long, like the example secret key in AWS's documentation, and spells out that it is fake.

1. Install Semgrep and keep it local

brew install semgrep

Without Homebrew, python3 -m pip install semgrep installs the same tool. Opengrep ships as a single binary on its releases page; with it, type opengrep wherever this page types semgrep. Before running Semgrep at all, turn off the two ways it contacts its servers by default:

export SEMGREP_SEND_METRICS=off SEMGREP_ENABLE_VERSION_CHECK=0
semgrep --version
mkdir -p ~/semgrep-lab/rules ~/semgrep-lab/code
cd ~/semgrep-lab

This page used 1.183.0. Semgrep reports usage metrics when a scan pulls rules from its registry or when you are logged in, and it asks semgrep.dev whether a newer version exists, even when all you run is semgrep --version. The two settings turn both off until you close this terminal; --metrics=off and --disable-version-check do the same for a single command. Nothing on this page needs a login or the registry: every rule is a local file, and the page avoids --config auto, p/ rulesets and --validate, which downloads a rule pack from the registry. While this page was being checked, Semgrep's connections went through a proxy that logged them. Run from a new home directory with only metrics off, Semgrep connected to semgrep.dev; with both settings it made no connection at all. Opengrep, run the same way, made none, and has no metrics setting.

2. A first rule, and a test file to prove it

Someone pasted an AWS secret access key into a script that copies reports to Amazon S3. Write the rule from the line you found:

cat > rules/boto3-hardcoded-secret.yaml <<'EOF'
rules:
  - id: boto3-hardcoded-secret
    languages: [python]
    severity: HIGH
    message: >-
      A literal AWS secret access key is passed to boto3. Anyone who can
      read this file can use the key. Rotate it, then let boto3 find
      credentials itself (environment, profile or IAM role).
    metadata:
      cwe: "CWE-798: Use of Hard-coded Credentials"
      category: security
    pattern: boto3.client("s3", aws_secret_access_key="...")
EOF

That is a complete rule. id names it in every finding and every test. languages decides which files it reads. severity is LOW, MEDIUM, HIGH or CRITICAL (the older INFO, WARNING and ERROR still work). message is what a developer reads next to their code, so it says what is wrong and what to do instead. metadata is free-form and never changes what matches: a CWE (Common Weakness Enumeration) number, a link, the owning team. The pattern holds the matching itself.

Before running the rule on any real code, give it a test file. A file with the same name and the language's extension sits beside the rule, and a comment on the line above each case says what should happen there: ruleid: means the rule must fire on the next line, ok: means it must not.

cat > rules/boto3-hardcoded-secret.py <<'EOF'
import os

import boto3

# ruleid: boto3-hardcoded-secret
s3 = boto3.client("s3", aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")

# ruleid: boto3-hardcoded-secret
s3 = boto3.client("s3", region_name="eu-west-1", aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")

# ok: boto3-hardcoded-secret
s3 = boto3.client("s3", aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"])

# ok: boto3-hardcoded-secret
s3 = boto3.client("s3")
EOF
semgrep --test rules/

0/1: 1 unit tests did not pass:
--------------------------------------------------------------------------------
	✖ boto3-hardcoded-secret
	missed lines: [9], incorrect lines: []
	test file path: /Users/you/semgrep-lab/rules/boto3-hardcoded-secret.py


No tests for fixes found.

The rule failed its own test, which is the point of writing the test first. missed lines are ruleid: cases it did not fire on, and incorrect lines are lines it fired on that carry no ruleid: annotation, such as the ok: cases. Line 9 is the call with region_name: written without ..., a pattern's argument list has to match the call's argument list exactly, so one extra argument is enough to hide the key. Opengrep words the same failure differently and reports the same line.

cat > rules/boto3-hardcoded-secret.yaml <<'EOF'
rules:
  - id: boto3-hardcoded-secret
    languages: [python]
    severity: HIGH
    message: >-
      A literal AWS secret access key is passed to boto3. Anyone who can
      read this file can use the key. Rotate it, then let boto3 find
      credentials itself (environment, profile or IAM role).
    metadata:
      cwe: "CWE-798: Use of Hard-coded Credentials"
      category: security
    pattern: boto3.client(..., aws_secret_access_key="...", ...)
EOF
semgrep --test rules/

1/1: ✓ All tests passed
No tests for fixes found.

3. What counts as the same code

Add the shapes the same mistake takes in real code: a call split over lines with single quotes and the arguments in another order, a key held in a constant, and two other boto3 calls that take the same key.

cat >> rules/boto3-hardcoded-secret.py <<'EOF'

# ruleid: boto3-hardcoded-secret
sqs = boto3.client(
    'sqs',
    aws_secret_access_key='csohFAKEsecretKEYforTHEsemgrepLAB0000000',
    region_name="eu-west-1",
)

SECRET = "csohFAKEsecretKEYforTHEsemgrepLAB0000000"
# ruleid: boto3-hardcoded-secret
s3 = boto3.client("s3", aws_secret_access_key=SECRET)

# ruleid: boto3-hardcoded-secret
session = boto3.Session(aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")

# ruleid: boto3-hardcoded-secret
table = boto3.resource("dynamodb", aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")
EOF
semgrep --test rules/

0/1: 1 unit tests did not pass:
--------------------------------------------------------------------------------
	✖ boto3-hardcoded-secret
	missed lines: [32, 29], incorrect lines: []
	test file path: /Users/you/semgrep-lab/rules/boto3-hardcoded-secret.py


No tests for fixes found.

The split, reordered call matched without any change to the rule, and so did the constant: Semgrep knows that SECRET holds a string, because it is assigned one and never changed, and matches it as that string. This is called constant propagation. What it missed, lines 29 and 32, are boto3.Session and boto3.resource. They take the same key, but they are different calls, and a pattern only knows the call it names. pattern-either matches if any one of its patterns does:

cat > rules/boto3-hardcoded-secret.yaml <<'EOF'
rules:
  - id: boto3-hardcoded-secret
    languages: [python]
    severity: HIGH
    message: >-
      A literal AWS secret access key is passed to boto3. Anyone who can
      read this file can use the key. Rotate it, then let boto3 find
      credentials itself (environment, profile or IAM role).
    metadata:
      cwe: "CWE-798: Use of Hard-coded Credentials"
      category: security
    pattern-either:
      - pattern: boto3.client(..., aws_secret_access_key="...", ...)
      - pattern: boto3.resource(..., aws_secret_access_key="...", ...)
      - pattern: boto3.Session(..., aws_secret_access_key="...", ...)
EOF
semgrep --test rules/

1/1: ✓ All tests passed
No tests for fixes found.

The test file now records every shape this rule has been taught. When someone finds another, it becomes a ruleid: line first and a pattern second.

4. Metavariables, metavariable-regex and pattern-not

A rule that fires on test fixtures stops being read. Tests that run against moto, a library that fakes AWS, use "testing" as the key, as moto's own documentation does, and documentation copies the example key AWS publishes. Add both as cases that must stay quiet:

cat >> rules/boto3-hardcoded-secret.py <<'EOF'

# ok: boto3-hardcoded-secret
mock = boto3.client("s3", aws_secret_access_key="testing")

# ok: boto3-hardcoded-secret
docs = boto3.client("s3", aws_secret_access_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY")
EOF
semgrep --test rules/

0/1: 1 unit tests did not pass:
--------------------------------------------------------------------------------
	✖ boto3-hardcoded-secret
	missed lines: [], incorrect lines: [35, 38]
	test file path: /Users/you/semgrep-lab/rules/boto3-hardcoded-secret.py


No tests for fixes found.

To say anything about the key's value, the rule has to capture it. A metavariable is a capital name after a $, such as $KEY: it matches any single expression, where ... matches any number of them, and it remembers what it matched, so later parts of the rule can test it and the message can quote it. Written inside quotes, "$KEY" matches any string, as "..." does, and captures the string's contents. patterns combines everything under it with AND:

cat > rules/boto3-hardcoded-secret.yaml <<'EOF'
rules:
  - id: boto3-hardcoded-secret
    languages: [python]
    severity: HIGH
    message: >-
      A literal AWS secret access key is passed to boto3. Anyone who can
      read this file can use the key. Rotate it, then let boto3 find
      credentials itself (environment, profile or IAM role).
    metadata:
      cwe: "CWE-798: Use of Hard-coded Credentials"
      category: security
    patterns:
      - pattern-either:
          - pattern: boto3.client(..., aws_secret_access_key="$KEY", ...)
          - pattern: boto3.resource(..., aws_secret_access_key="$KEY", ...)
          - pattern: boto3.Session(..., aws_secret_access_key="$KEY", ...)
      - metavariable-regex:
          metavariable: $KEY
          regex: ^[A-Za-z0-9/+]{40}$
      - pattern-not: boto3.$CALL(..., aws_secret_access_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", ...)
EOF
semgrep --test rules/

1/1: ✓ All tests passed
No tests for fixes found.

metavariable-regex keeps a finding only if the captured text matches the regular expression: 40 letters, digits, slashes and plus signs, the shape of the example key AWS publishes, which "testing" is not. It does not reject AWS's example key, which has that shape, so pattern-not removes the one call that passes it. pattern-not removes a finding when its pattern matches the same code as the finding, and here both are the whole call, which is why it works. Step 5 is what happens when they are not the same code.

Two details decide whether this matches anything at all. Without the quotes, $KEY captures the whole string literal including its quote marks, and this regular expression would never match. And the regular expression is anchored at the start of the captured text, so it tests whether the text begins with a match; it does not search inside it.

5. Terraform: pattern-inside, and the exclusion that excludes nothing

An IAM (Identity and Access Management) policy document whose statement names "*" as the principal grants its actions to anyone, unless a condition narrows it down, such as requiring the caller to belong to your organisation in AWS Organizations. Here are three statements: one open to anyone, one open to any principal in one organisation, and one granted to a single role.

cat > rules/iam-wildcard-principal.tf <<'EOF'
data "aws_iam_policy_document" "public_read" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.site.arn}/*"]
    # ruleid: iam-wildcard-principal
    principals {
      type        = "*"
      identifiers = ["*"]
    }
  }
}

data "aws_iam_policy_document" "org_read" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.site.arn}/*"]
    # ok: iam-wildcard-principal
    principals {
      type        = "*"
      identifiers = ["*"]
    }
    condition {
      test     = "StringEquals"
      variable = "aws:PrincipalOrgID"
      values   = ["o-a1b2c3d4e5"]
    }
  }
}

data "aws_iam_policy_document" "role_read" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.site.arn}/*"]
    # ok: iam-wildcard-principal
    principals {
      type        = "AWS"
      identifiers = ["arn:aws:iam::111122223333:role/reader"]
    }
  }
}
EOF

The first draft of the rule matches a principals block that lists "*", and tries to drop it when a condition block is present:

cat > rules/iam-wildcard-principal.yaml <<'EOF'
rules:
  - id: iam-wildcard-principal
    languages: [terraform]
    severity: HIGH
    message: >-
      This statement grants its actions to any principal, with no
      condition to narrow it. Name the principals, or add a condition
      such as aws:PrincipalOrgID.
    patterns:
      - pattern: |
          principals {
            ...
            identifiers = [..., "*", ...]
            ...
          }
      - pattern-not: |
          condition {
            ...
          }
EOF
semgrep --test rules/

1/2: 1 unit tests did not pass:
--------------------------------------------------------------------------------
	✖ iam-wildcard-principal
	missed lines: [], incorrect lines: [18]
	test file path: /Users/you/semgrep-lab/rules/iam-wildcard-principal.tf


No tests for fixes found.

Line 18 is the organisation-limited statement, and the rule still fires on it. The pattern-not matched the condition block, which is a different piece of code from the principals block the finding covers, so it removed nothing. Semgrep gives no warning for an exclusion that never applies. What the rule needs is a test of where the finding sits: pattern-inside keeps findings that lie inside a match of its pattern, and pattern-not-inside drops them.

cat > rules/iam-wildcard-principal.yaml <<'EOF'
rules:
  - id: iam-wildcard-principal
    languages: [terraform]
    severity: HIGH
    message: >-
      This statement grants its actions to any principal, with no
      condition to narrow it. Name the principals, or add a condition
      such as aws:PrincipalOrgID.
    patterns:
      - pattern-inside: |
          statement {
            ...
          }
      - pattern: |
          principals {
            ...
            identifiers = [..., "*", ...]
            ...
          }
      - pattern-not-inside: |
          statement {
            ...
            condition {
              ...
            }
            ...
          }
EOF
semgrep --test rules/

2/2: ✓ All tests passed
No tests for fixes found.

The ... before and after the condition block let it sit anywhere in the statement, before or after principals. In HCL, ... inside a block stands for any number of attributes and nested blocks.

6. Autofix, tested like everything else

An aws_s3_bucket_public_access_block with a setting left at false turns that protection off for the bucket, leaving only account or organisation settings, if there are any, to stop that kind of public access. The fix is mechanical, so the rule can carry it. fix: is the replacement text, and it can use metavariables:

cat > rules/s3-public-access-block-disabled.yaml <<'EOF'
rules:
  - id: s3-public-access-block-disabled
    languages: [terraform]
    severity: HIGH
    message: >-
      $SETTING is false, so this bucket's public access block does not
      stop that kind of public access. Set it to true.
    patterns:
      - pattern-inside: |
          resource "aws_s3_bucket_public_access_block" $NAME {
            ...
          }
      - pattern: $SETTING = false
      - metavariable-regex:
          metavariable: $SETTING
          regex: ^(block_public_acls|block_public_policy|ignore_public_acls|restrict_public_buckets)$
    fix: $SETTING = true
EOF

A fix gets a test too: a file named like the test file with .fixed before the extension, holding what the test file should look like once every fix is applied. This one is written by hand. Semgrep's documentation suggests copying the test file and running the fix over the copy; if you do, read the result line by line before saving it, because a .fixed file nobody checked only proves the fix does the same thing twice.

cat > rules/s3-public-access-block-disabled.tf <<'EOF'
resource "aws_s3_bucket_public_access_block" "site" {
  bucket = aws_s3_bucket.site.id
  # ruleid: s3-public-access-block-disabled
  block_public_acls = false
  # ruleid: s3-public-access-block-disabled
  block_public_policy = false
  # ok: s3-public-access-block-disabled
  ignore_public_acls = true
  # ok: s3-public-access-block-disabled
  restrict_public_buckets = true
}

locals {
  # ok: s3-public-access-block-disabled
  block_public_acls = false
}
EOF
cat > rules/s3-public-access-block-disabled.fixed.tf <<'EOF'
resource "aws_s3_bucket_public_access_block" "site" {
  bucket = aws_s3_bucket.site.id
  # ruleid: s3-public-access-block-disabled
  block_public_acls = true
  # ruleid: s3-public-access-block-disabled
  block_public_policy = true
  # ok: s3-public-access-block-disabled
  ignore_public_acls = true
  # ok: s3-public-access-block-disabled
  restrict_public_buckets = true
}

locals {
  # ok: s3-public-access-block-disabled
  block_public_acls = false
}
EOF
semgrep --test rules/

3/3: ✓ All tests passed
1/1: ✓ All fix tests passed

The locals block is the pattern-inside check: the same attribute outside the resource is left alone. To apply fixes to real files, add --autofix to a scan, or --autofix --dryrun to print them without writing. A fix replaces exactly the matched text, so block_public_acls = false lined up with its neighbours becomes block_public_acls = true; run terraform fmt afterwards.

7. GitHub Actions: metavariable-pattern, and the wrong language

A run: step that contains ${{ github.event.issue.title }} does not pass the title to the shell as data. GitHub pastes the title into the script before the shell starts, so an issue titled "; id; echo " makes the step run id, and whatever else an attacker types there runs the same way, with the job's permissions. The GitHub Actions page covers the risk; here is the rule. Start with a workflow to scan:

mkdir -p code/.github/workflows
cat > code/.github/workflows/triage.yml <<'EOF'
name: triage
on:
  issues:
    types: [opened]
jobs:
  greet:
    runs-on: ubuntu-latest
    steps:
      - name: Echo the title
        run: echo "New issue ${{ github.event.issue.title }}"
      - name: Check out the PR branch
        run: |
          git fetch origin
          git checkout ${{ github.head_ref }}
      - name: Through the environment
        env:
          TITLE: ${{ github.event.issue.title }}
        run: echo "New issue $TITLE"
EOF

The dangerous text is shell, so the tempting first rule is a shell rule. -e runs a single pattern from the command line, and -l names its language:

semgrep scan -l bash -e 'git checkout $REF' code/

The summary ends:

Ran 1 rule on 0 files: 0 findings.

Zero findings, from zero files, with a green tick above it. A rule reads only files Semgrep recognises as one of its languages, by extension or, for a script without one, by its #! line. .yml is YAML, so a Bash rule never opens a workflow file. Semgrep does not warn that a rule had nothing to read. Switch to -l yaml and the file is read, but the pattern still finds nothing: to Semgrep, the script is one YAML string, the value of a run key, and git checkout $REF is not the whole of that string.

metavariable-pattern takes the text a metavariable captured and runs another pattern over it, in a language you choose. generic is Semgrep's mode for text it has no parser for: it splits the text into words and punctuation, so spacing within a line does not matter, and in it ... stands for any run of text up to ten lines long.

cat > rules/gha-run-event-interpolation.yaml <<'EOF'
rules:
  - id: gha-run-event-interpolation
    languages: [yaml]
    severity: HIGH
    message: >-
      This run step pastes event data into the script before the shell
      starts, so whoever wrote that data (an issue title, a branch name)
      can run commands in this job. Pass it in through env: and use
      the variable instead.
    metadata:
      cwe: "CWE-78: OS Command Injection"
    patterns:
      - pattern-inside: "steps: [...]"
      - pattern: "run: $SCRIPT"
      - metavariable-pattern:
          metavariable: $SCRIPT
          language: generic
          pattern-either:
            - pattern: "${{ ... github.event ... }}"
            - pattern: "${{ ... github.head_ref ... }}"
EOF
cat > rules/gha-run-event-interpolation.test.yaml <<'EOF'
on: pull_request_target
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # ruleid: gha-run-event-interpolation
      - run: echo "${{ github.event.pull_request.title }}"
      # ruleid: gha-run-event-interpolation
      - run: |
          echo "checking out"
          git checkout ${{github.head_ref}}
      # ruleid: gha-run-event-interpolation
      - run: echo '${{ toJSON(github.event.pull_request.labels) }}'
      - env:
          TITLE: ${{ github.event.pull_request.title }}
        # ok: gha-run-event-interpolation
        run: echo "$TITLE"
      # ok: gha-run-event-interpolation
      - run: echo "${{ github.sha }}"
EOF
semgrep --test rules/

4/4: ✓ All tests passed
1/1: ✓ All fix tests passed

A YAML rule's test file ends in .test.yaml, so that Semgrep can tell it from the rule. The env: case is quiet because the event data sits under env, where GitHub hands it to the shell as a variable, and the run script only names the variable. github.sha is set by GitHub, not by whoever opened the pull request. The rule also flags event fields that cannot carry a command, such as an issue number; moving those into env: costs nothing, so it does not try to tell them apart. Now scan the workflow:

semgrep scan --config rules/gha-run-event-interpolation.yaml code/

It reports two findings, on lines 10 and 12 to 14 of triage.yml, and the summary line now says it read one file:

Ran 1 rule on 1 file: 2 findings.

8. Taint mode, and a source that never matches

Some mistakes are about where data goes rather than what a line says. A Lambda function behind Amazon API Gateway reads a query parameter and builds a shell command from it. The mistake is a path rather than a line: data from the event reaching a command that a shell will interpret. A taint rule describes that path in three parts: sources, where untrusted data enters; sinks, where it must not arrive; and sanitizers, which make it safe on the way. Semgrep reports a finding when data from a source reaches a sink without passing through a sanitizer. In the Community Edition, it follows the data within one function.

cat > rules/lambda-event-to-shell.py <<'EOF'
import shlex
import subprocess


def handler(event, context):
    params = event.get("queryStringParameters") or {}
    prefix = params.get("prefix", "")
    # ruleid: lambda-event-to-shell
    subprocess.run("aws s3 ls s3://reports/" + prefix, shell=True)
    # ok: lambda-event-to-shell
    subprocess.run("aws s3 ls s3://reports/" + shlex.quote(prefix), shell=True)
    # ok: lambda-event-to-shell
    subprocess.run(["aws", "s3", "ls", "s3://reports/" + prefix])
    # ok: lambda-event-to-shell
    subprocess.run("ls -l", shell=True, cwd="/tmp/" + prefix)
EOF
cat > rules/lambda-event-to-shell.yaml <<'EOF'
rules:
  - id: lambda-event-to-shell
    mode: taint
    languages: [python]
    severity: HIGH
    message: >-
      Data from the Lambda event reaches a shell command. Pass a list of
      arguments without shell=True, or quote the value with shlex.quote().
    pattern-sources:
      - pattern: event["queryStringParameters"][...]
    pattern-sinks:
      - patterns:
          - pattern: subprocess.run($CMD, ..., shell=True, ...)
          - focus-metavariable: $CMD
    pattern-sanitizers:
      - pattern: shlex.quote(...)
EOF
semgrep --test rules/

4/5: 1 unit tests did not pass:
--------------------------------------------------------------------------------
	✖ lambda-event-to-shell
	missed lines: [9], incorrect lines: []
	test file path: /Users/you/semgrep-lab/rules/lambda-event-to-shell.py


1/1: ✓ All fix tests passed

Three of the four cases passed, and they would pass for a rule that matches nothing at all, which on this file is what this one does. The source describes event["queryStringParameters"]["prefix"], the way the first handler someone looked at read its input. This handler reads it with .get(), so nothing in the file is a source, nothing is tainted, and the rule has nothing to follow. A scan of a whole codebase written this way reports nothing, which is also what a clean codebase reports. Only the ruleid: line noticed.

The fix is to make the source the thing every handler has in common, its event parameter, so that everything read from it is tainted however it is read:

cat > rules/lambda-event-to-shell.yaml <<'EOF'
rules:
  - id: lambda-event-to-shell
    mode: taint
    languages: [python]
    severity: HIGH
    message: >-
      Data from the Lambda event reaches a shell command. Pass a list of
      arguments without shell=True, or quote the value with shlex.quote().
    pattern-sources:
      - patterns:
          - pattern-inside: |
              def $HANDLER(event, context):
                ...
          - pattern: event
    pattern-sinks:
      - patterns:
          - pattern: subprocess.run($CMD, ..., shell=True, ...)
          - focus-metavariable: $CMD
    pattern-sanitizers:
      - pattern: shlex.quote(...)
EOF
semgrep --test rules/

5/5: ✓ All tests passed
1/1: ✓ All fix tests passed

focus-metavariable narrows the sink to the first argument, the command string. Without it the whole call is the sink, and the last case fires: a tainted cwd is a problem of its own, but not shell injection. The list form passes the prefix as a single argument with no shell to interpret it, and shlex.quote() is the sanitizer.

9. Scan a project, and see what was skipped

Now run the rules against a project. Give it a script with a key, using Session this time, and an integration test where someone pasted the staging key:

mkdir -p code/app code/tests/integration
cat > code/app/sync.py <<'EOF'
import boto3

session = boto3.Session(region_name="eu-west-1", aws_secret_access_key="csohFAKEsecretKEYforTHEsemgrepLAB0000000")
EOF
cat > code/tests/integration/test_sync.py <<'EOF'
import boto3


def test_sync_against_staging():
    s3 = boto3.client("s3", aws_secret_access_key="csohFAKEsecretKEYforTHEstagingBUCKET0000")
    assert s3.list_objects_v2(Bucket="reports-staging")
EOF
semgrep scan --config rules/ code/

The summary, before the findings themselves:

 • Findings: 3 (3 blocking)
 • Rules run: 3
 • Targets scanned: 2
 • Parsed lines: ~100.0%
 • Scan skipped:
   ◦ Files matching .semgrepignore patterns: 1
 • For a detailed list of skipped files and lines, run semgrep with the --verbose flag
Ran 3 rules on 2 files: 3 findings.

Three findings: sync.py and the two workflow steps. Two numbers deserve a second look. Three of the five rules ran, because the two Terraform rules had no .tf file to read. And the integration test is not among the findings, for the reason in the line that is easiest to read past: Files matching .semgrepignore patterns: 1.

There is no .semgrepignore in this project. When there is none, Semgrep uses a built-in list that skips, among others, version-control directories, build/, dist/, vendor/, node_modules/, .venv/, test/, tests/, testsuite/ and files ending _test.go. Inside a git repository it also skips untracked files that .gitignore excludes, and it skips any file over 1,000,000 bytes. --verbose lists every skipped path:

semgrep scan --config rules/ code/ --verbose 2>&1 | grep -A3 'Skipped by .semgrepignore'

  Skipped by .semgrepignore:
  - https://semgrep.dev/docs/ignoring-files-folders-code/#understand-semgrep-defaults

   • code/tests/integration

Skipping tests is a reasonable default for most code rules and the wrong one for credentials. A .semgrepignore file replaces the built-in list rather than adding to it, so a file naming only what you mean to skip brings tests/ back:

cd ~/semgrep-lab/code
printf 'node_modules/\n.venv/\n' > .semgrepignore
semgrep scan --config ../rules/ .
cd ~/semgrep-lab

Ran 3 rules on 3 files: 4 findings.

Run scans from the project's root directory. Semgrep 1.183.0 also found code/.semgrepignore when run from ~/semgrep-lab on code/; Opengrep 1.33.1 did not, and fell back to the built-in list. Naming a single file on the command line scans it even when an ignore pattern covers it.

A scan exits 0 even when it has findings. In a pipeline, add --error, which makes any finding exit 1:

cd ~/semgrep-lab/code
semgrep scan --error --config ../rules/ . > /dev/null 2>&1
echo $?
cd ~/semgrep-lab

1

Run semgrep --test rules/ in the same pipeline, before the scan, so that a rule that stops matching fails the build instead of quietly passing it.

Hands-on exercises

          pattern-either:
            - pattern: "${{ github.event. ... }}"
            - pattern: "${{ github.head_ref }}"

      - run: echo "${{ github.event['issue']['title'] }}"
      - run: echo '${{ toJSON(github.event.issue) }}'

resource "aws_s3_bucket_policy" "site" {
  bucket = aws_s3_bucket.site.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = "*"
      Action    = "s3:GetObject"
      Resource  = "${aws_s3_bucket.site.arn}/*"
    }]
  })
}

rules:
  - id: s3-policy-public-principal
    languages: [terraform]
    severity: HIGH
    message: This bucket policy allows any principal.
    patterns:
      - pattern-inside: |
          resource "aws_s3_bucket_policy" $NAME {
            ...
          }
      - pattern-either:
          - pattern: Principal = "*"
          - pattern: |
              Principal = { AWS = "*" }
          - pattern: |
              Principal = { AWS = [..., "*", ...] }

    pattern-sources:
      - patterns:
          - pattern-inside: |
              def $HANDLER($EVENT, $CONTEXT):
                ...
          - metavariable-regex:
              metavariable: $HANDLER
              regex: (lambda_)?handler$
          - focus-metavariable: $EVENT

Common mistakes

• Writing the pattern from the one example you found. Without ... it matches only that exact argument list, and it knows only the call you named.
• Testing only the cases that must stay quiet. A rule that matches nothing passes every ok: line. Every test file needs at least one ruleid: line, written in the shapes your real code uses.
• Using pattern-not to exclude something nearby. It removes a finding only when it matches the finding's own code. To exclude by surroundings, use pattern-not-inside.
• Forgetting the quotes in "$KEY". A bare $KEY captures the literal with its quote marks, and a regular expression written for the contents never matches.
• Treating metavariable-regex as a search. It is anchored at the start of the captured text. To find something in the middle, begin the expression with (?s).*; without (?s), . stops at a line break and a multi-line value slips through. Or use metavariable-pattern.
• Choosing languages by what the dangerous text is. Choose it by what the file is. A shell script inside a workflow is YAML to Semgrep.
• Assuming tests/ is scanned. It is not, unless a .semgrepignore of your own replaces the built-in list. Decide on purpose.
• Saving --autofix output as the .fixed file without reading it. The test then checks the fix against itself.
• Reading the findings count and not the file count. "0 findings" on "0 files" is a rule that did not run.
• Treating the registry's rules as free to redistribute. Rules maintained by Semgrep, Inc. are under the Semgrep Rules License, which allows internal use only. Rules from other authors in the registry carry their own licences.