diff --git a/README.md b/README.md index b415d8de1..03a171afa 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@

- - Fibratus + + Fibratus

@@ -9,63 +9,37 @@

Adversary tradecraft detection, protection, and hunting
- Get Started » + Get Started »

- Docs + Docs   •   - Rules + Rules   •   Filaments   •   - Download + Download   •   Discussions

-Fibratus detects, protects, and eradicates advanced adversary tradecraft by scrutinizing -and asserting a wide spectrum of system events against a behavior-driven [rule engine](https://www.fibratus.io/#/filters/rules) and [YARA](https://www.fibratus.io/#/yara/introduction) memory scanner. +Fibratus detects and eradicates advanced attacker tradecraft by scrutinizing and asserting a wide spectrum of [system events](https://fibratus.io/docs/telemetry/events) against a behavior-driven [rule engine](https://fibratus.io/docs/rules) and [YARA](https://fibratus.io/docs/yara) memory scanner. -Events can also be shipped to a wide array of [output sinks](https://www.fibratus.io/#/outputs/introduction) or dumped to [capture](https://www.fibratus.io/#/captures/introduction) files for local inspection and forensics analysis. You can use [filaments](https://www.fibratus.io/#/filaments/introduction) to extend Fibratus with your own arsenal of tools and so leverage the power of the Python ecosystem. +Events can be routed to a wide range of [output sinks](https://fibratus.io/docs/telemetry/outputs) or written to [capture](https://fibratus.io/docs/captures) files for local inspection and forensic analysis. With [filaments](https://fibratus.io/docs/filaments), you can extend Fibratus with your own tooling and tap into the full power of the Python ecosystem. -In a nutshell, the Fibratus mantra is defined by the pillars of **realtime behavior detection**, **memory scanning**, and **forensics** capabilities. +In a nutshell, the Fibratus mantra is built on three pillars: **realtime behavior detection**, **memory scanning**, and **forensics**. +

+ + Fibratus + +

-### Installation - -- Download the latest [MSI package](https://github.com/rabbitstack/fibratus/releases) and follow the [UI](https://www.fibratus.io/#/setup/installation) wizard or -alternatively install via `msiexec` in silent mode - -``` -$ msiexec /i fibratus-2.4.0-amd64.msi /qn -``` - -### Quick start - ---- - -- spin up a command line prompt -- list credentials from the vault by using the `VaultCmd` tool - -``` -$ VaultCmd.exe /listcreds:"Windows Credentials" /all -``` - -`Credential discovery via VaultCmd tool` rule should trigger and emit the alert to the [Eventlog](https://www.fibratus.io/#/alerts/senders/eventlog). Check the short demo [here](https://www.fibratus.io/alerts/senders/images/eventlog.gif). - -### Documentation - -To fully exploit and learn about Fibratus capabilities, read the [docs](https://www.fibratus.io). - -### Rules - -Detection rules live in the [`rules`](/rules) directory of this repository. The CLI provides a set of -commands to explore the rule catalog, validate the rules, or [create a new rule](https://github.com/rabbitstack/fibratus/tree/master/rules#structure) from the template. +### Installation and Quick start -To describe all rules in the catalog, use the `fibratus rules list` command. It is possible to pass the -`-s` flag to show rules summary by MITRE tactics and techniques. +For installation and [quick start](https://www.fibratus.io/docs/setup/quick-start) instructions, go [here](https://www.fibratus.io/docs/setup/installation). ### Contributing diff --git a/docs/.nojekyll b/docs/.nojekyll deleted file mode 100644 index e69de29bb..000000000 diff --git a/docs/CNAME b/docs/CNAME deleted file mode 100644 index 1a307fbab..000000000 --- a/docs/CNAME +++ /dev/null @@ -1 +0,0 @@ -www.fibratus.io \ No newline at end of file diff --git a/docs/README.md b/docs/README.md index 5a797bc87..45e34c7c0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,8 +1,32 @@ -# Welcome to Fibratus Docs! +# Documentation -Fibratus documentation is built with [docsify](https://docsify.js.org/#/) site generator. docsify generates the documentation website on the fly by parsing the Markdown files. To preview the docs while you're editing, you can use the `docsify` CLI tool. +The **`docs/` directory** serves as the canonical source of all user-facing documentation. It contains Markdown files that are not merely static content, but structured inputs consumed by Fibratus’ custom [documentation](https://fibratus.io/docs) engine, which powers the website experience. -- download [Node.JS](https://nodejs.org/en/download/) -- `npm install -g docsify` -- go to the `docs` directory and run `docsify serve` -- browse the docs on `http://localhost:3000` +At a high level, this directory is designed to balance **readability for contributors** with **rich rendering capabilities** on the site. While everything is written in Markdown, the files follow conventions and include custom extensions that the rendering engine interprets to produce a more dynamic and navigable documentation UI. + +### Purpose and Structure + +The `docs/` directory organizes documentation into logical sections such as: + +* **Getting started** (installation, quickstart guides) +* **Core concepts** (filaments, telemetry, rules engine) +* **Configuration** (YAML/CLI flags, environment setup) +* **Reference material** (field descriptions, functions, operators) + +Each section is typically represented as a subdirectory, with Markdown files forming individual pages. File names and folder hierarchy directly map to the website’s routing and sidebar navigation. + +### Custom Markdown Extensions + +Although the files use standard Markdown syntax, they also leverage custom directives and components understood by the Fibratus documentation engine. These may include: + +* **Admonitions** (e.g., notes, warnings, tips) +* **Code block enhancements** with syntax highlighting tailored to Fibratus DSLs +* **Cross-references** that resolve to internal documentation links + +### Linking and Navigation + +Internal links between documents are written using relative paths, but the engine resolves and validates them at build time. It also: + +* Builds the **sidebar tree** from directory structure +* Generates **breadcrumbs and navigation controls** +* Ensures **consistent URL routing** across the site diff --git a/docs/_coverpage.md b/docs/_coverpage.md deleted file mode 100755 index f8132ec34..000000000 --- a/docs/_coverpage.md +++ /dev/null @@ -1,20 +0,0 @@ - - -
- -
- -# fibratus 2.4.0 - -> Adversary tradecraft detection, protection, and hunting - -- Runtime behaviour detection -- Forensics capabilities -- Memory scanning - - Download - Get Started - -
- -
diff --git a/docs/_sidebar.md b/docs/_sidebar.md index 74c8de9dd..7d893b36c 100755 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -1,73 +1,53 @@ -* Overview - * [What is Fibratus?](overview/what-is-fibratus.md) -* Setup +* Setup * [Installation](setup/installation.md) * [Quick Start](setup/quick-start.md) * [Configuration](setup/configuration.md) - * [CLI](setup/cli.md) -* Events - * [Anatomy Of An Event](kevents/anatomy.md) - * [Process](kevents/process.md) - * [Thread](kevents/thread.md) - * [Image](kevents/image.md) - * [File](kevents/file.md) - * [Registry](kevents/registry.md) - * [Network](kevents/network.md) - * [Handle](kevents/handle.md) - * [Object](kevents/object.md) - * [Driver](kevents/driver.md) - * [Memory](kevents/mem.md) -* Filters and Rules - * [Needle In The Haystack](filters/introduction.md) - * [Prefiltering](filters/prefiltering.md) - * [Filtering](filters/filtering.md) - * [Operators](filters/operators.md) - * [Iterators](filters/iterators.md) - * [Functions](filters/functions.md) - * [Rules](filters/rules.md) - * [Fields](filters/fields.md) -* Captures - * [Immortalizing The Event Flux](captures/introduction.md) - * [Capturing](captures/capturing.md) - * [Replaying](captures/replaying.md) -* Filaments - * [Python Meets Kernel Events](filaments/introduction.md) - * [Executing](filaments/executing.md) - * [Internals](filaments/internals.md) - * [Writing Filaments](filaments/writing.md) -* Outputs - * [Transporting Events](outputs/introduction.md) - * [Console](outputs/console.md) - * [Null](outputs/null.md) - * [RabbitMQ](outputs/rabbitmq.md) - * [Elasticsearch](outputs/elasticsearch.md) - * [HTTP](outputs/http.md) - * [Eventlog](outputs/eventlog.md) -* Transformers - * [Parsing, Enriching, Transforming](transformers/introduction.md) - * [Remove](transformers/remove.md) - * [Rename](transformers/rename.md) - * [Replace](transformers/replace.md) - * [Tags](transformers/tags.md) - * [Trim](transformers/trim.md) -* Alerts - * [Firing Alerts](alerts/introduction.md) - * [Alert Senders](alerts/senders.md) - * [Mail](alerts/senders/mail.md) - * [Slack](alerts/senders/slack.md) - * [Systray](alerts/senders/systray.md) - * [Eventlog](alerts/senders/eventlog.md) - * [Filament Alerting](alerts/filaments.md) -* PE - * [Portable Executable Introspection](/pe/introduction.md) - * [Sections](/pe/sections.md) - * [Symbols](/pe/symbols.md) - * [Resources](/pe/resources.md) -* Yara - * [Pattern Matching Swiss Knife](/yara/introduction.md) - * [Scanning Processes](/yara/scanning.md) - * [Alerts](/yara/alerts.md) -* Troubleshooting - * [Logs](troubleshooting/logs.md) - * [Stats](troubleshooting/stats.md) - * [Profiling](troubleshooting/pprof.md) +* --- +* [Architecture](architecture.md) +* --- +* Telemetry + * [Events](telemetry/events.md) + * [Process](telemetry/events/process.md) + * [Thread](telemetry/events/thread.md) + * [Module](telemetry/events/module.md) + * [File](telemetry/events/file.md) + * [Registry](telemetry/events/registry.md) + * [Memory](telemetry/events/memory.md) + * [Network](telemetry/events/network.md) + * [Handle](telemetry/events/handle.md) + * [Object](telemetry/events/object.md) + * [Callstacks](telemetry/callstacks.md) + * [Evasions](telemetry/evasions.md) + * [Filtering](telemetry/filtering.md) + * [Outputs](telemetry/outputs.md) + * [Console](telemetry/outputs/console.md) + * [Null](telemetry/outputs/null.md) + * [RabbitMQ](telemetry/outputs/rabbitmq.md) + * [Elasticsearch](telemetry/outputs/elasticsearch.md) + * [HTTP](telemetry/outputs/http.md) + * [Eventlog](telemetry/outputs/eventlog.md) + * [Transformers](telemetry/transformers.md) + * [Remove](telemetry/transformers/remove.md) + * [Rename](telemetry/transformers/rename.md) + * [Replace](telemetry/transformers/replace.md) + * [Trim](telemetry/transformers/trim.md) + * [Tags](telemetry/transformers/tags.md) +* [Rule Language](rules.md) + * [Macros](rules/macros.md) + * [Operators](rules/operators.md) + * [Iterators](rules/iterators.md) + * [Sequences](rules/sequences.md) + * [Functions](rules/functions.md) + * [Fields](rules/fields.md) + * [Actions](rules/actions.md) + * [Alert](rules/actions/alert.md) + * [Kill](rules/actions/kill.md) + * [Isolate](rules/actions/isolate.md) +* --- +* [Captures](captures.md) +* [Filaments](filaments.md) +* [YARA](yara.md) +* --- +* [Troubleshooting](troubleshooting.md) +* --- +* [CLI](cli.md) diff --git a/docs/alerts/filaments.md b/docs/alerts/filaments.md deleted file mode 100644 index cfc7b9ddd..000000000 --- a/docs/alerts/filaments.md +++ /dev/null @@ -1,20 +0,0 @@ -# Filament Alerting - -Filaments produce alerts by invoking the `emit_alert` function. The alert is propagated to all active alert senders. - -The `emit_alert` function accepts two positional and two keyword arguments. Here is the signature of the function: - -```python -emit_alert(title, text, severity='normal', tags=[]) -``` - -An example of calling the `emit_alert` function to generate an alert from the filament that detects registry persistence attacks: - -```python -emit_alert( - f'Registry persistence gained via {kevent.kparams.key_name}', - text(kevent), - severity='medium', - tags=['registry persistence'] -) -``` diff --git a/docs/alerts/introduction.md b/docs/alerts/introduction.md deleted file mode 100644 index d829abc09..000000000 --- a/docs/alerts/introduction.md +++ /dev/null @@ -1,13 +0,0 @@ -# Firing Alerts - -Fibratus has the ability to generate security alerts when the detection or [YARA](/yara/scanning) rule matches. Additionally, alerts can be emitted directly from [filaments](/alerts/filaments). - -The alert has the following key components: - -- **id** the alert identifier represented as UUID. -- **title** summarizes the purpose of the alert. -- **text** is the message that further explains what this alert is about as well as actors involved. -- **tags** contains a sequence of tags for categorizing the alerts. -- **severity** determines the severity of the alert. Possible values are `normal`, `medium`, `critical`. - -To send alert notifications, use [alert senders](/alerts/senders). diff --git a/docs/alerts/senders.md b/docs/alerts/senders.md deleted file mode 100644 index 81214c967..000000000 --- a/docs/alerts/senders.md +++ /dev/null @@ -1,9 +0,0 @@ -# Alert Senders - -You can send alert notifications to your team through email, Slack, or incident response platforms. The notification can be sent to multiple alert senders. Alert senders configuration resides in the `alertsenders` section of the `yml` file. - -- [Mail](/alerts/senders/mail) -- [Slack](/alerts/senders/mail) -- [Systray](/alerts/senders/systray) -- [Eventlog](/alerts/senders/eventlog) - diff --git a/docs/alerts/senders/eventlog.md b/docs/alerts/senders/eventlog.md deleted file mode 100644 index a78838e64..000000000 --- a/docs/alerts/senders/eventlog.md +++ /dev/null @@ -1,24 +0,0 @@ -# Eventlog - -The `eventlog` alert sender sends alerts to the [Windows Eventlog](https://sematext.com/glossary/what-is-windows-event-log/). - -

- -

- -### Configuration {docsify-ignore} - -The `eventlog` alert sender configuration is located in the `alertsenders.eventlog` section. - -#### enabled - -Indicates whether the `eventlog` alert sender is enabled. - -**default**: `true` - -#### verbose - -Enables/disables the verbose mode. In verbose mode, the full event context, including all parameters and the process information are included -in the log message. - -**default**: `true` diff --git a/docs/alerts/senders/images/eventlog.gif b/docs/alerts/senders/images/eventlog.gif deleted file mode 100644 index c3af9fe41..000000000 Binary files a/docs/alerts/senders/images/eventlog.gif and /dev/null differ diff --git a/docs/alerts/senders/mail.md b/docs/alerts/senders/mail.md deleted file mode 100644 index 67237511a..000000000 --- a/docs/alerts/senders/mail.md +++ /dev/null @@ -1,42 +0,0 @@ -# Mail - -The mail alert sender emits alert notifications through the `SMTP` protocol. -!> If you are using a Gmail SMTP provider, you might need to configure [App Passwords](https://support.google.com/accounts/answer/185833?hl=en) for your account - -!> If you are using a Gmail SMTP provider, you might need to configure [App Passwords](https://support.google.com/accounts/answer/185833?hl=en) for your account - -### Configuration {docsify-ignore} - -The `mail` alert sender configuration is located in the `alertsenders.mail` section. - -#### enabled - -Indicates if the alert sender is enabled. - -**default**: `false` - -#### host - -Represents the host name of the SMTP server where the alert notification is sent. - -#### port - -Represents the port number of the SMTP server. - -**default**: `25` - -#### user - -Specifies the user name when authenticating to the SMTP server. - -#### password - -Specifies the password when authenticating to the SMTP server. - -#### from - -Determines the sender's email address. - -#### to - -Specifies the list of the recipient email addresses. diff --git a/docs/alerts/senders/slack.md b/docs/alerts/senders/slack.md deleted file mode 100644 index bbff202a9..000000000 --- a/docs/alerts/senders/slack.md +++ /dev/null @@ -1,29 +0,0 @@ -# Slack - -The `slack` alert sender forwards alerts to Slack workspaces. You'll have to [activate](https://slack.com/intl/en-es/help/articles/115005265063-Incoming-webhooks-for-Slack) incoming webhooks and associate the webhook to your Slack workspace to be able to emit the alerts. - -### Configuration {docsify-ignore} - -The `slack` alert sender configuration is located in the `alertsenders.slack` section. - -#### enabled - -Indicates whether the `slack` alert sender is enabled. - -**default**: `false` - -#### url - -Represents the Webhook URL of the workspace where alerts will be dispatched. The Webhook URL has the following format: `https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX`. - -#### workspace - -The name of the Slack workspace where alerts are routed. - -#### channel - -Is the slack channel in which to post alerts. - -#### emoji - -Represents the emoji icon surrounded in `:` characters for the Slack bot. diff --git a/docs/alerts/senders/systray.md b/docs/alerts/senders/systray.md deleted file mode 100644 index 42dfe8926..000000000 --- a/docs/alerts/senders/systray.md +++ /dev/null @@ -1,25 +0,0 @@ -# Systray - -The `systray` alert sender sends alerts to the systray notification area. - -### Configuration {docsify-ignore} - -The `systray` alert sender configuration is located in the `alertsenders.systray` section. - -#### enabled - -Indicates whether the `systray` alert sender is enabled. - -**default**: `false` - -#### sound - -Indicates if the associated sound is played when the balloon notification is shown. - -**default**: `true` - -#### quiet-mode - -Instructs not to display the balloon notification if the current user is in quiet time. During this time, most notifications should not be sent or shown. This lets a user become accustomed to a new computer system without those distractions. Quiet time also occurs for each user after an operating system upgrade or clean installation. - -**default**: `false` diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 000000000..da4ec2365 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,66 @@ +# Architecture + +##### The diagram illustrates the end-to-end runtime threat detection pipeline allowing Fibratus to unveil and neutralize stealthy attack chains. + +![Architecture](images/architecture.png "Architecture") + +### Event Sources + +The leftmost block represents the Windows host machine and the kernel instrumentation layer that feeds all telemetry into **Fibratus**. +The Windows host exposes kernel-level telemetry through the [Event Tracing for Windows](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/event-tracing-for-windows) subsystem - a high-performance, always-on tracing infrastructure built into the Windows NT kernel. Fibratus subscribes to ETW system providers as a realtime consumer, receiving a continuous stream of kernel events without requiring a kernel driver of its own. Fibratus captures a wide-spectrum of system events: + + +- [Process](telemetry/events/process): process creation, termination, and process object access +- [Thread](telemetry/events/thread): thread creation, termination, thread pool activity, thread object access and thread context manipulation +- [File](telemetry/eventss/file): file I/O operations including reads, writes, creates, deletions, file metadata manipulation and renames +- [Memory](telemetry/events/mem.md): virtual memory allocation and section mapping/unmapping +- [Network](telemetry/events/network.md): low level network operations such as TCP/UDP send and receive, connect and accept events +- [Registry](telemetry/events/registry.md): key creation, deletion, access, and value mutation +- [Module](telemetry/events/module.md): DLL/executable/driver load and unload events +- [DNS](telemetry/events/network.md): DNS query and response telemetry +- [Handle](telemetry/events/handle.md): object handle creation and duplication + + +### Event Pipeline + +Once raw ETW events are ingested, they pass through a two-stage pipeline before reaching the engine. + +##### Collection + +The **Collection** stage is responsible for receiving, parsing, and normalising raw ETW event records. Each event is deserialised from its binary ETW wire format into a structured internal event representation. Field extraction, type coercion, and sequence assembly happen here, turning low-level kernel signals into richly typed, queryable event objects ready for downstream processing. + +##### Enrichment + +The **Enrichment** stage decorates each event with contextual metadata that is not present in the raw ETW record but is essential for detection. This includes: + +**Process context injection**. The full [process state](telemetry/events/process.md) for the event's originating process is attached, including executable path, command-line arguments, user, session, token integrity level, process modules, and so on. + +**Parent process tree traversal**. The ancestry chain is resolved from live process snapshot cache, allowing rules to reason about parent-child relationships. + +**Callstack resolution**. Return addresses captured at event time are resolved against loaded module symbols, producing human-readable [callstack frames](telemetry/callstacks.md) that reveal the code path that triggered the event. + +**PE metadata enrichment**. Portable executable attributes such as digital signature status, entropy, and import table characteristics are attached where available. + +The process snapshot and process tree cache serves as the authoritative in-memory state for all enrichment lookups, ensuring sub-microsecond context resolution. + +##### Rule Engine and Memory Scanner + +After enrichment, each event is dispatched simultaneously to two parallel evaluators. **Rule Engine** evaluates the enriched event against the loaded detection rule set. [Rules](rules.md) are expressed in a behaviour-driven `YAML` DSL (Domain Specific Language) and compiled into an optimised filter/expression tree at startup. The rule engine supports boolean and sequence operators for multi-event correlation, field accessor expressions with full type awareness, pattern matching, glob expressions, IP/CIDR predicates and stateful sequence tracking for multi-step attack pattern detection. When a rule condition matches, the engine generates an alert and apply response actions such as terminating the process. + +**Memory Scanner** runs in parallel and is triggered on arrival of different signals such as suspiciuous memory allocations, section mappings or unsigned DLL loading. It performs [YARA](yara.md) rule scanning of process virtual memory, enabling detection of injected shellcode, unpacked malware, and in-memory IOCs that leave no filesystem artefact. + +### Outputs + +[Outputs](telemetry/outputs.md) consists of three branches that operate independently and can all fire simultaneously for a single matched event. + +##### Alert + +Is triggered when the rule engine or memory scanner produces a security alert. It dispatches a structured alert object containing the rule name, severity, [MITRE](https://attack.mitre.org/) tactic and technique tags, process metadata, and a callstack excerpt to one or more configured [alert senders](rules/actions/alert.md) such as Slack, Email or Eventlog. + +##### Send + +Streams every matched event to persistent storage or message broker infrastructure for retention, search, and downstream analytics. The event can be forwarded to [Elasticsearch](telemetry/outputs/elasticsearch.md), [RabbitMQ](telemetry/outputs/rabbitmq.md) message broker, [HTTP](telemetry/outputs/http.md) endpoints, or [console](telemetry/outputs/console.md) for local analysis. + +##### Actuate + +Executes protective response [actions](rules/actions.md) in direct response to a rule match. Response actions include killing the offending process or isolating the host by cutting off its ability to communicate with internal or external networks while preserving forensic state for investigation. diff --git a/docs/captures.md b/docs/captures.md new file mode 100644 index 000000000..0badd0c00 --- /dev/null +++ b/docs/captures.md @@ -0,0 +1,69 @@ +# Captures + +##### Captures allow you to **record and replay the full stream of events** together with enough system state to reconstruct what happened at a given point in time. A capture acts as a *time capsule* of system activity enabling deterministic, offline analysis of behaviors that originally occurred on another machine. + +This makes captures especially valuable for post-mortem incident investigations, malware analysis and reverse engineering, detection rule development and testing, and evidence material. + +A capture is a file with the `.cap` extension that contains the snapshot of system state and chronologically ordered stream of events. Together, these allow rebuilding process context during replay, as if events were happening live. Think of captures as a recording of the event pipeline that can be replayed with full analytical capabilities. + +## Capturing events + +Capturing is initiated via the `fibratus capture` command. + + +$ fibratus capture -o events + + + +This command starts recording the telemetry, writes all events to `events.cap` in the current directory. The `-o` (output) flag specifies the capture file name. To stop capturing, press `Ctrl + C` After stopping, a summary is printed including number of captured events, number of processes, and the capture size. + +?> The `capture` command automatically overwrites any existing capture file with the same name located in the current directory. + +You can restrict which events are recorded by providing a [filter](telemetry/filtering.md) expression. This is useful for reducing the capture size, focusing on specific subsystems (file, registry, network, etc.), and minimizing noise during analisys. Filtering at capture time is more efficient than filtering during replay when dealing with high event volumes + + +$ fibratus capture evt.category = 'file' -o fs-events + + +## Replaying captures + +Replaying a capture restores system state and reprocesses events as if they were happening in real time. Replaying is initiated via the `fibratus replay` command. + + +$ fibratus replay -k events + + + +During replay process and handle state are rebuilt. Events are emitted through the same pipeline as live data, so [filters](telemetry/filtering.md) or [filaments](filaments.md) can be applied. + +You can apply filters when replaying to focus on relevant events and drill down into specific behaviors iterating quickly during investigations. + + +$ fibratus replay file.path contains 'Temp' -k fs-events + + + +Captures can be used as input for [filaments](filaments.md) enabling offline analytics and automation. This is particularly useful for testing filaments against real-world data. The `-f` flag specifies the filament against which the capture events are processed. + + +$ fibratus replay -f watch_files -k fs-events + + + +## Capture format and internals + +Under the hood, captures are stored as [zstd](https://es.wikipedia.org/wiki/Zstandard) compressed streams. ZSTD provides a strong balance between the compression ratio and runtime overhead. + +A `.cap` file consists of: + +1. **Header** + * Magic + * Major/minor version + * Flags +2. **Handle snapshot** + * All active kernel handles at capture start +3. **Event stream** + * Ordered events + +The process state is not explicitly stored. Instead, it is reconstructed during replay by processing events such as process enumeration and lifecycle notifications. +Fibratus increments the major version of the `cap` format when breaking changes are introduced. Captures with mismatched major versions may not replay. This way, the compatibility is guaranteed between capture producer version and replay version. diff --git a/docs/captures/capturing.md b/docs/captures/capturing.md deleted file mode 100644 index 511b803a1..000000000 --- a/docs/captures/capturing.md +++ /dev/null @@ -1,21 +0,0 @@ -# Capturing - -Under the hood, captures are written to disk in the form of the [zstd](https://en.wikipedia.org/wiki/Zstandard) compressed streams. zstd provides a compelling balance between the capture file size and the compression runtime overhead. - -Each capture file consists of the header that represents the `kcap` magic, major/minor version, and some arbitrary flags. Next, the handle snapshot is stored with all allocated handles followed by kernel events. We can forgo persisting the process snapshot, because it can be reconstructed when replaying the capture and processing the `EnumProcess` events. - -Capturing is initiated by running the `fibratus capture` command. The `o` flag, that stands for `output`, specifies the `kcap` file where events are dumped. The capture file is stored in the current working directory. **Any already existing file is overwritten**. To above command would produce a capture and store all events in `events.kcap` file. - -``` -$ fibratus capture -o events -``` - -To stop capturing events, hit the `Ctrl-C` key combination. A short summary is displayed indicating the number of captured events, processes, handles, the size of the `kcap` file and so on. - -### Filtering {docsify-ignore} - -As already explained in [filtering](/filters/filtering), for a fine-grained control over which events are stored in the capture, you can provide a filter expression. - -``` -$ fibratus capture kevt.category = 'file' -o fs-events -``` diff --git a/docs/captures/introduction.md b/docs/captures/introduction.md deleted file mode 100644 index fbea34185..000000000 --- a/docs/captures/introduction.md +++ /dev/null @@ -1,7 +0,0 @@ -# Immortalizing The Event Flux - -Captures or `kcap` files aim for the capture-once replay-anywhere workflow. Captures contain the full state of processes at the time capture was taken as well as the originated event flux. This makes them a great companion in post-mortem investigations - generate the capture in the honeypot machine, grab the `.kcap` file, and you're ready to dive into the attacker kill chain by replaying the capture file on your laptop. - -With captures you "freeze" the shape of the event flux at a certain point in time. Do you need to troubleshoot an network issue and surface the root cause? Or maybe you need to determine what files were written by a malicious process? Replay the capture at any given time and drill down into the event flow to start investigating. - -You can harness the power of the filtering engine when replaying captures or even execute a filament on top of captured events. diff --git a/docs/captures/replaying.md b/docs/captures/replaying.md deleted file mode 100644 index 81d52b568..000000000 --- a/docs/captures/replaying.md +++ /dev/null @@ -1,25 +0,0 @@ -# Replaying - -Replaying essentially recovers the handle/process state and consumes the captured event flux. It is important to point out that Fibratus increments the major `kcap` version under relevant changes in the format structure. Because of this, old capture files might not be able to replay due to mismatch of the `kcap` major version digit. - -To replay the `kcap` file, you launch the following command. - -``` -$ fibratus replay -k events -``` - -### Filtering {docsify-ignore} - -To drill down into capture by filtering out valuable events, you can provide a filter. - -``` -$ fibratus replay file.name contains 'Temp' -k fs-events -``` - -### Filaments {docsify-ignore} - -Another compelling use case stems from running a filament on top of events living in the capture. To run a filament you supply the filament name via the `-f` or `--filament.name` option. - -``` -$ fibratus replay -f watch_files -k fs-events -``` diff --git a/docs/cli.md b/docs/cli.md new file mode 100644 index 000000000..65544b686 --- /dev/null +++ b/docs/cli.md @@ -0,0 +1,120 @@ +# CLI + +### `run` + +The primary command for starting Fibratus-whether running the rule engine, operating as an event forwarder, or executing a filament. It also accepts an optional filter expression. Examples: + +1. Collect all events + + +$ fibratus run --forward + + + +2. Run a filament + + +$ fibratus run -f watch_files + + + +3. Collect and filter events + + +$ fibratus run --forward evt.category = 'file' and ps.name = 'cmd.exe' + + + +### `capture` + +Writes the event stream to capture files. It accepts an optional filter expression. Example: + + +$ fibratus capture evt.category = 'net' and net.dip=172.17.2.3 -o events + + + +### `replay` + +Reconstructs the event stream from the capture file. It accepts an optional filter expression. Example: + + +$ fibratus replay net.sip=172.2.2.3 -k events + + + +### `rules` + +The root command that exposes various subcommands for listing/validating rules and creating detection rule templates. + +- #### `list` + +List all rules located in the `Rules` directory. + +- #### `validate` + +Validates rules for structural and syntactic correctness. + +- #### `create` + +Create a new rule template. The command requires a rule name and an optional MITRE tactic identifier, for example `TA0001`, that can be passed via the `--tactic-id` flag. + +### `config` + +Prints the options loaded from configuration sources including files, command line flags or environment variables. Sensitive data, such as passwords are masked out. + +### `service` + +This is the root command that exposes multiple subcommands for interacting with the Windows [Service Control Manager](https://learn.microsoft.com/en-us/windows/win32/services/service-control-manager). + +- #### `start` + +Starts the Fibratus service that previously registered with the Windows Service Control Manager. + +- #### `stop` + +Stops the Fibratus Windows service. + +- #### `restart` + +Restarts the Fibratus Windows service. + +- #### `remove` + +Removes the Fibratus service from the Windows Service Control Manager. + +- #### `status` + +Checks the status of the Fibratus Windows service. + +### `docs` + +Launches the default web browser and opens the Fibratus documentation site. + +### `list` + +The command consists of various subcommands to list available filaments, event types or filter fields. + +- #### `filaments` + +Displays available filaments. Filaments live in the `%PROGRAMFILES\Fibratus\Filaments` directory, but you can override this location with the `--filament.path` flag or the corresponding key in the `yaml` configuration file. + +- #### `fields` + +Shows all [field names](rules/fields.md) that can be used in rule conditions. + +- #### `events` + +Shows available event types. + +### `stats` + +Returns the runtime metrics that are exposed through the [expvar](https://golang.org/pkg/expvar/) HTTP endpoint. Useful for debugging. + +### `version` + +Displays the Fibratus version along with the commit hash and the Go compiler version. + +### `help` + +Displays detailed usage information for commands, including available flags and options. diff --git a/docs/filaments.md b/docs/filaments.md new file mode 100644 index 000000000..a782130bc --- /dev/null +++ b/docs/filaments.md @@ -0,0 +1,256 @@ +# Filaments + +##### Filaments are extensibility mechanism that allow you to write custom logic in Python and execute it on top of the live or replayed event stream. Filaments effectively turn Fibratus into a programmable security analytics engine, where you can build anything from simple event processors to complex detection pipelines. + +[Python](https://www.python.org/) is the _lingua franca_ of penetration testers, threat hunters, and SecOps engineers. A vast ecosystem of security tooling already exists in Python. Filaments let you bring that ecosystem directly into Fibratus. + +With filaments, you can reuse existing Python libraries and tooling, build custom detections beyond declarative rules, enrich events with external intelligence, or automate investigations. + ++> Filaments principle is: If you can write it in Python, you can run it inside Fibratus. + +## Execution model + +A filament is a Python script executed within Fibratus that processes the stream of events in real time or during replay. Filament has access to event data, process context, and internal Fibratus state. + +Filaments are executed on top of the event stream. Each incoming event is passed to the filament. The filament can inspect, transform, correlate, or act on it. This makes filaments ideal for stateful analysis and complex heuristics that are difficult to express in rules. + +## Filament internals + +From a technical standpoint each filament runs as a fully initialized Python interpreter instance. Fibratus interacts with the [CPython](https://github.com/python/cpython) API to bootstrap the interpreter, initialize the module from filament definition, declare functions, and other related tasks. + +### Event processing + +Filament's backbone is the `on_next_event` function. This function is invoked for each event in the stream. The parameter of this function is the Python dictionary that contains event data. Here is the structure of such a dictionary object: + +```json +{ + 'seq': 122344, + 'pid': 2034, + 'tid': 2453, + 'ppid': 45, + 'cwd': 'C:\Windows\system32', + 'exe': 'cmd.exe', + 'cmdline': 'cmd.exe rm /r', + 'sid': 'S-1-15-8', + 'cpu': 2, + 'name': 'CreateFile', + 'category': 'file', + 'timestamp': '2013-08-23 16:15:13.4323', + 'host': 'archrabbit', + 'description': 'Creates or opens a file or I/O device', + 'params': { + 'file_path': 'C:\WINDOWS\system32\config\systemprofile\AppData\WindowsApps\', + 'file_object': 'ffffa88c7ea077d0', + 'irp': 'ffffa88c746b2a88', + 'create_disposition': 'supersede', + 'share_mask': 'rw-', + 'type': 'directory' + } +} +``` + +For a more convenient dictionary accesses, you can annotate the function with the `dotdictify` decorator. + +```python +from utils.dotdict import dotdictify + +@dotdictify +def on_next_event(event): + print(f'{event.name} generated by {event.exe}') +``` + +### Initialization + +If the `on_init` function is declared in the filament, any logic wrapped inside this function is executed prior to event processing. This is a convenient place for configuring the table columns or establishing the `on_interval` function triggering intervals among other initialization tasks. + +```python +def on_init(): + interval(1) +``` + +### Termination + +The `on_stop` function is called right before the Python interpreter is teared down. You can place any code you would like to get executed when the filament is stopped. + +```python +def on_stop(): + f.close() +``` + +### Periodical actions + +Filament has built-in support for scheduling timers. The timer, associated with the `on_interval` function, is fired after the interval specified by the `interval` function elapses. The minimum interval granularity is one second. + +```python +def on_interval(): + for ip, count in __connections__.copy().items(): + f.write(f'{ip.count}') +``` + +### Filtering + +The `set_filter` function defines a filter expression for the life span of a filament. Filter can be constructed dynamically, like in the following code snippet that defines a filter from the list: + +```python +set_filter("ps.name in (%s)" % (', '.join([f'\'{ps}\'' for ps in __procs__]))) +``` + +### Table rendering + +Filaments are able to render tabular data on the console in a flicker-free fashion by using the frame buffers. To render a table, you start by defining the columns with the `columns` function. It's possible to sort the data by specifying the column via `sort_by` function. Finally, the `add_row` function appends rows to the table. When you're ready to draw the table, invoke the `render_table` function. + +```python +def on_init(): + columns(["Source", "Count"]) + sort_by('Count') + interval(1) + +def on_interval(): + for ip, count in __connections__.copy().items(): + add_row([ip, count]) + render_table() +``` + +## Running filaments + +Filaments are bootstrapped via the `fibratus run` command by specifying the filament name. Use the `-f` or `--filament.name` flags to indicate the filament you'd like to run. + + +$ fibratus run -f watch_files + + + +The filament will keep running until the keyboard interrupt signal is received. + +Filaments may require additional arguments to execute some conditional logic or set up a filter. Arguments are passed to a filament by specifying a list of comma-separated values after the filament name. + + +$ fibratus run -f "watch_files,powershell.exe" + + + +This populates the [sys.argv](https://docs.python.org/3/library/sys.html#sys.argv) list with the provided arguments, where `sys.argv[0]` is the filament name. + +Filters can be specified either via a command-line argument when running the filament or by calling the `set_filter` function during filament initialization. If a filter expression is provided in both, the one defined in `set_filter` takes precedence. + +## Listing filaments + +By default, filaments are located in the `%PROGRAMFILES%\Fibratus\Filaments` directory. This location can be overridden by specifying an alternative path using the `--filament.path` flag or by modifying the configuration file. + +To list available filaments, run the command below. + + +$ fibratus list filaments + + + +## Writing filaments + +The most effective way to understand filaments is to build one from scratch. In this walkthrough, we’ll create a filament that retrieves an IP blacklist database and uses it to detect outbound and inbound connections to botnets, command-and-control (C&C) servers, and other suspicious endpoints. + + + + + +We’ll start by defining a function that fetches the database and converts it into a list. Along the way, we’ll declare two regular expressions to validate standard IP addresses and those expressed in CIDR notation. The database is retrieved using the `requests` package. + +```python +def fetch_db(url): + with requests.get(url) as r: + ips = [ipaddress.ip_network(l) if '/' in l else \ + ipaddress.ip_address(l) \ + for l in r.text.splitlines() if IP_RE.match(l) \ + or IP_CIDR_RE.match(l)] + return ips +``` + + + + +Next, we invoke the `fetch_db` function during filament initialization and store the results in a global `__fishy_ips__` list. To keep the data up to date, we schedule a periodic sync that refreshes the IP database every hour, ensuring we always have the latest indicators of spam networks and malicious hosts. Since our focus is on network activity, we also configure a filter to capture only network-related events. + +```python +def on_init(): + __fishy_ips__ = fetch_db(IP_DB_URL) + interval(3600) + set_filter("evt.category = 'net'") + columns(["Source", "Destination", "Process"]) + +def on_interval(): + __fishy_ips__ = fetch_db(IP_DB_URL) +``` + + + + +With the data in place, we can implement the core logic. This involves checking whether observed network flows match any entries in our blacklist, indicating potential communication with compromised infrastructure such as C&C servers. When a match is found, we add a new row and render a table displaying the source and destination IP addresses, along with the process responsible for initiating the connection. + +```python +@dotdictify +def on_next_event(event): + sip = event.params.sip + dip = event.params.dip + if (sip in __fishy_ips__ or dip in __fishy_ips__) or \ + (sip or dip in net for net in __fishy_ips__ \ + if isinstance(net, ipaddress.IPv4Network)): + add_row([sip, dip, event.exe]) + render_table() +``` + + + + + +In a more advanced setup, this logic could be extended to generate [alerts](rules/actions/alert.md) and deliver them via channels such as Slack or email. + +The complete filament implementation closely resembles the snippet shown above. + +```python +""" +Pinpoints network communications with botnets or C&C servers. +""" + +import requests +import re +import ipaddress +from utils.dotdict import dotdictify + +IP_RE = re.compile(r"(? +$ fibratus run -f cc + + diff --git a/docs/filaments/executing.md b/docs/filaments/executing.md deleted file mode 100644 index c6cb9a53c..000000000 --- a/docs/filaments/executing.md +++ /dev/null @@ -1,39 +0,0 @@ -# Executing Filaments - -Filaments are bootstrapped via the `fibratus run` command by specifying the filament name. Use the `-f` or `--filament.name` flags to indicate the filament you'd like to run. - -``` -$ fibratus run -f watch_files -``` - -The filament will keep running until the keyboard interrupt signal is received. - -### Passing arguments to filaments {docsify-ignore} - -Filaments may require additional arguments to execute some conditional logic or set up a filter. Arguments are passed to a filament by specifying a list of comma-separated values after filament name: - -``` -$ fibratus run -f "watch_files,powershell.exe" -``` - -This populates the [sys.argv](https://docs.python.org/3/library/sys.html#sys.argv) list with the provided arguments, where `sys.argv[0]` is the filament name. - - -### Listing filaments {docsify-ignore} - -By default, filaments reside within the `%PROGRAMFILES%\Fibratus\Filaments` directory. It is possible to override this location by specifying an alternative directory via the `--filament.path` flag or by editing the config file. - -To list available filaments, run the below command. - -``` -$ fibratus list filaments -``` - -### Filters {docsify-ignore} - -Engaging filters in filaments can be accomplished in two ways: - -- the command line argument when running the filament -- the `kfilter` function during filament initialization - -If the filter expression is supplied in both the CLI argument and the `kfilter` function, the one set in the latter takes precedence. diff --git a/docs/filaments/internals.md b/docs/filaments/internals.md deleted file mode 100644 index 5c77759ac..000000000 --- a/docs/filaments/internals.md +++ /dev/null @@ -1,100 +0,0 @@ -# Internals - -Filaments are scheduled as independent Python Virtual Machines and thus they have their own memory and other resources allocated. Fibratus takes as an input a Python module consisting of various functions and converts them into a running instance of the Python interpreter that knows how to process incoming kernel events. It also augments the Python module with numerous functions that permit a filament to interact with alerts and other state exposed by Fibratus. - -### Event dispatching {docsify-ignore} - -The backbone of a filament is the `on_next_kevent` function. It's executed whenever a new kernel event arrives. The parameter of this function is the Python dictionary that contains event data. Here is the structure of such a dictionary object: - -```python -{ - 'seq': 122344, - 'pid': 2034, - 'tid': 2453, - 'ppid': 45, - 'cwd': 'C:\Windows\system32', - 'exe': 'cmd.exe', - 'comm': 'cmd.exe rm /r', - 'sid': 'archrabbit\SYSTEM', - 'cpu': 2, - 'name': 'CreateFile', - 'category': 'file', - 'timestamp': '2013-08-23 16:15:13.4323', - 'host': 'archrabbit', - 'description': 'Creates or opens a file or I/O device', - 'kparams': { - 'file_name': 'C:\WINDOWS\system32\config\systemprofile\AppData\WindowsApps\', - 'file_object': 'ffffa88c7ea077d0', - 'irp': 'ffffa88c746b2a88', - 'operation': 'supersede', - 'share_mask': 'rw-', - 'type': 'directory' - } -} -``` - -For a more convenient dictionary accesses, you can annotate the function with the `dotdictify` decorator. - -```python -from utils.dotdict import dotdictify - -@dotdictify -def on_next_kevent(kevent): - print(f'{kevent.name} generated by {kevent.exe}') -``` - -### Initialization {docsify-ignore} - -If the `on_init` function is declared in the filament, any logic wrapped inside this function is executed prior to event processing. This is a convenient place for configuring the table columns or establishing the `on_interval` function triggering intervals among other initialization tasks. - -```python -def on_init(): - interval(1) -``` - -### Termination {docsify-ignore} - -The `on_stop` function is called right before the Python interpreter is teared down. You can place any code you would like to get executed when the filament is stopped. - -```python -def on_stop(): - f.close() -``` - -### Periodical actions {docsify-ignore} - -Filament has built-in support for scheduling timers. The timer, associated with the `on_interval` function, is fired after the interval specified by the `interval` function elapses. The minimum interval granularity is one second. - -```python -def on_interval(): - for ip, count in __connections__.copy().items(): - f.write(f'{ip.count}') -``` - -### Filtering {docsify-ignore} - -The `kfilter` function defines a filter expression for the life span of a filament. Filaments give an appealing approach for constructing the filters dynamically. For example, this following code snippet defines a filter from the list: - -```python -kfilter("ps.name in (%s)" % (', '.join([f'\'{ps}\'' for ps in __procs__]))) -``` - -### Table rendering {docsify-ignore} - -Filaments are able to render tabular data on the console in a flicker-free fashion by using the frame buffers. To render a table, you start by defining the columns with the `columns` function. It's possible to sort the data by specifying the column via `sort_by` function. Finally, the `add_row` function appends rows to the table. When you're ready to draw the table, invoke the `render_table` function. - -```python -def on_init(): - columns(["Source", "Count"]) - sort_by('Count') - interval(1) - -def on_interval(): - for ip, count in __connections__.copy().items(): - add_row([ip, count]) - render_table() -``` - -### Python distribution and pip {docsify-ignore} - -Fibratus bundles embedded Python 3.7 distribution. Installing additional packages can be achieved by running the `pip` command from the `%PROGRAM FILES%\Fibratus\Python\Scripts` directory. diff --git a/docs/filaments/introduction.md b/docs/filaments/introduction.md deleted file mode 100755 index be2dd5ac0..000000000 --- a/docs/filaments/introduction.md +++ /dev/null @@ -1,9 +0,0 @@ -# Python Meets Kernel Events - -**Python** is the lingua franca of pen testers and other **SecOps**-driven individuals. Many security tools are written in Python language. -Wouldn't it be awesome to exploit the arsenal of those tools in Fibratus or build your own tools atop them? - -Fibratus incorporates a framework for painlessly extending the functionality and incorporating new features via Python scripts. These scripts are called **filaments**. You can also think of them as extension points with virtually endless possibilities. Whatever you are allowed to craft in Python, you can also implement in filaments. - -Filaments are executed on top of kernel event flux and thus they have access to all event's parameters, process state and so on. -From technical perspective, a filament is a full-fledged instance of the Python interpreter. Fibratus interacts with the **CPython** API to bootstrap the interpreter, initialize the module from filament definition, declare functions and other related tasks. diff --git a/docs/filaments/writing.md b/docs/filaments/writing.md deleted file mode 100644 index 02d16fa60..000000000 --- a/docs/filaments/writing.md +++ /dev/null @@ -1,97 +0,0 @@ -# Writing Filaments - -The best way to grasp filaments is by writing a new filament from scratch. The following is the walkthrough of building a filament that fetches the IP blacklist database and relies on it to detect outbound/inbound connections to botnets, C&C servers, and other fishy destinations. - -Let's first define the function for fetching the database and converting it into a list. We also declare two regular expressions for accepting only valid IP addresses or IP addresses in CIDR notation. Note that we use the `requests` package for fetching the database. - -```python -IP_RE = re.compile(r"(? 666` | -| kevt.pid | Process identifier generating the event | `kevt.pid = 6` | -| kevt.tid | Thread identifier generating the event | `kevt.tid = 1024` | -| kevt.cpu | Logical processor core where the event was generated | `kevt.cpu = 2` | -| kevt.name | Symbolical event name | `kevt.name = 'CreateThread'` | -| kevt.category | Category to which the event pertains | `kevt.category = 'registry'` | -| kevt.desc | Cursory event description | `kevt.desc contains 'Creates'` | -| kevt.host | Hostname on which the event was produced | `kevt.host contains 'dev'` | -| kevt.nparams | Number of event parameters | `kevt.nparams > 2` | -| kevt.time | Event timestamp as a time string | `kevt.time = '17:05:32'` | -| kevt.time.h | Hour within the day on which the event occurred | `kevt.time.h = 23` | -| kevt.time.m | Minute offset within the hour on which the event occurred | `kevt.time.m = 54` | -| kevt.time.s | Second offset within the minute on which the event occurred | `kevt.time.s = 0` | -| kevt.time.ns | Nanoseconds specified by the event timestamp | `kevt.time.ns > 1591191629102337000` | -| kevt.date | Event timestamp as a date string | `kevt.date = '2018-03-03'` | -| kevt.date.d | Day of the month on which the event occurred | `kevt.date.d = 12` | -| kevt.date.m | Month of the year on which the event occurred | `kevt.date.m = 11` | -| kevt.date.y | Year on which the event occurred | `kevt.date.y = 2020` | -| kevt.date.tz | Time zone associated with the event timestamp | `kevt.date.tz = 'UTC'` | -| kevt.date.week | Week number within the year on which the event occurred | `kevt.date.week = 2` | -| kevt.date.weekday | Week day on which the event occurred | `kevt.date.weekday = 'Monday'` | -| kevt.arg[] | Accesses a specific event parameter via internal name | `kevt.arg[exe] = 'C:\\Windows\\cmd.exe'` | - - -### Process -| Field Name | Description | Example | -| :--- | :---- | :---: | -| ps.pid | Process identifier generating the event. Alias for `kevt.pid` | `ps.pid = 1024` | -| ps.ppid | Parent process identifier of the process generating the event | `ps.ppid = 25` | -| ps.name | Process (image) path name that generates an event | `ps.name = 'cmd.exe'` | -| ps.cmdline | Process command line | `ps.cmdline contains '/E c:\\ads\\file.txt:regfile.reg'` | -| ps.exe | Full name of the process' executable | `ps.exe = 'C:\\Windows\\system32\\cmd.exe'` | -| ps.args | Process command line arguments | `ps.args in ('/cdir', '/-C')` | -| ps.cwd | Process current working directory | `ps.cwd = 'C:\\Users\\Default'` | -| ps.sid | Security identifier under which this process is run | `ps.sid = 'S-1-5-18'` | -| ps.domain | Process domain name | `ps.domain = 'NT AUTHORITY'` | -| ps.username | Process user name | `ps.username = 'SYSTEM'` | -| ps.sessionid | Unique identifier for the current session | `ps.sessionid = 1` | -| ps.access.mask | Process access rights | `ps.access.mask = '0x1000'` | -| ps.access.mask.names | Process access human-readable rights | `ps.access.mask.names in ('TERMINATE', 'QUERY_INFORMATION')` | -| ps.access.status | Process access status | `ps.access.status = 'success'` | -| ps.envs | Process environment variables | `ps.envs in ('MOZ_CRASHREPORTER_DATA_DIRECTORY')` | -| ps.envs[] | Accesses a specific environment variable. Prefix matches are supported | `ps.envs['MOZ_CRASHREPORTER'] = 'C:\\Program Files\\Firefox'` | -| ps.dtb | Process directory table base address | `ps.dtb = '7ffe0000'` | -| ps.handles | Allocated process handles | `ps.handles in ('\\BaseNamedObjects\\__ComCatalogCache__')` | -| ps.handle.types | Allocated process handle types | `ps.handle.types in ('Key', 'Mutant', 'Section')` | -| ps.modules | Modules loaded by the process | `ps.modules in ('crypt32.dll', 'xul.dll')` | -| ps.modules[] | Accesses a specific process module. Prefix matches are supported | `ps.modules['crypt'].size > 1024` | -| ps.parent.name | Parent process name | `ps.parent.name = 'powershell.exe'` | -| ps.parent.pid | Parent process identifier | `ps.parent.pid = 2340` | -| ps.parent.cmdline| Parent process command line | `ps.parent.cmdline contains 'attrib'` | -| ps.parent.exe | Full name of the parent process executable | `ps.parent.exe = 'C:\\Windows\\system32\\cmd.exe'` | -| ps.parent.cwd | Parent process current working directory | `ps.parent.cwd = 'C:\\Users\\Default'` | -| ps.parent.sid | Security identifier under which the parent process is run | `ps.parent.sid = 'S-1-5-18'` | -| ps.parent.domain | Parent process domain name | `ps.parent.domain = 'NT AUTHORITY'` | -| ps.parent.username | Parent process user name | `ps.parent.username = 'SYSTEM'` | -| ps.parent.sessionid | Unique identifier for the current session of the parent process | `ps.parent.session = 1` | -| ps.parent.dtb | Parent process directory table base address | `ps.parent.dtb = 'powershell.exe'` | -| ps.parent.envs | Parent process environment variables | `ps.parent.envs in ('PROCESSOR_LEVEL')'` | -| ps.parent.handles | Allocated parent process handles | `ps.parent.handles in ('\\...\\Cor_SxSPublic_IPCBlock')` | -| ps.parent.handle.types | Allocated parent process handles types | `ps.parent.handle.types in ('Key', 'Mutant', 'Section')` | -| ps.ancestor[] | Process ancestry traversing | `ps.ancestor[2].name in ('winword.exe', 'powershell.exe')` | -| ps.child.name | Child process name | `ps.child.name = 'cmd.exe'` | -| ps.child.pid | Child process identifier | `ps.child.id = 6050` | -| ps.child.cmdline | Child process command line | `ps.child.cmdline contains '/k /v'` | -| ps.child.exe | Child process executable full path | `ps.child.exe = 'C:\\Windows\\system32\\cmd.exe'` | -| ps.child.args | Child process command line arguments | `ps.child.args in ('C:\\Windows\\system32\\cmd.exe')` | -| ps.child.sid | Child process security identifier | `ps.child.sid = 'S-1-5-20'` | -| ps.child.sessionid | Child process session identifier | `ps.child.sessionid = 1` | -| ps.child.domain | Child process domain name | `ps.child.domain = 'NT AUTHORITY'` | -| ps.child.username | Child process user name | `ps.child.username = 'SYSTEM'` | -| ps.uuid | Unique process identifier resistant to repetition | `ps.uuid > 10000400` | -| ps.parent.uuid | Unique parent process identifier resistant to repetition | `ps.parent.uuid = 1843450000440` | -| ps.child.uuid | Unique child process identifier resistant to repetition | `ps.child.uuid > 20030000000` | -| ps.child.pe.file.name | Original file name of the child process executable supplied at compile-time | `ps.child.pe.file.name = 'NOTEPAD.EXE'` | -| ps.child.is_wow64 | Indicates if the 32-bit child process is created in 64-bit Windows system | `ps.child.is_wow64` | -| ps.child.is_packaged | Indicates if the child process is packaged with the MSIX technology | `ps.child.is_packaged` | -| ps.child.is_protected | Indicates if the child process is a protected process | `ps.child.is_protected` | -| ps.is_wow64 | Indicates if the process generating the event is a 32-bit child process is created in 64-bit Windows system | `ps.is_wow64` | -| ps.is_packaged | Indicates if the process process generating the event is packaged with the MSIX technology | `ps.is_packaged` | -| ps.is_protected | Indicates if the process generating the event is a protected process | `ps.is_protected` | -| ps.parent.is_wow64 | Indicates if the parent process generating the event is a 32-bit process created in 64-bit Windows system | `ps.parent.is_wow64` | -| ps.parent.is_packaged | Indicates if the parent process generating the event is packaged with the MSIX technology | `ps.parent.is_packaged` | -| ps.parent.is_protected | Indicates if the parent process generating the event is a protected process | `ps.parent.is_protected` | - - -### Thread -| Field Name | Description | Example | -| :--- | :---- | :---: | -| thread.prio | Scheduler priority of the thread | `thread.prio = 5` | -| thread.io.prio | I/O priority hint for scheduling I/O operations | `thread.io.prio = 4` | -| thread.page.prio | Memory page priority hint for memory pages accessed by the thread | `thread.page.prio = 12` | -| thread.kstack.base | Base address of the thread's kernel space stack | `thread.kstack.base = 'a65d800000'` | -| thread.kstack.limit | Limit of the thread's kernel space stack | `thread.kstack.limit = 'a85d800000'` | -| thread.ustack.base | Base address of the thread's user space stack | `thread.ustack.base = '7ffe0000'` | -| thread.ustack.limit | Limit of the thread's user space stack | `thread.ustack.limit = '8ffe0000'` | -| thread.start_address | Start address of the function to be executed by the thread | `thread.start_address = '7efe0000'` | -| thread.access.mask | Thread access rights | `thread.access.mask = '0x1800'` | -| thread.access.mask.names | Thread access human-readable rights | `thread.access.mask.names in ('QUERY_LIMITED_INFORMATION')` | -| thread.access.status | Thread access status | `thread.access.status = 'Success'` | -| thread.teb_address | The base address of the thread environment block | `thread.teb_address = '8f30893000'` | -| thread.start_address.symbol | Thread start address symbol | `thread.start_address.symbol = 'LoadImage'` | -| thread.start_address.module | Thread start address module | `thread.start_address.module endswith 'kernel32.dll'` | - - -### Threadpool - -| Field Name | Description | Example | -| :--- | :---- | :---: | -| threadpool.id | Thread pool identifier | `threadpool.id = '20f5fc02440'` | -| threadpool.task.id | Thread pool task identifier | `threadpool.task.id = '20f7ecd21f8'` | -| threadpool.callback.address | Thread pool callback address | `threadpool.callback.address = '7ff868739ed0'` | -| threadpool.callback.symbol | Thread pool callback address symbol | `threadpool.callback.symbol = 'RtlDestroyQueryDebugBuffer'` | -| threadpool.callback.module | Thread pool callback address module | `threadpool.callback.module contains 'ntdll.dll'` | -| threadpool.callback.context | Thread pool callback context address | `threadpool.callback.context = '1df41e07bd0'` | -| threadpool.callback.context.rip | Thread pool callback thread context instruction pointer | `threadpool.callback.context.rip = '1df42ffc1f8'` | -| threadpool.callback.context.rip.symbol | Thread pool callback thread context instruction pointer symbol | `threadpool.callback.context.rip.symbol = 'VirtualProtect'` | -| threadpool.callback.context.rip.module | Thread pool callback thread context instruction pointer module | `threadpool.callback.context.rip.module contains 'ntdll.dll'` | -| threadpool.subprocess_tag | Thread pool service identifier | `threadpool.subprocess_tag = '10d'` | -| threadpool.timer.duetime | Thread pool timer due time | `threadpool.timer.duetime > 10` | -| threadpool.timer.subqueue | Thread pool timer subqueue address | `threadpool.timer.subqueue = '1db401703e8'` | -| threadpool.timer.address | Thread pool timer address | `threadpool.timer.address = '3e8'` | -| threadpool.timer.period | Thread pool timer period | `threadpool.timer.period = 0` | -| threadpool.timer.window | Thread pool timer tolerate period | `threadpool.timer.window = 0` | -| threadpool.timer.is_absolute | Indicates if the thread pool timer is absolute or relative | `threadpool.timer.is_absolute = true` | - - -### Callstack -| Field Name | Description | Example | -| :--- | :---- | :---: | -| thread.callstack.summary | Callstack summary showing involved modules | `thread.callstack.summary contains 'ntdll.dll\|KERNELBASE.dll'` | -| thread.callstack.detail | Detailed information of each stack frame | `thread.callstack.detail contains 'KERNELBASE.dll!CreateProcessW'` | -| thread.callstack.modules | List of modules comprising the callstack | `thread.callstack.modules in ('C:\WINDOWS\System32\KERNELBASE.dll')` | -| thread.callstack.symbols | List of symbols comprising the callstack | `thread.callstack.symbols in ('ntdll.dll!NtCreateProcess')` | -| thread.callstack.allocation_sizes | Allocation sizes of private pages | `thread.callstack.allocation_sizes > 10000` | -| thread.callstack.protections | Page protections masks of each frame | `thread.callstack.protections in ('RWX', 'WX')'` | -| thread.callstack.callsite_leading_assembly | Callsite leading assembly instructions | `thread.callstack.callsite_leading_assembly in ('mov r10,rcx', 'syscall')` | -| thread.callstack.callsite_trailing_assembly | Callsite trailing assembly instructions | `thread.callstack.callsite_trailing_assembly in ('add esp, 0xab')` | -| thread.callstack.is_unbacked | Indicates if the callstack contains unbacked regions | `thread.callstack.is_unbacked` | -| thread.callstack.addresses | List of all callstack return addresses | `thread.callstack.addresses in ('7ffb5c1d0396')` | -| thread.callstack.final_user_module.name | The final user module name | `thread.callstack.final_user_module.name != 'ntdll.dll'` | -| thread.callstack.final_user_module.path | The final user module path | `thread.callstack.final_user_module.path imatches '?:\\Windows\\System32\\ntdll.dll'` | -| thread.callstack.final_user_symbol.name | The final user symbol name | `thread.callstack.final_user_symbol.name imatches 'CreateProcess*'` | -| thread.callstack.final_kernel_module.name | The final kernel module name | `thread.callstack.final_kernel_module.name = 'FLTMGR.SYS'` | -| thread.callstack.final_kernel_module.path | The final kernel module path | `thread.callstack.final_kernel_module.path imatches '?:\\WINDOWS\\System32\\drivers\\FLTMGR.SYS'` | -| thread.callstack.final_kernel_symbol.name | The final kernel symbol name | `thread.callstack.final_kernel_symbol.name = 'FltGetStreamContext'` | -| thread.callstack.final_user_module.signature.is_signed | Indicates if the final user module is signed | `thread.callstack.final_user_module.signature.is_signed = true` | -| thread.callstack.final_user_module.signature.is_trusted | Indicates if the final user module signature is trusted | `thread.callstack.final_user_module.signature.is_trusted = true` | -| thread.callstack.final_user_module.signature.cert.issuer | The final user module signature certificate issuer | `thread.callstack.final_user_module.signature.cert.issuer imatches '*Microsoft Corporation*'` | -| thread.callstack.final_user_module.signature.cert.subject | The final user module signature certificate subject | `thread.callstack.final_user_module.signature.cert.subject imatches '*Microsoft Windows*'` | - -### Image -| Field Name | Description | Example | -| :--- | :---- | :---: | -| image.path | Full image path | `image.name = 'C:\\Windows\\System32\\advapi32.dll'` | -| image.name | Image name | `image.name = 'advapi32.dll'` | -| image.base.address | Base address of the process in which the image is loaded | `image.base.address = 'a65d800000'` | -| image.checksum | Image checksum | `image.checksum = 746424` | -| image.size | Image size | `image.size > 1024` | -| image.default.address | Default image address | `image.default.address = '7efe0000'` | -| image.signature.type | Image signature type | `image.signature.type != 'NONE'` | -| image.signature.level | Image signature level | `image.signature.level = 'AUTHENTICODE'` | -| image.cert.serial | Image certificate serial number | `image.cert.serial = '330000023241fb59996dcc4dff000000000232'` | -| image.cert.subject | Image certificate subject | `image.cert.subject contains 'Washington, Redmond, Microsoft Corporation'` | -| image.cert.issuer | Image certificate CA | `image.cert.issuer contains 'US, Washington, Redmond, Microsoft Windows Production PCA 2011` | -| image.cert.after | Image certificate expiration date | `image.cert.after contains '2024-02-01 00:05:42 +0000 UTC'` | -| image.cert.before | Image certificate enrollment date | `image.cert.before contains '2024-02-01 00:05:42 +0000 UTC'` | -| image.is_driver_malicious | Indicates if the loaded driver is malicious | `image.is_driver_malicious` | -| image.is_driver_vulnerable | Indicates if the loaded driver is vulnerable | `image.is_driver_vulnerable` | -| image.is_dll | Indicates if the loaded image is a DLL | `image.is_dll` | -| image.is_driver | Indicates if the loaded image is a driver | `image.is_driver` | -| image.is_exec | Indicates if the loaded image is an executable | `image.is_exec` | -| image.is_dotnet | Indicates if the loaded image is a .NET assembly | `image.is_dotnet` | - -### File -| Field Name | Description | Example | -| :--- | :---- | :---: | -| file.object | File object address in the kernel space | `file.object = 18446738026482168384` | -| file.path | Full file path | `file.name = 'C:\\Windows\\Sytem32\\regedit.exe'` | -| file.name | File name | `file.name = 'regedit.exe'` | -| file.operation | Operation performed on the file or I/O device | `file.operation = 'OPEN'` | -| file.share.mask | File share mask | `file.share.mask = 'READ'` | -| file.io.size | I/O read/write size | `file.io.size > 512` | -| file.offset | Read/write position in the file | `file.offset = 1024` | -| file.type | File type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown` | `file.type = 'Directory'` | -| file.extension | File extension represents the file extension (e.g. .exe or .dll) | `file.extension = '.dll'` | -| file.attributes | List of file attributes | `file.attributes in ('HIDDEN', 'TEMPORARY')` | -| file.status | System status message of the `CreateFile` operation | `file.status = 'Success'` | -| file.view.base | Base address of the mapped/unmapped section view | `file.view.base = '25d42170000'` | -| file.view.size | Size of the mapped/unmapped section view | `file.view.size > 1024` | -| file.view.type | Type of the mapped/unmapped section view | `file.view.type = 'IMAGE'` | -| file.view.protection | Protection rights of the section view | `file.view.protection = 'READONLY'` | -| file.is_driver_malicious | Indicates if the dropped driver is malicious | `file.is_driver_malicious` | -| file.is_driver_vulnerable | Indicates if the dropped driver is vulnerable | `file.is_driver_vulnerable` | -| file.is_dll | Indicates if the created file is a DLL | `file.is_dll` | -| file.is_driver | Indicates if the created file is a driver | `file.is_driver` | -| file.is_exec | Indicates if the created file is an executable | `file.is_exec` | -| file.info_class | Identifies the file information class | `file.info_class = 'Allocation'` | -| file.info.allocation_size | Represents the file allocation size set via `NtSetInformationFile` syscall | `file.info.allocation_size > 645400` | -| file.info.eof_size | Represents the file EOF size set via `NtSetInformationFile` syscall | `file.info.eof_size > 1000` | -| file.info.is_disposition_file_delete | Indicates if the file is deleted when its handle is closed | `file.info.is_disposition_file_delete = true` | - - -### Registry -| Field Name | Description | Example | -| :--- | :---- | :---: | -| registry.path | Fully qualified registry path | `registry.path = 'HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services'` | -| registry.key.name | Base registry key name | `registry.key.name = 'Services'` | -| registry.key.handle | Registry key object address | `registry.key.handle = 'FFFFB905D60C2268'` | -| registry.value | Registry value content | `registry.value = '%SystemRoot%\\system32'` | -| registry.value.type | Registry value type | `registry.value.type = 'REG_SZ'` | -| registry.status | Registry operation status | `registry.status != 'Success'` | - -### Network -| Field Name | Description | Example | -| :--- | :---- | :---: | -| net.dip | Destination IP address | `net.dip = 172.17.0.3` | -| net.sip | Source IP address | `net.sip = 127.0.0.1` | -| net.dport | Destination port | `net.dport in (80, 443, 8080)` | -| net.sport |Source port | `net.sport != 3306` | -| net.dport.name | Destination port name as per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service assignation | `net.dport.name = 'dns'` | -| net.sport.name | Source port name as per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service assignation | `net.sport.name = 'http'` | -| net.l4.proto | Layer 4 protocol name | `net.l4.proto = 'TCP'` | -| net.size | Network packet size | `net.size > 512` | -| net.dip.names | List of destination IP address domain names | `net.dip.names in ('github.com.')` | -| net.sip.names | List of source IP address domain names | `net.sip.names in ('github.com.')` | - - -### Handle -| Field Name | Description | Example | -| :--- | :---- | :---: | -| handle.id | Handle identifier | `handle.id = 24` | -| handle.object | Handle kernel object address | `handle.object = 'FFFFB905DBF61988'` | -| handle.name | Handle name | `handle.name = '\\Device\\NamedPipe\\chrome.12644.28.105826381'` | -| handle.type | Handle type | `handle.type = 'Mutant'` | - - -### Memory -| Field Name | Description | Example | -| :--- | :---- | :---: | -| mem.address | Base address of the allocated region | `mem.address = '211d13f2000'` | -| mem.size | Size of the allocated region | `mem.size > 438272` | -| mem.alloc | Region allocation or release type | `mem.alloc = 'COMMIT'` | -| mem.type | Designates the page type of the allocated region | `mem.type = 'PRIVATE'` | -| mem.protection | Designates the protection type of the allocated region | `mem.protection = 'READWRITE'` | -| mem.protection.mask | Designates the allocated region protection in mask notation | `mem.protection.mask = 'RWX'` | - - -### DNS -| Field Name | Description | Example | -| :--- | :---- | :---: | -| dns.name | DNS query name | `dns.name = 'example.org'` | -| dns.rr | DNS resource record type | `dns.rr = 'AA'` | -| dns.options | DNS query options | `dns.options in ('ADDRCONFIG', 'DUAL_ADDR')` | -| dns.rcode | DNS response status | `dns.rcode = 'NXDOMAIN'` | -| dns.answers | DNS response answers | `dns.answers in ('o.lencr.edgesuite.net', 'a1887.dscq.akamai.net')` | - - -### PE -| Field Name | Description | Example | -| :--- | :---- | :---: | -| pe.nsections | Number of sections | `pe.nsections < 5` | -| pe.nsymbols | Number of entries in the symbol table | `pe.nsymbols > 230` | -| pe.address.base | Image base address | `pe.address.base = '140000000'` | -| pe.address.entrypoint | Address of the entrypoint function | `pe.address.entrypoint = '20110'` | -| pe.sections[].entropy | Specified section entropy | `pe.sections[.text].entropy > 6.2` | -| pe.sections[].size | Size in bytes of the specified section | `pe.sections[.text].size > 56000` | -| pe.sections[].md5 | MD5 hash of the specified section | `pe.sections[.text].md5 = '0464997eb36c70083164c666d53c6af3'` | -| pe.symbols | Imported symbols | `pe.symbols in ('GetTextFaceW', 'GetProcessHeap')` | -| pe.imports | Imported dynamic linked libraries | `pe.imports in ('msvcrt.dll', 'GDI32.dll')` | -| pe.resources | Version and other PE resources | `pe.resources[FileDescription] = 'Notepad'` | -| pe.company | Internal company name of the file provided at compile-time | `pe.company = 'Microsoft Corporation'` | -| pe.copyright | Copyright notice for the file emitted at compile-time | `pe.company = '© Microsoft Corporation'` | -| pe.description | Internal description of the file provided at compile-time | `pe.description = 'Notepad'` | -| pe.file.name | Original file name supplied at compile-time | `pe.file.name = 'NOTEPAD.EXE'` | -| pe.file.version | File version supplied at compile-time | `pe.file.version = '10.0.18362.693 (WinBuild.160101.0800)'` | -| pe.product | Internal product name of the file provided at compile-time | `pe.product = 'Microsoft® Windows® Operating System'` | -| pe.product.name | Internal product version of the file provided at compile-time | `pe.product.version = '10.0.18362.693'` | -| pe.is_dll | Indicates if the loaded image or a created file is a DLL | `pe.is_dll` | -| pe.is_driver | Indicates if the loaded image or a created file is a driver | `pe.is_driver` | -| pe.is_exec | Indicates if the loaded image or a created file is an executable | `pe.is_exec` | -| pe.is_dotnet | Indicates if the PE contains CLR (Common Language Runtime) data | `pe.is_dotnet` | -| pe.is_signed | Indicates if the PE has embedded or catalog signature | `pe.is_signed` | -| pe.is_trusted | Indicates if the PE certificate chain is trusted | `pe.is_trusted` | -| pe.imphash | Import hash | `pe.impash = '5d3861c5c547f8a34e471ba273a732b2'` | -| pe.anomalies | Contains PE anomalies detected during parsing | `pe.anomalies in ('number of sections is 0')` | -| pe.cert.serial | PE certificate serial number | `pe.cert.serial = '330000023241fb59996dcc4dff000000000232'` | -| pe.cert.subject | PE certificate subject | `pe.cert.subject contains 'Washington, Redmond, Microsoft Corporation'` | -| pe.cert.issuer | PE certificate CA | `pe.cert.issuer contains 'US, Washington, Redmond, Microsoft Windows Production PCA 2011'` | -| pe.cert.after | PE certificate expiration date | `pe.cert.after contains '2024-02-01 00:05:42 +0000 UTC'` | -| pe.cert.before | PE certificate enrollment date | `pe.cert.before contains '2024-02-01 00:05:42 +0000 UTC'` | -| pe.is_modified | Indicates if on-disk and in-memory PE headers differ | `pe.is_modified'` | -| pe.is_modified | Indicates if on-disk and in-memory PE headers differ | `pe.is_modified'` | diff --git a/docs/filters/filtering.md b/docs/filters/filtering.md deleted file mode 100755 index 63975cc89..000000000 --- a/docs/filters/filtering.md +++ /dev/null @@ -1,95 +0,0 @@ -# Filtering - -### Anatomy of a filter {docsify-ignore} - -At its simplest form, a filter is composed of the LHS (Left Hand Side) and RHS (Right Hand Side) expressions connected with an [operator](/filters/operators) that can pertain to binary, logical, or string operators. The LHS expression is usually a [field](/filters/fields) [path](/filters/paths), even though the use of fields is possible in Right Hand Side expressions as well. - -``` - LHS operator RHS -kevt.category = 'net' -``` - -Boolean fields, that is the fields that always evaluate to either `true` or `false` can appear alone in the filter expression. For example, `pe.is_dll` is the short form of `pe.is_dll = true` expression. - -The RHS expressions can be strings, numbers, IP addresses, boolean values, and fields. In the above snippet, the RHS is a simple string literal, but we could have written filters with the following RHS expressions: - -- IP addresses: `net.sip = 127.0.0.1` -- Booleans: `kevt.nparams != false` -- Numbers: `ps.pid = 4` -- Fields: `kevt.pid != ps.pid` - -### Running filters {docsify-ignore} - -As previously mentioned, filters can be engaged in various stages of event collection and processing. The filter expression is given to `run`, `capture`, and `replay` commands in form of the command line argument. - -The `run` command applies the filter expression to each inbound event and prevents the routing the the output sink if the event doesn't match the filter. The above command filters events that occur on `Monday` and are produced by `cmd.exe` and `svchost.exe` processes. - -``` -$ fibratus run --forward kevt.date.weekday = 'Monday' and ps.name in ('cmd.exe', 'svchost.exe') -``` - -!> If you're using PowerShell, it is necessary to enclose the entire filter expression in quotes. For example, `fibratus run --forward "kevt.category = 'net'"` - - -In a similar fashion, the `capture` command only dumps events that match the provided filter. In this case, the capture would boil down to `registry` events. - -``` -$ fibratus capture kevt.category = 'registry' -o events -``` - -When replaying events from the kcap file, you can also specify a filter to narrow down the replay context, for example, to filter out events that mutate registry values. - -``` -$ fibratus replay kevt.name = 'RegSetValue' -k events -``` - -Lastly, filtering is possible during filament execution. If the filter is set in both, the `run` command and through the `kfilter` function, the latter takes precedence. Filtering in filaments is thoroughly explained in [filaments](/filaments/introduction). - -### Escaping characters {docsify-ignore} - -As you might have noticed, string values are enclosed in single quotes `''`. If the string contains characters that would result in an illegal identifier, you'll have to escape the offending characters accordingly. For example, path delimiters (backslashes) or quotes need to be escaped: - -``` -$ fibratus run --forward file.name = 'C:\\Windows\\System32' -$ fibratus run --forward file.name contains '\"hosts\"' -``` - -Filter expressions can accept escape sequences, such as newline characters (`\n`). - -### Invalid filters {docsify-ignore} - -If a syntax error is present in the filter, a hint is given indicating the erroneous position in the expression. - -``` -kevt.name in ('RegCreateKey', 'RegDeleteKey', 'RegSetValue', 'RegDeleteValue') - and - registry.key.name icontains - ( - 'CurrentVersion\\Run', - 'Windows\\System\\Scripts', - 'CurrentVersion\\Windows\\Load', - 'CurrentVersion\\Windows\\Run', - 'CurrentVersion\\Winlogon\\Shell', - 'CurrentVersion\\Winlogon\\System', - 'UserInitMprLogonScript' - ) - or - registry.key.name istartswith - ( - 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon\\Notify', - 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon\\Shell', - 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon\\Userinit', - 'HKEY_LOCAL_MACHINE\\Software\\WOW6432Node\\Microsoft\\Windows NT\\CurrentVersion\\Drivers32', - HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\Session Manager\\BootExecute', -╭──────────^ -| 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows NT\\CurrentVersion\\AeDebug' -| ) -| or -| registry.key.name iendswith -| ( -| 'user shell folders\\startup' -| ) -| -| -╰─────────────────── expected field, string, number, bool, ip, function, pattern binding -``` diff --git a/docs/filters/functions.md b/docs/filters/functions.md deleted file mode 100644 index ca551a39d..000000000 --- a/docs/filters/functions.md +++ /dev/null @@ -1,494 +0,0 @@ -# Functions - -Functions expand the scope of the filtering language by bringing a plethora of capabilities. The function can return a primitive value, including integers, strings, and booleans. Function calls can be nested where the result of one function is used as an input in another function. For example, `lower(ltrim(file.name, 'C:'))`, removes the `C` drive letter from the file path and converts it to a lower case string. - -Additionally, some functions may return a collection of values. Function names are case insensitive. - -### Network functions - -#### cidr_contains - -`cidr_contains` determines if the specified IP is contained within the block referenced by the given CIDR mask. The first argument represents the IP address and the subsequent arguments are IP masks in CIDR notation. - -- **Specification** - ``` - cidr_contains(ip: , cidrs: ...) :: - ``` - - `ip`: The IP address in v4/v6 notation - - `cidrs`: The list of CIDR masks - - `return` a boolean value indicating whether the IP pertains to the CIDR block - -- **Examples** - - Assuming `net.sip` contains the `192.168.1.20` IP address - - ``` - cidr_contains(net.sip, '192.168.1.1/24', '172.17.1.1/8') = true - ``` - -### Hash functions - -#### md5 - -`md5` computes the MD5 hash of the given value. - -- **Specification** - ``` - md5(data: ) :: - ``` - - `data`: The string or the byte array for which to calculate the hash - - `return` a string representing the md5 hash - -- **Examples** - - Assuming `registry.key.name` contains the `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid` key. - - ``` - md5(registry.key.name) = 'eab870b2a516206575d2ffa2b98d8af5' - ``` - -### String functions - -#### concat - -`concat` concatenates string/integer input arguments. - -- **Specification** - ``` - concat(args: ...) :: - ``` - - `args`: Strings or integers to be concatenated. This function requires at least 2 input arguments - - `return` a concatenated string of all input arguments - -- **Examples** - - Assuming `ps.domain` field contains `NT_AUTHORITY` and `ps.username` field contains `admin`. - - ``` - concat(ps.domain, '-', ps.username) = 'NT_AUTHORITY-admin' - ``` - -#### ltrim - -`ltrim` trims the specified prefix from a string. - -- **Specification** - ``` - ltrim(string: , prefix: ) :: - ``` - - `string`: Input string - - `prefix`: Prefix that is removed from the original input string - - `return` a string with the specified prefix removed - -- **Examples** - - Assuming `registry.key.name` contains the `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid` key. - - ``` - ltrim(registry.key.name, 'HKEY_LOCAL_MACHINE\\') = 'SYSTEM\\Setup\\Pid' - ``` - -#### rtrim - -`rtrim` trims the specified suffix from a string. - -- **Specification** - ``` - rtrim(string: , suffix: ) :: - ``` - - `string`: Input string - - `prefix`: Suffix that is removed from the original string - - `return` a string with the specified suffix removed - -- **Examples** - - Assuming `registry.key.name` contains the `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid` key. - - ``` - rtrim(registry.key.name, '\\Pid') = 'HKEY_LOCAL_MACHINE\\SYSTEM\\Setup' - ``` - -#### lower - -`lower` converts the string with all Unicode letters mapped to their lower case. - -- **Specification** - ``` - lower(string: ) :: - ``` - - `string`: Input string - - `return` a string converted to lower case - -- **Examples** - - Assuming `registry.key.name` contains the `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid` key. - - ``` - lower(registry.key.name) = 'hkey_local_machine\\system\\setup' - ``` - -#### upper - -`upper` converts the string with all Unicode letters mapped to their upper case. - -- **Specification** - ``` - upper(string: ) :: - ``` - - `string`: Input string - - `return` a string converted to upper case - -- **Examples** - - Assuming `registry.key.name` contains the `HKEY_LOCAL_MACHINE\SYSTEM\Setup` key. - - ``` - upper(registry.key.name) = 'HKEY_LOCAL_MACHINE\\SYSTEM\\SETUP' - ``` - -#### replace - -`replace` replaces all occurrences in the string as given by arbitrary old/new replacement pairs. - -- **Specification** - ``` - replace(string: , old: , new: , ..., old-n: , new-n: ) :: - ``` - - `string`: Input string - - `old`: substring in the original string that is replaced with the `new` string - - `new`: the replacement string - - `return` a string with all occurrences replaced by old/new pairs - -- **Examples** - - Assuming `registry.key.name` contains the `HKEY_LOCAL_MACHINE\SYSTEM\Setup` key. - - ``` - replace(registry.key.name, 'HKEY_LOCAL_MACHINE', 'HKLM', 'SYSTEM', 'SYS') = 'HKLM\\SYS\\Setup' - ``` - -#### split - -`split` produces a slice of substrings separated by the given delimiter. - -- **Specification** - ``` - split(string: , sep: ) :: <[]string> - ``` - - `string`: Input string - - `prefix`: The separator that is used to split the string - - `return` a slice of substrings - -- **Examples** - - Assuming `file.name` contains the `C:\Windows\System32\kernel32.dll` path. - - ``` - split(file.name, '\\') in ('kernel32.dll', 'System32', 'Windows') - ``` - -#### length - -`length` returns the number of characters for string arguments and the size of the slice for slice arguments. - -- **Specification** - ``` - length(string: ) :: - ``` - - `string`: Input string or slice - - `return` the number of characters or array length - -- **Examples** - - Assuming `ps.domain` field contains `"こんにちは"`. - - ``` - length(ps.domain) = 5 - ``` - -#### indexof - -`indexof` returns the index of the instance of substring in a given string depending on the provided search order. - -- **Specification** - ``` - indexof(string: , substring: , order: ) :: - ``` - - `string`: Input string - - `prefix`: Substring for which the search is performed - - `order`: Specifies the string index search order. It can be `first`, `any`, `last`, `lastany`. This is an optional argument. - - `return` the index of the substring - -- **Examples** - - Assuming `ps.domain` contains `NT_AUTHORITY`. - - ``` - indexof(ps.domain, 'NT') = 0 - ``` - -#### substr - -`substr` creates a substring of a given string. - -- **Specification** - ``` - substr(string: , start: , end: ) :: - ``` - - `string`: Input string - - `start`: Substring start index - - `end`: Substring end index (optional) - - `return` a substring contained within start and end indices - -- **Examples** - - Assuming `file.name` contains the `\Device\HarddiskVolume2\Windows\system32\user32.dll` path. - - ``` - substr(file.name, indexof(file.name, '\\'), indexof(file.name, '\\Hard')) = '\\Device' - substr(file.name, indexof(file.name, 'system32')) = 'system32\\user32.dll' - ``` - -#### entropy - -`entropy` measures the string entropy. - -- **Specification** - ``` - entropy(string: , algo: ) :: - ``` - - `string`: Input string - - `algo`: The algorithm used to calculate the string entropy. `shannon` is the default entropy type. This argument is optional - - `return` the string entropy - -- **Examples** - - Assuming `file.name` contains the `\Device\HarddiskVolume2\Windows\system32\user32.dll` path. - - ``` - entropy(file.name) > 255 - ``` - -#### regex - -`regex` applies single/multiple regular expressions on the provided string argument. - -- **Specification** - ``` - regex(string: , patterns: ...) :: - ``` - - `string`: Input string - - `patterns`: Regular expression patterns - - `return` `true` if at least one regular expression matches or `false` otherwise - -- **Examples** - - Assuming `ps.name` contains `powershell.exe`. - - ``` - regex(ps.name, 'power.*(shell|hell).dll', '.*hell.exe') = true - ``` - -### File functions - -#### base - -`base` returns the last element of the path. - -- **Specification** - ``` - base(path: , ext: bool) :: - ``` - - `path`: The string or an array of strings representing file system path(s) - - `ext`: Determines whether the extension is retained in the file path. This parameter is optional and `true` by default - - `return` a string or a slice of strings with file names - -- **Examples** - - Assuming `file.name` contains the `C:\\Windows\\cmd.exe` path. - - ``` - base(file.name) = 'cmd.exe' - base(file.name, false) = 'cmd' - ``` - -#### dir - -`dir` returns all but the last element of the path, typically the path's directory. - -- **Specification** - ``` - dir(path: ) :: - ``` - - `path`: The string or an array of strings representing file system path(s) - - `return` a string or a slice of strings with directory names - -- **Examples** - - Assuming `file.name` contains the `C:\\Windows\\cmd.exe` path. - - ``` - dir(file.name) = 'C:\\Windows' - ``` - -#### ext - -`ext` returns the file name extension used by the path. - -- **Specification** - ``` - ext(path: , dot: bool) :: - ``` - - `path`: The string representing file system path - - `dot`: Indicates if the dot symbol is retained as part of extension. This parameter is optional and `true` by default - - `return` file name extension used by the path - -- **Examples** - - Assuming `file.name` contains the `C:\\Windows\\cmd.exe` path. - - ``` - ext(file.name) = '.exe' - ext(file.name, false) = 'exe' - ``` - -#### glob - -`glob` returns the names of all files matching the pattern. - -- **Specification** - ``` - glob(pattern: ) :: <[]string> - ``` - - `pattern`: Shell file name pattern as described [here](https://pkg.go.dev/path/filepath#Match) - - `return` returns the names of all files matching the pattern or an empty list if there is no matching file - -- **Examples** - - ``` - glob('C:\\Windows\\*.exe') in ('C:\\Windows\\notepad.exe') - ``` - -#### is_abs - -`is_abs` reports whether the path is absolute. - -- **Specification** - ``` - is_abs(path: ) :: bool - ``` - - `path`: The string representing file system path - - `return` `true` if `path` references an absolute path or `false` otherwise - -- **Examples** - - Assuming `file.name` contains the `Windows\\cmd.exe` path. - - ``` - is_abs(file.name) = false - ``` - -#### symlink - -`symlink` returns the path name after the evaluation of any symbolic links. - -- **Specification** - ``` - symlink(path: ) :: string - ``` - - `path`: The string representing file system path - - `return` the path name after the evaluation of any symbolic links, or the original path if any errors occur - -- **Examples** - - Assuming `file.name` contains the `C:\\Windows\\symlink.txt` path which is a symlink to `C:\Windows\target.txt`. - - ``` - symlink('C:\\Windows\\symlink.txt') = 'C:\\Windows\\target.txt' - ``` - -#### volume - -`volume` returns leading volume name. - -- **Specification** - ``` - volume(path: ) :: string - ``` - - `path`: The string representing file system path - - `return` leading volume name - -- **Examples** - - Assuming `file.name` contains the `C:\\Windows\\symlink.txt` path. - - ``` - volume(file.name) = 'C:' - ``` - -#### is_minidump - -`is_minidump` checks the signature of the provided file and returns `true` if the signature matches the `minidump` file. - -- **Specification** - ``` - is_mindump(path: ) :: - ``` - - `string`: The file path for which the minidump signature is checked - - `return` `true` if the file contains the `minidump` signature or `false` otherwise - -- **Examples** - - Assuming `file.name` contains the `C:\\Temp\\lsass.dmp` path with a valid `minidump` file. - - ``` - is_minidump(file.name) = true - ``` - -### Registry functions - -`get_reg_value` retrieves the content of the registry value. - -- **Specification** - ``` - get_reg_value(key: ) :: - ``` - - `key`: Is the fully-qualified registry key path including the value name. The root key can be expressed in abbreviated notation, e.g. instead of `HKEY_LOCAL_MACHINE` you can write `HKLM`. - - `return` depending on the registry value type, it can return a string, array of strings or an integer value. - -- **Examples** - - Assuming the `HKEY_CURRENT_USER\Volatile Environment\Envs` registry value contains a multi-size string with `dev\0staging` values. - - ``` - get_reg_value('HKCU\Volatile Environment\Envs') in ('dev', 'staging') - ``` - - -### YARA functions - -`yara` provides signature-based detection in filters and rules. YARA is a tool aimed at (but not limited to) helping malware -researchers to identify and classify malware samples. With YARA you can create descriptions of malware families based on textual -or binary patterns. Depending on the parameter type supplied to this function, the scan can be performed on the process, filename or a memory block. - -- **Specification** - ``` - yara(target: , rules: ) :: bool - ``` - - `target`: If this parameter is an integer value, it's assumed to be a pid for which the memory area is scanned. If it is a string, the scan is performed on the process image executable or arbitrary file system file. Otherwise, it is a stream of bytes that represents a memory block to be scanned. - - `rules`: a string containing YARA rules - - `return` if any rule defined in the `rules` parameter matches, the function returns `true`. Otherwise, it returns `false`. - -- **Examples** - - Assuming `file.name` contains `C:\\Windows\\notepad.exe`. - - ``` - yara(file.name, 'rule Notepad : notepad -{ - strings: - $c0 = "Notepad" fullword ascii - condition: - $c0 -}') = true - ``` diff --git a/docs/filters/images/rule-alert.png b/docs/filters/images/rule-alert.png deleted file mode 100644 index 32faeac16..000000000 Binary files a/docs/filters/images/rule-alert.png and /dev/null differ diff --git a/docs/filters/introduction.md b/docs/filters/introduction.md deleted file mode 100755 index d18f952d1..000000000 --- a/docs/filters/introduction.md +++ /dev/null @@ -1,46 +0,0 @@ -# Needle In The Haystack - -An overwhelming volume of events makes it challenging to dig out valuable insights or to narrow down the scope of a particular investigation or postmortem scenario. To streamline day-to-day tasks and to propel your workflow, Fibratus delivers a powerful engine for building sophisticated filtering expressions. Let's assume you want to catch events originated from `cmd.exe`, `powershell.exe`, `winword.exe` processes, or processes that export the `WORKER=1` environment variable and whose current working directory contains the `Users\Public` path. We can write such a filter as: - -``` -ps.name in ('cmd.exe', 'powershell.exe', 'winword.exe') - or ps.envs['WORKER'] = '1' - and ps.cwd contains 'Users\\Public' -``` - -It may look intimidating at first glance, but once you get familiar with the syntax and the field names you'll be able to write even the most intricate filters. - -Filters represent the foundation of the [rule engine](/filters/rules) that provides threat detection capabilities. For example, the following stanza detects the outbound communication followed by the execution of the command shell within one-minute time window. The action invokes the [alert sender](/alerts/senders) to emit the security alert via email, Slack, or a -different supported channel. - -```yaml -name: Remote connection followed by command shell execution -id: eddace20-4962-4381-884e-40dcdde66626 -version: 1.0.0 - -condition: > - sequence - maxspan 1m - by ps.uuid - |kevt.name = 'Connect' - and - not - cidr_contains( - net.dip, - '10.0.0.0/8', - '172.16.0.0/12' - ) - | - |kevt.name = 'CreateProcess' - and - ps.child.name in ('cmd.exe', 'powershell.exe') - | - -output: > - Command shell spawned after remote connection %2.ps.exe - process spawned a command shell after connecting to %1.net.dip - -severity: critical - -min-engine-version: 2.0.0 -``` diff --git a/docs/filters/operators.md b/docs/filters/operators.md deleted file mode 100755 index 2d6fc2a83..000000000 --- a/docs/filters/operators.md +++ /dev/null @@ -1,169 +0,0 @@ -# Operators - -The filter engine supports logical, arithmetic, and string operators. Operator names are case-insensitive. - -## Binary operators - -The filtering query language supports the following comparison binary operators: - -- `=` (equal) -- `!=` (not equal) -- `<` (less than) -- `>` (greater than) -- `>=` (greater or equal) -- `<=` (less or equal) -- `~=` (case-insensitive string comparison) - -## Logical operators - -Logical operators are applied on two or more binary expressions, except for `not` that acts as a unary operator. - -### or - -`or` (union) evalutes to true if either one of the LHS (Left Hand Side) or RHS (Right Hand Side) expressions are true. - -- **Example** - - Filter events where the originating process name is equal to `svchost.exe` or the process name contains the `svc` string - - ``` - fibratus run ps.name = 'svchost.exe' or ps.name contains ('svc') - ``` - -### and - -`and` (intersection) evalutes to true if both of the LHS (Left Hand Side) and RHS (Right Hand Side) expressions are true. - -- **Example** - - Filter events only when the originating process name is equal to `System` and the process identifier is equal to `4` - - ``` - fibratus run ps.name = 'System' and ps.pid = 4 - ``` - -### not - -`not` (negation) negates the result of the adjacent expression. - -- **Example** - - Filter events only when the originating process name is equal to `svchost.exe` and none of the process' command line arguments is equal to `/-C` or `/cdir` - - ``` - fibratus run ps.name = 'svchost.exe' and ps.args not in ('/-C', '/cdir') - ``` - -## String operators - -String operators are applied to string field types or string literals. - -### in, iin - -`in` operator validates the presence of a value in the string sequence. It can be applied to string literal sequences or dynamic string slices given by filter fields. `iin` is the case-insensitive variant of the `in` operator. - -- **Examples** - - Tests if the process name producing the event is either `cmd.exe` or `powershell.exe` - - ``` - $ fibratus run ps.name in ('cmd.exe', 'powershell.exe') - ``` - - Checks if any of the process modules contains the `kernel32` dynamic linked object - - ``` - $ fibratus run ps.modules in ('kernel32.dll') - ``` - -### contains, icontains - -`contains` operator checks whether a string field contains a sequence of characters. This operator works on both simple string values and lists of strings. `icontains` is the case-insensitive variant of the `contains` operator. - -- **Examples** - - Checks if the process' name contains the `cmd` or `power` substrings - - ``` - $ fibratus run ps.name contains ('cmd', 'power') - ``` - - Checks if the process' command line contains the `Windows Tasks` substring - - ``` - $ fibratus run ps.comm contains 'Windows Tasks' - ``` - - -### startswith, istartswith - -`startswith` checks whether a string field starts with a specified prefix. This operator works on both simple string values and lists of strings. `istartswith` is the case-insensitive variant of the `startswith` operator. - -- **Example** - - Filter events where the originating process name is equal to `svchost.exe` - - ``` - fibratus run ps.name startswith 'svchost' - ``` - - -### endswith, iendswith - -`endswith` checks whether a string field ends with a specified suffix. This operator works on both simple string values and lists of strings. `iendswith` is the case-insensitive variant of the `endswith` operator. - -- **Example** - - Filter events where the originating process name is equal to `svchost.exe` - - ``` - fibratus run ps.name endswith '.exe' - ``` - -### intersects, iintersects - -`intersects` operator and its case-insensitive `iintersects` variant operate on string slices. If all elements in the RHS slice are present in the slice given by LHS, the operator evaluates to `true`. Otherwise, it evaluates to `false`. - -- **Example** - - Filter events where the originating process command line arguments contain both `DcomLaunch` and `LSM` arguments - - ``` - fibratus run ps.args intersects ('DcomLaunch', 'LSM') - ``` - -### matches, imatches - -`matches` is the swiss army knife string matching operator. It allows string matching by using the wildcard characters similar to [globbing](https://en.wikipedia.org/wiki/Glob_(programming)). The `*` wildcard matches a sequence of characters, while the `?` wildcard matches a single character. `imatches` is the case-insensitive variant of the `matches` operator. - -- **Examples** - - To match events with file paths equal to `C:\\Windows\\System32\\lsass2.dmp` or `C:\\ProgramData\\Directory\\tmp\\anubis.dll` - - ``` - fibratus run file.name matches ('C:\\*\\lsass?.dmp', 'C:\\ProgramData\\*.dll') - ``` - - For filtering registry events with key names such as `HKEY_USERS\\S-1-5-21-2271034452-2606270099-984871569-1001\\Environment\\windir` - - ``` - fibratus run registry.key.name matches 'HKEY_USERS\\*\\Environment\\windir' - ``` - -### fuzzy, ifuzzy, fuzzynorm, ifuzzynorm - -`fuzzy` operators allow for flexibly matching a string with partial input based on [fuzzy matching](https://en.wikipedia.org/wiki/Fuzzy_matching_(computer-assisted_translation)) techniques. `fuzzynorm` applies Unicode normalization before running the matching phase. `ifuzzy` and `ifuzzynorm` are the case-insensitive variants of the respective fuzzy operators. - -- **Examples** - - To match events with file paths that contain `C:\\Windows\\System32\\user32.dll`, you could create an expression with the partial path input - - ``` - fibratus run file.name fuzzy 'C:\\Windows\\Sys\\ser3ll' - ``` - - `fuzzynorm` operates on normalized Unicode codepoints - - ``` - fibratus run file.name fuzzynorm 'C:\\Windows\\Sys\\sér3ll' - ``` \ No newline at end of file diff --git a/docs/filters/prefiltering.md b/docs/filters/prefiltering.md deleted file mode 100755 index 29e26d430..000000000 --- a/docs/filters/prefiltering.md +++ /dev/null @@ -1,23 +0,0 @@ -# Prefiltering - -Sometimes it is useful to drop certain events either by image (process) name or event type once the event is peeked from the tracing buffer. Besides this, the kernel stream consumer can be configured to ignore events at the `ETW` session level. This can drastically reduce the impact on the system load if you're not interested in events that may produce an immense volume of data. - -The above is the summary of configuration options that influence the collection of events. These options are placed in the `kstream` section of the configuration file. - -- `enable-thread` enables/disables the collection of the thread-related kernel events -- `enable-registry` enables/disables the collection of registry kernel events -- `enable-net` enables/disables the collection of network kernel events -- `enable-fileio` enables/disables the collection of the file system events -- `enable-image` enables/disables the collection of image loading/unloading events -- `enable-handle` enables/disables the collection of handle events -- `enable-audit-api` enables/disables kernel audit API calls events -- `enable-mem` enables/disables the collection of memory events -- `enable-dns` enables/disables DNS telemetry -- `enable-threadpool` enables/disables thread pool telemetry - -### Excluding processes or events {docsify-ignore} - -If you want to permanently exclude specific events or processes that produce them from the event flow, you can achieve this by defining the blacklist in the `kstream.blacklist` configuration section: - -- `events` contains a list of event names that are dropped from the event stream. -- `images` contains a list of case-sensitive process image names including the extension. Any event originated by the image specified in this list is dropped from the event stream. diff --git a/docs/filters/rules.md b/docs/filters/rules.md deleted file mode 100644 index 910593b5c..000000000 --- a/docs/filters/rules.md +++ /dev/null @@ -1,361 +0,0 @@ -# Rules - -Rules bring a set of capabilities for detecting and disrupting the adversary kill chain exploiting stealthy attacks and advanced threat models. Fibratus comes equipped with a decent catalog of [detection rules](https://github.com/rabbitstack/fibratus/tree/master/rules) built on top of the [MITRE ATT&CK](https://attack.mitre.org/) framework with each rule mapped to the most relevant tactics, techniques, and sub-techniques. - -Detection rules are organized in `yaml` files. Rules can live in different directories and there is no strict naming convention. However, the recommended practice is to follow the `tactic_name_rule_name.yml` naming nomenclature. Analysts are encouraged to follow design guidelines to produce optimal rules. - -#### Stick to naming nomenclature {docsify-ignore} - -It is highly recommended to name the rule files after the pattern explained in the above section. This facilitates the organization and searching through the detection rules catalog and fosters standardization. - -#### Include descriptions and labels {docsify-ignore} - -Rules should have a meaningful description. -For example, `Potential process injection via tainted memory section`. - -Additionally, there should exist labels attached to every rule describing the MITRE tactic, technique, and sub-technique. This information is used when rendering email rule alert templates as depicted in the image above. - -#### Rules should have a narrowed event scope {docsify-ignore} - -If a rule is declared without the scoped event conditions, such as `kevt.name` or `kevt.category`, you'll get a warning message in `Fibratus` logs informing you about unwanted side effects. **This always lead to the rule being utterly discarded by the engine!** - -#### Pay attention to the condition arrangement {docsify-ignore} - -As highlighted in the previous paragraph, all rules should have the event type condition. Additionally, condition arrangement may have important runtime performance impact because the rule engine can lazily evaluate binary expressions that comprise a rule. In general, costly evaluations or functions such as `get_reg_value` should go last to make sure they are evaluated after all other expressions have been visited. - -#### Prefer macros over raw conditions {docsify-ignore} - -Fibratus comes with a [macros](https://www.fibratus.io/#/filters/rules?id=macros) library to promote the reusability and modularization of rule conditions and lists. Before trying to spell out a raw rule condition, explore the library to check if there's already a macro you can pull into the rule. For example, detecting file accesses could be accomplished by declaring the `kevt.name = 'CreateFile' and file.operation = 'open'` expression. However, the macro library comes with the `open_file` macro that you can directly call in any rule. If you can't encounter a particular macro in the library, please consider creating it. Future detection engineers and rule writers could profit from those macros. - -#### Formatting styles {docsify-ignore} - -Pay attention to rule condition/action formatting style. If the rule consists of multiple or large expressions, it is desirable to split each spanning expression on a new line and properly indent the `and`, `or`, or `not` operators. By default, we use 1 space tab for indenting operators and rule actions. This notably improves readability and prevents formatting inconsistencies. - -### Loading rules - -By default, rules reside inside `%PROGRAM FILES%\Fibratus\Rules` directory. You can load any number of rule files from file system paths or URL locations. It is possible to utilize the wildcard expressions in file paths to enumerate multiple rule files from a single path specifier. - -Edit the main [configuration](/setup/configuration?id=files) file. Go to the `filters` section where the following `yaml` fragment is located: - -```yaml -filters: - rules: - from-paths: - - C:\Program Files\Fibratus\Rules\*.yml - from-urls: - - https://raw.githubusercontent.com/rabbitstack/fibratus/master/rules/credential_access_unusual_access_to_windows_credential_history.yml -``` - -- `from-paths` represents an array of file system paths pointing to the rule definition files -- `from-urls` is an array of URL resources that serve the rule definitions - -### Creating rules - -Let's have a glimpse at an example of a simple rule definition described in `yaml` format. When creating a new rule, use the `fibratus rules create` CLI command. It will create a `yaml` template with some required fields populated automatically. Run `fibratus rules create -h` to get extended help on this command. - -```yaml -name: Unusual access to Windows Credential history files -id: 9d94062f-2cf3-407c-bd65-4072fe4b167f -version: 1.0.0 -description: | - Detects unusual accesses to the Windows Credential history file. - The CREDHIST file contains all previous password-linked master key hashes used by - DPAPI to protect secrets on the device. Adversaries may obtain credentials - from the Windows Credentials Manager. -enabled: true -labels: - tactic.id: TA0006 - tactic.name: Credential Access - tactic.ref: https://attack.mitre.org/tactics/TA0006/ - technique.id: T1555 - technique.name: Credentials from Password Stores - technique.ref: https://attack.mitre.org/techniques/T1555/ - subtechnique.id: T1555.004 - subtechnique.name: Windows Credential Manager - subtechnique.ref: https://attack.mitre.org/techniques/T1555/004/ -tags: - - credential stealing -references: - - https://www.passcape.com/windows_password_recovery_dpapi_credhist - -condition: > - open_file - and - file.name imatches '?:\\Users\\*\\AppData\\*\\Microsoft\\Protect\\CREDHIST' - and - not - ps.exe imatches - ( - '?:\\Program Files\\*', - '?:\\Windows\\System32\\lsass.exe', - '?:\\Windows\\System32\\svchost.exe', - '?:\\Windows\\ccmcache\\*.exe' - ) - -output: | - Detected an attempt by `%ps.name` process to access and read - Windows Credential history file at `%file.name` -severity: critical - -action: - - name: kill - -min-engine-version: 2.0.0 - -notes: | - Anything else relevant for the analyst. -``` - -1. `name` (required). Short title to emphasize rule's intent. The rule name is primary message used in security alerts. -2. `id` (required). Anchors a unique identifier to the rule that never changes, while the name could change over time. -3. `version` (required). Describes the rule version. The version is useful to track regressions or breaking changes. -4. `description` (optional). A larger explanation of what the rule should detect. Provide as much context as possible to the analyst. -5. `enabled` (optional). Sometimes it comes in handy to temporarily disable a specific rule. If omitted, the default value is `true`. -6. `labels` (optional). Arbitrary key/value pairs. As per [best practices](https://github.com/rabbitstack/fibratus/tree/master/rules#guidelines) rule design guidelines, it is highly recommended to include labels for the MITRE tactic, technique/sub-technique. Of course, you are free to populate the labels attribute with any other useful data. -7. `tags` (optional). Unlike labels, tags represent a sequence of meaningful keywords that you may find useful for categorization purposes. -8. `references` (optional). List of web resources, documents, etc. with relevant information about adversary tactics, defensive strategy, and so on. -9. `condition` (required). Well-known filter expression we used to employ in the CLI filters. You might have spotted some additional constructs like `open_file`. They are called macros, and we'll dive into macros goodies in the next section. -11. `output` (optional) Provides additional context in the alert description. It is possible to use fields interpolation, as seen later. -12. `severity` (optional) Sets the alert severity. Possible values are `low`, `medium`, `high`, and `critical`. If omitted, `medium` severity is assumed. -13. `action` (optional) Rule action can perform a variety of operations, such killing a process involved in the matched rule condition. -14. `min-engine-version` (required) Identifies the minimum Fibratus version that is compatible with the rule. -15. `notes` (optional). Any notes or observations that you would like to communicate to the analyst. - -#### Macros - -Macros foment rule patterns reusability and a human-friendly domain-specific language (DSL). A vast majority of detection rules may require conditions to express process execution or file writes. Traditionally, one could spell out a raw filter expression such as `kevt.name = 'CreateProcess'`. This may lead to bloated and boilerplate rules. From the maintenance standpoint, introducing a small change in the rule condition would force us to update all the rules, while macros are a much more convenient mechanism for declaring reusable rule patterns. Fibratus ships with a [macros library](https://github.com/rabbitstack/fibratus/blob/master/rules/macros/macros.yml) containing a dozen of different macros ready to use. Macros library is loaded from the file system and can be split across multiple `yaml` files. The default location is designated by the `%PROGRAM FILES%\Fibratus\Rules\Macros` directory. - -```yaml -filters: - macros: - from-paths: - - C:\Program Files\Fibratus\Rules\Macros\*.yml -``` - -Macros come in two flavors: - -- **expression** macros encapsulate filter expressions. A macro declaration requires a unique macro name, the filter expression, and an optional description. - -```yaml -- macro: spawn_process - expr: kevt.name = 'CreateProcess' - description: Identifies the execution of a new process -``` - -Where macro expressions really shine is when combined with other macros to forge recursive macros. For example, the `spawn_msoffice_process` is composed of `spawn_process` expression macro and the `msoffice_binaries` list macro. - -```yaml -- macro: spawn_msoffice_process - expr: spawn_process and ps.sibling.exe iin msoffice_binaries - description: Identifies the execution of the MS Office process -``` - -- **list** macros declare a sequence of items such as file system paths, process names, or registry keys. List macros help to make the rules succinct and clean. An example of a macro list containing Microsoft Office process image names. Various operators, such as `in`, `matches`, or `startswith` can accept list macros as RHS (Right Hand Side) expressions. - -```yaml -- macro: msoffice_binaries - list: [EXCEL.EXE, WINWORD.EXE, MSACCESS.EXE, POWERPNT.EXE] -``` - -#### Templates {docsify-ignore} - -Both, rule and macro `yaml` files can include Go [template](https://pkg.go.dev/text/template) directives. This encompasses loops, conditional directives, pipelines, or functions. Fibratus ships with a collection of [predefined](http://masterminds.github.io/sprig/) functions for string and filepath manipulation, math, date, and cryptographic functions to name a few. - -To illustrate the use of templates, let's assume the rules we deploy should require a minimum engine version of `2.0.0` under the presence of a certain environment variable in the host. By combining different Go template directives, we can conditionally render a fragment of the `yaml` file if the `env` function which receives the `AZ` environment variable name returns the `za` value. - -```yaml -name: Execution via Microsoft Office process -id: a10ebe66-1b55-4005-a374-840f1e2933a3 -version: 1.0.0 -description: - Identifies the execution of the file dropped by Microsoft Office process. -labels: - tactic.id: TA0001 - tactic.name: Initial Access - tactic.ref: https://attack.mitre.org/tactics/TA0001/ - technique.id: T1566 - technique.name: Phishing - technique.ref: https://attack.mitre.org/techniques/T1566/ - subtechnique.id: T1566.001 - subtechnique.name: Spearphishing Attachment - subtechnique.ref: https://attack.mitre.org/techniques/T1566/001/ - -condition: > - sequence - maxspan 1h - |create_file - and - (file.extension iin executable_extensions or file.is_exec) - and - ps.name iin msoffice_binaries - | by file.name - |spawn_process - and - ps.name iin msoffice_binaries - | by ps.child.exe - -{{- if eq (env "AZ") "za" }} -min-engine-version: 2.0.0 -{{- else }} -min-engine-version: 2.2.0 -{{- end }} -``` - -### Actions - -Actions are responses executed as a consequence of rule matches. Actions provide alerting and prevention capabilities aim at stopping the adversary at the initial stages of the attack. - -#### Generating alerts - -Alerts are automatically generated when the rule matches. The alert is sent via all active [senders](/alerts/senders). [Systray](/alerts/senders/systray) alert sender is enabled by default. The rule name is used in the alert title. To provide extra context in the alert, the rule `output` attribute can be used in combination with **field interpolation**. In the following snippet, we use the `%ps.exe` format modifier to render the process executable path. Formats modifiers are well-known filter fields prefixed with the `%` symbol. - -```yaml -output: | - "%ps.exe process spawned a command shell after connecting to the remote endpoint -``` - -In the case of sequence rules, we can access the desired event reference by including the ordinal which points to the slot in the sequence. This way, `%1.ps.name` would yield the name of the process in the first sequence slot, while `%2.file.name` gives us the filename located in the event reference of the second slot. Both, title and text permit including Markdown or HTML tags. - -```yaml -output: | - Detected an attempt by %1.ps.name process to access - and read the memory of the **Local Security And Authority Subsystem Service** - and subsequently write the %2.file.name dump file to the disk device -``` - -[email](alerts/senders/mail.md) alert senders render beautiful responsive HTML emails as depicted in the image below. Email rule alerts deliver high-fidelity and incident-centric notifications that aim to prevent alert fatigue by summarizing key investigation insights. - -

- -

- -The email alerts have the following sections: - -- Date, time, and the hostname where the alert was triggered -- Alert severity given by the `severity` attribute -- Alert title as specified the `name` attribute -- Alert description as specified by the `output` attribute -- Links with MITRE tactic, technique, and sub-technique -- The list of security events involved in the incident. For each event, the name, timestamp, and excerpt are shown. Next, all event attributes and process state information is represented. - -#### Killing processes - -`kill` action terminates the process involved in matched rule condition. Fibratus needs to acquire the process handle with the `PROCESS_TERMINATE` access rights to successfully kill the process. - -```yaml -action: - - name: kill -``` - -#### Isolating endpoints - -`isolate` action isolates the host by installing **Windows Filtering Platform** (WFP) rules. To revert the effects of the `isolate` action, simply restart the Fibratus service. The action can be combined with other actions. For example, the following snippet will kill the process and isolate the endpoint. - - -```yaml -action: - - name: kill - - name: isolate -``` - -### Advanced patterns - -Adversaries often employ sophisticated techniques which may be daunting to detect without combining events from different data sources. For example, detecting a remote connection attempt followed by the execution of a command shell by the same process that initiated the connection can't be expressed with a simple rule expecting to match on a single event. Enter `sequence` rules. - -#### Sequences - -In a nutshell, sequences permit state tracking of an ordered series of events over a short period of time. Let's peer into the structure of sequence-powered rules. Sequence rules start with the `sequence` keyword as you can observe in the `LSASS memory dumping via legitimate or offensive tools` rule's condition. Sequence consists of two or more expressions surrounded by vertical bars or pipes. For the sequence to match, all expressions need to match successively in time. - -```yaml -name: LSASS memory dumping via legitimate or offensive tools -id: 335795af-246b-483e-8657-09a30c102e63 -version: 1.0.0 -description: | - Detects an attempt to dump the LSAAS memory to the disk by employing legitimate - tools such as procdump, Task Manager, Process Explorer or built-in Windows tools - such as comsvcs.dll. -labels: - tactic.id: TA0006 - tactic.name: Credential Access - tactic.ref: https://attack.mitre.org/tactics/TA0006/ - technique.id: T1003 - technique.name: OS Credential Dumping - technique.ref: https://attack.mitre.org/techniques/T1003/ - subtechnique.id: T1003.001 - subtechnique.name: LSASS Memory - subtechnique.ref: https://attack.mitre.org/techniques/T1003/001/ -references: - - https://redcanary.com/threat-detection-report/techniques/lsass-memory/ - - https://www.deepinstinct.com/blog/lsass-memory-dumps-are-stealthier-than-ever-before - -condition: > - sequence - maxspan 2m - by ps.uuid - |open_process - and - ps.access.mask.names in ('ALL_ACCESS', 'CREATE_PROCESS', 'VM_READ') - and - kevt.arg[exe] imatches '?:\\Windows\\System32\\lsass.exe' - and - not - ps.exe imatches - ( - '?:\\Windows\\System32\\svchost.exe', - '?:\\ProgramData\\Microsoft\\Windows Defender\\*\\MsMpEng.exe' - ) - | - |write_minidump_file| - -output: > - Detected an attempt by `%1.ps.name` process to access and read - the memory of the **Local Security And Authority Subsystem Service** - and subsequently write the `%2.file.name` dump file to the disk device -severity: critical - -min-engine-version: 2.0.0 -``` - -The sequence behavior can be controlled by the following statements: - -- `maxspan` defines the time window in duration units, such as `2s`, `2m`, or `2h` for two minutes, two hours, and two days respectively. The time window dictates how long each expression in the sequence is expecting to wait for events that could result in expression evaluating to true. For example, by examining the above snippet, the sequence starts by detecting process handle acquisition on the `lsass` process. Since this is the first expression in the sequence, the time window constraint doesn't kick in yet. After the first expression evaluates to true, the next one, expecting to detect creation of the minidump file, will evaluate only if the `CreateFile` event arrives within the 2 minutes time frame. Otherwise, the deadline is reached and the entire sequence is discarded. -- `by` enables event stitching by any of the [filter fields](filters/fields). It guarantees that only events sharing certain properties will be eligible for matching. Continuing the example from previous rule, the sequence can match only if `OpenProcess` and `CreateFile` events are generated by the same process. Specifically, events are joined by the `ps.uuid` field which is meant to offer a more robust version of the `ps.pid` field that is resistant to being repeated. A variation of the `by` statement allows establishing a joining criteria separately on each expression in the sequence. Let's take a look at the following rule: - -```yaml -sequence -maxspan 1h - |write_file - and - file.extension iin executable_extensions - and - ps.name iin msoffice_binaries - | by file.name - |spawn_process - and - ps.name iin msoffice_binaries - | by ps.child.exe -``` - -As we can observe, the `by` statement is anchored to each expression but using a different join field. This rule would only match if the file being written is equal to the spawned process executable image. -Of course, it is possible to omit both `maxspan` and `by` statements. However, such rules are rarely used to express behaviors that require relationships between events, instead, a mere temporally connection. - -#### Aliases - -In certain situations, expressing event stitching relations may require more complex heuristics. Imagine a detection rule checking the presence of a created filename against the list of values obtained in subsequent sequence expression. An avid reader may immediately realize this sort of joining is not attainable by means of the `by` statement. Luckily, a more flexible solution exists in form of the `as` statement. This statement allows creating aliases which can be referenced in sequence expressions by using **bound fields**. Bound field is essentially a regular filter field prefixed with an alias. Let's see another example. - -```yaml -sequence -maxspan 5m - |create_file - and - file.name imatches '?:\\Windows\\System32\\*.dll' - | as e1 - |modify_registry - and - registry.key.name ~= 'HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\Lsa\\Notification Packages' - and - get_reg_value(registry.key.name) iin (base($e1.file.name, false)) - | -``` - -The first expression in the sequence detects the creation of a DLL file in the system directory. Once this expression evaluates to true, the event that triggered it is accessible via the `e1` alias. The second expression will detect registry modifications on the specified value, and if eligible, it will use the `get_reg_value` function to query the value, which, in this case,contains the `MULTI_SZ` content. The retrieved list of strings is compared against the filename from the event matching the first expression. The `$e1.file.name` bound field is responsible for consulting the filename field value from the referenced expression's matching event. diff --git a/docs/images/architecture.png b/docs/images/architecture.png new file mode 100644 index 000000000..bb2d30437 Binary files /dev/null and b/docs/images/architecture.png differ diff --git a/docs/images/callstacks.png b/docs/images/callstacks.png new file mode 100644 index 000000000..d3103f43f Binary files /dev/null and b/docs/images/callstacks.png differ diff --git a/docs/images/fibratus-term.gif b/docs/images/fibratus-term.gif deleted file mode 100644 index 32027b7bb..000000000 Binary files a/docs/images/fibratus-term.gif and /dev/null differ diff --git a/docs/images/fibratus-term.png b/docs/images/fibratus-term.png deleted file mode 100755 index 4ed41e31b..000000000 Binary files a/docs/images/fibratus-term.png and /dev/null differ diff --git a/docs/images/filter.png b/docs/images/filter.png new file mode 100644 index 000000000..640612d2a Binary files /dev/null and b/docs/images/filter.png differ diff --git a/docs/images/invalid-filter.png b/docs/images/invalid-filter.png new file mode 100644 index 000000000..0d5b29a5f Binary files /dev/null and b/docs/images/invalid-filter.png differ diff --git a/docs/images/realtime-detection.gif b/docs/images/realtime-detection.gif new file mode 100644 index 000000000..a461ba54d Binary files /dev/null and b/docs/images/realtime-detection.gif differ diff --git a/docs/images/rule-format.png b/docs/images/rule-format.png new file mode 100644 index 000000000..4a2b49318 Binary files /dev/null and b/docs/images/rule-format.png differ diff --git a/docs/images/telemetry-exploration.gif b/docs/images/telemetry-exploration.gif new file mode 100644 index 000000000..165042789 Binary files /dev/null and b/docs/images/telemetry-exploration.gif differ diff --git a/docs/index.html b/docs/index.html deleted file mode 100755 index 26051212e..000000000 --- a/docs/index.html +++ /dev/null @@ -1,59 +0,0 @@ - - - - - Fibratus - - - - - - - - - - -
- - - - - - - - - - - diff --git a/docs/kevents/anatomy.md b/docs/kevents/anatomy.md deleted file mode 100755 index 2b9ef3199..000000000 --- a/docs/kevents/anatomy.md +++ /dev/null @@ -1,82 +0,0 @@ -# Anatomy Of An Event - -The **Event** or `kevent` as referred to in Fibratus internal lingo is the fundamental building block that encapsulates the state of the event. The [ETW](https://docs.microsoft.com/en-us/windows/win32/etw/event-tracing-portal) **Kernel Logger** provider produces the flow of events in their raw format. This means that the vast majority of the event's parameters lack a meaningful context needed for an intuitive human interpretation. To clarify the previous statement, here are some concrete examples: - -- registry keys containing partial paths are typically not very useful. Think of `Settings/Control Panel` key that actually represents the fully qualified `HKEY_CURRENT_USER/Settings/Control Panel` path. Fibratus makes sure to run a hook in the early stage of the kernel event stream to figure out the remaining key name. Aside from this, root keys are represented in the native object manager format, e.g. `REGISTRY\MACHINE` is the key name for the `HKEY_CURRENT_MACHINE` root level key. Fibratus normalizes native root keys to well-known names. -- missing file names in `ReadFile` or `WriteFile` events. These events only contain the address of the file object that obviously doesn't give any hints about the file involved in the I/O operation. Fibratus strives for a best-effort file name resolution. Similarly to native registry key names, the Kernel Logger emits files names in DOS device name format, e.g. `Device\HardDisk4\Windows\System32\kernel32.dll`. Fibratus maps DOS device names to drive letters. -- handle type names are given as integer values. Fibratus takes care of mapping type identifiers to human-readable handle type names such as `File`, `Mutant` or `Key`. - -However, this is not the only heavy-lifting task Fibratus does to produce high-quality events. Additionally, events are enriched with various parameters that are not originally present in the ETW payloads. - -### Canonical fields - -Each event contains a series of canonical fields that describe the nature of the event such as its name, the process identifier that generated the event and such. The following is the list of all canonical fields. - -- **Sequence** is a monotonically increasing integer value that uniquely identifies an event. The sequence value is guaranteed to increment monotonically as long as the machine is not rebooted. After the restart, the sequence is restored to the zero value. -- **PID** represents the process identifier that triggered the event. -- **TID** is the thread identifier connected to the event. -- **CPU** designates the logical CPU core on which the event was originated. -- **Name** is the human-readable event name such as `CreateProcess` or `RegOpenKey`. -- **Timestamp** denotes the timestamp expressed in nanosecond precision as the instant the event occurred. -- **Category** designates the category to which the event pertains, e.g. `file` or `thread`. Each particular category is explained thoroughly in the next - sections. Possible category types are: `registry`, `file`, `net`, `process`, `thread`, `image` and `handle`. -- **Description** is a short explanation about the purpose of the event. For example, `CreateFile` event creates or opens a file, directory, I/O device, pipe, console buffer or other block/pseudo device. -- **Host** represents the host name where the event was produced. - -### Parameters - -Also called as `kparams` in Fibratus parlance, contain each of the event's parameters. Internally, they are modeled as a collection of key/value pairs where the key is mapped to the structure consisting of parameter name, parameter type and the value. An example of the parameter tuple could be the `dip` parameter -that denotes a destination IP address with value `172.17.0.2` and therefore `IPv4` type. Additionally, parameter types can be scalar values, strings, slices, enumerations, and timestamps among others. - -### Process state - -Each event stores the process state that represents an extended information about the process including its allocated resources such as handles, dynamically-linked libraries, exported environment variables and other attributes. The process state internals are thoroughly explained in the [Process](/kevents/process) events section. - -### Callstack - -Callstack reconstructs the sequence of function calls that led to the current thread state. When stack enrichment is enabled (controlled by the `--kstream.stack-enrichment` configuration flag), return addresses obtained directly from kernel space, are symbolized and enriched with a vital context that can be leveraged to boost behavioral detections and reduce false positive alerts. A typical callstack summary is depicted in the snippet above: - -``` -0x7ffb5c1d0396 C:\WINDOWS\System32\KERNELBASE.dll!CreateProcessW+0x66 -0x7ffb5d8e61f4 C:\WINDOWS\System32\KERNEL32.DLL!CreateProcessW+0x54 -0x7ffb5c1d0396 C:\WINDOWS\System32\KERNELBASE.dll!CreateProcessW+0x61 -0x7ffb3138592e C:\Program Files\JetBrains\GoLand 2021.2.3\jbr\bin\java.dll!Java_java_lang_ProcessImpl_waitForTimeoutInterruptibly+0x3a2 -0x7ffb313853b2 C:\Program Files\JetBrains\GoLand 2021.2.3\jbr\bin\java.dll!Java_java_lang_ProcessImpl_create+0x10a -0x2638e59e0a5 unbacked!? -``` - -Each line is comprised of: - -- function call return address (e.g. `0x7ffb5c1d0396`) -- full path of the image/module containing the executed function. If the call is invoked from a floating memory region, then the module name is marked as `unbacked` -- name of the symbol mapping to the return address (e.g. `CreateProcessW`) -- offset within the function body - -Stack enrichment is performed for the following event set: - -- `CreateProcess` -- `CreateThread` -- `TerminateThread` -- `SetThreadContext` -- `LoadImage` -- `RegCreateKey` -- `RegDeleteKey` -- `RegSetValue` -- `RegDeleteValue` -- `CreateFile` -- `DeleteFile` -- `RenameFile` -- `VirtualAlloc` -- `OpenProcess` -- `OpenThread` -- `CreateSymbolicLinkObject` -- `SubmitThreadpoolWork` -- `SubmitThreadpoolCallback` -- `SetThreadpoolTimer` - - -To enable stack enrichment for kernel space return addresses, the `symbolize-kernel-addresses` config option needs to be set to `true`. Callstack data is used by [filter fields](/filters/fields?id=callstack) to permit crafting advanced detection rules. - -### Metadata - -Metadata are an arbitrary sequence of tags in form of key/value pairs that you can squash into the event on behalf of [transformers](/transformers/introduction). A tag can be virtually any string data that you find meaningful to either identify the event or apply filtering/grouping once event is persisted in the data store. diff --git a/docs/kevents/driver.md b/docs/kevents/driver.md deleted file mode 100644 index 9ceeab740..000000000 --- a/docs/kevents/driver.md +++ /dev/null @@ -1,15 +0,0 @@ -# Driver events - -#### LoadDriver - -Driver loading events are triggered when the driver object is loaded into the kernel. Regular `LoadImage` events are triggered when the kernel driver is loaded or when the driver is unloaded, the `UnloadImage` event is received. - -An alternative route for detecting driver loading events is based on observing the [handle manager](kevents/handle.md) events. For example, a filter expression for pinpointing such events could be arranged as follows. - -``` -kevt.name = 'CreateHandle' - and -handle.type = 'Driver' -``` - -Rule writers are encouraged to use the `load_driver` macro from the [macro library](https://github.com/rabbitstack/fibratus/blob/master/rules/macros/macros.yml). This macro seamlessly handles the detection of driver loading depending on whether direct events are available or handle manager tracking is required. diff --git a/docs/kevents/file.md b/docs/kevents/file.md deleted file mode 100755 index 4234ac20d..000000000 --- a/docs/kevents/file.md +++ /dev/null @@ -1,112 +0,0 @@ -# File events - -File events encompass a variety of file system related activities such as creating or opening directories/files/devices, writing or reading data, altering file metadata and so on. - -#### CreateFile - -The `CreateFile` event is triggered when the kernel serves create/open requests for files or I/O devices. The most commonly used I/O devices are as follows: file, file stream, directory, physical disk, volume, console buffer, tape drive, communications resource, mailslot, and pipe. `CreateFile` events have the following parameters: - -- `file_object` is the file object pointer as seen from the kernel perspective. You can use this parameter to correlate file events. -- `file_name` represents the file/directory or device name (e.g. `C:\ProgramData\AVG\Antivirus\psi.db-journal`) -- `irp` is the I/O request packet value that identifies the file system activity. -- `tid` is the thread identifier that initiated the I/O operation. -- `create_disposition` identifies the file system operation performed on the file or device: - - `SUPERSEDE` replaces the file if it already exists, otherwise creates a new file - - `OPEN` opens the file if it exists - - `CREATE` creates a new file or fails if the file already exists - - `OPENIF` opens the file if it already exists, otherwise creates a new file - - `OVERWRITE` opens and overwrites the file if it already exists - - `OVERWRITEIF` opens and overwrites the file if it already exists, otherwise creates a new file -- `create_options` the options to be applied when creating or opening the file, as a compatible combination of the following values: `DIRECTORY_FILE`,`WRITE_THROUGH`, `SEQUENTIAL_ONLY`, `NO_INTERMEDIATE_BUFFERING`, `SYNCHRONOUS_IO_ALERT`, `SYNCHRONOUS_IO_NONALERT`, `NON_DIRECTORY_FILE`, `CREATE_TREE_CONNECTION`, `COMPLETE_IF_OPLOCKED`, `NO_EA_KNOWLEDGE`, `OPEN_REMOTE_INSTANCE`, `RANDOM_ACCESS`, `DELETE_ON_CLOSE`,`OPEN_BY_FILE_ID`, `FOR_BACKUP_INTENT`, `NO_COMPRESSION`, `OPEN_REQUIRING_OPLOCK`,`DISALLOW_EXCLUSIVE`, `RESERVE_OPFILTER`, `OPEN_REPARSE_POINT`, `OPEN_NO_RECALL` and `OPEN_FOR_FREE_SPACE_QUERY`. -- `share_mask` specifies the sharing mode of the file or device, which can be the combination of `READ`, `WRITE`, and `DELETE` values. This flag determines the permission granularity which enables a process to share a file or device while another process has the file or device open. -- `type` defines the file type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown`. -- `attributes` denotes the file attributes. Possible values are `READONLY`, `HIDDEN`, `SYSTEM`, `DIRECTORY`, `COMPRESSED`, `ENCRYPTED`, `JUNCTION`, `SPARSE`,`TEMPORARY`, `DEVICE`, `NORMAL`, `OFFLINE`, `UNINDEXED`, `STREAM`, `VIRTUAL`, `NOSCRUB`, `RECALLOPEN`, `RECALLACCESS`, `PINNED`, `UNPINNED`, `UNKNOWN`. -- `status` represents the system status message (e.g. `Success`) -- `is_dll` determines if the created file is a DLL object. Only present when `create_disposition != OPEN`. -- `is_driver` determines if the created file is a driver. Only present when `create_disposition != OPEN`. -- `is_exec` determines if the created file is an executable image. Only present when `create_disposition != OPEN`. - -#### WriteFile and ReadFile - -These events occur when a process writes data to a file or reads data from the file or I/O device. They contain the following parameters: - -- `file_object` is the file object pointer as seen from the kernel perspective. You can use this parameter to correlate file events. -- `file_name` represents the file/directory or device name the data is written to or read from. -- `irp` is the I/O request packet value that identifies the file system activity. -- `io_size` specifies the number of bytes read or written. -- `offset` determines the offset in the file where the data is read or written. -- `type` defines the file type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown`. - - -#### DeleteFile - -Removes the file from the file system. This event contains the following parameters: - -- `file_object` is the file object pointer as seen from the kernel perspective. You can use this parameter to correlate file events. -- `file_name` represents the file/directory that was removed -- `irp` is the I/O request packet value that identifies the file system activity. -- `type` defines the file type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown`. - -#### RenameFile - -Renames the file or directory in the file system. - -- `file_object` is the file object pointer as seen from the kernel perspective. -- `file_name` represents the file/directory that was renamed. -- `irp` is the I/O request packet value that identifies the file system activity. -- `type` defines the file type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown`. - -#### CloseFile - -Closes the handle to opened file. This event is excluded by default. - -- `file_object` is the file object pointer as seen from the kernel perspective. -- `file_name` represents the file/directory that was closed. -- `irp` is the I/O request packet value that identifies the file system activity. -- `type` defines the file type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown`. - - -#### SetFileInformation - -Sets the file information for the file according to the file information class. - -- `class` identifies the file information class. For example, the `Basic` information class means the process altered file timestamps or basic attributes. Refer to [this](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class) link for a full list of the file information class enumerations. -- `file_object` is the file object pointer as seen from the kernel perspective. -- `file_name` represents the file whose information class was set. -- `irp` is the I/O request packet value that identifies the file system activity. -- `type` defines the file type. Possible values are `File`, `Pipe`, `Mailslot`, `Other`, `Unknown`. - - -#### EnumDirectory - -The `EnumDirectory` event is triggered in response to directory enumeration requests. - -- `dir` specifies the directory that was requested for enumeration. -- `file_name` is the pattern for directory enumeration. -- `class` identifies the requested directory enumeration class. -- `file_object` is the file object pointer as seen from the kernel perspective. -- `irp` is the I/O request packet value that identifies the file system activity. - -#### MapViewFile - -Maps a view of a file mapping into the process address space. These events contain the following parameters: - -- `file_key` is the address of the file object for which the mapping is performed. -- `offset` represents the file offset where the view is to begin. -- `pid` is the process identifier where the file mapping is performed. -- `protection` specifies the page protection of the file mapping object. Can be the compatible combination of the following values: `READONLY`, `EXECUTE`, `EXECUTE_READ`, `READWRITE`, `WRITECOPY`, `NOCACHE`, `EXECUTE_WRITECOPY` and `EXECUTE_READWRITE`. -- `section_type` describes the type of the mapped section. It can be `DATA`, `IMAGE`, `IMAGE_NO_EXECUTE`, `PAGEFILE` or `PHYSICAL`. -- `view_base` is the base memory address in the process address space where mapping begins. -- `view_size` represents the number of bytes of a file mapping to map to a view. - -#### UnmapViewFile - -Unmaps a mapped view of a file from the process's virtual address space. - -- `file_key` is the address of the file object for which the unmapping is performed. -- `offset` represents the file offset where the view to unmap begins. -- `pid` is the process identifier where the file unmapping is performed. -- `protection` specifies the page protection of the file mapping object that is being unmapped. Can be the compatible combination of the following values: `READONLY`, `EXECUTE`, `EXECUTE_READ`, `READWRITE`, `WRITECOPY`, `NOCACHE`, `EXECUTE_WRITECOPY` and `EXECUTE_READWRITE`. -- `section_type` describes the type of the unmapped section. It can be `DATA`, `IMAGE`, `IMAGE_NO_EXECUTE`, `PAGEFILE` or `PHYSICAL`. -- `view_base` is the base memory address in the process address space where unmapping begins. -- `view_size` represents the number of bytes of a file mapping to unmap. diff --git a/docs/kevents/handle.md b/docs/kevents/handle.md deleted file mode 100755 index a67144b17..000000000 --- a/docs/kevents/handle.md +++ /dev/null @@ -1,40 +0,0 @@ -# Handle events - -The object manager produces handle events to provision or dispose Windows resources allocated to processes. Resources can be threads, registry keys, files, synchronization primitives and so on. - -!> Handle events can be quite voluminous and they are disabled by default. To enable the collection of handle events either run Fibratus with the `--kstream.enable-handle=true` flag or activate them permanently by editing the config file. - -#### CreateHandle - -Provisions a new handle in the address space of the calling process. The following parameters are associated with the `CreateHandle` event: - -- `handle_id` represents the unique identifier of the handle. -- `handle_object` represents the address of the kernel object to which the handle is associated -- `handle_name` denotes the handle name. (e.g. `\RPC Control\OLEA61B27E13E028C4EA6C286932E80`) -- `type_id` designates the handle type. (e.g. `ALPC Port`, `File`, `Key`) - -#### CloseHandle - -The `CloseHandle` event is triggered when the handle is released by the process. It contains the same set of parameters found in the `CreateHandle` event. - -#### DuplicateHandle - -This event is fired when the process duplicates an object handle. The following parameters are present in `DuplicateHandle` events: - -- `handle_id` represents the duplicated handle identifier. -- `handle_object` represents the address of the duplicated kernel object -- `handle_source_id` represents the identifier of the handle to be duplicated. -- `type_id` designates the duplicated handle type. (e.g. `DxgkCompositionObject`) -- `pid`, `exe`, `name` represent the source process identifier, process executable path, and the process image name respectively. - -### Handle state {docsify-ignore} - -During bootstrap, Fibratus builds a snapshot of currently allocated handles. Similarly, when a new process is created Fibratus fetches its handles and attaches them to the process state. However, to optimize memory utilization, the initial handle snapshot and process handle table initialization are disabled by default. You can enable both features by modifying the `--handle.init-snapshot=true` and `--handle.enumerate-handles` config flags respectively or changing the corresponding key in the configuration file. - -The handle state contains: - -- handle name -- handle type -- the address of the kernel object -- handle identifier -- additional metadata such as `ALPC` port information or mutant count diff --git a/docs/kevents/image.md b/docs/kevents/image.md deleted file mode 100755 index fc700f936..000000000 --- a/docs/kevents/image.md +++ /dev/null @@ -1,17 +0,0 @@ -# Image events - -Image events occur when a process loads/unloads a dynamic linked library, executable or the kernel driver. The loading can happen in the local or remote process. These events are represented by `LoadImage` and `UnloadImage` types respectively. The following list describes all available parameters present in DLL events. - -- `file_name` denotes the full path name of the image file. (e.g. `C:\Windows\system32\kernel32.dll`) -- `image_size` represents the image size in bytes. -- `checksum` represents the image checksum digest. -- `base_address` is the base address of the process in which the image is loaded/unloaded. -- `default_address` represents the image's base address. -- `pid` that specifies the process identifier where the image was loaded/unloaded. - - -Additionally, `LoadImage` events contain the following parameters. - -- `is_dll` determines if the image is a DLL object. -- `is_driver` determines if the image is a kernel driver. -- `is_exec` determines if the image being loaded is an executable. diff --git a/docs/kevents/mem.md b/docs/kevents/mem.md deleted file mode 100644 index 54f160e95..000000000 --- a/docs/kevents/mem.md +++ /dev/null @@ -1,24 +0,0 @@ -# Memory events - -Memory events include reserving, committing, or freeing the state of a region of pages in the virtual process address space. - -#### VirtualAlloc - -Allocates, commits, or changes the state of a region of pages in the virtual address space. If the `pid` parameter is different than the calling process id, memory allocation is performed in the address space of another process. `VirtualAlloc` events carry the following parameters: - -- `alloc_type` designates the type of memory allocation. It can be the combination of `COMMIT`, `RESERVE`, `RESET`, `RESET_UNDO`, `PHYSICAL`, `LARGE_PAGES`, `TOP_DOWN`, and `WRITE_WATCH`. -- `base_address` is the starting address of the allocated region. -- `page_type` represents the type of pages in the allocated region. It can be one of `IMAGE`, `MAPPED`, or `PRIVATE`. -- `protection` designates the memory protection for the region of allocated pages. It can be the combination of `EXECUTE`, `EXECUTE_READ`, `EXECUTE_READWRITE`, `EXECUTE_WRITECOPY`, `NOACCESS`, `READONLY`, `READWRITE`, `WRITECOPY`, `TARGETS_INVALID`, `TARGETS_NO_UPDATE`, `GUARD`, `NOCACHE`, and `WRITECOMBINE`. -- `protection_mask` is an abbreviated form of the pages protection flag. e.g. `RWX` -- `region_size` is the size of the allocated region in bytes. -- `pid`, `exe`, `name` represent process identifier, process executable path, and the image name of the process into which the region is allocated. - -#### VirtualFree - -Releases, decommits, or releases and decommits a region of pages within the virtual process address space. If the `pid` parameter is different than the calling process id, memory release is performed in the address space of another process. `VirtualFree` events contain the following parameters: - -- `alloc_type` designates the type of free operation that can be `DECOMMIT` or `RELEASE`. -- `base_address` is the base address of the region of pages to be freed. -- `region_size` represents the size of the region of memory to be freed, in bytes. -- `pid`, `exe`, `name` represent process identifier, process executable path, and the image name of the process for which the pages are freed. diff --git a/docs/kevents/network.md b/docs/kevents/network.md deleted file mode 100755 index eacdd56d1..000000000 --- a/docs/kevents/network.md +++ /dev/null @@ -1,87 +0,0 @@ -# Network events - -Interprocess communication via TCP/UDP sockets produces all sorts of network events including, sending and receiving data, retransmitting TCP segments or connecting to sockets. - -#### Connect - -Establishes a connection to the stream-oriented socket. `Connect` events have the following parameters: - -- `dip` represents the destination IPv4/IPv6 address of the communication endpoint. -- `sip` represents the source IPv4/IPv6 address of the local process. -- `dport` identifies the destination port. -- `sport` identifies the source port. -- `l4_proto` denotes the type of the Layer 4 protocol. Possible values are `tcp`, `udp`. -- `dport_name` represents the destination port name per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service mappings. -- `sport_name` represents the source port name per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service mappings. - -The remaining network events share the same parameters as the `Connect` event. - -#### Accept - -Accepts the connection request from the socket queue. - -#### Send - -Sends data over the wire. The kernel generates `Send` events when a process transports data to local or remote endpoint. - -#### Recv - -Receives data from the socket. The kernel produces `Recv` events when a process is ready to consume data sent by local or remote socket. - -#### Disconnect - -Terminates data reception on the socket. - - -#### Reconnect - -Reconnects to the socket. - -#### Retransmit - -Retransmits unacknowledged TCP segments. The kernel networking stack generates retransmissions when packets are dropped due to network congestion, packets arriving out of order and other reasons. - -### DNS queries/responses - -DNS telemetry uncovers all DNS query/reply interactions. More specifically, `QueryDns` and `ReplyDns` events are fired when the process sends a query to the name server and when it receives the response from the DNS server, respectively. DNS events are collected by default, but it is possible to disable them by setting the `kstream.enable-dns` config flag to `false`. - -#### QueryDns - -Sends a query to the name server. This event has the following parameters: - -- `name` represents the DNS query (e.g. `www.iana.org`) -- `options` represents the DNS options. It can be the combination of the following values: `STANDARD`, `ACCEPT_TRUNCATED_RESPONSE`, `USE_TCP_ONLY`, `NO_RECURSION`, `BYPASS_CACHE`, `NO_WIRE_QUERY`, `NO_LOCAL_NAME`, `NO_NETBT`, -`WIRE_ONLY`, `RETURN_MESSAGE`, `MULTICAST_ONLY`, `NO_MULTICAST`, `TREAT_AS_FQDN`, `ADDRCONFIG`, `DUAL_ADDR`, `MULTICAST_WAIT`, -`MULTICAST_VERIFY`, `DONT_RESET_TTL_VALUES`, `DISABLE_IDN_ENCODING`, `APPEND_MULTILABEL`. -- `rr` specifies the type of the resource record. It can be one of `A`,`NS`, `MD`, `MF`, `CNAME`, `SOA`, `MB`, `MG`, `MR`, `NULL`, `WKS`, `PTR`,`HINFO`, `MINFO`, `MX`, `TEXT`, `RP`, `AFSDB`, `X25`, `ISDN`, `NSAPPTR`, `SIG`, `KEY`, `PX`, `GPOS`, -`AAAA`, `LOC`, `NXT`, `EID`, `NIMLOC`, `SRV`, `ATMA`, `NAPTR`, `KX`, `CERT`, `A6`, `DNAME`, `SINK`, `OPT`, `DS`, `RRSIG`, -`NSEC`, `DNSKEY` `DHCID`, `UINFO`, `UID`, `GID`, `UNSPEC`, `ADDRS`, `TKEY`, `TSIG`, `IXFR`, `AXFR`, `MAILB`, `MAILA`, `ANY`, -`WINS`, `WINSR`. - -#### ReplyDns - -Receives the response from the DNS server. DNS reply events contain the following parameters: - -- `name` represents the DNS query (e.g. `www.iana.org`) -- `answers` contains the response answers (e.g. `151.101.194.132`, `151.101.130.132`) -- `options` represents the DNS options. It can be the combination of the following values: `STANDARD`, `ACCEPT_TRUNCATED_RESPONSE`, `USE_TCP_ONLY`, `NO_RECURSION`, `BYPASS_CACHE`, `NO_WIRE_QUERY`, `NO_LOCAL_NAME`, `NO_NETBT`, -`WIRE_ONLY`, `RETURN_MESSAGE`, `MULTICAST_ONLY`, `NO_MULTICAST`, `TREAT_AS_FQDN`, `ADDRCONFIG`, `DUAL_ADDR`, `MULTICAST_WAIT`, -`MULTICAST_VERIFY`, `DONT_RESET_TTL_VALUES`, `DISABLE_IDN_ENCODING`, `APPEND_MULTILABEL`. -- `rr` specifies the type of the resource record. It can be one of `A`,`NS`, `MD`, `MF`, `CNAME`, `SOA`, `MB`, `MG`, `MR`, `NULL`, `WKS`, `PTR`,`HINFO`, `MINFO`, `MX`, `TEXT`, `RP`, `AFSDB`, `X25`, `ISDN`, `NSAPPTR`, `SIG`, `KEY`, `PX`, `GPOS`, -`AAAA`, `LOC`, `NXT`, `EID`, `NIMLOC`, `SRV`, `ATMA`, `NAPTR`, `KX`, `CERT`, `A6`, `DNAME`, `SINK`, `OPT`, `DS`, `RRSIG`, -`NSEC`, `DNSKEY` `DHCID`, `UINFO`, `UID`, `GID`, `UNSPEC`, `ADDRS`, `TKEY`, `TSIG`, `IXFR`, `AXFR`, `MAILB`, `MAILA`, `ANY`, -`WINS`, `WINSR`. -- `rcode` designates the DNS response code. It can be one of `NOERROR`, `FORMERR`, `SERVFAIL`, `NXDOMAIN`, `NOTIMP`, `REFUSED`, -`YXDOMAIN`, `YXRRSET`, `NXRRSET`, `NOTAUTH`, `NOTZONE`, `BADSIG`, `BADKEY`, `BADTIME`, `BADNAME`, `INVALID`, `NXDOMAIN`. - -### DNS reverse lookups - -Fibratus can perform reverse DNS lookups on IP addresses and return a list of domain names that correspond to a particular IP address. - -For example, `47.224.186.35.bc.googleusercontent.com` is the reverse lookup representation of the Google IP address. - -You can use domain names in [filters](filters/introduction) in conjunction with the `matches`, `contains` or `in` operators. For example, the following filter would match all events that have at least one domain ending with `.domain.` - -``` -net.sip.names matches ('*.domain.') -``` diff --git a/docs/kevents/object.md b/docs/kevents/object.md deleted file mode 100644 index beba65ab3..000000000 --- a/docs/kevents/object.md +++ /dev/null @@ -1,10 +0,0 @@ -# Object manager events - -#### CreateSymbolicLinkObject - -`CreateSymbolicLinkObject` event is fired when the symbolic link within the object manager directory. This event has the following parameters: - -- `source` identifies the parameter that represents the source symbolic link object or other kernel object. -- `target` identifies the parameter that represents the target symbolic link object or other kernel object. -- `desired_access` denotes the access rights for the target symbolic link object. Can be the combination of `DELETE`, `READ_CONTROL`, `WRITE_DAC`, `WRITE_OWNER`, `SYNCHRONIZE`, `STANDARD_RIGHTS_REQUIRED`, `STANDARD_RIGHTS_ALL`, `ACCESS_SYSTEM_SECURITY`, `MAXIMUM_ALLOWED`, `GENERIC_READ`, `GENERIC_WRITE`, `GENERIC_EXECUTE`, `GENERIC_ALL`. -- `status` represents the outcome of the operation. \ No newline at end of file diff --git a/docs/kevents/process.md b/docs/kevents/process.md deleted file mode 100755 index 5bacbb5d6..000000000 --- a/docs/kevents/process.md +++ /dev/null @@ -1,52 +0,0 @@ -# Process events - -#### CreateProcess and TerminateProcess - -Process events are fired up as a stimulus to the process' life-cycle changes. When the kernel puts into motion a process or terminates it, the `CreateProcess` and `TerminateProcess` events are emitted respectively. The following list summarizes all the distinct event parameters that are associated with process events. - -- `pid` is the process' identifier. This value is valid from the time a process is created until it is terminated. -- `tid` is the thread identifier inside process address space that generated the event. -- `ppid` represents the parent identifier of the child process. Process identifier numbers are reused, so they only identify a process for the lifetime of that process. It is possible that the process identified by `ppid` is terminated, so `ppid` may not refer to a running process. It is also possible that `ppid` incorrectly refers to a process that reuses a process identifier. -- `real_ppid` is the process identifier useful for detecting process spoofing. -- `name` is the process' image name including file extension (e.g. `cmd.exe`). -- `cmdline` is the full process' command line (e.g. `C:\Windows\system32\cmd.exe /cdir /-C /W`). -- `exe` is the full name of the process' executable (e.g. `C:\Windows\system32\cmd.exe`) -- `sid` is the security identifier under which this process is run. (e.g. `S-1-5-18`) -- `kproc` represents the address of the process object in the kernel. -- `directory_table_base` represents the address of the directory table that holds process' memory page mappings. -- `session_id` is the unique identifier for the current session under which process was started or terminated. -- `status` is the exit status of the started/stopped process. -- `start_time` designates the instant when the process was started. -- `domain` represents the domain name under which the process is started. -- `username` represents the username that started the process. -- `flags` represents process creation flags. Can be `WOW64`, `PROTECTED`, or `PACKAGED` to designate 32-bit process is created in 64-bit Windows system, process is to be run as a protected process, or a process packaged with the **MSIX** technology respectively. - -#### OpenProcess - -`OpenProcess` event is triggered when a process tries to acquire an existing local process object. This event contains the following parameters: - -- `desired_access` is the hexadecimal value that represents the desired access to the process object. -- `desired_access_names` is the list of human-readable desired access strings (e.g. `TERMINATE,QUERY_INFORMATION`). For a full list and detailed explanation of available access rights, head to the official [docs](https://docs.microsoft.com/en-us/windows/win32/procthread/process-security-and-access-rights). -- `name` is the name of the local process that was opened. -- `exe` is the full path of the local process object that was open. -- `pid` is the identifier of the local process that was opened. -- `status` contains the result of the process object open operation. (e.g. `Success`) - -### Process state {docsify-ignore} - -Fibratus keeps a snapshot of all running processes including their state such as basic process attributes, allocated file handles, dynamically-linked libraries, PE (Portable Executable) metadata and other resources. The snapshot is updated dynamically as processes get spawn or die. Each time a kernel event is triggered, its process' state is fetched from the snapshot and attached to the event. This state machine semantically enriches each individual event with the aim on providing a powerful context for [filtering](/filters/introduction.md) and [scripting](/filaments/introduction.md). - -Process state comprises the following attributes and resources: - -- process name -- process identifier as well as its parent process identifier -- process command line -- current working directory -- process SID -- session identifier -- environment variables -- threads -- modules -- handles -- memory mappings -- PE metadata diff --git a/docs/kevents/registry.md b/docs/kevents/registry.md deleted file mode 100755 index 041f777a1..000000000 --- a/docs/kevents/registry.md +++ /dev/null @@ -1,49 +0,0 @@ -# Registry events - -Registry events are triggered when processes access or modify the registry structure. - -#### RegCreateKey - -Creates a new registry key or opens the key if it already exists. This event has the following parameters: - -- `key_handle` represents the handle to the registry key. In reality, this value represents the address of the KCB (Key Control Block) structure in kernel space. -- `key_name` is the name of the registry key including the root key. (e.g. `HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control`) -- `status` contains the result of the registry operation (e.g. `key not found`) - -#### RegDeleteKey - -Deletes a subkey and all its values. This event has the same group of parameters as the `RegCreateKey` event. - -#### RegOpenKey - -Opens a registry key. This event has the same group of parameters as the `RegCreateKey` event. - -#### RegCloseKey - -Closes the registry key. This event has the same group of parameters as the `RegCreateKey` event. - -#### RegQueryKey - -Enumerates the subkeys of the specified key. The first subkey is referenced by index number 0, the second key by 1, and so on. - -- `key_handle` represents the handle to the registry key. -- `key_name` is the name of the registry key whose subkeys are enumerated. -- `status` contains the result of the registry operation (e.g. `key not found`) - -#### RegQueryValue - -Fetches the data associated with the value of a registry key. This event has the same group of parameters as the `RegCreateKey` event, except that the base name of the registry key path is the value name. - -#### RegSetValue - -Sets the data associated with the value of a registry key. This event contains the following parameters: - -- `key_handle` represents the handle to the registry key. -- `key_name` represents the fully qualified name of the registry value whose data is modified -- `status` contains the result of the registry operation (e.g. `success`) -- `value` contains the payload of the value being set -- `type` represents the registry value type. Possible value are: `REG_DWORD`, `REG_QWORD`, `REG_SZ`, `REG_EXPAND_SZ`, `REG_MULTI_SZ`, `REG_BINARY`, `UNKNOWN`. - -#### RegDeleteValue - -Deletes the registry value. This event has the same parameters as the `RegCreateKey` event. diff --git a/docs/kevents/thread.md b/docs/kevents/thread.md deleted file mode 100755 index 0e9bf8715..000000000 --- a/docs/kevents/thread.md +++ /dev/null @@ -1,78 +0,0 @@ -# Thread events - -#### CreateThread and TerminateThread - -Thread events notify the creation (`CreateThread`) or termination (`TerminateThread`) of threads within the process' address space. In situations where a process spawns a remote thread that runs in other process' address space, the `CreateThread` event is also triggered. -Thread events are comprised of the following parameters: - -- `pid` identifies the process where the thread was created. If this value is not equal to the `pid` of the process generating the event, then the thread is created in remote process. -- `tid` represents the identifier of the freshly spawn thread. -- `base_prio` is the scheduler priority of the thread. -- `page_prio` is a memory page priority hint for memory pages accessed by the thread. -- `io_prio` represents an I/O priority hint for scheduling I/O operations generated by the thread. -- `ustack_base` is the base address of the thread's user space stack. -- `ustack_limit` is the limit of the thread's user space stack. -- `kstack_base` is the base address of the thread's kernel space stack. -- `kstack_limit` is the limit of the thread's kernel space stack. -- `start_address` is the start address of the function to be executed by the thread. -- `start_address_symbol` the symbol the represents the thread start address (e.g. `LoadLibraryEx`). Only present in `CreateThread` events. -- `start_address_module` the module that maps to the thread start address (e.g. `ntdll.dll`). Only present in `CreateThread` events. -- `teb` is the address of the Thread Environment Block (TEB). - - -#### OpenThread - -`OpenProcess` event is triggered when a process opens an existing local thread object. This event contains the following parameters: - -- `desired_access` is the hexadecimal value that represents the desired access to the thread object. -- `desired_access_names` is the list of human-readable desired access strings (e.g. `QUERY_LIMITED_INFORMATION`). For a full list and detailed explanation of available access rights, head to the official [docs](https://docs.microsoft.com/en-us/windows/win32/procthread/thread-security-and-access-rights). -- `name` is the name of the local process whose thread object was open. -- `exe` is the full path of the local process image whose thread object was open. -- `pid` is the identifier of the local process whose thread object was opened. -- `tid` is the identifier of the local thread that was opened. -- `status` contains the result of the thread object open operation. (e.g. `Success`) - -#### SetThreadContext - -`SetThreadContext` sets the thread context. Thread context represents the set of CPU registers. - -- `status` contains the result of operation. (e.g. `Success`) - -## Thread pool events {docsify-ignore} - -A thread pool is a collection of worker threads that efficiently execute asynchronous callbacks on behalf of the application. The thread pool is primarily used to reduce the number of application threads and provide management of the worker threads. - - -#### SubmitThreadpoolWork - -Enqueues the work item to the thread pool. This event has the following parameters: - -- `pool_id` represents the thread pool identifier. -- `task_id` represents the thread pool task identifier. -- `callback` represents the address of the callback function. -- `context` represents the address of the callback context. -- `subprocess_tag` represents the thread pool identifier. -- `pool_id` represents the service identifier associated with the thread pool. -- `callback_symbol` represents the callback symbol (e.g. `RtlCaputreContext`) -- `callback_module` represents the module containing the callback symbol (e.g. `C:\Windows\System32\ntdll.dll`) -- `context_rip` represents the value of instruction pointer contained in the callback context. -- `context_rip_symbol` represents the symbol name associated with the instruction pointer in callback context. -- `context_rip_module` represents the module name associated with the instruction pointer in callback context. - - -#### SubmitThreadpoolCallback - -Submits the thread pool callback for execution within the work item. This event has the same parameter set as the `SubmitThreadpoolWork` events. - - -#### SetThreadpoolTimer - -Sets the thread pool timer object. This event consists of the following parameters: - -- `duetime` represents the timer due time. -- `subqueue` represents the memory address of the timer subqueue. -- `timer` represents the memory address of the timer object. -- `period` represents the period of the timer. -- `window` represents the timer tolerate period. -- `absolute` indicates if the timer is absolute or relative. - diff --git a/docs/logo.png b/docs/logo.png deleted file mode 100644 index 14546ce8a..000000000 Binary files a/docs/logo.png and /dev/null differ diff --git a/docs/nav.md b/docs/nav.md deleted file mode 100755 index a095bc7ad..000000000 --- a/docs/nav.md +++ /dev/null @@ -1 +0,0 @@ - diff --git a/docs/outputs/console.md b/docs/outputs/console.md deleted file mode 100644 index 587ed860c..000000000 --- a/docs/outputs/console.md +++ /dev/null @@ -1,81 +0,0 @@ -# Console - -Console is the default output. It renders the inbound event flow onto the console. The console output tends to reduce the number of I/O syscalls by buffering the incoming event lines. - -### Configuration {docsify-ignore} - -The console output configuration is located in the `outputs.console` section. - -#### enabled - -Indicates whether the console output is active. - -**default**: `true` - -#### format - -Specifies the console output format. The `pretty` format dictates that formatting is accomplished by replacing the specifiers in the template. The `json` format outputs the event as a raw JSON string. - -**default**: `pretty` - -#### kv-delimiter - -Specifies the separator that's rendered between the event parameter's key and its value. - -**default**: `➜` - -#### template - -Defines the template that's used in the event formatter. For more details, see the next section. - -**default**: `{{ .Seq }} {{ .Timestamp }} - {{ .CPU }} {{ .Process }} ({{ .Pid }}) - {{ .Type }} ({{ .Kparams }})` - -### Templates {docsify-ignore} - -The template consists of a collection of named placeholders that event formatter replaces with desired values. The syntax of the template resembles the Go [template](https://golang.org/pkg/text/template/) engine constructs, excepts the event formatter lacks advanced templating features such as loops , functions or `if` statements. - -The following field modifiers are supported: - -- `.Seq` -- `.Timestamp` -- `.Pid` -- `.Ppid` -- `.Pexe` -- `.Pcomm` -- `.Pname` -- `.Cwd` -- `.Exe` -- `.Comm` -- `.Tid` -- `.Sid` -- `.Process` -- `.Category` -- `.Description` -- `.CPU` -- `.Type` -- `.Kparams` -- `.Meta` -- `.Host` -- `.PE` -- `.Kparams.` -- `.Callstack` - -**Examples** - -- `{{ .Type }} on file ({{ .Kparams.File_name }})` - - renders - - `CreateFile on file C:\ProgramData\AVG\Antivirus\psi.db-journal` - -- `{{ .Seq }} {{ .Process }} ({{ .Cwd }}) {{ .Ppid }} ({{ .Sid }})` - - renders - - `1999 cmd.exe (C:/Windows/System32) 2324 (nedo/archrabbit)` - -- `{{ .Seq }} {{.CPU}} - ({{.Type}}) Pid: {{ .Kparams.Pid }} {{ .Meta }}` - - renders - - `1999 4 - (CreateProcess) Pid: 1232 env: prod, az: east` diff --git a/docs/outputs/elasticsearch.md b/docs/outputs/elasticsearch.md deleted file mode 100644 index 546899845..000000000 --- a/docs/outputs/elasticsearch.md +++ /dev/null @@ -1,173 +0,0 @@ -# Elasticsearch - -The Elasticsearch output ships kernel events to the `_bulk` [API endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). Events are batched and flushed when the interval specified by `flush-period` elapses. - -### Configuration {docsify-ignore} - -The Elasticsearch output configuration is located in the `outputs.elasticsearch` section. - -#### enabled - -Indicates whether the Elasticsearch output is enabled. - -**default**: `false` - -#### servers - -Defines the URL endpoints of the Elasticsearch nodes. - -**default**: `http://localhost:9200` - -#### timeout - -Represents the initial HTTP connection timeout when connecting to the Elasticsearch cluster. - -**default**: `5s` - -#### flush-period - -Specifies when to flush the bulk at the end of the given interval. - -**default**: `1s` - -#### bulk-workers - -Determines the number of workers that commit docs to Elasticsearch. Higher values maximize the throughout at the cost of increased CPU utilization. - -**default**: `1` - -#### healthcheck - -Enables or disables nodes health checking. - -**default**: `true` - -#### healthcheck-interval - -Specifies the interval for checking if the Elasticsearch nodes are available. - -**default**: `10s` - -#### healthcheck-timeout - -Specifies the timeout for periodic node health checks. - -**default**: `5s` - -#### username - -Identifies the user name for the basic HTTP authentication. - -#### password - -Identifies the password for the basic HTTP authentication. - -#### sniff - -Enables the discovery of all Elasticsearch nodes in the cluster. This avoids populating the list of available Elasticsearch nodes. - -**default**: `false` - -#### trace-log - -Determines if the Elasticsearch client trace log is enabled. Useful for troubleshooting. - -**default**: `false` - -#### gzip-compression - -Determines if the `gzip` compression is enabled for Elasticsearch documents. - -**default**: `false` - -#### template-name - -Specifies the name of the index template. - -**default**: `fibratus` - -#### template-config - -Contains the full JSON body of the index template. For more information refer to [index templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html). - -**default**: - -``` -{ - "index_patterns": [ "{{ fibratus* }}" ], - "settings": { - "index": { - "refresh_interval": "5s", - "number_of_shards": 1, - "number_of_replicas": 1 - } - }, - "mappings": { - "properties": { - "seq": { "type": "long" }, - "pid": { "type": "long" }, - "tid": { "type": "long" }, - "cpu": { "type": "short" }, - - "name": { "type": "keyword" }, - "category": { "type": "keyword" }, - "description": { "type": "text" }, - "host": { "type": "keyword" }, - - "timestamp": { "type": "date" }, - - "kparams": { - "type": "nested", - "properties": { - "dip": { "type": "ip" }, - "sip": { "type": "ip" } - } - }, - - "ps": { - "type": "nested", - "properties": { - "pid": { "type": "long" }, - "ppid": { "type": "long" }, - "name": { "type": "keyword" }, - "comm": { "type": "text" }, - "exe": { "type": "text" }, - "cwd": { "type": "text" }, - "sid": { "type": "keyword" }, - "sessionid": { "type": "short" } - } - } - } - } -} -``` - -#### index-name - -Represents the target index for kernel events. It allows time specifiers to create indices per time frame. For example, `fibratus-%Y-%m` generates the index name with current year and month. Supported time specifiers are: - -- `%Y` current year in `YYYY` format (`2020`) -- `%y` current year in `YY` format (`20`) -- `%m` current month (`01`) -- `%d` current day (`02`) -- `%H` current hour (`15`) - -**default**: `fibratus` - -#### tls-key - -Path to the public/private key file. - -#### tls-cert - -Path to the certificate file. - -#### tls-ca - -Represents the path of the certificate file that is associated with the Certification Authority (CA). - -#### tls-insecure-skip-verify - -Indicates if the chain and host verification stage is skipped. - -**default**: `false` diff --git a/docs/outputs/eventlog.md b/docs/outputs/eventlog.md deleted file mode 100644 index e1dbc7b4a..000000000 --- a/docs/outputs/eventlog.md +++ /dev/null @@ -1,139 +0,0 @@ -# Eventlog - -Exports events via [Windows Event Log](https://docs.microsoft.com/en-us/windows/win32/wes/windows-event-log) API that can be explored with the [Event Viewer](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc766042(v=ws.11)) management tool. The screenshots below illustrate event logs produced by Fibratus. The `General` tab reveals the event type that was generated. Each log event pertains to the specific `Task Category` that directly maps to the internal event category. Similar to [sysmon](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon), Fibratus classifies each event with a custom `Event ID`. - -

- -

- -The `Details` tab shows extended event data including event parameters, process, and thread information. It is possible to customize the rendering template to influence the constructed event data. This is achieved by changing the [`eventlog.template`](outputs/eventlog?id=template) configuration property. - -

- -

- -### Configuration {docsify-ignore} - -The Eventlog output configuration is located in the `outputs.eventlog` section. - -#### enabled - -Indicates whether the Eventlog output is enabled. - -**default**: `false` - - -#### level - -Specifies the eventlog level associated with the event logs produced by Fibratus. You can specify one of the following values: - -- `info`, `INFO` -- `warn`, `warning`, `WARN`, `WARNING` -- `erro`, `ERRO`, `error`, `ERROR` - -**default**: `info` - -#### remote-host - -Address of the remote eventlog intake. - -#### template - -Go [template](https://pkg.go.dev/text/template) for rendering the eventlog message. - -**default**: - -``` -Name: {{ .Kevt.Name }} -Sequence: {{ .Kevt.Seq }} -Process ID: {{ .Kevt.PID }} -Thread ID: {{ .Kevt.Tid }} -Cpu: {{ .Kevt.CPU }} -Params: {{ .Kevt.Kparams }} -Category: {{ .Kevt.Category }} - -{{- if .Kevt.PS }} - -Process: {{ .Kevt.PS.Name }} -Exe: {{ .Kevt.PS.Exe }} -Pid: {{ .Kevt.PS.PID }} -Ppid: {{ .Kevt.PS.Ppid }} -Cmdline: {{ .Kevt.PS.Comm }} -Cwd: {{ .Kevt.PS.Cwd }} -SID: {{ .Kevt.PS.SID }} -Session ID: {{ .Kevt.PS.SessionID }} -{{ if and (.SerializeEnvs) (.Kevt.PS.Envs) }} -Env: - {{- with .Kevt.PS.Envs }} - {{- range $k, $v := . }} - {{ $k }}: {{ $v }} - {{- end }} - {{- end }} -{{ end }} -{{ if .SerializeThreads }} -Threads: - {{- with .Kevt.PS.Threads }} - {{- range . }} - {{ . }} - {{- end }} - {{- end }} -{{ end }} -{{ if .SerializeImages }} -Modules: - {{- with .Kevt.PS.Modules }} - {{- range . }} - {{ . }} - {{- end }} - {{- end }} -{{ end }} -{{ if and (.SerializeHandles) (.Kevt.PS.Handles) }} -Handles: - {{- with .Kevt.PS.Handles }} - {{- range . }} - {{ . }} - {{- end }} - {{- end }} -{{ end }} - -{{ if and (.SerializePE) (.Kevt.PS.PE) }} -Entrypoint: {{ .Kevt.PS.PE.EntryPoint }} -Image base: {{ .Kevt.PS.PE.ImageBase }} -Build date: {{ .Kevt.PS.PE.LinkTime }} - -Number of symbols: {{ .Kevt.PS.PE.NumberOfSymbols }} -Number of sections: {{ .Kevt.PS.PE.NumberOfSections }} - -Sections: - {{- with .Kevt.PS.PE.Sections }} - {{- range . }} - {{ . }} - {{- end }} - {{- end }} -{{ if .Kevt.PS.PE.Symbols }} -Symbols: - {{- with .Kevt.PS.PE.Symbols }} - {{- range . }} - {{ . }} - {{- end }} - {{- end }} -{{ end }} -{{ if .Kevt.PS.PE.Imports }} -Imports: - {{- with .Kevt.PS.PE.Imports }} - {{- range . }} - {{ . }} - {{- end }} - {{- end }} -{{ end }} -{{ if .Kevt.PS.PE.VersionResources }} -Resources: - {{- with .Kevt.PS.PE.VersionResources }} - {{- range $k, $v := . }} - {{ $k }}: {{ $v }} - {{- end }} - {{- end }} - {{ end }} -{{ end }} -{{- end }} - -``` diff --git a/docs/outputs/images/eventlog-details.png b/docs/outputs/images/eventlog-details.png deleted file mode 100644 index c97b7bcc4..000000000 Binary files a/docs/outputs/images/eventlog-details.png and /dev/null differ diff --git a/docs/outputs/images/eventlog-general.png b/docs/outputs/images/eventlog-general.png deleted file mode 100644 index 3b04497c8..000000000 Binary files a/docs/outputs/images/eventlog-general.png and /dev/null differ diff --git a/docs/outputs/introduction.md b/docs/outputs/introduction.md deleted file mode 100644 index 11df133f0..000000000 --- a/docs/outputs/introduction.md +++ /dev/null @@ -1,13 +0,0 @@ -# Transporting Events - -Fibratus delivers a diverse array of output sinks to route the events. When captures are not enough, you may opt for forwarding the event stream to remote destinations such as RabbitMQ brokers or Elasticsearch clusters. Outputs expose a rich set of configuration knobs that enable to fine-tune the behaviour of the event flow transmission. - -### Event serialization tweaking {docsify-ignore} - -JSON is the default serialization format for events. Since the event state contains a vast of attributes, you can specify which fields are serialized through configuration properties located in the `kevent` section. - -- `serialize-threads` indicates whether the threads metadata are serialized as part of the process, and consequently, the event state -- `serialize-images` decides whether modules such as Dynamic Linked Libraries are serialized as part of the process state -- `serialize-handles` determines whether allocated process handles are serialized as part of the process state -- `serialize-pe` indicates if PE (Portable Executable) metadata are serialized as part of the process state -- `serialize-envs` indicates if environment variables are serialized as part of the process state diff --git a/docs/outputs/null.md b/docs/outputs/null.md deleted file mode 100644 index f9f21aea2..000000000 --- a/docs/outputs/null.md +++ /dev/null @@ -1,3 +0,0 @@ -# Null - -The null output devours kernel events the same way a black hole swallows the light. This output is useful in case you want to snooze the event stream. diff --git a/docs/overview/what-is-fibratus.md b/docs/overview/what-is-fibratus.md deleted file mode 100755 index 0770d4170..000000000 --- a/docs/overview/what-is-fibratus.md +++ /dev/null @@ -1,8 +0,0 @@ -# What is Fibratus? - -[Fibratus](https://github.com/rabbitstack/fibratus) detects, protects, and eradicates advanced adversary tradecraft by scrutinizing -and asserting a wide spectrum of system events against a behavior-driven [rule engine](/filters/rules) and [YARA](/yara/introduction) memory scanner. - -Events can also be shipped to a wide array of [output sinks](/outputs/introduction) or dumped to [capture](/captures/introduction) files for local inspection and forensics analysis. You can use [filaments](/filaments/introduction) to extend Fibratus with your own arsenal of tools and so leverage the power of the Python ecosystem. - -In a nutshell, the Fibratus mantra is defined by the pillars of **realtime behavior detection**, **memory scanning**, and **forensics capabilities**. diff --git a/docs/pe/introduction.md b/docs/pe/introduction.md deleted file mode 100644 index 0c25dd7f1..000000000 --- a/docs/pe/introduction.md +++ /dev/null @@ -1,9 +0,0 @@ -# Portable Executable Introspection - -[Portable Executable](https://en.wikipedia.org/wiki/Portable_Executable) (PE) is the Windows file format that describes the layout of the executable code. PE is frequently subject to weaponization ranging from reverse shell payload injections to several obfuscation techniques used by malware creators. - -Fibratus natively supports the scanning of the PE format data that, for example, underpins the [PE filters](/filters/fields?id=pe). To enable the PE introspection, it is necessary to edit the `pe.enabled` key in the configuration file or provide the `--pe.enabled=true` command line flag. - -### Excluding executables {docsify-ignore} - -When the PE introspection is enabled, Fibratus will try to obtain the PE data for every running process in the system. This happens during Fibratus bootstrap stage, but also when a new process is spawn. To skip gathering the PE data for some process images, you can add the image name under the `excluded-images` key in the configuration file. Avoiding the parsing of the PE format data for some process images, alleviates the pressure on Fibratus and reduces resource usage. diff --git a/docs/pe/resources.md b/docs/pe/resources.md deleted file mode 100644 index 306dd69a7..000000000 --- a/docs/pe/resources.md +++ /dev/null @@ -1,18 +0,0 @@ -# Resources - -Fibratus reads version resources from the PE resource directory. The resource contains information about the exeucutable as its version number, its intended operating system, and its original filename. An example of version resources read by Fibratus: - -``` -CompanyName: Microsoft Corporation -FileDescription: Notepad -FileVersion: 10.0.18362.693 (WinBuild.160101.0800) -InternalName: Notepad -LegalCopyright: © Microsoft Corporation. All rights reserved. -OriginalFilename: NOTEPAD.EXE -ProductName: Microsoft® Windows® Operating System -ProductVersion: 10.0.18362.693 -``` - -You can use any of these resource entries in filter expressions. For example, the `pe.resources[FileDescription] = 'Notepad'` filter matches any event where the `FileDescription` resource is equal to `Notepad`. - -To enable resource parsing, it is necessary to set the `read-resources` option to `true`. diff --git a/docs/pe/sections.md b/docs/pe/sections.md deleted file mode 100644 index a5d91013a..000000000 --- a/docs/pe/sections.md +++ /dev/null @@ -1,11 +0,0 @@ -# Sections - -Sections are the fundamental building block of the PE data. They contain the content of the file, including the executable code, data, resources and other executable artifacts. - -Fibratus obtains the number of sections, and for each section encountered in the PE data, its name and size is fetched. For example, the code section is called `.text` and the data section is called `.data`. Sometimes the malware specimens tamper the PE structure and alter sections. You can hunt for a non-standard number of sections in the executable or detect extraneous sections by checking the data surfaced by Fibratus. - -### Reading extended section data {docsify-ignore} - -For a full-blown section parsing, you can enable the `read-sections` option. This instructs the PE parser to read the underlying section bytes for the purpose of computing the `md5` hash of each section. It is possible to write filter expressions that involve evaluating the section attributes. - -For example, to match against specific `.text` section `md5` hash, you would write the `pe.sections[.text].md5 = '0464997eb36c70083164c666d53c6af3'` filter. diff --git a/docs/pe/symbols.md b/docs/pe/symbols.md deleted file mode 100644 index a73dd51c4..000000000 --- a/docs/pe/symbols.md +++ /dev/null @@ -1,7 +0,0 @@ -# Symbols - - Symbols or functions referred to by the binary that are expected to be satisfied by other libraries at dynamic load time are located in the Import Address Table (IAT). Fibratus parses this table and extracts all symbols names as well as the dynamic libraries referenced by the binary. Symbol names can be `URLDownloadToFileA` or `WriteFile`. - - To activate symbol parsing it is necessary to enable the `read-symbols` option. - - From the filtering perspective, you can write `pe.symbols in ('GetTextFaceW', 'GetProcessHeap')` or `pe.imports in ('msvcrt.dll', 'GDI32.dll')` to filter events where the originating process contains the provided symbols or imports in its binary PE data. diff --git a/docs/rules.md b/docs/rules.md new file mode 100644 index 000000000..916bed483 --- /dev/null +++ b/docs/rules.md @@ -0,0 +1,200 @@ +# Rules + +##### Fibratus rules define how behavioral patterns are detected from system telemetry. They allow expressing conditions over events and optionally trigger [response actions](rules/actions.md). Inspired by declarative detection formats like [Sigma](https://sigmahq.io/), Fibratus rules are designed to be readable, expressive, and tightly integrated with the event model. + +## Rule Structure + +A rule is defined in `YAML` format and consists of **metadata**, a detection **condition**, and optional response **actions**. + +![Rule Format](images/rule-format.png "Rule Format") + +The recommended rule naming nomenclature dictates that the rule file starts with the [MITRE](https://attack.mitre.org/tactics/enterprise/) tactic followed by the rule name. All lower case tokens separated by `_` delimiter. For example, `defense_evasion_dll_loaded_via_apc_queue.yml` + +By default, rules reside inside `%PROGRAM FILES%\Fibratus\Rules` directory. You can load any number of rule files from file system paths or URL locations. It is possible to utilize wildcard expressions in file paths to enumerate multiple rule files from a single path specifier. + +Edit the main [configuration](../setup/configuration?id=files) file. Go to the `filters` section where the `from-paths` fragment represents an array of file system paths pointing to the rule definition files: + +```yaml +filters: + rules: + from-paths: + - C:\Program Files\Fibratus\Rules\*.yml +``` + +Here is the full YAML illustrating the rule format: + +```yaml +name: File access to SAM database +id: e3dace20-4962-4381-884e-40dcdde66626 +version: 1.0.5 +description: | + Identifies access to the Security Account Manager on-disk database. +labels: + tactic.id: TA0006 + tactic.name: Credential Access + tactic.ref: https://attack.mitre.org/tactics/TA0006/ + technique.id: T1003 + technique.name: OS Credential Dumping + technique.ref: https://attack.mitre.org/techniques/T1003/ + subtechnique.id: T1003.002 + subtechnique.name: Security Account Manager + subtechnique.ref: https://attack.mitre.org/techniques/T1003/002/ + +condition: > + open_file and + file.path imatches + ( + '?:\\WINDOWS\\SYSTEM32\\CONFIG\\SAM', + '\\Device\\HarddiskVolumeShadowCopy*\\*SAM' + ) and + ps.exe not imatches + ( + '?:\\Program Files\\*', + '?:\\Program Files (x86)\\*', + '?:\\Windows\\System32\\lsass.exe', + '?:\\Windows\\System32\\srtasks.exe' + ) +action: + - name: kill + +severity: high + +min-engine-version: 3.0.0 +``` + + +## Metadata Fields + +### `name` + +A short, descriptive title for the rule. The rule name is primary message used in security alerts. + +### `id` + +A unique identifier, typically a [UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier), used to track the rule across versions. Rule identifier never changes, while the name could change over time. + +### `version` + +The rule version using semantic versioning. Increment this when modifying logic or metadata. + +### `authors` + +List of contributors or organizations responsible for the rule. + +### `description` + +A detailed explanation of what the rule detects and why it matters. This should provide enough context for analysts to understand the behavior. + +### `enabled` + +Controls whether the rule is active. Disabled rules are ignored during evaluation. + + +## Classification and Context + +### `labels` + +Structured metadata used to classify the rule. Commonly aligned with frameworks such as [MITRE ATT&CK](https://attack.mitre.org/). +Typical fields include `tactic.id`, `tactic.name`, `technique.id`, `technique.name`, `subtechnique.id` and `subtechnique.name` + +### `tags` + +Free-form labels for categorization, for example, `credential stealing` or `lateral movement` + +### `references` + +External resources that provide additional context, such as research articles or vendor reports. + + +## Detection Logic + +### `condition` + +The `condition` field defines the rule logic using a domain-specific expression language. It evaluates event fields and supports logical operators, pattern matching, and event correlation. + +```yaml +condition: > + open_file and + file.path imatches '?:\\Users\\*\\AppData\\*\\Microsoft\\Protect\\CREDHIST' and + ps.exe not imatches + ( + '?:\\Program Files\\*', + '?:\\Windows\\System32\\lsass.exe' + ) +``` + +#### Key concepts + +* **Event predicates** are expressions like `open_file` that match specific event types. To be more precise, `open_file` is a [macro](macros.md) that expands into `evt.name = 'OpenProcess'` event condition +* **Field accessors** populate [fields](fields.md) such as `file.path` or `ps.exe` from event attributes or process context +* **Operators** include `and`, `or`, `not` logical composition [operators](operators.md) or `imatches` operator for case-insensitive pattern matching with wildcards +* **Grouping** allows combining multiple expressions with parentheses + +## Output and severity + +### `output` + +Defines the alert message emitted when the rule matches. Supports field interpolation using `%field.name`, for example, `Detected an attempt by %ps.name process to access %file.path` + +### `severity` + +Indicates the importance of the security alert. Common levels include `low`, `medium`, `high`, and `critical` + +## Response actions + +### `action` + +Specifies automated responses triggered when the rule matches. + +For more details, see [actions](actions/alerts.md). + +## Engine compatibility + +### `min-engine-version` + +Specifies the minimum Fibratus version required to run the rule. This ensures compatibility with newer features or syntax. + + +## Analyst notes + +### `notes` + +Optional field for additional insights, investigation tips, or known false positives. This section is not used by the engine but is valuable for human operators. + + +## Design guidelines + +#### Stick to naming nomenclature + +It is highly recommended to name the rule files after the pattern explained in the above section. This facilitates the organization and searching through the detection rules catalog and fosters standardization. + +The CLI provides a command to create a new rule from the template. For example, if you want to create the `Potential Process Doppelganging` rule that belongs to the `Defense Evasion` MITRE tactic, you would use the following command. + + +$ fibratus rules create "Potential Process Doppelganging Injection" -t TA0005 + + + +The `-t` flag specifies the MITRE tactic id. The end result is the `defense_evasion_potential_process_doppelganging_injection.yml` file with the most required attributes such as rule identifier, name, and the minimum engine version, filled out automatically. + +#### Include descriptions and labels + +Rules should have a meaningful description. For example, `Potential process injection via tainted memory section` + +Additionally, there should exist labels attached to every rule describing the MITRE tactic, technique, and sub-technique. This information is used when rendering email rule alert templates as depicted in the image above. + +#### Rules should have a narrowed event scope + +If a rule is declared without the scoped event conditions, such as `evt.name` or `evt.category`, you'll get a warning message in `Fibratus` logs informing you about unwanted side effects. **This always lead to the rule being utterly discarded by the engine!** + +#### Pay attention to the condition arrangement + +As highlighted in the previous paragraph, all rules should have the event type condition. Additionally, condition arrangement may have important runtime performance impact because the rule engine can lazily evaluate binary expressions that comprise a rule. In general, costly evaluations or functions such as `get_reg_value` should go last to make sure they are evaluated after all other expressions have been visited. + +#### Prefer macros over raw conditions + +Fibratus comes with a [macros](https://www.fibratus.io/#/filters/rules?id=macros) library to promote the reusability and modularization of rule conditions and lists. Before trying to spell out a raw rule condition, explore the library to check if there's already a macro you can pull into the rule. For example, detecting file accesses could be accomplished by declaring the `evt.name = 'CreateFile' and file.operation = 'open'` expression. However, the macro library comes with the `open_file` macro that you can directly call in any rule. If you can't encounter a particular macro in the library, please consider creating it. Future detection engineers and rule writers could profit from those macros. + +#### Formatting styles + +Pay attention to rule condition/action formatting style. If the rule consists of multiple or large expressions, it is desirable to split each spanning expression on a new line. This notably improves readability and prevents formatting inconsistencies. \ No newline at end of file diff --git a/docs/rules/actions.md b/docs/rules/actions.md new file mode 100644 index 000000000..05b28d2e6 --- /dev/null +++ b/docs/rules/actions.md @@ -0,0 +1,6 @@ +# Actions + +##### Response actions are a core mechanism that allow detection rules to actively react to security-relevant events. Actions are responses executed as a consequence of rule matches. They provide alerting and prevention capabilities aim at stopping the adversary at the initial stages of the attack. + + +Instead of merely observing and alerting, response actions enable to perform automated operations when a rule matches an event condition. This triggers immediate reaction to suspicious behavior such as process creation, injection attempts, or anomalous file activity. diff --git a/docs/rules/actions/alert.md b/docs/rules/actions/alert.md new file mode 100644 index 000000000..8e9194d7d --- /dev/null +++ b/docs/rules/actions/alert.md @@ -0,0 +1,168 @@ +# Alert + +##### Fibratus can generate security alerts via multiple delivery channels when a detection rule or a [YARA](../../yara.md) rule matches. Alerts can also be emitted programmatically from [filaments](../../filaments.md), enabling fully customizable alerting workflows. + +Each alert consists of the following key components: + +| COMPONENT | DESCRIPTION | +| :--- | :---- | +| `id` | A unique identifier for the alert, represented as a UUID. For detection alerts, this identifier is immutable, meaning it remains constant for a given rule instance. | +| `title` | A short, human-readable summary describing the alert. | +| `text` | A detailed message providing additional context, such as the activity observed and the entities involved. | +| `description`| Provides additional context about the alert, typically derived from the rule description. | +| `tags` | A list of tags used to categorize and filter alerts. | +| `lables` | A list of labels in key/value format. | +| `severity` | Indicates the importance of the alert. Supported values are `low`, `medium`, `high` and `critical` | +| `events` | List of events that participated in the alert containing full process context and callstacks. | + +Alerts are automatically generated when a rule matches and are dispatched through all configured alert senders. By default, the `Eventlog` sender is enabled, publishing security alerts to the Windows Event Log. + +## Customizing alerts + +By default, the rule name is used as the alert title. To provide richer context, rules can define an `output` attribute, which is used as the alert message. The `output` field supports [Markdown](https://www.markdownguide.org/) formatting, enabling enhanced presentation and better readability in alerts. + +The `output` field supports field interpolation, allowing dynamic insertion of event data into the alert. Fields are referenced using format modifiers prefixed with `%`. In this example, `%ps.exe` is resolved to the full path of the process executable associated with the event. + +```yaml +output: > + %ps.exe process spawned a command shell after connecting to the remote endpoint +``` + +Format modifiers correspond to well-known event fields and provide a concise way to reference structured data within alerts. They enable rules to produce highly contextual and actionable messages without hardcoding values. +For sequence rules, alert messages can reference specific events within the sequence by using ordinal prefixes. The ordinal represents the position of the event in the sequence. For example, `%1.ps.name` derives the process name from the first event in the sequence, while `%2.file.name` extracts the file name from the second event in the sequence. + +This allows precise attribution of actions across multiple related events. As an example: + +```yaml +output: > + Detected an attempt by %1.ps.name process to access + and read the memory of the **Local Security Authority Subsystem Service**, + followed by writing the %2.file.name dump file to disk. +``` + +## Publishing alerts + +Alert notifications can be delivered via email, Slack, Eventlog and other alert senders. Alerts may be sent through multiple senders simultaneously. Alert sender configuration is defined in the `alertsenders` section of the YAML configuration file. + +### `Eventlog` + +The `eventlog` alert sender sends alerts to the [Windows Eventlog](https://sematext.com/glossary/what-is-windows-event-log/). Eventlog alert sender configuration is located in the `alertsenders.eventlog` section. Here are the available configuration knobs. + +##### `enabled` + +Indicates whether the `eventlog` alert sender is enabled. + +##### `verbose` + +Enables/disables the verbose mode. In verbose mode, the full event context, including all parameters and the process information are included +in the log message. + +##### `format` + +Can be `pretty` or `json` + +### `Mail` + +The mail alert sender emits alert notifications through the `SMTP` protocol. + +!> If you are using a Gmail SMTP provider, you might need to configure [App Passwords](https://support.google.com/accounts/answer/185833?hl=en) for your account. + +The `mail` alert sender configuration is located in the `alertsenders.mail` section. Here are the available configuration knobs. + +#### `enabled` + +Indicates if the alert sender is enabled. + +#### `host` + +Represents the host name of the SMTP server where the alert notification is sent. + +#### `port` + +Represents the port number of the SMTP server. + +#### `user` + +Specifies the user name when authenticating to the SMTP server. + +#### `password` + +Specifies the password when authenticating to the SMTP server. + +#### `from` + +Determines the sender's email address. + +#### `to` + +Specifies the list of the recipient email addresses. + +#### `use-template` + +Indicates if the alert is rendered using the built-in HTML template. + +### `Slack` + +The `slack` alert sender forwards alerts to Slack workspaces. You'll have to [activate](https://slack.com/intl/en-es/help/articles/115005265063-Incoming-webhooks-for-Slack) incoming webhooks and associate the webhook to your Slack workspace to be able to emit the alerts. The `slack` alert sender configuration is located in the `alertsenders.slack` section. + +#### `enabled` + +Indicates whether the `slack` alert sender is enabled. + +#### `url` + +Represents the Webhook URL of the workspace where alerts will be dispatched. The Webhook URL has the following format: `https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX` + +#### `workspace` + +The name of the Slack workspace where alerts are routed. + +#### `channel` + +Is the slack channel in which to post alerts. + +#### `emoji` + +Represents the emoji icon surrounded in `:` characters for the Slack bot. + +### `Systray` + +The `systray` alert sender sends alerts to the systray notification area. Alert sender configuration is located in the `alertsenders.systray` section. + +#### `enabled` + +Indicates whether the `systray` alert sender is enabled. + +#### `sound` + +Indicates if the associated sound is played when the balloon notification is shown. + +#### `quiet-mode` + +Instructs not to display the balloon notification if the current user is in quiet time. During this time, most notifications should not be sent or shown. This lets a user become accustomed to a new computer system without those distractions. Quiet time also occurs for each user after an operating system upgrade or clean installation. + +### `Filaments` + +Filaments can generate alerts by invoking the `emit_alert` function. Once emitted, the alert is automatically propagated to all active alert senders. The `emit_alert` function accepts two required positional arguments and two optional keyword arguments: + +```python +emit_alert(title, text, severity='normal', tags=[]) +``` + +| ARGUMENT | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | +| `title` | A short summary of the alert. | yes | +| `text` | A detailed message describing the alert. | yes | +| `severity` | Alert severity level. Defaults to `normal` | no | +| `tags` | A list of tags used to categorize the alert. | no | + +The following example demonstrates how to emit an alert from a filament that detects registry persistence activity. In this example, the alert title dynamically includes the registry key path, while the `text` function provides additional context derived from the event. + +```python +emit_alert( + f'Registry persistence detected in path {event.params.registry_path}', + text(event), + severity='medium', + tags=['registry persistence'] +) +``` \ No newline at end of file diff --git a/docs/rules/actions/isolate.md b/docs/rules/actions/isolate.md new file mode 100644 index 000000000..8653ece4c --- /dev/null +++ b/docs/rules/actions/isolate.md @@ -0,0 +1,9 @@ +# Isolate + +##### The isolate action restricts the host’s network connectivity to contain potential threats. + +When triggered, this action applies network isolation policies that block outbound and/or inbound connections, effectively quarantining the host from the network. The `isolate` action leverages [Windows Filtering Platform](https://learn.microsoft.com/en-us/windows/win32/fwp/windows-filtering-platform-start-page) (WFP) to define firewall policies. Here is an example of using the action in the rule definition. + +```yaml +action: + - name: isolate diff --git a/docs/rules/actions/kill.md b/docs/rules/actions/kill.md new file mode 100644 index 000000000..6c033a1ca --- /dev/null +++ b/docs/rules/actions/kill.md @@ -0,0 +1,12 @@ +# Kill + +##### The kill action terminates processes associated with the event that triggered the rule. + +When a rule with the kill action matches, Fibratus attempts to stop the target process by its process identifier (PID). This is useful for halting malicious execution before it can propagate or cause further damage. + +The action is defined in the rule `action` YAML field. + +```yaml +action: + - name: kill +``` \ No newline at end of file diff --git a/docs/rules/fields.md b/docs/rules/fields.md new file mode 100644 index 000000000..35cbfa81f --- /dev/null +++ b/docs/rules/fields.md @@ -0,0 +1,353 @@ +# Fields + +##### In the Fibratus rule language, fields represent structured attributes extracted from process metadata, file paths, registry keys, or callstacks. + +Each field evaluates to a specific data type, which determines how it can be compared, which operators are valid, and which functions can be applied. Understanding field types is essential for writing correct and efficient rules. Fields are accessed using `dot` notation, which reflects their hierarchical structure. This hierarchy mirrors the underlying event schema and helps keep rules expressive and readable. Some fields support optional or required arguments, accessed using `bracket` notation, for example, `evt.arg[exe]` + +## Supported data types + +### String + +String values represent textual data such as: + +* File paths `file.name` or `file.path` +* Process metadata `ps.name` or `ps.exe` +* Registry keys `registry.path` + +Strings must be enclosed in single quotes. Special characters must be escaped. String comparisons are case-sensitive by default. Use case-insensitive operators or normalization [functions](functions.md). + +### Number + +Numeric fields can be integers or floating-point numbers. Numbers use dot (`.`) notation for decimals. + +### Boolean + +Boolean fields represent truth values including `true` and `false`. Booleans are commonly used in flags, security attributes and state indicators. + +### IP address + +IP fields represent `IPv4` addresses. These fields support equality comparisons and range checks depending on operators/functions available. + +!> IP values are not strings and they are parsed and compared as structured addresses. + +### Collections + +Certain filter fields return a sequence of items rather than a primitive value. For example, `ps.modules` returns an array of process DLLs. + +## Filter fields + +The following tables summarize available field names that can be employed in detection rules. + +### Event + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `evt.seq` | Monotonic event sequence number | `evt.seq > 666` | +| `evt.pid` | Process identifier generating the event | `evt.pid = 6` | +| `evt.tid` | Thread identifier generating the event | `evt.tid = 1024` | +| `evt.cpu` | Logical processor core where the event was generated | `evt.cpu = 2` | +| `evt.name` | Symbolical event name | `evt.name = 'CreateThread'` | +| `evt.category` | Category to which the event pertains | `evt.category = 'registry'` | +| `evt.desc` | Cursory event description | `evt.desc contains 'Creates'` | +| `evt.host` | Hostname on which the event was produced | `evt.host contains 'dev'` | +| `evt.nparams` | Number of event parameters | `evt.nparams > 2` | +| `evt.time` | Event timestamp as a time string | `evt.time = '17:05:32'` | +| `evt.time.h` | Hour within the day on which the event occurred | `evt.time.h = 23` | +| `evt.time.m` | Minute offset within the hour on which the event occurred | `evt.time.m = 54` | +| `evt.time.s` | Second offset within the minute on which the event occurred | `evt.time.s = 0` | +| `evt.time.ns` | Nanoseconds specified by the event timestamp | `evt.time.ns > 1591191629102337000` | +| `evt.date` | Event timestamp as a date string | `evt.date = '2018-03-03'` | +| `evt.date.d` | Day of the month on which the event occurred | `evt.date.d = 12` | +| `evt.date.m` | Month of the year on which the event occurred | `evt.date.m = 11` | +| `evt.date.y` | Year on which the event occurred | `evt.date.y = 2020` | +| `evt.date.tz` | Time zone associated with the event timestamp | `evt.date.tz = 'UTC'` | +| `evt.date.week` | Week number within the year on which the event occurred | `evt.date.week = 2` | +| `evt.date.weekday` | Week day on which the event occurred | `evt.date.weekday = 'Monday'` | +| `evt.arg[]` | Accesses a specific event parameter via internal name | `evt.arg[exe] = 'C:\\Windows\\cmd.exe'` | +| `evt.is_direct_syscall` | Indicates if this event is originated by a direct syscall | `evt.is_direct_syscall` | +| `evt.is_indirect_syscall` | Indicates if this event is originated by an indirect syscall | `evt.is_indirect_syscall` | + +### Process + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `ps.pid` | Process identifier generating the event. Alias for `evt.pid` | `ps.pid = 1024` | +| `ps.ppid` | Parent process identifier of the process generating the event | `ps.ppid = 25` | +| `ps.name` | Process (image) path name that generates an event | `ps.name = 'cmd.exe'` | +| `ps.cmdline` | Process command line | `ps.cmdline contains '/E c:\\ads\\file.txt:regfile.reg'` | +| `ps.exe` | Full name of the process' executable | `ps.exe = 'C:\\Windows\\system32\\cmd.exe'` | +| `ps.args` | Process command line arguments | `ps.args in ('/cdir', '/-C')` | +| `ps.cwd` | Process current working directory | `ps.cwd = 'C:\\Users\\Default'` | +| `ps.sid` | Security identifier under which this process is run | `ps.sid = 'S-1-5-18'` | +| `ps.domain` | Process domain name | `ps.domain = 'NT AUTHORITY'` | +| `ps.username` | Process user name | `ps.username = 'SYSTEM'` | +| `ps.sessionid` | Unique identifier for the current session | `ps.sessionid = 1` | +| `ps.access.mask` | Process access rights | `ps.access.mask = '0x1000'` | +| `ps.access.mask.names` | Process access human-readable rights | `ps.access.mask.names in ('TERMINATE', 'QUERY_INFORMATION')` | +| `ps.access.status` | Process access status | `ps.access.status = 'success'` | +| `ps.envs` | Process environment variables | `ps.envs in ('MOZ_CRASHREPORTER_DATA_DIRECTORY')` | +| `ps.envs[]` | Accesses a specific environment variable. Prefix matches are supported | `ps.envs['MOZ_CRASHREPORTER'] = 'C:\\Program Files\\Firefox'` | +| `ps.dtb` | Process directory table base address | `ps.dtb = '7ffe0000'` | +| `ps.handles` | Allocated process handles | `ps.handles in ('\\BaseNamedObjects\\__ComCatalogCache__')` | +| `ps.handle.types` | Allocated process handle types | `ps.handle.types in ('Key', 'Mutant', 'Section')` | +| `ps.modules` | Modules loaded by the process | `ps.modules in ('C:\\Windows\\System32\\crypt32.dll')` | +| `ps.parent.name` | Parent process name | `ps.parent.name = 'powershell.exe'` | +| `ps.parent.pid` | Parent process identifier | `ps.parent.pid = 2340` | +| `ps.parent.cmdline` | Parent process command line | `ps.parent.cmdline contains 'attrib'` | +| `ps.parent.exe` | Full name of the parent process executable | `ps.parent.exe = 'C:\\Windows\\system32\\cmd.exe'` | +| `ps.parent.cwd` | Parent process current working directory | `ps.parent.cwd = 'C:\\Users\\Default'` | +| `ps.parent.sid` | Security identifier under which the parent process is run | `ps.parent.sid = 'S-1-5-18'` | +| `ps.parent.domain` | Parent process domain name | `ps.parent.domain = 'NT AUTHORITY'` | +| `ps.parent.username` | Parent process user name | `ps.parent.username = 'SYSTEM'` | +| `ps.parent.sessionid` | Unique identifier for the current session of the parent process | `ps.parent.session = 1` | +| `ps.parent.dtb` | Parent process directory table base address | `ps.parent.dtb = 'powershell.exe'` | +| `ps.parent.envs` | Parent process environment variables | `ps.parent.envs in ('PROCESSOR_LEVEL')'` | +| `ps.parent.handles` | Allocated parent process handles | `ps.parent.handles in ('\\...\\Cor_SxSPublic_IPCBlock')` | +| `ps.parent.handle.types` | Allocated parent process handles types | `ps.parent.handle.types in ('Key', 'Mutant', 'Section')` | +| `ps.ancestor` | Process ancestors | `ps.ancestor in ('winword.exe', 'powershell.exe')` | +| `ps.ancestor[]` | Access an ancestor at the specified level | `ps.ancestor[1] = 'winword.exe'` | +| `ps.is_wow64` | Indicates if the process generating the event is a 32-bit child process is created in 64-bit Windows system | `ps.is_wow64` | +| `ps.is_packaged` | Indicates if the process process generating the event is packaged with the MSIX technology | `ps.is_packaged` | +| `ps.is_protected` | Indicates if the process generating the event is a protected process | `ps.is_protected` | +| `ps.parent.is_wow64` | Indicates if the parent process generating the event is a 32-bit process created in 64-bit Windows system | `ps.parent.is_wow64` | +| `ps.parent.is_packaged` | Indicates if the parent process generating the event is packaged with the MSIX technology | `ps.parent.is_packaged` | +| `ps.parent.is_protected` | Indicates if the parent process generating the event is a protected process | `ps.parent.is_protected` | +| `ps.token.integrity_level` | Process token integrity level | `ps.token.integrity_level = 'HIGH'` | +| `ps.token.elevation_type` | Process token elevation type | `ps.token.elevation_type = 'LIMITED'` | +| `ps.token.is_elevated` | Indicates if the process token is elevated | `ps.token.is_elevated` | +| `ps.parent.token.integrity_level` | Process parent token integrity level | `ps.parent.token.integrity_level = 'HIGH'` | +| `ps.parent.token.elevation_type` | Process parent token elevation type | `ps.parent.token.elevation_type = 'LIMITED'` | +| `ps.parent.token.is_elevated` | Indicates if the parent process token is elevated | `ps.parent.token.is_elevated` | +| `ps.signature.exists` | Indicates if the process executable is signed | `ps.signature.signed` | +| `ps.signature.trusted` | Indicates if the process executable is trusted | `ps.signature.trusted` | +| `ps.signature.serial` | Process executable signature certificate serial number | `ps.signature.serial = '330000023241fb59996dcc4dff000000000232'` | +| `ps.signature.subject` | Process executable signature certificate subject | `ps.signature.subject contains 'Washington, Redmond, Microsoft Corporation'` | +| `ps.signature.issuer` | Process executable signature certificate CA | `ps.signature.issuer contains 'US, Washington, Redmond, Microsoft Windows Production PCA 2011'` | +| `ps.signature.after` | Process executable signature certificate expiration date | `ps.signature.after contains '2024-02-01 00:05:42 +0000 UTC'` | +| `ps.signature.before` | Process executable signature certificate enrollment date | `ps.signature.before contains '2024-02-01 00:05:42 +0000 UTC'` | + + +### Thread + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `thread.prio` | Scheduler priority of the thread | `thread.prio = 5` | +| `thread.io.prio` | I/O priority hint for scheduling I/O operations | `thread.io.prio = 4` | +| `thread.page.prio` | Memory page priority hint for memory pages accessed by the thread | `thread.page.prio = 12` | +| `thread.kstack.base` | Base address of the thread's kernel space stack | `thread.kstack.base = 'a65d800000'` | +| `thread.kstack.limit` | Limit of the thread's kernel space stack | `thread.kstack.limit = 'a85d800000'` | +| `thread.ustack.base` | Base address of the thread's user space stack | `thread.ustack.base = '7ffe0000'` | +| `thread.ustack.limit` | Limit of the thread's user space stack | `thread.ustack.limit = '8ffe0000'` | +| `thread.start_address` | Start address of the function to be executed by the thread | `thread.start_address = '7efe0000'` | +| `thread.access.mask` | Thread access rights | `thread.access.mask = '0x1800'` | +| `thread.access.mask.names` | Thread access human-readable rights | `thread.access.mask.names in ('QUERY_LIMITED_INFORMATION')` | +| `thread.access.status` | Thread access status | `thread.access.status = 'Success'` | +| `thread.teb_address` | The base address of the thread environment block | `thread.teb_address = '8f30893000'` | +| `thread.start_address.symbol` | Thread start address symbol | `thread.start_address.symbol = 'LoadImage'` | +| `thread.start_address.module` | Thread start address module | `thread.start_address.module endswith 'kernel32.dll'` | + + +### Threadpool + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `threadpool.id` | Thread pool identifier | `threadpool.id = '20f5fc02440'` | +| `threadpool.task.id` | Thread pool task identifier | `threadpool.task.id = '20f7ecd21f8'` | +| `threadpool.callback.address` | Thread pool callback address | `threadpool.callback.address = '7ff868739ed0'` | +| `threadpool.callback.symbol` | Thread pool callback address symbol | `threadpool.callback.symbol = 'RtlDestroyQueryDebugBuffer'` | +| `threadpool.callback.module` | Thread pool callback address module | `threadpool.callback.module contains 'ntdll.dll'` | +| `threadpool.callback.context` | Thread pool callback context address | `threadpool.callback.context = '1df41e07bd0'` | +| `threadpool.callback.context.rip` | Thread pool callback thread context instruction pointer | `threadpool.callback.context.rip = '1df42ffc1f8'` | +| `threadpool.callback.context.rip.symbol` | Thread pool callback thread context instruction pointer symbol | `threadpool.callback.context.rip.symbol = 'VirtualProtect'` | +| `threadpool.callback.context.rip.module` | Thread pool callback thread context instruction pointer module | `threadpool.callback.context.rip.module contains 'ntdll.dll'` | +| `threadpool.subprocess_tag` | Thread pool service identifier | `threadpool.subprocess_tag = '10d'` | +| `threadpool.timer.duetime` | Thread pool timer due time | `threadpool.timer.duetime > 10` | +| `threadpool.timer.subqueue` | Thread pool timer subqueue address | `threadpool.timer.subqueue = '1db401703e8'` | +| `threadpool.timer.address` | Thread pool timer address | `threadpool.timer.address = '3e8'` | +| `threadpool.timer.period` | Thread pool timer period | `threadpool.timer.period = 0` | +| `threadpool.timer.window` | Thread pool timer tolerate period | `threadpool.timer.window = 0` | +| `threadpool.timer.is_absolute` | Indicates if the thread pool timer is absolute or relative | `threadpool.timer.is_absolute = true` | + + +### Callstack + +| Field Name | Description | Example | +| :--- | :---- | :--- | +| `thread.callstack.summary` | Callstack summary showing involved modules | `thread.callstack.summary contains 'ntdll.dll\|KERNELBASE.dll'` | +| `thread.callstack.detail` | Detailed information of each stack frame | `thread.callstack.detail contains 'KERNELBASE.dll!CreateProcessW'` | +| `thread.callstack.modules` | List of modules comprising the callstack | `thread.callstack.modules in ('C:\WINDOWS\System32\KERNELBASE.dll')` | +| `thread.callstack.symbols` | List of symbols comprising the callstack | `thread.callstack.symbols in ('ntdll.dll!NtCreateProcess')` | +| `thread.callstack.allocation_sizes` | Allocation sizes of private pages | `thread.callstack.allocation_sizes > 10000` | +| `thread.callstack.protections` | Page protections masks of each frame | `thread.callstack.protections in ('RWX', 'WX')'` | +| `thread.callstack.callsite_leading_assembly` | Callsite leading assembly instructions | `thread.callstack.callsite_leading_assembly in ('mov r10,rcx', 'syscall')` | +| `thread.callstack.callsite_trailing_assembly` | Callsite trailing assembly instructions | `thread.callstack.callsite_trailing_assembly in ('add esp, 0xab')` | +| `thread.callstack.is_unbacked` | Indicates if the callstack contains unbacked regions | `thread.callstack.is_unbacked` | +| `thread.callstack.addresses` | List of all callstack return addresses | `thread.callstack.addresses in ('7ffb5c1d0396')` | +| `thread.callstack.final_user_module.name` | The final user module name | `thread.callstack.final_user_module.name != 'ntdll.dll'` | +| `thread.callstack.final_user_module.path` | The final user module path | `thread.callstack.final_user_module.path imatches '?:\\Windows\\System32\\ntdll.dll'` | +| `thread.callstack.final_user_symbol.name` | The final user symbol name | `thread.callstack.final_user_symbol.name imatches 'CreateProcess*'` | +| `thread.callstack.final_kernel_module.name` | The final kernel module name | `thread.callstack.final_kernel_module.name = 'FLTMGR.SYS'` | +| `thread.callstack.final_kernel_module.path` | The final kernel module path | `thread.callstack.final_kernel_module.path imatches '?:\\WINDOWS\\System32\\drivers\\FLTMGR.SYS'` | +| `thread.callstack.final_kernel_symbol.name` | The final kernel symbol name | `thread.callstack.final_kernel_symbol.name = 'FltGetStreamContext'` | +| `thread.callstack.final_user_module.signature.is_signed` | Indicates if the final user module is signed | `thread.callstack.final_user_module.signature.is_signed = true` | +| `thread.callstack.final_user_module.signature.is_trusted` | Indicates if the final user module signature is trusted | `thread.callstack.final_user_module.signature.is_trusted = true` | +| `thread.callstack.final_user_module.signature.cert.issuer` | The final user module signature certificate issuer | `thread.callstack.final_user_module.signature.cert.issuer imatches '*Microsoft Corporation*'` | +| `thread.callstack.final_user_module.signature.cert.subject` | The final user module signature certificate subject | `thread.callstack.final_user_module.signature.cert.subject imatches '*Microsoft Windows*'` | + +### Module + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `module.path` | Full module path | `module.path = 'C:\\Windows\\System32\\advapi32.dll'` | +| `module.path.stem` | Module path without extension | `module.path.stem = 'C:\\Windows\\System32\\advapi32'` | +| `module.name` | Module name | `module.name = 'advapi32.dll'` | +| `module.base` | Base address of the process in which the module is loaded | `module.base = 'a65d800000'` | +| `module.checksum` | Module checksum | `module.checksum = 746424` | +| `module.size` | Module VA size | `module.size > 1024` | +| `module.default_address` | Default module address | `module.default_address = '7efe0000'` | +| `module.signature.type` | Module signature type | `module.signature.type != 'NONE'` | +| `module.signature.level` | Module signature level | `module.signature.level = 'AUTHENTICODE'` | +| `module.signature.exists` | Indicates if module signature exists | `module.signature.exists` | +| `module.signature.trusted` | Indicates if module signature is trusted | `module.signature.trusted` | +| `module.signature.serial` | Module certificate serial number | `module.signature.serial = '330000023241fb59996dcc4dff000000000232'` | +| `module.signature.subject` | Module certificate subject | `module.signature.subject contains 'Washington, Redmond, Microsoft Corporation'` | +| `module.signature.issuer` | Module certificate CA | `module.signature.issuer contains 'US, Washington, Redmond, Microsoft Windows Production PCA 2011` | +| `imodule.signature.after` | Module certificate expiration date | `module.signature.after contains '2024-02-01 00:05:42 +0000 UTC'` | +| `module.signature.before` | Module certificate enrollment date | `module.signature.before contains '2024-02-01 00:05:42 +0000 UTC'` | +| `image.is_driver_malicious` | Indicates if the loaded driver is malicious | `module.is_driver_malicious` | +| `image.is_driver_vulnerable` | Indicates if the loaded driver is vulnerable | `module.is_driver_vulnerable` | +| `module.is_dll` | Indicates if the loaded module is a DLL | `module.is_dll` | +| `module.is_driver` | Indicates if the loaded module is a driver | `module.is_driver` | +| `module.is_exec` | Indicates if the loaded module is an executable | `module.is_exec` | +| `module.pe.is_dotnet` | Indicates if the loaded module is a .NET assembly | `module.pe.is_dotnet` | +| `dll.path` | Same as `module.path` but for DLL modules | `dll.path = 'C:\\Windows\\System32\\advapi32.dll'` | +| `dll.path.stem` | Same as `module.path.stem` but for DLL modules | `dll.path.stem = 'C:\\Windows\\System32\\advapi32'` | +| `dll.name` | Same as `module.name` but for DLL modules| `dll.name = 'advapi32.dll'` | +| `dll.base` | Same as `module.base` but for DLL modules | `dll.base = 'a65d800000'` | +| `dll.size` | Same as `module.size` but for DLL modules | `dll.size > 1024` | +| `dll.signature.type` | Same as `module.signature.type` but for DLL modules | `dll.signature.type != 'NONE'` | +| `dll.signature.level` | Same as `module.signature.level` but for DLL modules | `dll.signature.level = 'AUTHENTICODE'` | +| `dll.signature.exists` | Same as `module.signature.exists` but for DLL modules | `dll.signature.exists` | +| `dll.signature.trusted` | Same as `module.signature.trusted` but for DLL modules | `dll.signature.trusted` | +| `dll.signature.serial` | Same as `module.signature.serial` but for DLL modules | `dll.signature.serial = '330000023241fb59996dcc4dff000000000232'` | +| `dll.signature.subject` |Same as `module.signature.subject` but for DLL modules| `dll.signature.subject contains 'Washington, Redmond, Microsoft Corporation'` | +| `dll.signature.issuer` | Same as `module.signature.issuer` but for DLL modules | `dll.signature.issuer contains 'US, Washington, Redmond, Microsoft Windows Production PCA 2011` | +| `dll.signature.after` | Same as `module.signature.after` but for DLL modules | `dll.signature.after contains '2024-02-01 00:05:42 +0000 UTC'` | +| `dll.signature.before` | Same as `module.signature.before` but for DLL modules | `dll.signature.before contains '2024-02-01 00:05:42 +0000 UTC'` | +| `dll.pe.is_dotnet` | Same as `module.pe.is_dotnet` but for DLL modules | `dll.pe.is_dotnet` | + + +### File + +| Field Name | Description | Example | +| :--- | :---- | :--- | +| `file.object` | File object address in the kernel space | `file.object = 18446738026482168384` | +| `file.path` | Full file path | `file.path = 'C:\\Windows\\Sytem32\\regedit.exe'` | +| `file.path.stem` | File path without extension | `file.path.stem = 'C:\\Windows\\Sytem32\\regedit'` | +| `file.name` | File name | `file.name = 'regedit.exe'` | +| `file.operation` | Operation performed on the file or I/O device | `file.operation = 'OPEN'` | +| `file.share.mask` | File share mask | `file.share.mask = 'READ'` | +| `file.io.size` | I/O read/write size | `file.io.size > 512` | +| `file.offset` | Read/write position in the file | `file.offset = 1024` | +| `file.type` | File type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown` | `file.type = 'Directory'` | +| `file.extension` | File extension represents the file extension (e.g. .exe or .dll) | `file.extension = '.dll'` | +| `file.attributes` | List of file attributes | `file.attributes in ('HIDDEN', 'TEMPORARY')` | +| `file.status` | System status message of the `CreateFile` operation | `file.status = 'Success'` | +| `file.view.base` | Base address of the mapped/unmapped section view | `file.view.base = '25d42170000'` | +| `file.view.size` | Size of the mapped/unmapped section view | `file.view.size > 1024` | +| `file.view.type` | Type of the mapped/unmapped section view | `file.view.type = 'IMAGE'` | +| `file.view.protection` | Protection rights of the section view | `file.view.protection = 'READONLY'` | +| `file.is_driver_malicious` | Indicates if the dropped driver is malicious | `file.is_driver_malicious` | +| `file.is_driver_vulnerable` | Indicates if the dropped driver is vulnerable | `file.is_driver_vulnerable` | +| `file.is_dll` | Indicates if the created file is a DLL | `file.is_dll` | +| `file.is_driver` | Indicates if the created file is a driver | `file.is_driver` | +| `file.is_exec` | Indicates if the created file is an executable | `file.is_exec` | +| `file.info_class` | Identifies the file information class | `file.info_class = 'Allocation'` | +| `file.info.allocation_size` | Represents the file allocation size set via `NtSetInformationFile` syscall | `file.info.allocation_size > 645400` | +| `file.info.eof_size` | Represents the file EOF size set via `NtSetInformationFile` syscall | `file.info.eof_size > 1000` | +| `file.info.is_disposition_file_delete` | Indicates if the file is deleted when its handle is closed | `file.info.is_disposition_file_delete = true` | + + +### Registry + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `registry.path` | Fully qualified registry path | `registry.path = 'HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services'` | +| `registry.key.name` | Registry key name | `registry.key.name = 'Services'` | +| `registry.key.handle` | Registry key object address | `registry.key.handle = 'FFFFB905D60C2268'` | +| `registry.value` | Registry value name | `registry.value = 'Version'` | +| `registry.value.type` | Registry value type | `registry.value.type = 'REG_SZ'` | +| `registry.data` | Registry value data | `registry.data = '%windir%\system32\rundll32.exe'` | +| `registry.status` | Registry operation status | `registry.status != 'Success'` | + +### Network + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `net.dip` | Destination IP address | `net.dip = 172.17.0.3` | +| `net.sip` | Source IP address | `net.sip = 127.0.0.1` | +| `net.dport` | Destination port | `net.dport in (80, 443, 8080)` | +| `net.sport` | Source port | `net.sport != 3306` | +| `net.dport.name` | Destination port name as per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service assignation | `net.dport.name = 'dns'` | +| `net.sport.name` | Source port name as per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service assignation | `net.sport.name = 'http'` | +| `net.l4.proto` | Layer 4 protocol name | `net.l4.proto = 'TCP'` | +| `net.size` | Network packet size | `net.size > 512` | +| `net.dip.names` | List of destination IP address domain names | `net.dip.names in ('github.com.')` | +| `net.sip.names` | List of source IP address domain names | `net.sip.names in ('github.com.')` | + + +### Handle + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `handle.id` | Handle identifier | `handle.id = 24` | +| `handle.object` | Handle kernel object address | `handle.object = 'FFFFB905DBF61988'` | +| `handle.name` | Handle name | `handle.name = '\\Device\\NamedPipe\\chrome.12644.28.105826381'` | +| `handle.type` | Handle type | `handle.type = 'Mutant'` | + + +### Memory + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `mem.address` | Base address of the allocated region | `mem.address = '211d13f2000'` | +| `mem.size` | Size of the allocated region | `mem.size > 438272` | +| `mem.alloc` | Region allocation or release type | `mem.alloc = 'COMMIT'` | +| `mem.type` | Designates the page type of the allocated region | `mem.type = 'PRIVATE'` | +| `mem.protection` | Designates the protection type of the allocated region | `mem.protection = 'READWRITE'` | +| `mem.protection.mask` | Designates the allocated region protection in mask notation | `mem.protection.mask = 'RWX'` | + + +### DNS + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `dns.name` | DNS query name | `dns.name = 'example.org'` | +| `dns.rr` | DNS resource record type | `dns.rr = 'AA'` | +| `dns.options` | DNS query options | `dns.options in ('ADDRCONFIG', 'DUAL_ADDR')` | +| `dns.rcode` | DNS response status | `dns.rcode = 'NXDOMAIN'` | +| `dns.answers` | DNS response answers | `dns.answers in ('o.lencr.edgesuite.net', 'a1887.dscq.akamai.net')` | + + +### PE + +| Field Name | Description | Example | +| :--- | :---- | :---: | +| `ps.pe.nsections` | Number of sections | `ps.pe.nsections < 5` | +| `ps.pe.nsymbols` | Number of entries in the symbol table | `ps.pe.nsymbols > 230` | +| `ps.pe.address.base` | Image base address | `ps.pe.address.base = '140000000'` | +| `ps.pe.address.entrypoint` | Address of the entrypoint function | `pe.address.entrypoint = '20110'` | +| `ps.pe.symbols` | Imported symbols | `ps.pe.symbols in ('GetTextFaceW', 'GetProcessHeap')` | +| `ps.pe.imports` | Imported dynamic linked libraries | `ps.pe.imports in ('msvcrt.dll', 'GDI32.dll')` | +| `ps.pe.imphash` | Import hash | `ps.pe.impash = '5d3861c5c547f8a34e471ba273a732b2'` | +| `ps.pe.resources` | Version and other PE resources | `ps.pe.resources[FileDescription] = 'Notepad'` | +| `ps.pe.company` | Internal company name of the file provided at compile-time | `ps.pe.company = 'Microsoft Corporation'` | +| `ps.pe.copyright` | Copyright notice for the file emitted at compile-time | `ps.pe.company = '© Microsoft Corporation'` | +| `ps.pe.description` | Internal description of the file provided at compile-time | `ps.pe.description = 'Notepad'` | +| `ps.pe.file.name` | Original file name supplied at compile-time | `ps.pe.file.name = 'NOTEPAD.EXE'` | +| `ps.pe.file.version` | File version supplied at compile-time | `ps.pe.file.version = '10.0.18362.693 (WinBuild.160101.0800)'` | +| `ps.pe.product` | Internal product name of the file provided at compile-time | `ps.pe.product = 'Microsoft® Windows® Operating System'` | +| `ps.pe.product.name` | Internal product version of the file provided at compile-time | `ps.pe.product.version = '10.0.18362.693'` | +| `ps.pe.is_dotnet` | Indicates if the PE contains CLR (Common Language Runtime) data | `ps.pe.is_dotnet` | +| `ps.pe.is_modified` | Indicates if on-disk and in-memory PE headers differ | `ps.pe.is_modified'` | +| `ps.pe.anomalies` | Contains PE anomalies detected during parsing | `ps.pe.anomalies in ('number of sections is 0')` | diff --git a/docs/rules/functions.md b/docs/rules/functions.md new file mode 100644 index 000000000..c5c3ece5a --- /dev/null +++ b/docs/rules/functions.md @@ -0,0 +1,593 @@ +# Functions + +##### Functions significantly extend the expressive power of the rule language by enabling **data transformation, enrichment, and dynamic evaluation** at runtime. A function can return a primitive values such as string/boolean or a collection of primitive values. + +Functions can be used anywhere a value is expected inside comparisons, logical expressions, or even as inputs to other functions. +For example, this expression removes the `C:` prefix from `file.path` and converts the resulting string to lowercase. + +```python +lower(ltrim(file.path, 'C:')) +``` + +Functions can be nested arbitrarily, allowing to build complex transformations step by step. In this example, the execution proceeds from the innermost function outward. `replace` normalizes path separators. Next,`trim` removes leading/trailing whitespace, and finally, `lower` ensures case-insensitive comparasion. This makes it easy to normalize data before applying detection logic. + +```python +lower(trim(replace(file.path, '\\', '/'))) +``` + +Some functions return collections (lists) instead of single values. These are especially useful when combined with operators like `in`, `iin`, `contains`, or `intersects`. In this example, `get_reg_value` may return multiple strings, for example from a `MULTI_SZ` registry value. The `iin` operator checks whether any element matches the provided list. + +```python +get_reg_value(registry.path) iin ('mimikatz.dll', 'rpcrt4.dll') +``` + +?> Each function expects arguments of specific types. Passing an incompatible type results in rule compilation error. + + +## Network functions + +### `cidr_contains` + +Determines if the specified IP is contained within the block referenced by the given CIDR masks. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `ip` | ip | IP address in v4/v6 notation. | yes | +| `cidrs` | array | List of IP masks in CIDR notation. | yes | + +##### Return + +> `return` Boolean Indicates whether the IP pertains to the CIDR block + +##### Usage + +``` +cidr_contains(net.sip, '192.168.1.1/24', '172.17.1.1/8') = true +``` + +## Hash functions + +### `md5` + +Computes the MD5 hash of the given value. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `data` | string or byte | The input string or byte array used to compute the MD5 hash. | yes | + +##### Return + +> `return` String MD5 hash in string format + +##### Usage + +``` +md5(registry.path) = 'eab870b2a516206575d2ffa2b98d8af5' +``` + +## String functions + +### `concat` + +Concatenates string/integer input arguments. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `tokens` | strings or integers | Strings or integers to be concatenated. | yes | + +##### Return + +> `return` String Concatenated string of all input tokens + +##### Usage + +``` +concat(ps.domain, '-', ps.username) = 'NT_AUTHORITY-SYSTEM' +``` +--- + +### `ltrim` + +Trims the prefix from the string. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `prefix` | string | Prefix to be removed. | yes | + +##### Return + +> `return` String Resulting string with the prefix removed + +##### Usage + +``` +ltrim(registry.path, 'HKEY_LOCAL_MACHINE\\') = 'SYSTEM\\Setup\\Pid' +``` +--- + +### `rtrim` + +Trims the suffix from the string. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `suffix` | string | Suffix to be removed. | yes | + +##### Return + +> `return` String Resulting string with the suffix removed + +##### Usage + +``` +rtrim(registry.path, '\\Pid') = 'HKEY_LOCAL_MACHINE\\SYSTEM\\Setup' +``` +--- + +### `lower` + +Converts the string with all Unicode letters mapped to their lower case. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | + +##### Return + +> `return` String Resulting string converted to lower case + +##### Usage + +``` +lower(registry.path) = 'hkey_local_machine\\system\\setup' +``` + +--- + +### `upper` + +Converts the string with all Unicode letters mapped to their upper case. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | + +##### Return + +> `return` String Resulting string converted to upper case + +##### Usage + +``` +upper(registry.path) = 'HKEY_LOCAL_MACHINE\\SYSTEM\\SETUP' +``` + +--- + +### `replace` + +Replaces all occurrences in the string as given by arbitrary old/new replacement pairs. You can specify multiple `old`/`new` pairs to be replaced. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `old` | string | Substring in the original string that is replaced with the `new` string. | yes | +| `new` | string | Replacement string. | yes | + +##### Return + +> `return` String Resulting string with all occurrences replaced by old/new pairs + +##### Usage + +``` +replace(registry.path, 'HKEY_LOCAL_MACHINE', 'HKLM') = 'HKLM\\SYSTEM\\Setup' +``` + +--- + +### `count` + +Counts the number of items in the slice or substrings in the string by matching a wildcard pattern. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` or `slice` | string or array | Input string or string array. | yes | +| `pattern` | string | Wildcard pattern used to match the string. `*` matches any sequence of characters, while `?` matches a single character. | yes | +| `case_insensitive` | boolean | Indicates if case insensitive matching is performed when counting substrings. | no | + +##### Return + +> `return` Integer The count of matched substring occurrences + +##### Usage + +``` +count(ps.modules, '?:\\*ntdll.dll') >= 2 +``` + +--- + +### `split` + +Produces a slice of substrings separated by the given delimiter. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `separator` | string | The separator used for splitting the string. | yes | + +##### Return + +> `return` Array Collection of substrings contained in the string + +##### Usage + +``` +split(file.path, '\\') in ('kernel32.dll', 'System32', 'Windows') +``` +--- + +### `length` + +Returns the number of characters for string arguments and the size of the slice for slice arguments. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` or `slice` | string or array | Input string or array. | yes | + +##### Return + +> `return` Integer The number of characters in the string or array length + +##### Usage + +``` +length(ps.cmdline) > 200 +``` + +--- + +### `indexof` + +Returns the index of the instance of substring in a given string depending on the provided search order. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `substring` | string | Substring for which the search is performed. | yes | +| `order` | string | Specifies the string index search order. It can be `first`, `any`, `last`, `lastany` | no | + +##### Return + +> `return` Integer The index of the substring or -1 if the substring is not found + +##### Usage + +``` +indexof(ps.domain, 'NT') = 0 +``` +--- + +### `substr` + +Creates a substring of a given string. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `start` | integer | Substring start index. | yes | +| `end` | integer | Substring end index.| no | + +##### Return + +> `return` String Substring contained within start and end indices + +##### Usage + +``` +substr(file.path, indexof(file.path, '\\'), indexof(file.path, '\\Hard')) = '\\Device' +``` + +--- + +### `entropy` + +Calculates the string entropy. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `algo` | integer | The algorithm used to calculate the string entropy. `shannon` is the default entropy type. | no | + +##### Return + +> `return` Integer String entropy value + +##### Usage + +``` +entropy(file.path) > 255 +``` + +--- + +### `regex` + +Applies single/multiple regular expressions on the provided string. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `string` | string | Input string. | yes | +| `patterns` | string | [Go](https://pkg.go.dev/regexp/syntax) compatible regular expression patterns. | yes | + +##### Return + +> `return` Boolean True if at least one regular expression matches or false otherwise + +##### Usage + +``` +regex(ps.name, 'power.*(shell|hell).dll', '.*hell.exe') = true +``` + +## File functions + +### `base` + +Returns the last element of the path. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string or array | The string or an array of strings representing file system path(s). | yes | +| `ext` | boolean | Determines whether the extension is retained in the file path. Default `true`| no | + +##### Return + +> `return` String|Array String or a slice of strings with file name(s) + +##### Usage + +``` +base(file.path) = 'cmd.exe' +``` +--- + +### `dir` + +Returns all but the last element of the path, typically the path's directory. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string or array | The string or an array of strings representing file system path(s). | yes | + +##### Return + +> `return` String|Array String or a slice of strings with directory name(s) + +##### Usage + +``` +dir(file.path) = 'C:\\Windows' +``` +--- + +### `ext` + +Returns the file name extension used by the path. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string or array | The string representing file system path. | yes | +| `dot` | boolean | Indicates if the dot symbol is retained as part of extension. Default `true` | no | + +##### Return + +> `return` String File name extension used by the path + +##### Usage + +``` +ext(file.name) = '.exe' +``` +--- + +### `glob` + +Returns the names of all files in the file system matching the pattern. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `pattern` | string | Shell file name pattern as described [here](https://pkg.go.dev/path/filepath#Match). | yes | + +##### Return + +> `return` Array Names of all files matching the pattern or an empty list if there is no matching file + +##### Usage + +``` +glob('C:\\Windows\\*.exe') in ('C:\\Windows\\notepad.exe') +``` + +--- + +### `is_abs` + +Reports whether the path is absolute. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string | The string representing file system path. | yes | + +##### Return + +> `return` Boolean True if the path references an absolute path or false otherwise + +##### Usage + +``` +is_abs(file.path) = false +``` +--- + +### `symlink` + +Returns the path name after the evaluation of any symbolic links. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string | The string representing file system path. | yes | + +##### Return + +> `return` String The path name after the evaluation of any symbolic links, or the original path if any errors occur + +##### Usage + +``` +symlink('C:\\Windows\\symlink.txt') = 'C:\\Windows\\target.txt' +``` +--- + +### `volume` + +Returns leading volume name. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string | The string representing file system path. | yes | + +##### Return + +> `return` String Leading volume name + +##### Usage + +``` +volume(file.path) = 'C:' +``` + +--- + +### `is_minidump` + +Checks the signature of the provided file and returns `true` if the signature matches the `minidump` header. + +Returns leading volume name. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `path` | string | The string representing file system path. | yes | + +##### Return + +> `return` Boolean True if the file contains the minidump header or false otherwise + +##### Usage + +``` +is_minidump(file.path) = true +``` + +--- + +## Registry functions + +### `get_reg_value` + +Reads the content of the registry value. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `key` | string | Fully-qualified registry key path including the value name. The root key can be expressed in abbreviated notation, for example, `HKLM` instead of `HKEY_LOCAL_MACHINE` | yes | + +##### Return + +> `return` Variant Depending on the registry value type, it can return a string, array of strings, blob, or an integer value. + +##### Usage + +``` + get_reg_value('HKCU\Volatile Environment\Envs') in ('SYSTEM', 'ROOT') +``` + +## YARA functions + +### `yara` + +Provides signature-based detection in rules. [YARA](https://virustotal.github.io/yara/) is a tool aimed at (but not limited to) helping malware +researchers to identify and classify malware samples. With YARA you can create descriptions of malware families based on textual or binary patterns. Depending on the parameter type supplied to this function, the scan can be performed on the process, file path or a memory block. + +##### Arguments + +| ARGUMENT | TYPE | DESCRIPTION | REQUIRED? | +| :--- | :---- | :---- | :---- | +| `target` | int, string or byte array | If the parameter is an integer value, it's assumed to be a `pid` for which the memory area is scanned. If it is a string, the scan is performed on the process executable or arbitrary file system file. Otherwise, it is a stream of bytes that represents a memory block to be scanned. | yes | +| `rules` | string | YARA rule definitions. | yes | + +##### Return + +> `return` Boolean If any rule the function returns true or false othwerise + +##### Usage + +``` +yara(file.path, 'rule Notepad : notepad +{ + strings: + $c0 = "Notepad" fullword ascii + condition: + $c0 +}') = true +``` diff --git a/docs/filters/iterators.md b/docs/rules/iterators.md similarity index 73% rename from docs/filters/iterators.md rename to docs/rules/iterators.md index f3933a135..286a596af 100644 --- a/docs/filters/iterators.md +++ b/docs/rules/iterators.md @@ -1,6 +1,6 @@ # Iterators -`foreach` idiom adds iteration capabilities to the rule language. Under the hood, `foreach` is implemented as a function that accepts three required and multiple optional arguments. The first argument is the `iterable` value typically yielded by the pseudo field. +##### `foreach` idiom adds iteration capabilities to the rule language. Under the hood, `foreach` is implemented as a function that accepts three required and multiple optional arguments. The first argument is the `iterable` value typically yielded by the pseudo field. The function recognizes process internal state collections such as modules, threads, memory mappings, or thread stack frames. Obviously, it is also possible to iterate over simple string slices. The second argument represents the `bound variable` which is an item associated with every element in the slice. The bound variable is accessed in the third argument, the `predicate`. It is usually followed by the `segment` that denotes the accessed value. Unsurprisingly, the predicate is commonly a binary expression that can be formed of `not/paren` expressions, other functions, and so on. The predicate is executed on every item in the slice. If the predicate evaluates to true, the function also returns the true value. @@ -10,21 +10,21 @@ Some examples of the `foreach` usage: - Traverses process modules and return true if the module path matches the pattern -``` +```python foreach(ps._modules, $mod, $mod.path imatches '?:\\Windows\\System32\\us?r32.dll') ``` - For each process ancestor, check if the ancestor is `services.exe` and the current process is protected. In this example, the `ps.is_protected` field is captured before its usage in the predicate -``` +```python foreach(ps._ancestors, $proc, $proc.name = 'services.exe' and ps.is_protected, ps.is_protected) ``` -## Process iterators {docsify-ignore} +## Process iterators The `ps.ancestor` returns all ancestor names of the process generating the event. Alternatively, the filter field can accept an argument. In case of the `ps.ancestor` field, the argument indicates the ancestor level. Given the process tree below and assuming the current process generating the event is `cmd.exe`, the field with an optional level argument yields the values as follows: -``` +```python ├───wininit.exe │ └───services.exe │ └───svchost.exe @@ -39,7 +39,7 @@ The `ps.ancestor` returns all ancestor names of the process generating the event If the argument is omitted, the slice with all ancestor names is returned. The `ps.ancestor` field can only yield a single process attribute - process name. To build complex conditions involving different process attribute, we can use the `foreach` construct. The bound variable associated with the `ps._ancestors` pseudo field can have the any of the segments: -| Segment Name | Description | +| SEGMENT | DESCRIPTION | | :--- | :---- | |`pid` | Process identifier | |`name` | Process name | @@ -52,27 +52,27 @@ If the argument is omitted, the slice with all ancestor names is returned. The ` |`username` | User name associated with the process security context | |`domain` | Domain associated with the process security context | -Examples +Examples: - Check if the ancestor has one of the particular process identifiers and the pid belongs to the `services.exe` process -``` +```python foreach(ps._ancestors, $proc, $proc.pid in (2034, 343) and $proc.name = 'services.exe') ``` - Check if the ancestor starts with the specific security identifier and the pid belongs to the `svchost.exe` process -``` +```python foreach(ps._ancestors, $proc, $proc.sid imatches `S-1-5*` and $proc.name = 'svchost.exe') ``` -### Modules {docsify-ignore} +### Modules The `ps._modules` pseudo field returns the process modules iterable. Available module segments are: -| Segment Name | Description | +| SEGMENT | DESCRIPTION | | :--- | :---- | |`address` | Base address of the process in which the module is loaded| |`checksum` | Module checksum | @@ -80,20 +80,20 @@ The `ps._modules` pseudo field returns the process modules iterable. Available m |`name` | Module name | |`path` | Full module path | -Examples +Examples: - Check the virtual memory space size of the specific module -``` +```python foreach(ps._modules, $mod, $mod.size >= 212354 and $mod.name imatches '*winhttp.dll') ``` -### Threads {docsify-ignore} +### Threads The `ps._threads` pseudo field yields all of the process running threads. Available thread segments are: -| Segment Name | Description | +| SEGMENT | DESCRIPTION | | :--- | :---- | |`tid` | Thread identifier | |`start_address` | The address of the function executed by the thread | @@ -102,11 +102,11 @@ The `ps._threads` pseudo field yields all of the process running threads. Availa |`kernel_stack_base` | The base address of the thread kernel stack | |`kernel_stack_limit` | he address denoting the thread kernel stack limit | -### Memory mappings {docsify-ignore} +### Memory mappings Process memory mappings (also known as sections) can be accessed via the `ps._mmaps` pseudo field. Available memory mappings segments are: -| Segment Name | Description | +| SEGMENT | DESCRIPTION | | :--- | :---- | |`address` | Address where the section is mapped within the process address space | |`type` | The type of the memory mapping. For example, `DATA`. | @@ -114,55 +114,69 @@ Process memory mappings (also known as sections) can be accessed via the `ps._mm |`protection` | Protection attributes of the mapped memory section | |`path` | If the memory mapping is backed by a physical file, indicates the path of the file | -### Environment variables {docsify-ignore} +### Environment variables You can access process environment variables by providing the name of the environment variable. Alternatively, you can provide the prefix. -``` +```python ps.envs['MOZ_CRASHREPORTER'] = 'C:\\Program Files\\Firefox' ``` Or, supplying the prefix -``` +```python ps.envs['MOZ_CRASH'] = 'C:\\Program Files\\Firefox' ``` It is also possible to retrieve all environment variables as a list of colon separated key/value pairs. Example using the `foreach` idiom: -``` +```python foreach(ps.envs, $env, substr($env, 0, indexof($env, ':')) = 'OS') ``` -## Portable Executable iterators {docsify-ignore} +## PE iterators + +[Portable Executable](https://en.wikipedia.org/wiki/Portable_Executable) (PE) is the Windows file format that describes the layout of the executable code. PE is frequently subject to weaponization ranging from reverse shell payload injections to several obfuscation techniques. Fibratus introspects the PE metadata and populates filter fields that can be used in detection rules. -[Portable Executable](/pe/introduction) introspection allows for utilizing the PE metadata in filters. See other [fields](filters/fields?id=pe) that can be used to narrow down events by PE data. +### Sections -### Sections {docsify-ignore} +Sections are the fundamental building blocks of PE data. They contain the contents of the file, including executable code, data, resources, and other runtime artifacts. -The `pe._sections` pseudo field yields all of the executable image PE sections. Available section segments are: +The `ps.pe._sections` pseudo field yields all of the executable image PE sections. Available section segments are: -| Segment Name | Description | +| SEGMENT | DESCRIPTION | | :--- | :---- | |`name` | Section name. For example, `.debug$` | |`size` | Section size in bytes | |`entropy` | Section entropy | |`md5` | Section MD5 hash | -### Resources {docsify-ignore} +### Resources -PE [resources](/pe/resources) can be accessed by the resource name. Alternatively, it is possible to obtain all the resources as a list separated by the colon delimiter: +PE resources can be accessed by the resource name. Alternatively, it is possible to obtain all the resources as a list separated by the colon delimiter: -``` +```python pe.resources iin ('FileDescription:Notepad') ``` +Fibratus reads version resources from the PE resource directory. The resource contains information about the exeucutable as its version number, its intended operating system, and its original filename. An example of version resources: -## Callstack {docsify-ignore} +```python +CompanyName: Microsoft Corporation +FileDescription: Notepad +FileVersion: 10.0.18362.693 (WinBuild.160101.0800) +InternalName: Notepad +LegalCopyright: © Microsoft Corporation. All rights reserved. +OriginalFilename: NOTEPAD.EXE +ProductName: Microsoft® Windows® Operating System +ProductVersion: 10.0.18362.693 +``` -[Stack enrichment](/kevents/anatomy?id=callstack) attaches call frames that can be accessed by the `thread._callstack` pseudo field. Available callstsack segments are: +## Callstack iterators -| Segment Name | Description | +[Stack enrichment](../telemetry/callstacks.md) attaches call frames that can be accessed by the `thread._callstack` pseudo field. Available callstsack segments are: + +| SEGMENT | DESCRIPTION | | :--- | :---- | |`address` | Symbol address | |`offset` | Symbol offset | @@ -173,8 +187,8 @@ pe.resources iin ('FileDescription:Notepad') |`is_unbacked` | Indicates if the frame is unbacked | |`callsite_leading_assembly` | Callsite leading assembly instructions | |`callsite_trailing_assembly` | Callsite trailing assembly instructions| -|`module.signature.is_signed` | Indicates if the frame module is signed | -|`module.signature.is_trusted` | Indicates if the frame module signature is trusted | +|`module.signature.exists` | Indicates if the frame module is signed | +|`module.signature.trusted` | Indicates if the frame module signature is trusted | |`module.signature.cert.subject` | Frame module signature certificate subject | |`module.signature.cert.issuer` | Frame module signature certificate issuer | @@ -182,12 +196,12 @@ Examples: - Determine if the frame protection is RWX (Read-Write-Execute) -``` +```python foreach(thread._callstack, $frame, $frame.protection = 'RWX') ``` - Determine if the frame trailing assembly contain the `syscall` instruction and the frame resides in the floating memory region -``` +```python foreach(thread._callstack, $frame, $frame.callsite_trailing_assembly matches '*mov r10, rcx|mov eax, 0x*|syscall*' and $frame.module = 'unbacked') -``` \ No newline at end of file +``` diff --git a/docs/rules/macros.md b/docs/rules/macros.md new file mode 100644 index 000000000..86b2b6bbe --- /dev/null +++ b/docs/rules/macros.md @@ -0,0 +1,40 @@ +# Macros + +##### Macros enable reusable rule patterns and provide a more expressive, human-friendly domain-specific language (DSL). Many detection rules rely on common conditions, such as process execution or file creation. For example, a rule might include an expression like `evt.name = 'CreateProcess'` + +Repeatedly embedding such conditions can lead to verbose, boilerplate-heavy rules. From a maintenance perspective, even a small change would require updating every affected rule. Macros solve this problem by offering a convenient way to define and reuse common patterns. Fibratus includes a built-in [macros library](https://github.com/rabbitstack/fibratus/blob/master/rules/macros/macros.yml) with a variety of ready-to-use definitions. The macros library is loaded from the file system and can be organized across multiple `yaml` files. By default, macros are located in the `%PROGRAM FILES%\Fibratus\Rules\Macros` directory. + +```yaml +filters: + macros: + from-paths: + - C:\Program Files\Fibratus\Rules\Macros\*.yml +``` + +## Macro types + +### Expressions + +Expression macros encapsulate rule conditions. A macro declaration requires a unique macro name, the expression, and an optional description. + +```python +- macro: spawn_process + expr: evt.name = 'CreateProcess' + description: Identifies the execution of a new process +``` + +Macros can be composed with other macros to build more complex, recursive patterns. For example, the `spawn_msoffice_process` macro combines the `spawn_process` expression macro and the `msoffice_binaries` list macro. + +```python +- macro: spawn_msoffice_process + expr: spawn_process and ps.sibling.exe iin msoffice_binaries + description: Identifies the execution of the MS Office process +``` +### Lists + +List macros define a sequence of values such as file system paths, process names, or registry keys. They help keep rules concise, readable, and easier to maintain. For example, a list macro can contain Microsoft Office process executable names. Operators such as `in`, `matches`, or `startswith` can accept list macros as right-hand side (RHS) expressions in rule conditions. + +```python +- macro: msoffice_binaries + list: [EXCEL.EXE, WINWORD.EXE, MSACCESS.EXE, POWERPNT.EXE] +``` \ No newline at end of file diff --git a/docs/rules/operators.md b/docs/rules/operators.md new file mode 100644 index 000000000..547551dac --- /dev/null +++ b/docs/rules/operators.md @@ -0,0 +1,84 @@ +# Operators + +##### Operators define how expressions are evaluated in the Fibratus rule language. They are used to compare values, combine conditions, and perform advanced string matching. + +Operator names are case-insensitive, so `AND`, `and`, and `And` are equivalent. Fibratus supports several classes of operators including binary, logical, and string. + +## Binary operators + +Binary operators compare two values and evaluate the relationship between them, producing a boolean result that can be used in rule conditions. In Fibratus, binary operators are used to express comparisons such as equality, inequality or ordering, forming the building blocks of more complex detection logic. + + +| OPERATOR | DESCRIPTION | +| :--- | :---- | +| `=` | Equal | +| `!=` | Not equal | +| `<` | Less than | +| `>` | Greater than | +| `<=` | Less than or equal | +| `>=` | Greater than or equal | +| `~=` | Case-insensitive equality (strings) | + +!> `~=` is useful when you want equality semantics without manually normalizing case. + + +## Logical operators + +Logical operators combine multiple expressions into a single condition, allowing rules to express more complex logic in a clear and structured way. In Fibratus, binary logical operators are used to evaluate relationships between two boolean expressions and determine the overall outcome of a condition. These operators enable combining checks such as comparisons, membership tests, or pattern matches into a unified rule, making detection logic more expressive and precise. + +### Intersection + +The intersection operator is denoted by the `and` keyword and evaluates to `true` only if both expressions are true. + +```python +ps.name = 'System' and ps.pid = 4 +``` + +### Union + +The union operator is denoted by the `or` keyword and evaluates to `true` if at least one expression is true. + +```python +ps.name = 'svchost.exe' or ps.name contains 'svc' +``` + +### Negation + +The negation operator is denoted by the `not` keyword and it negates the result of an expression. + +```python +ps.name = 'svchost.exe' and ps.args not in ('/c', '/k') +``` + +## String operators + +String operators apply to string [fields](fields.md) or collections of strings, enabling evaluation and comparison of textual values within rules. In Fibratus, these operators are used to perform operations such as pattern matching, prefix or suffix checks, containment tests, and exact or case-insensitive comparisons. They help build precise detection logic by allowing rules to express conditions based on the structure and content of string data. + + +| OPERATOR | DESCRIPTION | EXAMPLE | +| :--- | :---- | :---- | +| `in` | Checks if a value exists in a collection. | `ps.name in ('cmd.exe', 'powershell.exe')` | +| `iin` | Checks if a value exists in a collection but ignores case sensitivity. | `ps.modules iin ('kernel32.dll')` | +| `contains` | Checks whether a string or any element in a list contains a substring. | `ps.name contains 'cmd'` | +| `icontains` | Checks whether a string or any element in a list contains a substring but ignores case sensitivity. | `ps.cmdline icontains 'windows tasks'` | +| `startswith` | Checks if a string begins with a prefix. | `ps.name startswith 'svchost'` | +| `istartswith` | Checks if a string begins with a prefix but ignores case sensitivity. | `ps.name istartswith 'cmd'` | +| `endswith` | Checks if a string ends with a suffix. | `file.path endswith 'Windows'` | +| `iendswith` | Checks if a string ends with a suffix but ignores case sensitivity. | `file.path iendswith '.exe'` | +| `intersects` | Checks whether all elements in RHS exist in the LHS collection. | `ps.args intersects ('DcomLaunch', 'LSM')` | +| `iintersects` | Checks whether all elements in RHS exist in the LHS collection but ignores case sensitivity. | `ps.args iintersects ('dcomLaunch', 'LSM')` | +| `matches` | Wildcard-based matching similar to globbing. `*` matches any sequence of characters, while `?` matches a single character. | `registry.path matches 'HKEY_USERS\\*\\Environment\\?'` | +| `imatches` | Wildcard-based matching but ignores case sensitivity. `*` matches any sequence of characters, while `?` matches a single character. | `file.path imatches ('?:\\*\\lsass?.dmp', '?:\\ProgramData\\*.dll')` | + + +## Fuzzy operators + +Fuzzy operators enable approximate string matching, which is useful for detecting obfuscation or minor variations in textual data. They help identify values that are not identical but remain similar enough to indicate a potential match, even when attackers introduce small modifications such as typos, character substitutions, or encoding tricks. This makes fuzzy matching particularly valuable for uncovering evasive behaviors and reducing false negatives in detection rules. + + +| OPERATOR | DESCRIPTION | EXAMPLE | +| :--- | :---- | :---- | +| `fuzzy` | Approximate match. | `file.path fuzzy 'C:\\Windows\\Sys\\ser3ll'` matches `C:\Windows\System32\user32.dll` | +| `ifuzzy` | Case-insensitive fuzzy match. | `file.path ifuzzy 'C:\\Windows\\Sys\\ser3ll'` matches `C:\WINDOWS\System32\user32.dll`| +| `fuzzynorm` | Fuzzy match with Unicode normalization. | `file.path fuzzynorm 'C:\\Windows\\Sys\\sér3ll'` matches `C:\Windows\System32\usér32.dll` | +| `ifuzzynorm` | Case-insensitive and Unciode normalized. | `file.path ifuzzynorm 'C:\\Windows\\Sys\\sér3ll'` matches `C:\Windows\System32\usér32.dll`| diff --git a/docs/rules/sequences.md b/docs/rules/sequences.md new file mode 100644 index 000000000..5f3ced6aa --- /dev/null +++ b/docs/rules/sequences.md @@ -0,0 +1,99 @@ +# Sequences + +##### In a nutshell, sequence rules allow to model behaviors that unfold over time by tracking an ordered chain of events. Instead of matching a single event in isolation, sequences let you express causality: this happened, and then shortly after, that happened. + +A sequence rule always starts with the `sequence` keyword and is composed of two or more expressions separated by vertical bars (`|`). Each expression represents a step in the behavioral chain. A match occurs only if all expressions evaluate to `true` and they do so in the declared order occurring within the allowed time window (if specified). + +Let’s revisit a real-world example: + +```python +condition: > + sequence + maxspan 2m + by ps.uuid + |open_process and + ps.access.mask.names in ('ALL_ACCESS', 'CREATE_PROCESS', 'VM_READ') and + evt.arg[exe] imatches '?:\\Windows\\System32\\lsass.exe' and + ps.exe not imatches + ( + '?:\\Windows\\System32\\svchost.exe', + '?:\\ProgramData\\Microsoft\\Windows Defender\\*\\MsMpEng.exe' + ) + | + |create_new_file and file.extension iin ('.dmp', '.mdmp', '.dump')| +``` + +This rule detects a classic [LSASS](https://en.wikipedia.org/wiki/Local_Security_Authority_Subsystem_Service) memory dump pattern: + +1. A process opens a handle to `lsass.exe` with suspicious access rights +2. The same process writes a minidump file shortly after + +This is a strong behavioral signal because neither event alone is necessarily malicious, but their combination in sequence is. + +## Execution model + +A sequence behaves like a state machine. Each incoming event is evaluated against the current stage(s) of active sequences. When the first expression matches, a new sequence instance is created. That instance waits for the next expression to match. If all expressions match in order the sequence fire. If constraints like time or process termination are violated the sequence is discarded. Multiple sequence instances can exist concurrently for different processes or entities. + +## Controlling sequence behavior + +### `maxspan` + +`maxspan` defines the maximum time allowed between the first and last expression in the sequence. Supported units for time window duration are exprssed as `2s`, `2m`, or `2h` for two minutes, two hours, and two days respectively. The time window dictates how long each expression in the sequence is expecting to wait for events that could result in expression evaluating to true. For example, by examining the above snippet, the sequence starts by detecting process handle acquisition on the `lsass` process. Since this is the first expression in the sequence, the time window constraint doesn't kick in yet. After the first expression evaluates to true, the next one, expecting to detect creation of the minidump file, will evaluate only if the `CreateFile` event arrives within the 2 minutes time frame. Otherwise, the deadline is reached and the entire sequence is discarded. + +?> Without `maxspan` sequences can live indefinitely, which is usually undesirable in high-throughput systems. + +### `by` + +`by` clause enables event stitching ensuring that only related events participate in the same sequence. Continuing the example from previous rule, the sequence can match only if `OpenProcess` and `CreateFile` events are generated by the same process. Specifically, events are joined by the `ps.uuid` field which is meant to offer a more robust version of the `ps.pid` field that is resistant to being repeated. A variation of the `by` statement allows establishing a joining criteria separately on each expression in the sequence. + +```python +sequence +maxspan 1h + |write_file and + file.extension iin executable_extensions and + ps.name iin msoffice_binaries + | by file.path + |spawn_process and ps.name iin msoffice_binaries| by ps.exe +``` + +As we can observe, the `by` statement is anchored to each expression but using a different join key. This rule would match only if the file being written is equal to the spawned process executable image. This effectively expresses, **the file that was written must be the same file that gets executed later**. +Of course, it is possible to omit both `maxspan` and `by` statements. However, such rules are rarely used to express behaviors that require relationships between events, instead, a loose temporal correlation. + +`by` also supports event correlation by mutlitple join keys. For example, this rule matches when section unmapping and executable loading events arrive from the same process and section base address. + +```python +sequence +maxspan 40s + |unmap_view_of_section and + evt.pid != 4 and ps.sid not in ('S-1-5-18', 'S-1-5-19', 'S-1-5-20') and + file.view.size > 20000 and file.view.protection != 'READONLY' + | by ps.uuid, file.view.base + |load_executable and + module.path not imatches '?:\\Windows\\SoftwareDistribution\\*.exe' + | by ps.uuid, module.base +``` + +## Aliases + +Sometimes, simple equality joins with the `by` clause are not enough. You may need to compare values across steps, perform transformations, or match against derived data. +To address this limitation, Fibratus provides the more flexible `as` statement. It allows you to assign aliases to sequence expressions, which can later be referenced using bound fields. A bound field is simply a regular filter field prefixed with an alias, enabling access to values from previously matched events. + +Let’s look at an example: + +```python +sequence +maxspan 5m + |create_file and + file.name imatches '?:\\Windows\\System32\\*.dll' + | as e1 + |modify_registry and + registry.path ~= 'HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\Lsa\\Notification Packages' and + get_reg_value(registry.path) iin (base($e1.file.path, false)) + | +``` + +The first expression in the sequence detects the creation of a DLL file in the system directory. When this expression evaluates to true, the matching event is stored and becomes accessible through the `e1` alias. + +The second expression detects modifications to a specific registry value. If it matches, the rule retrieves the registry data using the `get_reg_value` function. In this case, the value is a `MULTI_SZ` entry containing a list of strings. + +This list is then compared against the file path captured by the first expression. The `$e1.file.path` bound field is used to reference the file path from the previously matched event, enabling correlation across sequence steps. diff --git a/docs/setup/cli.md b/docs/setup/cli.md deleted file mode 100644 index 6d419b7a2..000000000 --- a/docs/setup/cli.md +++ /dev/null @@ -1,127 +0,0 @@ -## CLI {docsify-ignore} - -Invoking the `fibratus` binary without any parameters reveals available CLI commands. You can obtain help information for each available command by appending the `--help` or `-h` option after the command name. Let's briefly describe available commands. - -### run - -The main command for bootstrapping Fibratus (either in rule engine or event forwarder mode) or running a filament. It accepts an optional filter expression. Examples: - -- collect all events - ``` - $ fibratus run --forward - ``` - -- run the `watch_files` filament - ``` - $ fibratus run -f watch_files - ``` - -- collect fs events originated from the `cmd.exe` process - ``` - $ fibratus run --forward kevt.category = 'file' and ps.name = 'cmd.exe' - ``` - -- collect fs events and enable PE introspection - ``` - $ fibratus run --forward kevt.category = 'file' --pe.enabled=true - ``` - -### capture - -Dumps the event flow to specialized kcap (capture) file. It accepts an optional filter expression. Examples: - -- capture all events to `events.kcap` capture file - ``` - $ fibratus capture -o events - ``` - -- capture network events from the specific destination IP address - ``` - $ fibratus capture kevt.category = 'net' and net.dip = 172.17.2.3 -o events - ``` - -### replay - -Replays the event flow from the kcap file. It accepts an optional filter expression. Examples: - -- replay all events from the `events.kcap` capture file - ``` - $ fibratus replay -k events - ``` - -- replay events that contain a specific resource name in the PE resource directory - ``` - $ fibratus replay pe.resources[Company] contains 'blackwater' -k events - ``` - -### rules - -The root command that exposes various subcommands for listing/validating rules and creating detection rule templates. - -- #### list - -List all rules present in the `Rules` directory. - -- #### validate - -Validates rules for structural and syntactic correctness. - -- #### create - -Create a new rule template. The command requires a rule name and an optional MITRE tactic identifier (e.g. `TA0001`) that can be passed via `--tactic-id` flag. - -### config - -Prints the options loaded from configuration sources including files, command line flags or environment variables. Sensitive data, such as passwords are masked out. - -### service - -This is the root command that exposes multiple subcommands for interacting with the **Windows Service Control Manager**. - -- #### start - -Starts the Fibratus service that was previously registered within the Windows Service Control Manager. - -- #### stop - -Stops the Fibratus Windows service. - -- #### restart - -Restarts the Fibratus Windows service. - -- #### remove - -Removes the Fibratus service from the Windows Service Control Manager. - -- #### status - -Checks the status of the Fibratus Windows service. - -### docs - -Launches the default web browser and opens the Fibratus documentation site. - -### list - -The command consists of various subcommands: - -- #### filaments - -Displays available filaments. Filaments live in the `%PROGRAMFILES\Fibratus\Filaments` directory, but you can override this location with the `--filament.path` flag or the corresponding key in the `yaml` configuration file. - -- #### fields - -Shows all [field names](/filters/fields) that can be used in filter expressions. - -- #### kevents - -Shows available event types. - -### stats - -Returns the runtime metrics that are exposed through the [expvar](https://golang.org/pkg/expvar/) HTTP endpoint. Useful for debugging. - -### version - -Displays the Fibratus version along with the commit hash and the Go compiler version. diff --git a/docs/setup/configuration.md b/docs/setup/configuration.md index 6fc8dd07f..7251d04aa 100755 --- a/docs/setup/configuration.md +++ b/docs/setup/configuration.md @@ -1,21 +1,20 @@ # Configuration -You can parametrize Fibratus from various configuration sources including `yaml`/`json` files, environment variables and command line flags. -Properties that you specify via command line flags or environment variables take precedence over the configuration file. +##### You can parametrize Fibratus from various configuration sources including `yaml`/`json` files, environment variables and command line flags. Properties that you specify via command line flags or environment variables take precedence over the configuration file. -### Files {docsify-ignore} +## Files -By default, configuration files are stored in `%PROGRAM FILES%\Fibratus\Config` directory. If you prefer to keep them in a different location, you can override the configuration file path via the `--config-file` command line flag when starting Fibratus. The template, `fibratus.yml` configuration file with all options documented is available in the `%PROGRAM FILES%\Fibratus\Config` directory. +By default, configuration files are stored inside `%PROGRAM FILES%\Fibratus\Config` directory. If you prefer to keep them in a different location, you can override the configuration file path via the `--config-file` command line flag when starting Fibratus. The default [`fibratus.yml`](https://github.com/rabbitstack/fibratus/blob/master/configs/fibratus.yml) configuration file with all documented config keys is available in the Fibratus repository. -### Flags {docsify-ignore} +## Flags Each CLI command accepts a set of config flags. For example, running the `fibratus run -h` command displays flag names, their default value (if any), and a short description. Command line flags take precedence over environment variables and configuration files. !> Some configuration options are not exposed via command line flags and can only be tuned in the configuration file. -### Environment variables {docsify-ignore} +## Environment variables To set a certain configuration property via an environment variable, a simple rule of thumb needs to be followed: remove the leading `--` characters in the flag name, convert all `.` and `-` characters to `_` symbol, capitalize the environment variable name and you're ready to go. -Let's suppose we want to set the value of the `--kstream.buffer-size` flag via an environment variable. The resulting environment variable would get converted to `KSTREAM_BUFFER_SIZE`. +Let's suppose we want to set the value of the `--eventsource.buffer-size` flag via an environment variable. The resulting environment variable is converted to `EVENTSOURCE_BUFFER_SIZE`. diff --git a/docs/setup/images/fibratus-msi.png b/docs/setup/images/fibratus-msi.png deleted file mode 100644 index 2566deb9e..000000000 Binary files a/docs/setup/images/fibratus-msi.png and /dev/null differ diff --git a/docs/setup/installation.md b/docs/setup/installation.md index a317f86b7..d280688bc 100755 --- a/docs/setup/installation.md +++ b/docs/setup/installation.md @@ -1,101 +1,182 @@ # Installation -### System requirements {docsify-ignore} +### Minimum system requirements -- **Windows 10** and higher or **Windows Server 2016** or higher -- 40 MB of free disk space -- 1 (V)CPU -- 90 MB of available physical memory +- AMD64 processor architecture +- **Windows 10** and higher or **Windows Server 2016** and higher +- 60 MB of free disk space +- 100 MB of available physical memory -### Permission requirements {docsify-ignore} +### Permission requirements Fibratus requires **administrator** or **SYSTEM** privileges to capture system events from the ETW subsystem. During execution, Fibratus performs the following operations on your system: -- takes a snapshot of allocated system handles. You can control this option through [configuration](/kevents/handle?id=handle-state) flags. Disabled by default. +- takes a snapshot of allocated system handles. You can control this option through [configuration](../telemetry/events/handle.md) flags. Disabled by default. - periodically writes the current event sequence into volatile registry value - writes logs to disk. The default logs directory location is `%PROGRAMFILES%\Fibratus\Logs` - grants the `SeDebugPrivilege` to its process token. However, you can disable granting this privilege by setting the `debug-privilege` option to `false` -- transports event messages over the wire if the eligible output sink is active -- inspects process image [PE](/pe/introduction.md) metadata. Again, you can disable this feature through [config](/pe/introduction) file -- executes [YARA](/yara/introduction.md) rules on freshly created process images or other image files when the [YARA scanner](/yara/introduction) is enabled -- spins up an embedded Python interpreter to run [filaments](/filaments/introduction) -- accesses raw disk devices to read file data +- transports event messages over the wire if the eligible [output](../telemetry/outputs.md) sink is active +- executes [YARA](../yara.md) rules on freshly created process images or other image files when the [YARA scanner](../yara.md) is enabled +- spins up an embedded Python interpreter to run [filaments](../filaments.md) -### Installation {docsify-ignore} +### Installation -The easiest way to get started with Fibratus is by downloading the Windows installer. Head over to the [releases](https://github.com/rabbitstack/fibratus/releases) and pick your download. Latest releases are recommended as they ship with new features, bug fixes and tend to improve the performance. -Windows installers are automatically built by the CI platform each time new Fibratus release is published. + -

- -

+ -There are two flavors of Windows MSI installers: +Head over to the [downloads](https://fibratus.io/downloads) page and pick your release artifact. Latest releases are recommended as they ship with new features, bug fixes and tend to improve the performance. -- __full installers__ ship with all features ([captures](captures/introduction), [filaments](filaments/introduction), [yara](yara/introduction)) and bundle the embedded Python distribution -- __slim installers__ support less features but are more portable and have lower disk space requirements ++> To confirm the integrity of the downloaded MSI, calculate its SHA256 hash and compare it with the hash listed on the release download page. Use this command to compute the SHA256 hash `Get-FileHash -Path fibratus-[version]-amd64.msi -Algorithm SHA256` + +!> There are two flavors of Windows MSI packages: **full** and **slim** installers. Full installers ship with all features including [captures](../captures.md), [filaments](../filaments.md), [yara](../yara.md) and bundle the embedded **Python** distribution. Slim installers lack the aforementioned features but have lower disk space footprint. + + + + + +Double-click the MSI package and follow the UI wizard or alternatively install via `msiexec` in silent mode: + + +$ msiexec /i fibratus-[version]-amd64.msi /qn + + + + + The installer will automatically register and start Fibratus as **Windows Service**. To verify if the service is running correctly, spin up a command line prompt and execute the following command: -``` + $ fibratus service status + Fibratus service is running -``` -If you're able to see the output like in the snippet above, congratulations! You have successfully installed Fibratus. Jump to [quick start](/setup/quick-start). + + + + + + +If you're able to see the output like in the terminal above, congratulations! You have successfully installed Fibratus. Jump to [quick start](/setup/quick-start). + + -### Uninstall {docsify-ignore} +
-To remove Fibratus from your system, head to the **Control Panel > Programs and Features** and start the uninstall process. The uninstaller will make sure to stop/remove the Windows Service and get rid of all installation data. +### Uninstall -## Building from source {docsify-ignore} +To remove Fibratus from your system, head to the **Control Panel > Programs and Features** and start the uninstall process. The uninstaller will make sure to stop/remove the Windows Service and get rid of all installation data. Alternatively, uninstall from the command line with `msiexec` + + +$ msiexec.exe /x fibratus-[version]-amd64.msi + + + +## Building from source To build Fibratus directly from source code you have to satisfy the following requirements: -- Go compiler 1.21+ +- [Go 1.26+](https://go.dev/doc/install) +- [git](https://git-scm.com/install/windows) - C compiler (optional) - Python headers (optional) - [libyara](https://github.com/VirusTotal/yara/tree/master/libyara) (optional) -### Installing dependencies {docsify-ignore} +### Installing dependencies + +!> You can skip this step if you're not interested in capture, YARA, and filaments features, as they require interoperability with [`cgo`](https://go.dev/wiki/cgo) + + + + + +[Download](https://www.msys2.org/) the `msys2` installer and follow the instructions [here](https://www.msys2.org/). + + + + +Launch the `msys2` shell which by default located in `C:\msys2\msys2.exe`. You can also access it from the `MSYS2 64-bit` Start Menu item. + + + + +Install the `MinGW` compiler toolchain along with other dependencies. + + +$ pacman -S base-devel mingw-w64-x86_64-openssl mingw-w64-x86_64-gcc mingw-w64-x86_64-pkg-config automake libtool autoconf + + + + + + +[Download](https://www.python.org/ftp/python/3.7.9/python-3.7.9-amd64.exe) and install `Python 3.7`. Assuming the Python distribution was installed in `C:\Python37`, set the `PKG_CONFIG_PATH` environment variable to the location of the `pkg-config` directory within the `fibratus` directory. + + +$ set PKG_CONFIG_PATH=[pkg-config absolute path] + + + + + + +Clone the `yara` repository into the path visible to the `msys2` environment. This is ideally done from the `MSYS2 64-bit` shell. Next, build the `libyara` library. + + + +$ pacman -S git +$ git clone https://github.com/VirusTotal/yara.git +$ cd yara +$ autoreconf -fiv +$ ./configure --host=x86_64-w64-mingw32 +$ make install + + + + + + + +### Building + +Optional dependencies are only required if you plan to build features that interoperate with C code. By default, the Go compiler ignores any features that rely on [cgo](https://golang.org/cmd/cgo/), but you can control which features are included in Fibratus using the following build flags: + +- `filament` compiles Fibratus with [filaments](../filaments.md) support +- `cap` compiles Fibratus with support for capturing/replaying [capture](../captures.md) files +- `yara` builds Fibratus with [YARA](https://virustotal.github.io/yara/) memory scanning capabilities + +!> The build flags are injected via the `TAGS` environment variable. It needs to be set prior to running the build script, for example, running the folllwing snippet from Powershell terminal: `$env:TAGS="filament,cap,yara"` + +To build the binary run the following commands from `Powershell` terminal. + + + + + +Clone the Fibratus repository to the location of your choice. + + +$ git clone https://github.com/rabbitstack/fibratus.git -!> You can skip this step if you're not interested in building features that interop with the C code. + -1. Download the `msys2` installer and follow the instructions [here](https://www.msys2.org/). - - open the `msys2` shell (by default located in `C:\msys2\msys2.exe`). You can also access it from the `MSYS2 64-bit` Start Menu item. - - install the `MinGW` compiler toolchain along with other dependencies: - - `pacman -S base-devel mingw-w64-x86_64-openssl mingw-w64-x86_64-gcc mingw-w64-x86_64-pkg-config automake libtool autoconf` -2. [Download](https://www.python.org/ftp/python/3.7.9/python-3.7.9-amd64.exe) and install the `Python 3.7`. Assuming the Python distribution was installed in `C:\Python37`, set the `PKG_CONFIG_PATH` environment variable to the location of the `pkg-config` directory within the `fibratus` directory. - - `set PKG_CONFIG_PATH=` -3. Build `libyara` - - clone the `yara` repository into the path visible to the `msys2` environment. This is ideally done from the `MSYS2 64-bit` shell. - - `pacman -S git` - - `git clone https://github.com/VirusTotal/yara.git` - - go to the `yara` repository you previously cloned. Run the following commands: - - `autoreconf -fiv` - - `./configure --host=x86_64-w64-mingw32` - - `make install` + -### Building the executable {docsify-ignore} + -The **optional dependencies are only needed** if you'll be building features that interop with the C code. The Go compiler is instructed to ignore all features that trigger the [cgo](https://golang.org/cmd/cgo/), but you can control which features are built into Fibratus through the following build flags: +Launch the `make` script to initiate the build process. -- `filament`: compiles Fibratus with filaments support -- `kcap`: compiles Fibratus with support for capturing/replaying kcap files -- `yara`: builds Fibratus with support for [Yara](https://virustotal.github.io/yara/) pattern matching + -To build the Fibratus binary without filament, kcap nor yara features, run the following command from the `cmd` shell and within the`fibratus` directory: +$ cd fibratus +$ ./make -``` -$ make -``` + -To produce the Fibratus binary with the filaments support, you would run the following commands: +After compilation completes, the `fibratus.exe` binary can be found in the `cmd\fibratus` directory. -``` -$ set TAGS=filament -$ make -``` + -In either case, the resulting binary is placed in the `cmd\fibratus` directory. + diff --git a/docs/setup/quick-start.md b/docs/setup/quick-start.md index 4f401b0c0..46d4f2529 100644 --- a/docs/setup/quick-start.md +++ b/docs/setup/quick-start.md @@ -1,59 +1,68 @@ -# Quick start +# Quick Start -By default, Fibratus operates in rule engine mode. It loads the rule set from the `%PROGRAM FILES%\Fibratus\Rules` directory and sends security alerts to [Eventlog](/alerts/senders/eventlog) or other configurable [sinks](/alerts/senders). Optionally, it takes response actions when the rule is fired, such as killing the process. To see Fibratus in action, we can trigger a rule by performing the following actions: +##### By default, Fibratus operates in rule engine mode observing the system activity and alerting when the rule policy is violated. Fibratus sends security alerts to [Eventlog](../rules/actions/alert.md) or other configurable [senders](../rules/actions/alert.md) and remmediates threats via response actions. To see Fibratus in action, we can trigger a rule by executing a native Windows binary. -- spin up a command line prompt -- list credentials from the vault by using the `VaultCmd` tool -``` + +## Realtime Detection + + + + + +Spin up a command line prompt and run the `VaultCmd.exe` command. This command lists the credentials stored in the Credential Manager. + + $ VaultCmd.exe /listcreds:"Windows Credentials" /all -``` + + + + + + +Hit `⊞ Win` + `R` key combination to bring the Run dialog. Type `evtwvr.msc` to open the Event Viewer snap-in. + + + + + -- `Credential discovery via VaultCmd tool` rule should trigger displaying the alert in the Eventlog `Application` channel. +Navigate to the Eventlog `Application` channel. `Credential Manager access via known tools` rule should trigger under `Fibratus` event source. -To learn more about detection rules, head to [rules](/filters/rules). +![Detection](images/realtime-detection.gif "Detection") -### Event forwarding {docsify-ignore} + -To start forwarding events to [output](/outputs/introduction) sinks, run Fibratus from the command line in event forwarding mode: + -``` -$ fibratus service stop -$ fibratus run --forward -``` +To learn more about detection rules, head to [rules](../rules.md). -The continuous stream of events will start rendering on the console. +## Telemetry Exploration -``` - ... -681951 2020-11-07 14:24:57.1839809 +0100 CET - 2 cmd.exe (6328) - CreateFile (file_name➜ C:\WINDOWS\system32\config\systemprofile\AppData\Local\Microsoft\WindowsApps\, file_object➜ ffffa88c7ea077d0, irp➜ ffffa88c746b2a88, operation➜ supersede, share_mask➜ rw-, type➜ directory) -681952 2020-11-07 14:24:57.1840451 +0100 CET - 2 cmd.exe (6328) - RegOpenKey (key_handle➜ 0, key_name➜ HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\WindowsStore, status➜ key not found) -681953 2020-11-07 14:24:57.1840626 +0100 CET - 2 cmd.exe (6328) - RegOpenKey (key_handle➜ 0, key_name➜ HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\StateSeparation\RedirectionMap\Keys, status➜ key not found) -681954 2020-11-07 14:24:57.1840751 +0100 CET - 2 cmd.exe (6328) - RegOpenKey (key_handle➜ 0, key_name➜ HKEY_LOCAL_MACHINE\Software\Microsoft\LanguageOverlay\OverlayPackages\en-US, status➜ key not found) -681955 2020-11-07 14:24:57.1841104 +0100 CET - 2 cmd.exe (6328) - CreateFile (file_name➜ C:\WINDOWS\system32\en-US\cmd.exe.mui, file_object➜ ffffa88c7ea077d0, irp➜ ffffa88c746b2a88, operation➜ open, share_mask➜ r-d, type➜ directory) -681956 2020-11-07 14:24:57.1848044 +0100 CET - 2 cmd.exe (6328) - TerminateThread (base_prio➜ 8, entrypoint➜ 7ff7762382b0, io_prio➜ 2, kstack➜ fffff10cf0785000, kstack_limit➜ fffff10cf077e000, page_prio➜ 5, pid➜ 6328, tid➜ 11716, ustack➜ d020700000, ustack_limit➜ d020604000) -681957 2020-11-07 14:24:57.1848713 +0100 CET - 2 cmd.exe (6328) - UnloadImage (base_address➜ 7ff776220000, default_address➜ 7ff776220000, file_name➜ C:\Windows\System32\cmd.exe, image_size➜ 413696, pid➜ 6328) -681958 2020-11-07 14:24:57.1848779 +0100 CET - 2 cmd.exe (6328) - UnloadImage (base_address➜ 7fffaaba0000, default_address➜ 7fffaaba0000, file_name➜ C:\Program Files\AVG\Antivirus\aswhook.dll, image_size➜ 73728, pid➜ 6328) -681959 2020-11-07 14:24:57.1848954 +0100 CET - 2 cmd.exe (6328) - UnloadImage (base_address➜ 7fffc97a0000, default_address➜ 7fffc97a0000, file_name➜ C:\Windows\System32\KernelBase.dll, image_size➜ 2764800, pid➜ 6328) -681967 2020-11-07 14:24:57.184997 +0100 CET - 2 erl.exe (5236) - TerminateProcess (comm➜ C:\WINDOWS\system32\cmd.exe /c handle.exe /accepteula -s -p 5236 2> nul, directory_table_base➜ 2300cb000, exe➜ C:\WINDOWS\system32\cmd.exe, exit_status➜ 1, kproc➜ ffffa88c70ee7080, name➜ cmd.exe, pid➜ 6328, ppid➜ 5236, session_id➜ 0, sid➜ NT AUTHORITY\SYSTEM) -681968 2020-11-07 14:24:57.1853111 +0100 CET - 2 (6328) - RegOpenKey (key_handle➜ ffffc980980b55f0, key_name➜ HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\bam\State\UserSettings\S-1-5-18, status➜ success) -681969 2020-11-07 14:24:57.1853224 +0100 CET - 2 (6328) - RegQueryValue (key_handle➜ ffffc980abd657d0, key_name➜ HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\bam\State\UserSettings\S-1-5-18\\Device\HarddiskVolume4\Windows\System32\cmd.exe, status➜ key not found) -681970 2020-11-07 14:24:57.1853581 +0100 CET - 5 aswidsagent.exe (7812) - CreateFile (file_name➜ C:\WINDOWS\SYSTEM32\CMD.EXE, file_object➜ ffffa88c7a8053e0, irp➜ ffffa88c7b711608, operation➜ open, share_mask➜ rw-, type➜ file) -681971 2020-11-07 14:24:57.185952 +0100 CET - 4 AVGSvc.exe (4000) - CreateFile (file_name➜ C:\ProgramData\AVG\Antivirus\psi.db-journal, file_object➜ ffffa88c7ea02500, irp➜ ffffa88c73ff8a88, operation➜ supersede, share_mask➜ rw-, type➜ directory) -681972 2020-11-07 14:24:57.1860706 +0100 CET - 4 AVGSvc.exe (4000) - ReadFile (file_name➜ C:\ProgramData\AVG\Antivirus\psi.db, file_object➜ ffffa88c72c7c260, io_size➜ 16, irp➜ ffffa88c73ff8a88, offset➜ 24, type➜ file) - ... -``` +**Detection** and **Protection** is Fibratus' primary operating mode. However, you can access live kernel telemetry by running Fibratus in event forwarder mode as a `CLI` tool. -Each line is comprised of the following fields: +!> If Fibratus is already running as Windows Service, **stop** the service before running the CLI tool. + +To gain access to the live kernel telemetry, run the following command from an elevataed terminal. + + +$ fibratus run "ps.name = 'svchost.exe'" --forward + + + +The argument passed to the `run` command is a [filter](../telemetry/filtering.md) expression. You’ll learn more about filters in the upcoming section, but in short, they enable fine-grained inspection of kernel events. In this example, only events generated by the `svchost.exe` process are accepted and streamed to the [output](../telemetry/outputs/outputs.md) sinks. + +Once the command is run, a continuous stream of events is rendered on the console, as shown below. [Console](../telemetry/outputs/console.md) is the default output sink, but you can also route the telemetry to Elasticsearch, HTTP or other destinations. + +![Telemetry](images/telemetry-exploration.gif "Telemetry") + + +Each individual event line is comprised of the following segments: - monotonic sequence value -- timestamp of event occurrence +- timestamp of event occurrence and the delta from previous event - CPU where the event was captured -- the process name and the pid that produced the event +- process name and the `pid` that produced the event - event type - event parameters -A different [rendering template](/outputs/console?id=templates) can be used to customize the line format or you can opt to change the output format to, for example, JSON. - -The console output is the default events output, even though you can route the event flow to [Elasticsearch](https://www.elastic.co/elasticsearch/) or [RabbitMQ](https://www.rabbitmq.com/) sinks, just to name a few. [Learn](/outputs/introduction) more about output sinks. - -To stop Fibratus, hit the `Ctr-C` key combination. +To stop Fibratus CLI, hit the `Ctr` + `C` key combination. diff --git a/docs/telemetry/callstacks.md b/docs/telemetry/callstacks.md new file mode 100644 index 000000000..32fc9ae14 --- /dev/null +++ b/docs/telemetry/callstacks.md @@ -0,0 +1,42 @@ +# Callstacks + +##### Callstack reconstructs the sequence of function calls that led to the current thread state. Return addresses obtained directly from kernel space, are symbolized and enriched with a vital context that can be leveraged to boost behavioral detections and reduce false positive alerts. + +By resolving raw instruction pointers into human-readable symbols such as module paths, function names, and offsets, Fibratus enables analysts to trace behavior back to its true origin, even across user-mode and kernel-mode boundaries. This capability is particularly valuable for detecting stealthy or indirect activity as it exposes the full execution path rather than relying solely on the initiating process. Callstacks can be filtered, symbolized, and enriched with metadata, allowing them to be incorporated into [detection rules](../rules/fields.md) and forensic workflows for high-fidelity behavioral analysis. + +A typical callstack summary is depicted in the screenshot below, delineating kernel, system, and user frames provenance. + +!> For the [console](outputs/console.md) output to render callstack frames, the template shall define the `{{ .Callstack }}` segment. Segments can be customized by overriding the default [rendering template](outputs/console?id=templates). + +![Callstacks](images/callstacks.png "Callstacks") + +Each line corresponds to the frame stack consisting of: + +- function call return address, for example, `0x7ffb5c1d0396` +- full path of the module containing the executed function, such as `C:\Windows\System32\kernel32.dll`. If the call is invoked from a floating memory region, then the module name is marked as `unbacked` +- name of the symbol mapping to the return address, for example, `CreateProcessAsUserW` +- offset within the symbol + +Stack enrichment is applied to the following event types: + +- `CreateProcess` +- `CreateThread` +- `TerminateThread` +- `SetThreadContext` +- `LoadImage` +- `RegCreateKey` +- `RegDeleteKey` +- `RegSetValue` +- `RegDeleteValue` +- `CreateFile` +- `DeleteFile` +- `RenameFile` +- `VirtualAlloc` +- `OpenProcess` +- `OpenThread` +- `CreateSymbolicLinkObject` +- `SubmitThreadpoolWork` +- `SubmitThreadpoolCallback` +- `SetThreadpoolTimer` + +Stack enrichment is enabled by default, but can be controlled via `--eventsource.stack-enrichment` configuration flag. To enable stack enrichment for kernel space return addresses, the `symbolize-kernel-addresses` config option needs to be set to `true` diff --git a/docs/telemetry/evasions.md b/docs/telemetry/evasions.md new file mode 100644 index 000000000..62325bc59 --- /dev/null +++ b/docs/telemetry/evasions.md @@ -0,0 +1,23 @@ +# Evasions + +##### The evasion scanner is a behavioral analysis component designed to detect techniques that attempt to bypass traditional monitoring and security controls. It introduces a modular architecture that leverages a set of pluggable *evasion detectors*, each responsible for identifying a specific class of evasion behavior at runtime. + +At its core, the evasion scanner operates as a lightweight layer that integrates with the event processing pipeline. Rather than generating standalone alerts, it enriches existing events with additional context when suspicious behavior is observed. This design allows evasion signals to be correlated with other telemetry, preserving full execution context. + +Each evasion scanner encapsulates logic for identifying a particular technique. Available scannners focus on detecting [direct](https://docs.redteamleaders.com/offensive-security/defense-evasion/direct-syscall-execution-in-windows) and [indirect](https://redops.at/en/blog/direct-syscalls-vs-indirect-syscalls) system call evasion, where user-mode hooks are bypassed by invoking system calls outside of standard API pathways. + +### Evasion behaviour enrichment + +When an event is captured, the evasion scanner passes it through the registered detectors. If a detector identifies a known evasion pattern, then the event is annotated with **evasion metadata**. This metadata describes the type of evasion technique observed. +The enrichment occurs inline, without interrupting the event flow ensuring minimal overhead while still surfacing high-value behavioral signals. +The evasion metadata is designed to be consumable by the **rule engine**, enabling **precise** and **expressive** detections. + +### Direct syscall + +**Direct** syscall scanner hunts for adversary techniques to bypass traditional user-mode API monitoring and security +hooks by invoking system calls directly, but does so in a way that evades detection or analysis. +A direct syscall bypasses Windows API functions and calls the underlying system call directly using the `syscall` instruction, skipping the `NTDLL` stub that normally performs the transition to kernel mode. + +### Indirect syscall + +**Indirect** syscall scanner flags attempts to execute the `syscall` instruction diverting the execution flow into a legitimate, clean ntdll stub that performs the syscall on process behalf. This achieves code origin legitimacy, since the execution lands in text of a signed Microsoft `NTDLL` module. Stack frames look identical to a normal API call, which achieves call stack normalization. diff --git a/docs/telemetry/events.md b/docs/telemetry/events.md new file mode 100644 index 000000000..b1db9803f --- /dev/null +++ b/docs/telemetry/events.md @@ -0,0 +1,38 @@ +# Anatomy Of An Event + +**Event** encapsulates the state that is fundamental for assertion against the rule engine. Most [ETW](https://docs.microsoft.com/en-us/windows/win32/etw/event-tracing-portal) providers such as **Kernel Logger** produce the flow of events in raw format. This means that the vast majority of the event's parameters lack a meaningful context needed for an intuitive human interpretation. To clarify the previous statement, here are some concrete examples: + +- Registry keys containing partial paths are typically not very useful. Think of `Settings/Control Panel` key that actually represents the fully qualified `HKEY_CURRENT_USER/Settings/Control Panel` path. Fibratus makes sure to run a hook in the early stage of the event stream to reconstruct the full registry path. Aside from this, root keys are represented in the native object manager format, for example, `REGISTRY\MACHINE` is the key name for the `HKEY_CURRENT_MACHINE` root level key. Fibratus normalizes native root keys to well-known names. +- Missing file paths in `ReadFile` or `WriteFile` events. These events contain only the address of the file object that obviously doesn't give any hints about the file path involved in the I/O operation. Fibratus strives for a best-effort file name resolution. Similarly to native registry key names, the Kernel Logger emits files names in DOS device name format, for example, `Device\HardDisk4\Windows\System32\kernel32.dll`. Fibratus maps DOS device names to drive letters. +- Handle type names are given as integer values. Fibratus takes care of mapping type identifiers to human-readable handle type names such as `File`, `Mutant` or `Key`. + +This is not the only heavy-lifting tasks Fibratus performs to produce high-quality events. Additionally, events are enriched with various parameters that are not originally present in ETW payloads. + +### Canonical fields + +Each event contains a series of canonical fields that describe the nature of the event such as its name, the process identifier that generated the event and such. Canonical fields include: + +- `Sequence` is a monotonically increasing integer value that uniquely identifies an event. The sequence value is guaranteed to increment monotonically as long as the machine is not rebooted. On machine reboot, the sequence value is reset. +- `PID` represents the process identifier that triggered the event +- `TID` is the thread identifier connected to the event +- `CPU` designates the logical CPU core on which the event was originated +- `Name` is the human-readable event name such as `CreateProcess` or `VirtualAlloc` +- `Timestamp` denotes the timestamp expressed in nanosecond precision as the instant the event occurred +- `Category` designates the category to which the event pertains, for example, `file` or `thread` +- `Host` represents the host name where the event was produced + +### Parameters + +Internally, event parameters are modeled as a collection of key/value pairs. The key is mapped to the structure consisting of parameter name, parameter type and the value. An example of the parameter tuple could be the `dip` parameter that denotes a destination IP address with value `172.17.0.2` and therefore `IPv4` type. Parameter types can be scalar values, strings, slices, enumerations, and timestamps among others. + +### Process state + +Each event has attached process state that represents the contextual information about the process including its name, command line, user, token integrity level, allocated resources such as handles, dynamically-linked libraries, environment variables and other attributes. The process state internals are thoroughly explained in the [Process](events/process.md) events section. + +### Callstacks + +[Callstacks](callstacks.md) provide detailed insight into the execution context of system events by capturing the sequence of function calls (stack frames) that led to a particular action, such as a file write, process creation, or registry state manipulation. + +### Metadata + +Metadata are arbitrary tags composed of key/value pairs. Event metadata are decorated via [transformers](transformers.md). Tags can hold any string values employed for filtering, grouping or other purposes. diff --git a/docs/telemetry/events/file.md b/docs/telemetry/events/file.md new file mode 100644 index 000000000..4bca2bbd7 --- /dev/null +++ b/docs/telemetry/events/file.md @@ -0,0 +1,72 @@ +# File Events + +##### File events encompass a variety of file system related activities such as creating or opening directories/files/devices, writing or reading data, altering file metadata and so on. + +### `CreateFile` + +The `CreateFile` event is triggered when the kernel serves create/open requests for files or I/O devices. The most commonly used I/O devices are as follows: file, file stream, directory, physical disk, volume, console buffer, tape drive, communications resource, mailslot, and pipe. `CreateFile` events have the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `file_object` | File object pointer as seen from the kernel perspective. You can use this parameter to correlate file events. | +| `file_path` | File/directory path or device name, for example, `C:\ProgramData\AVG\Antivirus\psi.db-journal` | +| `irp` | I/O request packet value that identifies the file system activity. | +| `tid` | Thread identifier that initiated the I/O operation. | +| `create_disposition` | Identifies the file system operation performed on the file or device. Can be `SUPERSEDE` (replaces the file if it already exists, otherwise creates a new file), `OPEN` (opens the file if it exists), `CREATE` (creates a new file or fails if the file already exists), `OPENIF` (opens the file if it already exists, otherwise creates a new file), `OVERWRITE` (opens and overwrites the file if it already exists) and `OVERWRITEIF` (opens and overwrites the file if it already exists, otherwise creates a new file). | +| `create_options` | Options to be applied when creating or opening the file, as a compatible combination of the following values: `DIRECTORY_FILE`, `WRITE_THROUGH`, `SEQUENTIAL_ONLY`, `NO_INTERMEDIATE_BUFFERING`, `SYNCHRONOUS_IO_ALERT`, `SYNCHRONOUS_IO_NONALERT`, `NON_DIRECTORY_FILE`, `CREATE_TREE_CONNECTION`, `COMPLETE_IF_OPLOCKED`, `NO_EA_KNOWLEDGE`, `OPEN_REMOTE_INSTANCE`, `RANDOM_ACCESS`, `DELETE_ON_CLOSE`,`OPEN_BY_FILE_ID`, `FOR_BACKUP_INTENT`, `NO_COMPRESSION`, `OPEN_REQUIRING_OPLOCK`,`DISALLOW_EXCLUSIVE`, `RESERVE_OPFILTER`, `OPEN_REPARSE_POINT`, `OPEN_NO_RECALL` and `OPEN_FOR_FREE_SPACE_QUERY` | +| `share_mask` | Specifies the sharing mode of the file or device, which can be the combination of `READ`, `WRITE`, and `DELETE` values. This flag determines the permission granularity which enables a process to share a file or device while another process has the file or device open. | +| `type` | Idefines the file type. Possible values are `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown` | +| `attributes` | Denotes the file attributes. Possible values are `READONLY`, `HIDDEN`, `SYSTEM`, `DIRECTORY`, `COMPRESSED`, `ENCRYPTED`, `JUNCTION`, `SPARSE`,`TEMPORARY`, `DEVICE`, `NORMAL`, `OFFLINE`, `UNINDEXED`, `STREAM`, `VIRTUAL`, `NOSCRUB`, `RECALLOPEN`, `RECALLACCESS`, `PINNED`, `UNPINNED`, `UNKNOWN` | +| `status` | Represents the system status message, for example, `Success` | + + +### `WriteFile` `ReadFile` + +`WriteFile` and `ReadFile` events occur when a process writes data to a file or reads data from the file or I/O device respectively. They contain the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `file_object` | File object pointer as seen from the kernel perspective. You can use this parameter to correlate file events. | +| `file_path` | Represents the file/directory or device name the data is written to or read from. | +| `irp` | I/O request packet value that identifies the file system activity. | +| `io_size` | Specifies the number of bytes read or written. | +| `offset` | Offset in the file where the data is read or written. | +| `type` | Defines the file type. Possible values include `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown` | + + +### `DeleteFile` `RenameFile` `CloseFile` + +`DeleteFile` is fired when the file is removed from the file system. `RenameFile` signals file or directory renaming. `CloseFile` fires when file handle is closed. These events share the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `file_object` | File object pointer as seen from the kernel perspective. You can use this parameter to correlate file events. | +| `file_path` | Represents the file path/directory involved in the operation. | +| `irp` | I/O request packet value that identifies the file system activity. | +| `type` | Defines the file type. Possible values include `File`, `Directory`, `Pipe`, `Console`, `Mailslot`, `Other`, `Unknown` | + + +### `SetFileInformation` + +`SetFileInformation` is published when the file metadata are manipulated for the given information class. The event contains the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `class` | File information class. For example, the `Basic` information class means the process altered file timestamps or basic attributes. Refer to [this](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ne-wdm-_file_information_class) link for a full list of the file information class enumerations. | +| `file_object` | File object pointer as seen from the kernel perspective. | +| `file_path` | Represents the file whose information class is manipulated. | +| `irp` | I/O request packet value that identifies the file system activity. | +| `type` | Defines the file type. Possible values include `File`, `Pipe`, `Mailslot`, `Other`, `Unknown` | + + +### `EnumDirectory` + +The `EnumDirectory` event is triggered in response to directory enumeration requests. The event has the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `dir` | Specifies the directory that was requested for enumeration. | +| `file_path` | Pattern for directory enumeration, for example, `*` | +| `class` | Identifies the requested directory enumeration class. | +| `file_object` | File object pointer as seen from the kernel perspective. | +| `irp` | I/O request packet value that identifies the file system activity. | diff --git a/docs/telemetry/events/handle.md b/docs/telemetry/events/handle.md new file mode 100644 index 000000000..cd4900787 --- /dev/null +++ b/docs/telemetry/events/handle.md @@ -0,0 +1,53 @@ +# Handle Events + +##### The object manager produces handle events to provision or dispose resources allocated to processes. Resources can be threads, registry keys, files, synchronization primitives and so on. **Fibratus** captures handle creation, destruction, and duplication operations. + +?> Handle events are voluminous and they are disabled by default. To enable the collection of handle events either run Fibratus with the `--eventsource.enable-handle=true` flag or activate it permanently by editing the config file. + +### `CreateHandle` + +`CreateHandle` is captured when the object manager provisions a new handle in the address space of the calling process. The following parameters are associated with the `CreateHandle` event: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `handle_id` | Unique identifier of the handle. | +| `handle_object` | Address of the kernel object to which the handle is associated. | +| `type_id` | Handle type, for example, `File` or `Key` | + +### `CloseHandle` + +`CloseHandle` event is triggered when the handle is released by the process. It contains the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `handle_id` | Unique identifier of the handle. | +| `handle_name` | Handle name, for example, `\RPC Control\OLEA61B27E13E028C4EA6C286932E80` | +| `handle_object` | Address of the kernel object to which the handle is associated. | +| `type_id` | Handle type, for example, `File` or `Key` | + + +### `DuplicateHandle` + +`DuplicateHandle` event is fired when the process duplicates an object handle. The following parameters are present in this event: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `handle_id` | Duplicated handle identifier. | +| `handle_object` | Address of the duplicated kernel object. | +| `handle_source_id` | Identifier of the handle to be duplicated. | +| `type_id` | Duplicated handle type, for example, `DxgkCompositionObject` | +| `pid` | Process identifier from which the handle is duplicated. | +| `exe` | Process executable path from which the handle is duplicated. | +| `name` | Process name from which the handle is duplicated. | + +### Handle state + +During bootstrap, Fibratus builds a snapshot of currently allocated handles. Similarly, when a new process is created Fibratus fetches its handles and attaches them to the process state. However, to optimize memory utilization, the initial handle snapshot and process handle table initialization are disabled by default. You can enable both features by modifying the `--handle.init-snapshot=true` and `--handle.enumerate-handles` config flags respectively or changing the corresponding key in the configuration file. + +The handle state contains: + +- handle name +- handle type +- the address of the kernel object +- handle identifier +- additional metadata such as `ALPC` port information or mutant count diff --git a/docs/telemetry/events/memory.md b/docs/telemetry/events/memory.md new file mode 100644 index 000000000..185f67139 --- /dev/null +++ b/docs/telemetry/events/memory.md @@ -0,0 +1,61 @@ +# Memory Events + +##### Memory events include reserving, committing, or freeing the state of a region of pages, and also, mapping/unmapping the section view into/from the virtual process address space. + +### `VirtualAlloc` + +`VirtualAlloc` event is published in response to memory allocation of a region of pages in the virtual address space. If the `pid` parameter is different than the calling process id, memory allocation is performed in the address space of another process. `VirtualAlloc` events carry the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `alloc_type` | designates the type of memory allocation. It can be the combination of `COMMIT`, `RESERVE`, `RESET`, `RESET_UNDO`, `PHYSICAL`, `LARGE_PAGES`, `TOP_DOWN`, and `WRITE_WATCH` | +| `base_address` | Starting address of the allocated region. | +| `page_type` | Type of pages in the allocated region. It can be one of `IMAGE`, `MAPPED`, or `PRIVATE` | +| `protection` | Memory protection for the region of allocated pages. It can be the combination of `EXECUTE`, `EXECUTE_READ`, `EXECUTE_READWRITE`, `EXECUTE_WRITECOPY`, `NOACCESS`, `READONLY`, `READWRITE`, `WRITECOPY`, `TARGETS_INVALID`, `TARGETS_NO_UPDATE`, `GUARD`, `NOCACHE`, and `WRITECOMBINE` | +| `protection_mask` | Abbreviated form of the pages protection flag, for example, `RWX` | +| `region_size` | Size of the allocated region in bytes. | +| `pid` | Process identifier where the memory allocation occurs. | +| `exe` | Process executable path where the memory allocation occurs. | +| `name` | Process name where the memory allocation occurs. | + +### `VirtualFree` + +`VirtualFree` event is captured when the memory manager releases, decommits, or releases and decommits a region of pages within the virtual process address space. If the `pid` parameter is different than the calling process id, memory release is performed in the address space of another process. `VirtualFree` events contain the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `alloc_type` | designates the type of a freed memory region. It can be the combination of `COMMIT`, `RESERVE`, `RESET`, `RESET_UNDO`, `PHYSICAL`, `LARGE_PAGES`, `TOP_DOWN`, and `WRITE_WATCH` | +| `base_address` | Starting address of the freed region. | +| `region_size` | Size of the freed region in bytes. | +| `pid` | Process identifier where the memory release occurs. | +| `exe` | Process executable path where the memory release occurs. | +| `name` | Process name where the memory release occurs. | + + +### `MapViewFile` + +`MapViewFile` is published when the view of a file mapping is mapped into the process address space. This event contain the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `file_key` | Address of the file object for which the mapping is performed. | +| `offset` | File offset where the view is to begin. | +| `pid` | Process identifier where the file mapping is performed. | +| `protection` | Specifies the page protection of the file mapping object. Can be the compatible combination of the following values: `READONLY`, `EXECUTE`, `EXECUTE_READ`, `READWRITE`, `WRITECOPY`, `NOCACHE`, `EXECUTE_WRITECOPY` and `EXECUTE_READWRITE` +| `section_type` | Type of the mapped section. It can be `DATA`, `IMAGE`, `IMAGE_NO_EXECUTE`, `PAGEFILE` or `PHYSICAL` | +| `view_base` | Base memory address in the process address space where mapping begins. | +| `view_size` | Number of bytes of a file mapping to map to a view. | + +### `UnmapViewFile` + +`UnmapViewFile` trigger as a response to unmapping a mapped view of a file from the process's virtual address space. This event has the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `file_key` | Address of the file object for which the unmapping is performed. | +| `offset` | File offset where the view to unmap begins. | +| `pid` | Process identifier where the file unmapping is performed. | +| `protection` | specifies the page protection of the file mapping object that is being unmapped. Can be the compatible combination of the following values: `READONLY`, `EXECUTE`, `EXECUTE_READ`, `READWRITE`, `WRITECOPY`, `NOCACHE`, `EXECUTE_WRITECOPY` and `EXECUTE_READWRITE` | +| `section_type` | Type of the unmapped section. It can be `DATA`, `IMAGE`, `IMAGE_NO_EXECUTE`, `PAGEFILE` or `PHYSICAL` | +| `view_base` | Base memory address in the process address space where unmapping begins. | +| `view_size` | Number of bytes of a file mapping to unmap. | diff --git a/docs/telemetry/events/module.md b/docs/telemetry/events/module.md new file mode 100644 index 000000000..870b0e436 --- /dev/null +++ b/docs/telemetry/events/module.md @@ -0,0 +1,16 @@ +# Module Events + +##### Module events occur when a process loads/unloads a dynamic linked library, executable or a kernel driver. The loading can happen in the local or remote process. These events are represented by `LoadModule` and `UnloadModule` types respectively. The following list describes all available parameters present in module events captured by **Fibratus** + +### `LoadModule` `UnloadModule` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `file_path` | Full path name of the module file, for example, `C:\Windows\system32\kernel32.dll` | +| `module_size` | Represents the size of the mapped module region. | +| `checksum` | Represents the module checksum digest. | +| `base_address` | Base address of the process in which the image is loaded/unloaded. | +| `default_address` | Represents the module base address. | +| `pid` | Specifies the process identifier where the module is loaded/unloaded. | +| `signature_type` | Describes the type of the digital signature. Can be `NONE`, `EMBEDDED`, `CACHED`, `CATALOG_CACHED`, `CATALOG_UNCACHED`, `CATALOG_HINT`, `PACKAGE_CATALOG` and `FILE_VERIFIED`. This parameter is a best-effort hint. The kernel doesn't always verify signatures on module load and can report false positives for the signature type. | +| `signature_level` | Describes the signature level. This parameter is a best-effort hint. The kernel doesn't always verify signatures on module load and can report false positives for the signature level. | diff --git a/docs/telemetry/events/network.md b/docs/telemetry/events/network.md new file mode 100644 index 000000000..b2cd3b1e7 --- /dev/null +++ b/docs/telemetry/events/network.md @@ -0,0 +1,57 @@ +# Network Events + +##### Interprocess communication via TCP/UDP sockets produces all sorts of network events including, sending and receiving data, retransmitting TCP segments or connecting to sockets. DNS events are also classified as a subset of network events. + +### TCP/UDP events + +Fibratus captures TCP and UDP network events at a low level, providing detailed visibility into how processes communicate over the network. By leveraging Windows kernel telemetry, it records connection attempts, accepted connections, data transfers, and endpoint metadata such as IP addresses and ports, all correlated back to the originating process. The following network signals are surfaced: + +- `Connect` establishes a connection to the stream-oriented socket. +- `Accept` accepts the connection request from the socket queue. +- `Send` transports data to local or remote endpoint. +- `Recv` consumes data sent by local or remote socket. +- `Disconnect` terminates data reception on the socket. +- `Reconnect` reconnects to the socket. +- `Retransmit` retransmits unacknowledged TCP segments. + +All TPC/UDP network events share the same parameter set: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `dip` | Destination IPv4/IPv6 address of the communication endpoint. | +| `sip` | Source IPv4/IPv6 address of the local process. | +| `dport` | Destination port. | +|`sport` | Source port. | +| `l4_proto` | Type of the Layer 4 protocol. Possible values are `tcp`, `udp` | +| `dport_name` | Destination port name per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service mappings. | +| `sport_name` | Source port name per [IANA](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt) port to service mappings. | + + +### DNS events + +DNS telemetry uncovers all DNS query/reply interactions. More specifically, `QueryDns` and `ReplyDns` events are fired when the process sends a query to the name server and when it receives the response from the DNS server, respectively. DNS events are collected by default, but it is possible to disable them by setting the `eventsource.enable-dns` config flag to `false`. + +`QueryDns` event is generated when a query is sent to the name server. This event has the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `name` | DNS query, for example, `www.iana.org` | +| `options` | DNS query options. It can be the combination of the following values: `STANDARD`, `ACCEPT_TRUNCATED_RESPONSE`, `USE_TCP_ONLY`, `NO_RECURSION`, `BYPASS_CACHE`, `NO_WIRE_QUERY`, `NO_LOCAL_NAME`, `NO_NETBT`, `WIRE_ONLY`, `RETURN_MESSAGE`, `MULTICAST_ONLY`, `NO_MULTICAST`, `TREAT_AS_FQDN`, `ADDRCONFIG`, `DUAL_ADDR`, `MULTICAST_WAIT`, `MULTICAST_VERIFY`, `DONT_RESET_TTL_VALUES`, `DISABLE_IDN_ENCODING`, `APPEND_MULTILABEL` | +| `rr` | Type of the resource record. It can be one of `A`,`NS`, `MD`, `MF`, `CNAME`, `SOA`, `MB`, `MG`, `MR`, `NULL`, `WKS`, `PTR`,`HINFO`, `MINFO`, `MX`, `TEXT`, `RP`, `AFSDB`, `X25`, `ISDN`, `NSAPPTR`, `SIG`, `KEY`, `PX`, `GPOS`,`AAAA`, `LOC`, `NXT`, `EID`, `NIMLOC`, `SRV`, `ATMA`, `NAPTR`, `KX`, `CERT`, `A6`, `DNAME`, `SINK`, `OPT`, `DS`, `RRSIG`,`NSEC`, `DNSKEY` `DHCID`, `UINFO`, `UID`, `GID`, `UNSPEC`, `ADDRS`, `TKEY`, `TSIG`, `IXFR`, `AXFR`, `MAILB`, `MAILA`, `ANY`, `WINS`, `WINSR` | + + +`ReplyDns` event is captured when the the response is received by the DNS server. DNS reply events contain the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `name` | DNS query linked to the response, for example, `www.iana.org` | +| `answers` | Response answers, for example, `151.101.194.132` +| `options` | DNS response options. It can be the combination of the following values: `STANDARD`, `ACCEPT_TRUNCATED_RESPONSE`, `USE_TCP_ONLY`, `NO_RECURSION`, `BYPASS_CACHE`, `NO_WIRE_QUERY`, `NO_LOCAL_NAME`, `NO_NETBT`, `WIRE_ONLY`, `RETURN_MESSAGE`, `MULTICAST_ONLY`, `NO_MULTICAST`, `TREAT_AS_FQDN`, `ADDRCONFIG`, `DUAL_ADDR`, `MULTICAST_WAIT`, `MULTICAST_VERIFY`, `DONT_RESET_TTL_VALUES`, `DISABLE_IDN_ENCODING`, `APPEND_MULTILABEL`. +| `rr` | Type of the resource record. It can be one of `A`,`NS`, `MD`, `MF`, `CNAME`, `SOA`, `MB`, `MG`, `MR`, `NULL`, `WKS`, `PTR`,`HINFO`, `MINFO`, `MX`, `TEXT`, `RP`, `AFSDB`, `X25`, `ISDN`, `NSAPPTR`, `SIG`, `KEY`, `PX`, `GPOS`,`AAAA`, `LOC`, `NXT`, `EID`, `NIMLOC`, `SRV`, `ATMA`, `NAPTR`, `KX`, `CERT`, `A6`, `DNAME`, `SINK`, `OPT`, `DS`, `RRSIG`, `NSEC`, `DNSKEY` `DHCID`, `UINFO`, `UID`, `GID`, `UNSPEC`, `ADDRS`, `TKEY`, `TSIG`, `IXFR`, `AXFR`, `MAILB`, `MAILA`, `ANY`, `WINS`, `WINSR` | +| `rcode` | DNS response code. It can be one of `NOERROR`, `FORMERR`, `SERVFAIL`, `NXDOMAIN`, `NOTIMP`, `REFUSED`,`YXDOMAIN`, `YXRRSET`, `NXRRSET`, `NOTAUTH`, `NOTZONE`, `BADSIG`, `BADKEY`, `BADTIME`, `BADNAME`, `INVALID`, `NXDOMAIN` | + +### DNS reverse lookups + +Fibratus supports reverse DNS resolution for IP addresses, enabling the retrieval of domain names associated with a given network endpoint. For instance, `47.224.186.35.bc.googleusercontent.com` represents the reverse DNS entry for a Google-owned IP address. + +Resolved domain names can be incorporated into detection rules via [filter fields](../../rules/fields.md) to enhance network event analysis. diff --git a/docs/telemetry/events/object.md b/docs/telemetry/events/object.md new file mode 100644 index 000000000..dfbace29b --- /dev/null +++ b/docs/telemetry/events/object.md @@ -0,0 +1,16 @@ +# Object Manager Events + +##### **Fibratus** captures the `CreateSymbolicLinkObject` event when a symbolic link object is created via native APIs. The Windows kernel emits an event that includes key metadata such as the link name, target path, and the process responsible for the operation. + +### `CreateSymbolicLinkObject` + +This visibility is particularly valuable because symbolic links are often leveraged by both legitimate system components and adversaries to redirect access to sensitive resources or obscure execution paths. By capturing these events in real time and correlating them with other system activity, Fibratus allows analysts to detect anomalous link creation patterns, trace their origin, and incorporate them into behavioral detections. + +`CreateSymbolicLinkObject` has the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `source` | Source symbolic link object or other kernel object, for example, `DosDevices\PROCEXP152` | +| `target` | Target symbolic link object or other kernel object, for example, `\Device\PROCEXP152` | +| `desired_access` | Access rights for the target symbolic link object. Can be the combination of `DELETE`, `READ_CONTROL`, `WRITE_DAC`, `WRITE_OWNER`, `SYNCHRONIZE`, `STANDARD_RIGHTS_REQUIRED`, `STANDARD_RIGHTS_ALL`, `ACCESS_SYSTEM_SECURITY`, `MAXIMUM_ALLOWED`, `GENERIC_READ`, `GENERIC_WRITE`, `GENERIC_EXECUTE`, `GENERIC_ALL` | +| `status` | System status code that represents the outcome of the operation. | diff --git a/docs/telemetry/events/process.md b/docs/telemetry/events/process.md new file mode 100644 index 000000000..0f400a2ce --- /dev/null +++ b/docs/telemetry/events/process.md @@ -0,0 +1,58 @@ +# Process Events + +##### Process events are fired as a stimulus to the process life-cycle changes. When the kernel puts into motion a process or terminates it, the `CreateProcess` and `TerminateProcess` events are emitted respectively. `OpenProcess` event fires when the process attemps to acquire an existing local process object. The following sections summarize all the distinct event parameters that are associated with process events captured by **Fibratus** + +### `CreateProcess` `TerminateProcess` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `pid` | Process identifier. This value is valid from the time a process is created until it is terminated. | +| `tid` | Thread identifier inside process address space that generated the event. | +| `ppid` | Parent identifier of the child process. Process identifier numbers are reused, so they only identify a process for the lifetime of that process. It is possible that the process identified by `ppid` is terminated, so `ppid` may not refer to a running process. It is also possible that `ppid` incorrectly refers to a process that reuses a process identifier. | +| `real_ppid` | Real parent process identifier useful for detecting process spoofing. | +| `name` | Process name including file extension, for example, `cmd.exe` | +| `cmdline` | Full process command line, for example, `C:\Windows\system32\cmd.exe /cdir /-C /W`) | +| `exe` | Full name of the process executable, for example, `C:\Windows\system32\cmd.exe` | +| `sid` | Security identifier under which this process runs, for example, `S-1-5-18` | +| `kproc` | Represents the address of the `KPROCESS` object in the kernel. | +| `directory_table_base` | Represents the address of the directory table that holds process memory page mappings. | +| `session_id` | Unique identifier for the current session under which process was started or terminated. | +| `status` | Exit status of the started/stopped process. | +| `start_time` | Designates the instant when the process was started. Start time is available only in `CreateProcess` events. | +| `domain` | Represents the domain name under which the process is started. | +| `username` | Represents the username that started the process. | +| `flags` | Represents process creation flags. Can be `WOW64`, `PROTECTED`, or `PACKAGED` to designate 32-bit process is created in 64-bit Windows system, process is to be run as a protected process, or a process packaged with the [MSIX](https://learn.microsoft.com/en-us/windows/msix/overview) technology respectively. | +| `token_integrity_level` | Process token integrity level. Can be `PROTECTED`, `SYSTEM`, `HIGH`, `MEDIUM`, `MEDIUM+`, `LOW` and `UNTRUSTED`. | +| `token_is_elevated` | Indicates if the process token is elevated. | +| `token_elevation_type` | Indicates the process token elevation type. Can be `FULL` or `LIMITED`. | + +### `OpenProcess` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `desired_access` | Value that represents the desired access bitmask to the process object. | +| `desired_access_names` | List of human-readable desired access strings, for example, `TERMINATE,QUERY_INFORMATION`. For a full list and detailed explanation of available access rights, head to the official [docs](https://docs.microsoft.com/en-us/windows/win32/procthread/process-security-and-access-rights). | +| `name` | Name of the local process that was opened. | +| `exe` | Full path of the local process object that was open. | +| `pid` | Process identifier of the local process that was opened. | +| `status` | System status of the open operation, for example, `Success` | + +## Process state + +Fibratus keeps a snapshot of all running processes including their state such as basic process attributes, allocated file handles, dynamically-linked libraries, PE (Portable Executable) metadata and other resources. The snapshot is updated dynamically as processes get spawn or die. Each time an event is captured, its **process state** is fetched from the snapshot and attached to the event. This state machine semantically enriches each individual event with the aim on providing a powerful context for [rules](../../rules.md), [filtering](../filtering.md), and [scripting](../../filaments.md). + +Process state comprises the following attributes and resources: + +- process name +- process identifier as well as its parent process identifier +- process command line +- current working directory +- process SID +- session identifier +- process token properties +- environment variables +- threads +- modules +- handles +- memory mappings +- PE metadata diff --git a/docs/telemetry/events/registry.md b/docs/telemetry/events/registry.md new file mode 100644 index 000000000..272d131fc --- /dev/null +++ b/docs/telemetry/events/registry.md @@ -0,0 +1,47 @@ +# Registry Events + +##### Registry events are triggered when processes access or modify the registry structure, such as creating new keys, altering registry key values or opening a handle to the registry key. + +### `RegCreateKey` `RegDeleteKey` `RegOpenKey` `RegCloseKey` `RegQueryKey` + +Creates a new registry key or opens the key if it already exists. Deletes a subkey and all its values. Opens a registry key. Closes the registry key. Enumerates the subkeys of the specified key. All of these events share a common parameter schema: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `key_handle` | Represents the address of the [KCB](https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/-regkcb) (Key Control Block) structure in kernel space. | +| `key_path` | Full registry path involved in the operation, for example, `HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control` | +| `status` | System status code of the registry operation, for example, `More data is available` | + +### `RegQueryValue` + +`RegQueryValue` is captured when the process retrieves the data from registry value. This event contains the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `key_handle` | Represents the address of the [KCB](https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/-regkcb) (Key Control Block) structure in kernel space. | +| `key_path` | Full path of the registry value, for example, `HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\System\GpSvcDebugLevel` | +| `status` | System status code of the registry operation, for example, `The system cannot find the file specified` | + + +### `RegSetValue` + +`RegSetValue` event is triggered when registry data is set in the value. This event contains the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `key_handle` | Represents the address of the [KCB](https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/-regkcb) (Key Control Block) structure in kernel space. | +| `key_path` | Full path of the registry value, for example, `HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\System\GpSvcDebugLevel` | +| `status` | System status code of the registry operation, for example, `Success` | +| `data` | Value data being stored. | +| `value_type` | Registry value type. Possible values include `REG_DWORD`, `REG_QWORD`, `REG_SZ`, `REG_EXPAND_SZ`, `REG_MULTI_SZ`, `REG_BINARY`, `UNKNOWN` | + +### `RegDeleteValue` + +`RegDeleteValue` is captured when the registry values is deleted. This event contains the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `key_handle` | Represents the address of the [KCB](https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/-regkcb) (Key Control Block) structure in kernel space. | +| `key_path` | Full path of the registry value, for example, `HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\System\GpSvcDebugLevel` | +| `status` | System status code of the registry operation, for example, `Success` | +| `value_type` | Registry value type. Possible values include `REG_DWORD`, `REG_QWORD`, `REG_SZ`, `REG_EXPAND_SZ`, `REG_MULTI_SZ`, `REG_BINARY`, `UNKNOWN` | \ No newline at end of file diff --git a/docs/telemetry/events/thread.md b/docs/telemetry/events/thread.md new file mode 100644 index 000000000..b5217636b --- /dev/null +++ b/docs/telemetry/events/thread.md @@ -0,0 +1,91 @@ +# Thread Events + +##### Thread events notify the creation or termination of threads within the process address space via `CreateThread` and `TerminateThread` events respectively. When the thread object is acquired, the `OpenThread` event is fired. Processes can manipulate thread context of local or remote threads. Under this circumstances the `SetThreadContext` is triggered. The following sections summarize all the distinct event parameters that are associated with thread events captured by **Fibratus** + +### `CreateThread` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `pid` | Process identifier where the thread has been spawned. If this value is not equal to the `pid` of the process generating the event, then the thread is created in the virtual address space of another process using the [`CreateRemoteThread`](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createremotethread) API function. | +| `tid` | Represents the identifier of the freshly spawn thread. | +| `base_prio`| Scheduler priority for the new thread. | +|`page_prio` | Memory page priority hint for memory pages accessed by the thread. | +| `io_prio` | I/O priority hint for scheduling I/O operations generated by the thread. | +| `ustack_base` | Base address of the thread's user space stack. | +| `ustack_limit`| Limit of the thread's user space stack. | +| `kstack_base` | Base address of the thread's kernel space stack. | +| `kstack_limit` | Limit of the thread's kernel space stack. | +| `start_address` | Start address of the function to be executed by the thread. | +| `start_address_symbol` | Symbol the represents the thread start address, for example, `LoadLibraryEx` | +| `start_address_module` | Module that maps to the thread start address, for example, `ntdll.dll` | +| `teb` | Address of the [`TEB`](https://learn.microsoft.com/en-us/windows/win32/api/winternl/ns-winternl-teb) (Thread Environment Block) structure that describes the state of a thread. | + +### `TerminateThread` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `pid` | Process identifier where the thread has been terminated. | +| `tid` | Represents the identifier of the terminated thread. | +| `base_prio`| Scheduler priority of the thread. | +|`page_prio` | Memory page priority hint for memory pages accessed by the thread. | +| `io_prio` | I/O priority hint for scheduling I/O operations generated by the thread. | +| `ustack_base` | Base address of the thread's user space stack. | +| `ustack_limit`| Limit of the thread's user space stack. | +| `kstack_base` | Base address of the thread's kernel space stack. | +| `kstack_limit` | Limit of the thread's kernel space stack. | +| `start_address` | Start address of the function thread used to execute. | +| `teb` | Address of the [`TEB`](https://learn.microsoft.com/en-us/windows/win32/api/winternl/ns-winternl-teb) (Thread Environment Block) structure that describes the state of a thread. | + +### `OpenThread` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `desired_access` | Value that represents the desired access to the thread object. | +| `desired_access_names` | List of human-readable desired access strings, for example, `QUERY_LIMITED_INFORMATION`. For a full list and detailed explanation of available access rights, head to the official [docs](https://docs.microsoft.com/en-us/windows/win32/procthread/thread-security-and-access-rights). +| `name` | Name of the local process whose thread object was open. | +| `exe` | Full path of the local process image whose thread object was open. | +| `pid` | Identifier of the local process whose thread object was opened. | +| `tid` | Identifier of the local thread that was opened. | +| `status` | System status of the thread object open operation, for example, `Success` | + +### `SetThreadContext` + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `status` | System status of the thread context set operation, for example, `Access denied` | + +## Thread pool events + +A thread pool is a collection of worker threads that efficiently execute asynchronous callbacks on behalf of the application. The thread pool is primarily used to reduce the number of application threads and provide management of the worker threads. Fibratus collects the following thread pool telemetry. + + +### `SubmitThreadpoolWork` `SubmitThreadpoolCallback` + +`SubmitThreadpoolWork` enqueues the work item to the thread pool. `SubmitThreadpoolCallback` submits the thread pool callback for execution within the work item. These events has the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `pool_id` | Represents the thread pool identifier. | +| `task_id` | Represents the thread pool task identifier. | +| `callback`| Represents the address of the callback function | +| `context` | Represents the address of the callback context. | +| `subprocess_tag` | Represents the service identifier associated with the thread pool. | +| `callback_symbol` | Represents the callback symbol, for example, `RtlCaputreContext` | +| `callback_module` | Represents the module containing the callback symbol, for example, `C:\Windows\System32\ntdll.dll` | +| `context_rip` | Represents the value of instruction pointer contained in the callback context | +| `context_rip_symbol` | Represents the symbol name associated with the instruction pointer in callback context. | +| `context_rip_module` | Represents the module name associated with the instruction pointer in callback context. | + + +### `SetThreadpoolTimer` + +Sets the thread pool timer object. This event consists of the following parameters: + +| PARAMETER | DESCRIPTION | +| :--- | :---- | +| `duetime` | Represents the timer due time. | +| `subqueue` | Represents the memory address of the timer subqueue. | +| `timer` | Represents the memory address of the timer object. | +| `period` | Represents the period of the timer. | +| `window` | Represents the timer tolerate period. | +| `absolute` | Indicates if the timer is absolute or relative. | diff --git a/docs/telemetry/filtering.md b/docs/telemetry/filtering.md new file mode 100644 index 000000000..226e1c8b5 --- /dev/null +++ b/docs/telemetry/filtering.md @@ -0,0 +1,140 @@ +# Filtering + +##### Modern systems generate an **overwhelming volume of events**, making it difficult to extract meaningful insights or focus on specific behaviors. **Fibratus** provides a powerful and expressive filtering engine that allows to narrow down event streams, isolate suspicious activity and drive investigations or postmortem analysis. Filters are the foundation of detection [rules](../rules.md) logic. + +## Anatomy of a filter + +At its core, a filter expression consists of `LHS` (Left Hand Side) and `RHS` (Right Hand Side) components connected by a binary, logical, or string operator as depicited in the image below. + +![Filter](images/filter.png "Filter") + +!> A special case are boolean fields that can appear alone in the filter expression. For example, `module.is_dll` is the boolean shortcut of `module.is_dll = true` + +Suppose we want to filter all events generated by `cmd.exe`, `powershell.exe` or `winword.exe` processes. We would write the following filter expression. + +```python +ps.name in ('cmd.exe', 'powershell.exe', 'winword.exe') +``` + +### LHS + +The LHS component is typically a [field](../rules/fields.md) however it can also be a [function](../rules/functions.md) result. Examples of field names are `ps.name` or `evt.pid`. Fields contain values extracted from event parameters, callstack frames, or process context. + +### RHS + +The RHS component can be of several types including strings, numbers, IP addresses, boolean values, field references and collections. Here are some examples: + +| TYPE | EXPRESSION | +| :--- | :---- | +| string | `file.name = 'cmd.exe'` | +| number | `ps.pid = 4` | +| boolean | `dll.signature.exists = true` | +| network address | `net.sip = 127.0.0.1` | +| field reference | `evt.pid != ps.pid` | +| collection | `ps.name in ('cmd.exe', 'powershell.exe')` | + + +String values must be enclosed in single quotes `''`. If a string contains characters that would make it an invalid identifier, those characters need to be escaped. For instance, path delimiters (backslashes) and quotation marks require escaping: + +```python +file.path = 'C:\\Windows\\System32' +``` + +Filter expressions also support standard escape sequences, such as the newline character. + +### Operators + +Operators define how LHS and RHS are evaluated. They fall into several categories: + +| TYPE | OPERATORS | +| :--- | :---- | +| Comparison | `=` `!=` `>` `<` `>=` `<=` | +| Logical | `and` `or` `not` | +| String | `and` `or` `not` | +| Logical | `contains` `startswith` `endswith` `matches` `icontains` `istartswith` `iendswith` `imatches` | +| Membership | `in` `iin` | +| Negation | `not` | + +Operator precedence determines how expressions are evaluated. For example, `A or B and C` is interpreted as `A or (B and C)`. +When necessary, use parentheses to make intent explicit and avoid subtle bugs. + +### Functions + +[Functions](../rules/functions.md) can be used on either side of the expression, or even nested. This enables on-the-fly transformation of event fields and constants, making filters more robust and adaptable to variations in data. For instance, `lower(file.name) = 'cmd.exe'` ensures the comparison is case-insensitive, while `base(ltrim(file.path, 'C:'), false) in suspicious_files` demonstrates function composition, where the file path is first trimmed and then reduced to its base name before being matched against a list of suspicious files. + +### Errors + +If a filter contains a syntax error, a hint indicates the position of the error in the expression. By highlighting the exact offset where parsing fails, the hint enables precise debugging of filters, helping users quickly identify issues such as token mismatches, missing delimiters, invalid string escapes, or invalid operator usage without manually inspecting the entire expression. + +![Invalid Filter](images/invalid-filter.png "Invalid Filter") + +## Filtering in different contexts + +As described in the [quick start](../setup/quick-start.md), filters can be applied at different stages of event collection and processing. Filter expressions are passed as command-line arguments to the `run`, `capture`, and `replay` commands. + +!> If you're using PowerShell, wrap the entire filter expression in quotes `fibratus run --forward "evt.category = 'net'"` + +### Filtering with `run` + +The `run` command evaluates the filter expression against each incoming event. Events that don’t match are discarded and never forwarded to the output sink. + +For example, the following command captures only events that originate from the `cmd.exe` or `svchost.exe` processes. + + +$ fibratus run --forward ps.name in ('cmd.exe', 'svchost.exe') + + + +### Filtering with `capture` + +The `capture` command persists only those events that match the filter. [Captures](../captures.md) are thoroughly explained in the upcoming section. In this example, only `registry` events are written to the output file. + + +$ fibratus capture evt.category = 'registry' -o events + + + +### Filtering with `replay` + +When [replaying](../captures.md) events from a capture file, filters can narrow down the replay scope. For instance, the following command replays only events that modify registry values: + + +$ fibratus replay evt.name = 'RegSetValue' -k events + + + +### Filtering in `filaments` + +Filters can also be applied during [filament](../filaments.md) execution. If a filter is defined both in the `run` command and via the `set_filter` function, the filament-level filter takes precedence. + +## Exclusions + +In some cases, you may want to exclude specific events, either by process image name or event type after they are captured from kernel buffers. Additionally, the kernel stream consumer can be configured to ignore entire categories of events at the [Event Tracing for Windows](https://docs.microsoft.com/en-us/windows/win32/etw/event-tracing-portal) session level. + +Disabling event categories at this stage can significantly reduce system overhead, especially when dealing with high-volume telemetry that isn’t relevant to your use case. + +### Event source configuration + +These options are defined in the `eventsource` section of the configuration file and control which events are collected: + +* `enable-thread` collect thread-related events +* `enable-registry` collect registry events +* `enable-net` collect network events +* `enable-fileio` collect file system events +* `enable-module` collect module load/unload events +* `enable-handle` collect handle events +* `enable-audit-api` collect kernel audit API calls +* `enable-mem` collect memory events +* `enable-dns` collect DNS telemetry +* `enable-threadpool` collect thread pool telemetry + +Disabling unnecessary categories helps reduce both event volume and processing cost. + +### Excluding events and processes + +To permanently exclude specific events or processes from the event stream, use the `eventsource.blacklist` section: + +* `events` defines a list of event names to drop, for example, `CloseFile` +* `images` defines a list of case-sensitive process image names (including extensions). Any event generated by these processes is discarded + +This approach is useful for filtering out noisy or irrelevant sources before they reach downstream processing. diff --git a/docs/telemetry/outputs.md b/docs/telemetry/outputs.md new file mode 100644 index 000000000..1acb17a5b --- /dev/null +++ b/docs/telemetry/outputs.md @@ -0,0 +1,19 @@ +# Outputs + +##### Fibratus provides a wide range of output sinks for transmitting telemetry events. While local [captures](../../captures.md) are often sufficient for inspection and analysis, the event stream can also be forwarded to external systems such as message brokers or search and analytics platforms, for example, RabbitMQ or Elasticsearch, enabling centralized processing, storage, and observability at scale. + +Each output exposes a comprehensive set of configuration options, allowing you to fine-tune how events are transmitted and integrated with downstream systems. + +### Event serialization + +Events are serialized in JSON format by default. Since each event may contain a large number of attributes, you can control which fields are included in the serialized output via the `event` section of the configuration file. + +The following options determine which parts of the process state are included: + +* `serialize-threads` include thread metadata +* `serialize-modules` include loaded modules (e.g. DLLs) +* `serialize-handles` include allocated process handles +* `serialize-pe` include [Portable Executable](https://en.wikipedia.org/wiki/Portable_Executable) (PE) metadata +* `serialize-envs` include environment variables + +Adjusting these settings allows you to balance the level of detail against performance and storage considerations. diff --git a/docs/telemetry/outputs/console.md b/docs/telemetry/outputs/console.md new file mode 100644 index 000000000..4f0aa63bd --- /dev/null +++ b/docs/telemetry/outputs/console.md @@ -0,0 +1,64 @@ +# Console + +##### The console is the default output sink. It renders incoming events directly to standard output, providing a real-time view of the event stream. To improve performance, the console output is buffered, reducing the number of I/O system calls required to write event data. + +## Configuration + +The console output configuration is located in the `outputs.console` section. + +### `enabled` + +Indicates whether the console output is active. + +### `format` + +Specifies the console output format. The `pretty` format dictates that formatting is accomplished by replacing the specifiers in the template. The `json` format outputs the event as a raw JSON string. + +### `kv-delimiter` + +Specifies the separator rendered between the event parameter's key and its value. + + +### `template` + +Defines the template used in the event formatter. For more details, see the next section. + +### `colorize` + +Indicates if the console output is colorized. + +## Templates + +The template consists of a collection of named placeholders that event formatter replaces with desired values. The syntax of the template resembles the Go [template](https://golang.org/pkg/text/template/) engine constructs, excepts the event formatter lacks advanced templating features such as loops, functions or `if` statements. + +The following field modifiers are supported: + +- `.Seq` +- `.Timestamp` +- `.Pid` +- `.Ppid` +- `.Pexe` +- `.Pcmd` +- `.Pname` +- `.Cwd` +- `.Exe` +- `.Cmd` +- `.Tid` +- `.Sid` +- `.Process` +- `.Category` +- `.Description` +- `.CPU` +- `.Type` +- `.Params` +- `.Meta` +- `.Host` +- `.PE` +- `.Params.` +- `.Callstack` + +The default template is defined as: + +```python +{{ .Seq }} {{ .Timestamp }} - {{ .CPU }} {{ .Process }} ({{ .Pid }}) - {{ .Type }} ({{ .Params }}) +``` diff --git a/docs/telemetry/outputs/elasticsearch.md b/docs/telemetry/outputs/elasticsearch.md new file mode 100644 index 000000000..1078e0fb8 --- /dev/null +++ b/docs/telemetry/outputs/elasticsearch.md @@ -0,0 +1,93 @@ +# Elasticsearch + +##### The Elasticsearch output ships events to the Elasticsearch `_bulk` [API endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). Events are collected into batches and flushed at intervals defined by the `flush-period` config option, ensuring efficient indexing while minimizing request overhead. + +## Configuration + +The Elasticsearch output configuration is located in the `outputs.elasticsearch` section. + +### `enabled` + +Indicates whether the Elasticsearch output is enabled. + +### `servers` + +Defines the URL endpoints of the Elasticsearch nodes. + +### `timeout` + +Represents the initial HTTP connection timeout when connecting to the Elasticsearch cluster. + +### `flush-period` + +Specifies when to flush the bulk at the end of the given interval. + +### `bulk-workers` + +Determines the number of workers that commit docs to Elasticsearch. Higher values maximize the throughout at the cost of increased CPU utilization. + +### `healthcheck` + +Enables or disables nodes health checking. + +### `healthcheck-interval` + +Specifies the interval for checking if the Elasticsearch nodes are available. + +### `healthcheck-timeout` + +Specifies the timeout for periodic node health checks. + +### `username` + +Identifies the user name for the basic HTTP authentication. + +### `password` + +Identifies the password for the basic HTTP authentication. + +### `sniff` + +Enables the discovery of all Elasticsearch nodes in the cluster. This avoids populating the list of available Elasticsearch nodes. + +### `trace-log` + +Determines if the Elasticsearch client trace log is enabled. Useful for troubleshooting. + +### `gzip-compression` + +Determines if the `gzip` compression is enabled for Elasticsearch documents. + +### `template-name` + +Specifies the name of the index template. + +### `template-config` + +Contains the full JSON body of the index template. For more information refer to [index templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html). + +### `index-name` + +Represents the target index for the telemetry. It allows time specifiers to create indices per time frame. For example, `fibratus-%Y-%m` generates the index name with current year and month. Supported time specifiers are: + +- `%Y` current year in `YYYY` format (`2020`) +- `%y` current year in `YY` format (`20`) +- `%m` current month (`01`) +- `%d` current day (`02`) +- `%H` current hour (`15`) + +### `tls-key` + +Path to the public/private key file. + +### `tls-cert` + +Path to the certificate file. + +### `tls-ca` + +Represents the path of the certificate file that is associated with the Certification Authority (CA). + +### `tls-insecure-skip-verify` + +Indicates if the chain and host verification stage is skipped. diff --git a/docs/telemetry/outputs/eventlog.md b/docs/telemetry/outputs/eventlog.md new file mode 100644 index 000000000..ca6955466 --- /dev/null +++ b/docs/telemetry/outputs/eventlog.md @@ -0,0 +1,23 @@ +# Eventlog + +##### Exports events via [Windows Event Log](https://docs.microsoft.com/en-us/windows/win32/wes/windows-event-log) API that can be explored with the [Event Viewer](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc766042(v=ws.11)) management tool. Fibratus classifies each event with a custom `Event ID` + +## Configuration + +The Eventlog output configuration is located in the `outputs.eventlog` section. + +### `enabled` + +Indicates whether the Eventlog output is enabled. + +### `level` + +Specifies the eventlog level associated with the event logs produced by Fibratus. You can specify one of the following values `info`, `INFO`, `warn`, `warning`, `WARN`, `WARNING`, `erro`, `ERRO`, `error`, `ERROR` + +### `remote-host` + +Address of the remote eventlog service. + +### `template` + +Custom Go [template](https://pkg.go.dev/text/template) for rendering the eventlog message. diff --git a/docs/outputs/http.md b/docs/telemetry/outputs/http.md similarity index 58% rename from docs/outputs/http.md rename to docs/telemetry/outputs/http.md index 4c8fdb276..608b75897 100644 --- a/docs/outputs/http.md +++ b/docs/telemetry/outputs/http.md @@ -1,83 +1,71 @@ # HTTP -Sends events to local/remote endpoints via HTTP protocol. Requests are serialized to the data format specified by the `serializer` property, which by default encodes events as `JSON` payloads. HTTP requests are randomly load-balanced across endpoints defined in the `endpoints` config property. +##### Sends events to local/remote endpoints via HTTP protocol. Requests are serialized to the data format specified by the `serializer` property, which by default encodes events as `JSON` payloads. HTTP requests are randomly load-balanced across endpoints defined in the `endpoints` config property. -### Configuration {docsify-ignore} +## Configuration The HTTP output configuration is located in the `outputs.http` section. -#### enabled +### `enabled` Indicates whether the HTTP output is enabled. -**default**: `false` - -#### endpoints +### `endpoints` Specifies a list of endpoints to which the events are forwarded. Each of the endpoints must contain the HTTP protocol scheme, that can be `http` or `https`. -#### timeout +### `timeout` Represents the timeout for the HTTP requests. -**default**: `5s` - -#### proxy-url +### `proxy-url` Specifies the HTTP proxy URL. It overrides the HTTP proxy URL as indicated by the environment variables. -#### proxy-username +### `proxy-username` The username for HTTP proxy authentication. -#### proxy-password +### `proxy-password` The password for HTTP proxy authentication. -#### method +### `method` Determines the HTTP verb to use in requests. -**default**: `POST` - -#### serializer +### `serializer` -Specifies the event serializer type. +Specifies the event serializer type. `json` is the default serializer. -**default**: `json` - -#### username +### `username` Username for the basic HTTP authentication. -#### password +### `password` Password for the basic HTTP authentication. -#### enable-gzip +### `enable-gzip` If enabled, the HTTP body is compressed with the `gzip` compression. -**default**: `false` - -#### headers +### `headers` Represents a list of arbitrary headers to include in HTTP requests. -#### tls-key +### `tls-key` Path to the public/private key file. -#### tls-cert +### `tls-cert` Path to the certificate file. -#### tls-ca +### `tls-ca` Represents the path of the certificate file that is associated with the Certification Authority (CA). -#### tls-insecure-skip-verify +### `tls-insecure-skip-verify` Indicates if the chain and host verification stage is skipped. - -**default**: `false` diff --git a/docs/telemetry/outputs/null.md b/docs/telemetry/outputs/null.md new file mode 100644 index 000000000..c2d6a431e --- /dev/null +++ b/docs/telemetry/outputs/null.md @@ -0,0 +1,3 @@ +# Null + +##### The null output devours kernel events the same way a black hole swallows the light. This output is useful in case you want to snooze the event stream. diff --git a/docs/outputs/rabbitmq.md b/docs/telemetry/outputs/rabbitmq.md similarity index 67% rename from docs/outputs/rabbitmq.md rename to docs/telemetry/outputs/rabbitmq.md index 94612b313..ce0bfb0fc 100644 --- a/docs/outputs/rabbitmq.md +++ b/docs/telemetry/outputs/rabbitmq.md @@ -1,96 +1,83 @@ # RabbitMQ -The RabbitMQ output sends events to the [RabbitMQ](https://www.rabbitmq.com/) message broker. Various events are buffered and sent as part of a single AMQP message. +##### The RabbitMQ output transmits events to the [RabbitMQ](https://www.rabbitmq.com/) messaging system. Events are buffered and published in batches as a single AMQP message to improve transmission efficiency and reduce overhead. -### Configuration {docsify-ignore} + +## Configuration The RabbitMQ output configuration is located in the `outputs.amqp` section. -#### enabled +### `enabled` Specifies whether the RabbitMQ output sink is enabled. -**default**: `false` -#### url +### `url` Represents the AMQP connection string. -**default**: `amqp://localhost:5672` -#### timeout +### `timeout` Specifies the AMQP connection timeout. -**default**: `5s` -#### exchange +### `exchange` Specifies the target AMQP exchange name that receives inbound event message flow. -**default**: `fibratus` -#### exchange-type +### `exchange-type` Represents the AMQP exchange type. Available exchange type include common types are `direct`, `fanout`, `topic`, `header`, and `x-consistent-hash`. To learn more about exchange types, refer to the RabbitMQ [docs](https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchanges). -#### routing-key +### `routing-key` Represents the static routing key to link exchanges with queues. -**default**: `fibratus` - -#### vhost +### `vhost` Represents the AMQP virtual host name. -**default**: `/` - -#### durable +### `durable` Indicates if the exchange is marked as durable. Durable exchanges can survive broker restarts. -**default**: `false` -#### passive +### `passive` Indicates if the server checks whether the exchange already exists and raises an error if it doesn't exist. -**default**: `false` -#### delivery-mode +### `delivery-mode` Determines if a published message is persistent or transient. -**default**: `transient` - -#### username +### `username` The username for the plain authentication method. -#### password +### `password` The password for the plain authentication method. -#### headers +### `headers` Designates a collection of static headers that are added to each published message. -#### tls-key +### `tls-key` Path to the public/private key file. -#### tls-cert +### `tls-cert` Path to the certificate file. -#### tls-ca +### `tls-ca` Represents the path of the certificate file that is associated with the Certification Authority (CA). -#### tls-insecure-skip-verify +### `tls-insecure-skip-verify` Indicates if the chain and host verification stage is skipped. - -**default**: `false` diff --git a/docs/telemetry/transformers.md b/docs/telemetry/transformers.md new file mode 100644 index 000000000..dec03c5e0 --- /dev/null +++ b/docs/telemetry/transformers.md @@ -0,0 +1,5 @@ +# Transformers + +##### Transformers are responsible for mutating, parsing, or enriching events before they hit the output sink. Transformers are applied sequentially to every event routed to the output sink. + +You can parameterize transformers via the `yml` configuration in the `transformers` section. diff --git a/docs/telemetry/transformers/remove.md b/docs/telemetry/transformers/remove.md new file mode 100644 index 000000000..010ec3517 --- /dev/null +++ b/docs/telemetry/transformers/remove.md @@ -0,0 +1,15 @@ +# Remove + +##### The `remove` transform is used to exclude specific parameters from an event before it is forwarded or persisted. This is useful for reducing payload size, removing sensitive fields, or tailoring events to the needs of downstream consumers. + +## Configuration + +The `remove` transformer configuration is located in the `transformers.remove` section. + +### `enabled` + +Indicates if the `remove` transformer is enabled. + +### `params` + +Represents the list of parameters that are removed from the event. diff --git a/docs/telemetry/transformers/rename.md b/docs/telemetry/transformers/rename.md new file mode 100644 index 000000000..c60087ef5 --- /dev/null +++ b/docs/telemetry/transformers/rename.md @@ -0,0 +1,23 @@ +# Rename + +##### The `rename` transform is used to rename one or more event parameters before the event is emitted or forwarded. This is useful for standardizing field names, resolving naming conflicts, or adapting the event schema to match downstream expectations. + +## Configuration + +The `rename` transformer configuration is located in the `transformers.rename` section. + +### `enabled` + +Indicates if the `rename` transformer is enabled. + +### `params` + +Defines a list of mappings between original and new parameter names. The `old` key specifies the existing parameter name in the event, while the `new` key defines the name it should be renamed to during transformation. As an example, the following configuration renames the `username` parameter to `user` and `dport` parameter to `dst_port` + +```yaml +params: + - old: username + new: user + - old: dport + new: dst_post +``` diff --git a/docs/telemetry/transformers/replace.md b/docs/telemetry/transformers/replace.md new file mode 100644 index 000000000..24c77017f --- /dev/null +++ b/docs/telemetry/transformers/replace.md @@ -0,0 +1,23 @@ +# Replace + +##### The `replace` transformer substitutes all non-overlapping occurrences of a specified substring within string parameters. This is useful for normalizing values, redacting sensitive information, or standardizing event data before it is forwarded or stored. The transformation operates on a simple find-and-replace basis, scanning the target parameter and replacing every match with the configured replacement string. + + +## Configuration + +The `replace` transformer configuration is located in the `transformers.replace` section. + +### `enabled` + +Indicates if the `replace` transformer is enabled. + +### `replacements` + +Defines a list of parameter replacement rules. Each rule targets a specific event parameter identified by the `param` key. Within that parameter, every occurrence of the substring specified in `old` is replaced with the value defined in `new`. As an example, the following config replaces the `HKEY_LOCAL_MACHINE` substring in the `key_path` parameter value with `HKLM`. + +```yaml +replacements: + - param: key_path + old: HKEY_LOCAL_MACHINE + new: HKLM +``` diff --git a/docs/telemetry/transformers/tags.md b/docs/telemetry/transformers/tags.md new file mode 100644 index 000000000..174fa1b8c --- /dev/null +++ b/docs/telemetry/transformers/tags.md @@ -0,0 +1,18 @@ +# Tags + +##### The `tags` transformer appends custom tags to an event’s metadata field. This allows events to be enriched with additional contextual labels that can later be used for filtering, classification, or routing. Tags are typically used to mark events with domain-specific or operational metadata—such as environment, source category, or processing stage without modifying the original event payload. + + +## Configuration + +The `tags` transformer configuration is located in the `transformers.tags` section. + +### `enabled` + +Indicates if the `tags` transformer is enabled. + +### `tags` + +Defines the list of tags to be appended to the event metadata. Each value is added as a static label, enriching the event with additional contextual information. + +Tag values can also be dynamically resolved from environment variables by enclosing the variable name in `%`, for example, `%ENV_VAR%`. This allows tags to reflect runtime configuration without requiring changes to the static configuration file. diff --git a/docs/telemetry/transformers/trim.md b/docs/telemetry/transformers/trim.md new file mode 100644 index 000000000..21722a07d --- /dev/null +++ b/docs/telemetry/transformers/trim.md @@ -0,0 +1,30 @@ +# Trim + +##### The `trim` transformer removes specified prefixes and/or suffixes from string event parameters. This is useful for normalizing values by stripping redundant or contextual markers, such as path segments, protocol prefixes, or formatting artifacts. The transformation operates only on string parameters and ensures that only the defined leading or trailing substrings are removed, leaving the core value intact. + +## Configuration + +The `trim` transformer configuration is located in the `transformers.trim` section. + +### `enabled` + +Indicates if the `trim` transformer is enabled. + +### `prefixes` + +Contains the list of parameter names and prefixes that are trimmed from the parameter's value. + +### `suffixes` + +Contains the list of parameter names and suffixes that are trimmed from the parameter's value. + +As an example, the following config trims the `HKEY_LOCAL_MACHINE` prefix and the `Keys` suffix from the `key_path` parameter value. + +```yaml +prefixes: +- param: key_path + trim: HKEY_LOCAL_MACHINE +suffixes: +- param: key_path + trim: Keys +``` diff --git a/docs/themes/fibratus.css b/docs/themes/fibratus.css deleted file mode 100755 index 243857082..000000000 --- a/docs/themes/fibratus.css +++ /dev/null @@ -1,1194 +0,0 @@ -@import url("https://fonts.googleapis.com/css?family=Roboto+Mono|Source+Code+Pro|Oxygen+Mono|Nova+Mono|Source+Sans+Pro:300,400,600"); -* { - -webkit-font-smoothing: antialiased; - -webkit-overflow-scrolling: touch; - -webkit-tap-highlight-color: rgba(0, 0, 0, 0); - -webkit-text-size-adjust: none; - -webkit-touch-callout: none; - box-sizing: border-box -} - -body:not(.ready) { - overflow: hidden -} - -body:not(.ready) .app-nav, -body:not(.ready)>nav, -body:not(.ready) [data-cloak] { - display: none -} - -div#app { - font-size: 30px; - font-weight: lighter; - margin: 40vh auto; - text-align: center -} - -div#app:empty:before { - content: "Loading Fibratus Docs..." -} - -footer span { - font-size: 10px; -} - -footer { - text-align: center; -} - -.search input { - background-color: #F7F7F7; - border-radius:5px; - text-indent: 23px; - -moz-border-radius:5px; - -webkit-border-radius:5px; -} - -.search .input-wrap::before { - font-family: "FontAwesome"; - content: "\f002"; - position: relative; - left: 8px; - top: -2px; - color: #d6e0f0; - height: 24px; - width: 0; - overflow: visible; - display: inline-block; - line-height: 24px; - z-index: 1; -} - -.fast-icon { - color: #ffba00; -} - -.extensible-icon { - color: #5D99E5; -} - -.comprehensive-icon { - color: #738FD6; -} - -.operator > p { - background-color: rgba(0, 27, 61, 0.03); - border-left: 4px solid #6da9e2 !important; - border-radius: 5px; - color: #001b3d; - font-weight: 600; - font-size: .95em; - padding: 0.7em 0.6em; -} - - -p.center { - text-align: center; - font-size: 30px; -} - -.inner-icon { - border-top: 1px solid #ffba00; - color: #ffba00; - font-size: 1.5em; - margin-bottom: -.5em; - margin-top: 1em; - text-align: center; -} - -.emoji { - height: 1.2rem; - vertical-align: middle -} - -.progress { - background-color: var(--theme-color, #42b983); - height: 2px; - left: 0; - position: fixed; - right: 0; - top: 0; - transition: width .2s, opacity .4s; - width: 0; - z-index: 999999 -} - -.hero-icon-left { - font-size: 100px; - color: #b6d4f0; - float: left; - margin-left: -10px; - margin-bottom: 2px; - margin-top: -10px; - margin-right: 15px; - --ionicon-stroke-width: 16px; -} - -.search .search-keyword, -.search a:hover { - color: var(--theme-color, #42b983) -} - -.search .search-keyword { - font-style: normal; - font-weight: 700 -} - -.hydrated { - top: 3px; - position: relative; -} - -body, -html { - height: 100% -} - -body { - -moz-osx-font-smoothing: grayscale; - -webkit-font-smoothing: antialiased; - color: #34495e; - font-family: Jost, Source Sans Pro, Helvetica Neue, Arial, sans-serif; - font-size: 18px; - letter-spacing: 0; - margin: 0; - overflow-x: hidden -} - -img { - max-width: 100% -} - -a[disabled] { - cursor: not-allowed; - opacity: .6 -} - -kbd { - border: 1px solid #ccc; - border-radius: 3px; - display: inline-block; - font-size: 12px!important; - line-height: 12px; - margin-bottom: 3px; - padding: 3px 5px; - vertical-align: middle -} - -li input[type=checkbox] { - margin: 0 .2em .25em 0; - vertical-align: middle -} - -.app-nav { - margin: 0; - top: 0; - position: fixed; - right: 0; - text-align: right; - z-index: 10; - background-color: #6da9e2; - width: 100%; - height: 0px; - display: none; -} - -.app-nav.no-badge { - margin-right: 0px -} - -.app-nav p { - margin: 0 -} - -.app-nav>a { - margin: 0 0rem; - padding: 5px 0 -} - -.app-nav li, -.app-nav ul { - display: inline-block; - list-style: none; - top: -12px; - margin-right: 10px; -} - -.app-nav a { - border-radius: 1rem; - border: 1px solid var(--theme-color, #42b983); - box-sizing: border-box; - color: var(--theme-color, #42b983); - display: inline-block; - font-size: 0.6rem; - font-weight: 600; - letter-spacing: .1rem; - margin: .1rem .1rem; - padding: .4em .4rem; - text-decoration: none; - transition: all .15s ease; - background-color: #4787BE; - color: #F7F7F7 -} - -.sidebar-margin-nocover { - margin-top: 0px; -} - -.app-nav a.active { - color: #fff; -} -.app-nav a:hover { - background-color: #5894ce; - opacity: .8 -} - -.app-nav li { - display: inline-block; - margin: 0 0.2rem; - padding: 0px 0; - position: relative -} - -.app-nav li ul { - background-color: #fff; - border: 1px solid; - border-color: #ddd #ddd #ccc; - border-radius: 4px; - box-sizing: border-box; - display: none; - max-height: calc(100vh - 81px); - overflow-y: auto; - padding: 10px 0; - position: absolute; - right: -15px; - text-align: left; - top: 100%; - white-space: nowrap -} - -.app-nav li ul li { - display: block; - font-size: 14px; - line-height: 1rem; - margin: 8px 14px; - white-space: nowrap -} - -.app-nav li ul a { - display: block; - font-size: inherit; - margin: 0; - padding: 0; -} - -.app-nav li ul a.active { - border-bottom: 0; -} - -.app-nav li:hover ul { - display: block -} - -.app-nav .logo { - color: #F7F7F7; - font-size: 1.7rem; - font-weight: 400; - margin-left: 25px; - margin-top: 2px; - position: absolute; - text-align: right; - display: none; - cursor: pointer; - letter-spacing: 2px; -} - -.github-corner { - border-bottom: 0; - position: fixed; - right: 0; - text-decoration: none; - top: 0; - z-index: 1 -} - -.github-corner:hover .octo-arm { - -webkit-animation: octocat-wave .56s ease-in-out; - animation: octocat-wave .56s ease-in-out -} - -.github-corner svg { - color: #fff; - fill: var(--theme-color, #42b983); - height: 80px; - width: 80px -} - -main { - display: block; - position: relative; - width: 100vw; - height: 100%; - z-index: 0 -} - -main.hidden { - display: none -} - -.anchor { - display: inline-block; - text-decoration: none; - transition: all .3s -} - -.anchor span { - color: #34495e -} - -.anchor span:h { - color: #6da9e2 -} - -.anchor:hover { - text-decoration: none; -} - -.ml-auto { - margin-left: auto; -} - -.mr-auto { - margin-right: auto; -} - -.divider-custom { - margin: 1.25rem 0 1.5rem; - width: 100%; - display: flex; - justify-content: center; - align-items: center; -} - -.divider-custom.divider-light .divider-custom-line { - background-color: #2c3e50; -} - -.divider-custom .divider-custom-line { - width: 100%; - max-width: 7rem; - height: 0.25rem; - background-color: #2c3e50; - border-radius: 1rem; - border-color: #2c3e50 !important; -} - -.divider-custom .divider-custom-icon { - color: #2c3e50 !important; - font-size: 2rem; -} - -.sidebar { - border-right: 1px solid rgba(0, 0, 0, .07); - overflow-y: auto; - padding: 10px 10px 0; - position: absolute; - top: 0; - bottom: 0; - left: 0; - transition: transform .25s ease-out; - width: 300px; - z-index: 20; - background-color: #F7F7F7; -} - -.sidebar>h1 { - margin: 0 auto 1rem; - font-size: 1.5rem; - font-weight: 300; - text-align: center -} - -.sidebar>h1 a { - color: inherit; - text-decoration: none -} - -.sidebar>h1 .app-nav { - display: block; - position: static -} - -.sidebar .sidebar-nav { - line-height: 1.7em; - padding-bottom: 40px -} - -.sidebar li.collapse .app-sub-sidebar { - display: none -} - -.sidebar ul { - margin: 0 0 0 8px; - padding: 0; -} - -.sidebar li>p { - font-weight: 700; - margin: 0 -} - -.sidebar ul, -.sidebar ul li { - list-style: none; - line-height: 20px; - margin-bottom: -4px; -} - -.sidebar ul li:hover { - cursor: pointer; - color: #6da9e2; -} - -.sidebar ul +ul { - max-height:0; - overflow:hidden; - transition:0.5s linear; -} - -.sidebar ul li { - border-bottom: none; - display: block; - padding: 4px 8px; - font-size: 19px; - font-weight: 500; -} - -.sidebar ul li ul { - border-left: 1px solid rgba(0, 0, 0, .07); -} - -.sidebar ul li ul { - padding-left: 1px -} - -.sidebar::-webkit-scrollbar { - width: 4px -} - -.sidebar::-webkit-scrollbar-thumb { - background: transparent; - border-radius: 4px -} - -.sidebar:hover::-webkit-scrollbar-thumb { - background: hsla(0, 0%, 53.3%, .4) -} - -.sidebar:hover::-webkit-scrollbar-track { - background: hsla(0, 0%, 53.3%, .1) -} - -.sidebar-toggle { - background-color: transparent; - border: 0; - outline: none; - padding: 10px; - position: absolute; - bottom: 0; - left: 0; - text-align: center; - transition: opacity .3s; - width: 284px; - z-index: 30 -} - -.sidebar-toggle .sidebar-toggle-button:hover { - opacity: .4 -} - -.sidebar-toggle span { - background-color: var(--theme-color, #42b983); - display: block; - margin-bottom: 4px; - width: 16px; - height: 2px -} - -body.sticky .sidebar, -body.sticky .sidebar-toggle { - margin-top: 0px; - position: fixed; -} - -body.sticky .page_toc { - margin-top: 15px; -} - -body.sticky .app-nav .logo { - display: none; -} - -.page_toc ul a:hover span { - color: #6da9e2 !important; -} - -.page_toc ul a { - font-size: 17px !important; -} - -.page_toc .active { - border-left: 3px solid !important; -} - -.page_toc { - padding-left: 12px !important; -} - -.content { - padding-top: 30px; - position: absolute; - top: 0; - right: 0; - bottom: 0; - left: 300px; - transition: left .25s ease; - background-color: #F7F7F7; -} - -.markdown-section { - margin: 0 auto; - max-width: 80%; - padding: 30px 15px 40px; - position: relative; -} - -.markdown-section>* { - box-sizing: border-box; - font-size: inherit -} - -.markdown-section>:first-child { - margin-top: 0!important -} - -.markdown-section hr { - border: none; - border-bottom: 1px solid #eee; - margin: 2em 0 -} - -.markdown-section iframe { - border: 1px solid #eee; - width: 1px; - min-width: 100% -} - -.markdown-section table { - border-collapse: collapse; - border-spacing: 0; - display: block; - margin-bottom: 1rem; - overflow: auto; - width: 100% -} - -.markdown-section th { - font-weight: 700 -} - -.markdown-section td, -.markdown-section th { - border: 1px solid #ddd; - padding: 6px 13px -} - -.markdown-section tr { - border-top: 1px solid #ccc -} - -.markdown-section p.tip, -.markdown-section tr:nth-child(2n) { - background-color: rgba(0, 27, 61, 0.03); -} - -.markdown-section p.tip { - border-bottom-right-radius: 2px; - border-left: 4px solid #f66; - border-top-right-radius: 2px; - margin: 2em 0; - padding: 12px 24px 12px 30px; - position: relative -} - -.markdown-section p.tip:before { - background-color: #f66; - border-radius: 100%; - color: #fff; - content: "!"; - font-family: Dosis, Source Sans Pro, Helvetica Neue, Arial, sans-serif; - font-size: 14px; - font-weight: 700; - left: -12px; - line-height: 20px; - position: absolute; - height: 20px; - width: 20px; - text-align: center; - top: 14px -} - -.markdown-section p.tip code { - background-color: #efefef -} - -.markdown-section p.tip em { - color: #34495e -} - -.markdown-section p.warn { - background: rgba(66, 185, 131, .1); - border-radius: 2px; - padding: 1rem -} - -.markdown-section ul.task-list>li { - list-style-type: none -} - -body.close .sidebar { - transform: translateX(-300px) -} - -body.close .sidebar-toggle { - width: auto -} - -body.close .content { - left: 0 -} - -@media print { - .app-nav, - .github-corner, - .sidebar, - .sidebar-toggle { - display: none - } -} - - -@media screen and (max-width: 1300px) { - .page_toc { - display: none; - } -} - -@media screen and (max-width: 768px) { - .github-corner, - .sidebar, - .sidebar-toggle { - position: fixed; - } - .app-nav { - margin-top: 16px; - } - .app-nav li ul { - top: 30px; - } - main { - height: auto; - overflow-x: hidden; - } - .sidebar { - left: -300px; - transition: transform 0.25s ease-out; - } - .content { - left: 0; - max-width: 100vw; - position: static; - padding-top: 20px; - transition: transform 0.25s ease; - } - .app-nav, - .github-corner { - transition: transform 0.25s ease-out; - } - .sidebar-toggle { - background-color: transparent; - width: auto; - padding: 30px 30px 10px 10px; - } - body.close .sidebar { - transform: translateX(300px); - } - body.close .sidebar-toggle { - background-color: hsla(0, 0%, 100%, 0.8); - transition: background-color 1s; - width: 284px; - padding: 10px; - } - body.close .content { - transform: translateX(300px); - } - body.close .app-nav, - body.close .github-corner { - display: none; - } - .github-corner:hover .octo-arm { - -webkit-animation: none; - animation: none; - } - .github-corner .octo-arm { - -webkit-animation: octocat-wave 0.56s ease-in-out; - animation: octocat-wave 0.56s ease-in-out; - } - .page_toc { - display: none; - } -} - -@keyframes octocat-wave { - 0%, - to { - transform: rotate(0) - } - 20%, - 60% { - transform: rotate(-25deg) - } - 40%, - 80% { - transform: rotate(10deg) - } -} - -section.cover { - align-items: center; - background-position: 50%; - background-repeat: no-repeat; - background-size: cover; - height: 205vh; - display: none; -} - -section.cover.show { - display: flex; - background: linear-gradient(#1180e0,#c7def4) !important; -} - -section.cover.has-mask .mask { - background-color: #fff; - opacity: .8; - position: absolute; - top: 0; - height: 100%; - width: 100% -} - -section.cover .cover-main { - flex: 1; - margin: -20px 16px 0px; - text-align: center; - position: relative; -} - -section.cover .cover-main img { - margin-top: 45px; -} - -section.cover a { - color: inherit -} - -section.cover a, -section.cover a:hover { - text-decoration: none -} - -section.cover p { - line-height: 1.5rem; - margin: 1em 0 -} - -section.cover h1 { - color: #fff; - font-size: 2.5rem; - font-weight: 300; - margin: .625rem 0 2.5rem; - position: relative; - text-align: center -} - -section.cover h1 a { - display: block -} - -section.cover h1 small { - bottom: -.4375rem; - font-size: 1rem; - position: absolute -} - -section.cover blockquote { - font-size: 1.5rem; - text-align: center -} - -section.cover ul { - line-height: 1.8; - list-style-type: none; - margin: 1em auto; - max-width: 500px; - padding: 0 -} - -section.cover .cover-main>p:nth-last-child(2) a { - background: linear-gradient(340.87deg,#777ce0 -5.97%,#60cefd 100.83%); - box-shadow: 0px 20px 30px -3px rgba(112,153,234,.45); - border-radius: 2rem; - border: 1px solid var(--theme-color, #42b983); - box-sizing: border-box; - color: #34495e; - display: inline-block; - font-size: 1.05rem; - letter-spacing: .1rem; - margin: .5rem 1rem; - padding: .75em 2rem; - text-decoration: none; - transition: .3s all ease-in-out; -} - -section.cover .cover-main { - position: relative; - top: 0px; -} - -section.cover .cover-main>p:nth-last-child(2) a:hover { - color: inherit; - opacity: .8; -} - -section.cover .cover-main h1 { - font-size: 3.5rem; -} - -section.cover blockquote>p>a { - border-bottom: 2px solid var(--theme-color, #42b983); - transition: color .3s -} - -section.cover blockquote>p>a:hover { - color: var(--theme-color, #42b983) -} - -section.cover .cover-main div { - position: relative; -} - -.sidebar, -body { - background-color: #F7F7F7; -} - -.sidebar { - color: #364149 -} - -.sidebar li { - margin: 10px 0 -} - -.sidebar ul li a { - color: #505d6b; - font-size: 16px; - font-weight: 500; - overflow: hidden; - text-decoration: none; - text-overflow: ellipsis; - white-space: nowrap; -} - -.sidebar ul li a:hover { - text-decoration: underline -} - -.sidebar ul li ul { - padding: -1; - margin: 0 -} - -.sidebar ul li.active>a { - color: var(--theme-color, #42b983); - font-weight: 600 -} - -.app-sub-sidebar li:before { - padding-right: 4px; - float: left -} - -.markdown-section h1, -.markdown-section h2, -.markdown-section h3, -.markdown-section h4, -.markdown-section strong { - color: #2c3e50; - font-weight: 600 -} - -.markdown-section h4 { - border-bottom: 1px solid rgba(0, 0, 0,.045); -} - -.markdown-section h2.center { - color: #2c3e50; - font-weight: 600; - text-align: center; -} - -.markdown-section a { - color: var(--theme-color, #42b983); - font-weight: 600 -} - -.markdown-section h1 { - font-size: 2rem; - margin: 0 0 1rem; - border-bottom: 1px solid rgba(0, 0, 0,.075); -} - -.markdown-section h2 { - font-size: 1.75rem; - margin: 45px 0 .8rem -} - -.markdown-section h3 { - font-size: 1.5rem; - margin: 40px 0 .6rem -} - -.markdown-section h4 { - font-size: 1.25rem -} - -.markdown-section h5 { - font-size: 1rem -} - -.markdown-section h6 { - color: #777; - font-size: 1rem -} - -.markdown-section figure, -.markdown-section p { - margin: 1.2em 0 -} - -.markdown-section ol, -.markdown-section p, -.markdown-section ul { - line-height: 1.6rem; - word-spacing: .05rem -} - -.markdown-section ol, -.markdown-section ul { - padding-left: 1.5rem -} - -.markdown-section blockquote { - border-left: 4px solid var(--theme-color, #42b983); - color: #858585; - margin: 2em 0; - padding-left: 20px -} - -.markdown-section blockquote p { - font-weight: 600; - margin-left: 0 -} - -.markdown-section iframe { - margin: 1em 0 -} - -.markdown-section em { - color: #6da9e2 -} - -.pagination-item-title:hover { - color: #6da9e2 -} - -.markdown-section code { - border-radius: 5px; - color: #5894ce; - font-size: .8rem; - margin: 0 2px; - padding: 3px 5px; - white-space: pre-wrap; - font-weight: 600; -} - -.markdown-section code, -.markdown-section pre { - background-color: rgba(0, 27, 61, 0.03); - font-family: Nova Mono, Oxygen Mono, Source Code Pro, Roboto Mono, Monaco, courier, monospace -} - -.markdown-section pre { - font-family: Nova Mono, Source Code Pro, Roboto Mono, Monaco, courier, monospace; - -moz-osx-font-smoothing: initial; - -webkit-font-smoothing: initial; - line-height: 1.5rem; - margin: 1.2em 0; - overflow: auto; - padding: 0 1.4rem; - position: relative; - word-wrap: normal; - border-radius: 5px; -} - -.token.cdata, -.token.comment, -.token.doctype, -.token.prolog { - color: #8e908c -} - -.token.namespace { - opacity: .7 -} - -.token.boolean, -.token.number { - color: #c76b29 -} - -.token.punctuation { - color: #525252 -} - -.token.property { - color: #c08b30 -} - -.token.tag { - color: #2973b7 -} - -.token.string { - color: var(--theme-color, #42b983) -} - -.token.selector { - color: #6679cc -} - -.token.attr-name { - color: #2973b7 -} - -.language-css .token.string, -.style .token.string, -.token.entity, -.token.url { - color: #22a2c9 -} - -.token.attr-value, -.token.control, -.token.directive, -.token.unit { - color: var(--theme-color, #42b983) -} - -.token.function, -.token.keyword { - color: #e96900 -} - -.token.atrule, -.token.regex, -.token.statement { - color: #22a2c9 -} - -.token.placeholder, -.token.variable { - color: #3d8fd1 -} - -.token.deleted { - text-decoration: line-through -} - -.token.inserted { - border-bottom: 1px dotted #202746; - text-decoration: none -} - -.token.italic { - font-style: italic -} - -.token.bold, -.token.important { - font-weight: 700 -} - -.token.important { - color: #c94922 -} - -.token.entity { - cursor: help -} - -.markdown-section pre>code { - -moz-osx-font-smoothing: initial; - -webkit-font-smoothing: initial; - background-color: #f8f8f8a3; - border-radius: 0px; - color: #5994CE; - display: block; - font-family: Nova Mono, Oxygen Mono, Source Code Pro, Roboto Mono, Monaco, courier, monospace; - font-size: .85rem; - line-height: inherit; - margin: 0 2px; - max-width: inherit; - overflow: inherit; - padding: 1.2em 5px; - white-space: inherit -} - -.markdown-section code:after, -.markdown-section code:before { - letter-spacing: .05rem -} - -code .token { - -moz-osx-font-smoothing: initial; - -webkit-font-smoothing: initial; - min-height: 1.5rem; - position: relative; - left: auto -} - -pre:after { - color: #ccc; - content: attr(data-lang); - font-size: .6rem; - font-weight: 600; - height: 15px; - line-height: 15px; - padding: 5px 10px 0; - position: absolute; - right: 0; - text-align: right; - top: 0 -} diff --git a/docs/transformers/introduction.md b/docs/transformers/introduction.md deleted file mode 100644 index bf413dc7c..000000000 --- a/docs/transformers/introduction.md +++ /dev/null @@ -1,5 +0,0 @@ -# Parsing, Enriching, Transforming - -Transformers are responsible for mutating, parsing, or enriching kernel events before they hit the output sink. They offer a fair amount of flexibility to shape the structure of the event parameters. Transformers are applied sequentially to every event routed to the output sink. - -You can parameterize transformers via the `yml` configuration in the `transformers` section. diff --git a/docs/transformers/remove.md b/docs/transformers/remove.md deleted file mode 100644 index 69279ca72..000000000 --- a/docs/transformers/remove.md +++ /dev/null @@ -1,50 +0,0 @@ -# Remove - -The `remove` transform drops parameters from the event. Given the following event parameters: - -``` -{ - 'file_name': 'C:\WINDOWS\system32\config\systemprofile\AppData\WindowsApps\', - 'file_object': 'ffffa88c7ea077d0', - 'irp': 'ffffa88c746b2a88', - 'operation': 'supersede', - 'share_mask': 'rw-', - 'type': 'directory' -} -``` - -And the `remove` transformer configuration: - -``` -transformers: - remove: - enabled: true - kparams: - - irp - - share_mask - - file_object -``` - -The event will contain the following parameters: - -``` -{ - 'file_name': 'C:\WINDOWS\system32\config\systemprofile\AppData\WindowsApps\', - 'operation': 'supersede', - 'type': 'directory' -} -``` - -### Configuration {docsify-ignore} - -The `remove` transformer configuration is located in the `transformers.remove` section. - -#### enabled - -Indicates if the `remove` transformer is enabled. - -**default**: `false` - -#### kparams - -Represents the list of parameters that are removed from the event. diff --git a/docs/transformers/rename.md b/docs/transformers/rename.md deleted file mode 100644 index 10ceca259..000000000 --- a/docs/transformers/rename.md +++ /dev/null @@ -1,54 +0,0 @@ -# Rename - -The `rename` transform rename one or more parameters. Given the following event parameters: - -``` -{ - 'file_name': 'C:\WINDOWS\system32\config\systemprofile\AppData\WindowsApps\', - 'file_object': 'ffffa88c7ea077d0', - 'irp': 'ffffa88c746b2a88', - 'operation': 'supersede', - 'share_mask': 'rw-', - 'type': 'directory' -} -``` - -And the `rename` transformer configuration: - -``` -transformers: - rename: - enabled: true - kparams: - - old: file_name - new: file - - old: file_object - new: fobj -``` - -The event will contain the following parameters: - -``` -{ - 'name': 'C:\WINDOWS\system32\config\systemprofile\AppData\WindowsApps\', - 'fobj': 'ffffa88c7ea077d0', - 'irp': 'ffffa88c746b2a88', - 'operation': 'supersede', - 'share_mask': 'rw-', - 'type': 'directory' -} -``` - -### Configuration {docsify-ignore} - -The `rename` transformer configuration is located in the `transformers.rename` section. - -#### enabled - -Indicates if the `rename` transformer is enabled. - -**default**: `false` - -#### kparams - -Contains the list of old/new parameter name mappings. `old` key represents the original parameter name, while `new` is the new parameter name. diff --git a/docs/transformers/replace.md b/docs/transformers/replace.md deleted file mode 100644 index f7e96e6d3..000000000 --- a/docs/transformers/replace.md +++ /dev/null @@ -1,46 +0,0 @@ -# Replace - -The `replace` transformer replaces all non-overlapping instances of string parameters with the specified substring. Given the following event parameters: - -``` -{ - 'key_handle': 0, - 'key_name': 'HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\StateSeparation\Keys', - 'status': 'key not found' -} -``` - -And the `replace` transformer configuration: - -``` -replace: - enabled: true - replacements: - - kparam: key_name - old: HKEY_LOCAL_MACHINE - new: HKLM -``` - -The transformer produces the following parameters: - -``` -{ - 'key_handle': 0, - 'key_name': 'HKLM\System\CurrentControlSet\Control\StateSeparation\Keys', - 'status': 'key not found' -} -``` - -### Configuration {docsify-ignore} - -The `replace` transformer configuration is located in the `transformers.replace` section. - -#### enabled - -Indicates if the `replace` transformer is enabled. - -**default**: `false` - -#### replacements - -Contains the list of parameter replacements. For each target event parameter identified by the `kparam` key, `old` represent the substring that will be replaced by the `new` substring. diff --git a/docs/transformers/tags.md b/docs/transformers/tags.md deleted file mode 100644 index 43f071b8f..000000000 --- a/docs/transformers/tags.md +++ /dev/null @@ -1,27 +0,0 @@ -# Tags - -The `tags` transformer appends custom tags to event's metadata field. - -### Configuration {docsify-ignore} - -The `tags` transformer configuration is located in the `transformers.tags` section. - -#### enabled - -Indicates if the `tags` transformer is enabled. - -**default**: `false` - -#### tags - -Contains the list of tags that are appended to event metadata. Values can be fetched from environment variables by enclosing them in `%`. Example: - -``` -tags: - enabled: true - tags: - - key: env - value: prod - - key: drive - value: %HOMEDRIVE% -``` diff --git a/docs/transformers/trim.md b/docs/transformers/trim.md deleted file mode 100644 index 224c325a2..000000000 --- a/docs/transformers/trim.md +++ /dev/null @@ -1,52 +0,0 @@ -# Trim - -The `trim` transformer cuts off specified prefixes/suffixes from the string event parameters. Given the following event parameters: - -``` -{ - 'key_handle': 0, - 'key_name': 'HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\StateSeparation\Keys', - 'status': 'key not found' -} -``` - -And the `trim` transformer configuration: - -``` -replace: - enabled: true - prefixes: - - kparam: key_name - trim: HKEY_LOCAL_MACHINE - suffixes: - - kparam: key_name - trim: Keys -``` - -The transformer produces the following parameters: - -``` -{ - 'key_handle': 0, - 'key_name': '\System\CurrentControlSet\Control\StateSeparation\', - 'status': 'key not found' -} -``` - -### Configuration {docsify-ignore} - -The `trim` transformer configuration is located in the `transformers.trim` section. - -#### enabled - -Indicates if the `trim` transformer is enabled. - -**default**: `false` - -#### prefixes - -Contains the list of parameter names and prefixes that are trimmed from the parameter's value. - -#### suffixes - -Contains the list of parameter names and suffixes that are trimmed from the parameter's value. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 000000000..fa7ae8664 --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,74 @@ +# Troubleshooting + +## Logs + +Fibratus logs various diagnostics and error messages to log files residing in the `%PROGRAM FILES%\Fibratus\Logs` directory. Logs serve as an invaluable resource for debugging or chasing code bugs. + +It's possible to influence the behavior of how log file are written. The configuration options are stored in the `logging` section of the `yml` file. + +### `level` + +Specifies the minimum allowed log level. Anything logged below this log level will not get dumped to a file or stdout stream. Possible values are `debug`, `info`, `warn`, `error`, `panic`. + +### `max-age` + +Represents the maximum number of days to retain old log files based on the timestamp encoded in their filename. By default, all log files are retained. + + +### `max-backup` + +Specifies the maximum number of old log files to retain. + + +### `max-size` + +Specifies the maximum size in megabytes of the log file before it gets rotated. + +### `formatter` + +Represents the log file format. Possible values are `text` or `json`. By default, Fibratus will dump the logs in JSON format. + + +### `path` + +Represents the alternative paths for storing the logs. + +### `log-stdout` + +Indicates whether log lines are written to standard output in addition to writing them to log files. + + +## Profiling + +[pprof](https://golang.org/pkg/net/http/pprof/) is an extremely useful profiling facility that lets you collect CPU profiles, traces and heap allocation profiles among others. With `pprof` it is easy to spot top CPU consumers or find opportunities for code optimizations. + +To get the profile, you can use the `go tool pprof` tool. The pprof HTTP handlers are exposed on `localhost:8482` by default. To override the TCP port or the transport protocol, modify the `api.transport` configuration option. For example, getting the CPU profile could be accomplished with the following command: + + +$ go tool pprof http://localhost:8482/debug/pprof/profile + + + +The profile can be saved to the disk by typing `proto` in the interactive `pprof` CLI. + +## Stats + +Sometimes you need to go beyond surface-level visibility and understand how Fibratus itself is behaving under the hood. Especially when troubleshooting performance issues, validating pipeline behavior, or tuning rules. For that, Fibratus exposes a rich set of internal telemetry and runtime metrics. + +These metrics are made available via Go’s [expvar](https://golang.org/pkg/expvar/) interface, which provides a lightweight, structured way to inspect counters, gauges, and other runtime values directly from within the application. + +To quickly explore the metrics, you can use the built-in CLI command: + + +$ fibratus stats + + + +This command surfaces a snapshot of Fibratus’ internal state, including event throughput, queue depths, dropped events, and other operational signals. It’s particularly useful for: + +* Diagnosing bottlenecks in event processing pipelines +* Verifying that filters and rules are behaving as expected +* Monitoring resource utilization trends over time +* Gaining confidence in system stability during high event volumes + +Because these metrics are exposed via [expvar](https://golang.org/pkg/expvar/), they can also be integrated with external observability tools or scraped programmatically, making it easier to incorporate Fibratus into a broader monitoring and alerting ecosystem. diff --git a/docs/troubleshooting/logs.md b/docs/troubleshooting/logs.md deleted file mode 100644 index 7e2c18e52..000000000 --- a/docs/troubleshooting/logs.md +++ /dev/null @@ -1,47 +0,0 @@ -# Logs - -Fibratus logs various diagnostics and error messages to log files residing in the `%PROGRAM FILES%\Fibratus\Logs` directory. Logs serve as an invaluable resource for debugging or chasing down issues in Fibratus. - -### Configuration {docsify-ignore} - -It's possible to influence the behavior of how log file are written. The configuration options are stored in the `logging` section of the `yml` file. - -#### level - -Specifies the minimum allowed log level. Anything logged below this log level will not get dumped to a file or stdout stream. Possible values are `debug`, `info`, `warn`, `error`, `panic`. - -**default**: `info` - -#### max-age - -Represents the maximum number of days to retain old log files based on the timestamp encoded in their filename. By default, all log files are retained. - -**default**: `0` - -#### max-backup - -Specifies the maximum number of old log files to retain. - -**default**: `15` - -#### max-size - -Specifies the maximum size in megabytes of the log file before it gets rotated. - -**default**: `100` - -#### formatter - -Represents the log file format. Possible values are `text` or `json`. By default, Fibratus will dump the logs in JSON format. - -**default**: `json` - -#### path - -Represents the alternative paths for storing the logs. - -#### log-stdout - -Indicates whether log lines are written to standard output in addition to writing them to log files. - -**default**: `false` diff --git a/docs/troubleshooting/pprof.md b/docs/troubleshooting/pprof.md deleted file mode 100644 index a97391bf5..000000000 --- a/docs/troubleshooting/pprof.md +++ /dev/null @@ -1,9 +0,0 @@ -# Profiling - -[pprof](https://golang.org/pkg/net/http/pprof/) is an extremely useful profiling facility that lets you collect CPU profiles, traces and heap allocation profiles among others. With `pprof` it is easy to spot top CPU consumers or find opportunities for code optimizations. - -To get the profile, you can use the `go tool pprof` tool. The pprof HTTP handlers are exposed on `localhost:8482` by default. To override the TCP port or the transport protocol, modify the `api.transport` configuration option. For example, getting the CPU profile could be accomplished with the following command: - -``` -$ go tool pprof http://localhost:8482/debug/pprof/profile -``` diff --git a/docs/troubleshooting/stats.md b/docs/troubleshooting/stats.md deleted file mode 100644 index 07a296943..000000000 --- a/docs/troubleshooting/stats.md +++ /dev/null @@ -1,3 +0,0 @@ -# Stats - -Sometimes it is useful to dive into the internal Fibratus telemetry to get various metrics about its inner workings. Fibratus exposes its internal metrics through the [expvar](https://golang.org/pkg/expvar/) interface. To explore the metrics you can execute the `fibratus stats` command. diff --git a/docs/yara.md b/docs/yara.md new file mode 100644 index 000000000..5d0d4b797 --- /dev/null +++ b/docs/yara.md @@ -0,0 +1,97 @@ +# Memory scanning + +##### [YARA](https://virustotal.github.io/yara/) is a widely adopted framework for binary pattern matching and malware classification. It allows to define rules that describe suspicious or malicious artifacts based on strings, byte patterns, and binary format properties. + +Fibratus integrates YARA directly into its event processing pipeline to enable real-time scanning of memory and binaries, focusing on in-memory threats such as fileless malware, suspicious [PE](https://learn.microsoft.com/en-us/windows/win32/debug/pe-format) artifacts, and abnormal memory behavior. + +Memory or binary scanning is triggered in response to various behavioral signals: + +- new process creation +- loading of an unsigned/untrusted executable/DLL or when the executable/DLL is loaded from the unbacked memory region +- creation of executable, DLL, or driver PE files +- creation of [ADS](https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/e2b19412-a925-4360-b009-86e3b8a020c8) (Alternate Data Streams) +- RWX memory allocations +- mapping of a suspicious view of section +- writing a binary registry value + +Fibratus embeds [`libyara`](https://yara.readthedocs.io/en/stable/capi.html) via native bindings. The YARA engine is statically linked so no additional installation is required. Rules are compiled and executed inside the Fibratus runtime. + +!> Fibratus detection rules use a custom, purpose-built format tailored to its event processing and detection model. In contrast, YARA rules follow a widely adopted industry standard for pattern matching and malware classification. Because they serve different purposes and operate independently, Fibratus does not include built-in YARA rules. However, you can integrate and run community-maintained [YARA rules](https://github.com/yara-rules/rules) alongside Fibratus. + + +## Scanning + +The YARA scanner is disabled by default. To enable it, set the `yara.enabled` option in the configuration file. + +For the scanner to function, YARA rules must be compiled and loaded into the engine. You can do this by specifying file system paths in the `yara.rule.paths` configuration option. These directories are scanned recursively for `.yar` files. Alternatively, rules can be defined inline as strings directly within the configuration file. + +Alerts generated from rule matches are automatically dispatched through all configured alert senders. +When an event matches a YARA rule, its metadata is enriched with the corresponding match details. The `yara.matches` field contains a JSON array where each object represents a YARA rule match. + +## Configuration + +YARA scanner related options are located in the `yara` section of the configuration file. + +### `enabled` + +Indicates if the YARA scanner is enabled. + +### `rule` + +The `rule` key contains various nested keys. The `paths` key identifies directories that contain YARA rule definitions. It is also possible to link each directory to YARA namespace. The `strings` key allows defining inline YARA rules. Example: + +```python +rule: + paths: + - path: C:\\yara-rules + namespace: default + - path: C:\\pdf-rules + namespace: pdf + + strings: + - string: rule test : tag1 { meta: author = \"Hilko Bengen\" strings: $a = \"abc\" fullword condition: $a } + namespace: notepad +``` + +### `alert-template` + +Specifies templates for the alert text in Go [templating](https://golang.org/pkg/text/template) language. + +### `fastscan` + +Determines when multiple matches of the same string can be avoided when not necessary. + +### `scan-timeout` + +Specifies the timeout for the scanner. If the timeout is reached, the scan operation is cancelled. + +### `skip-files` + +Indicates whether file scanning is disabled. This affects the scan triggered by image loading, create file, and file mapping operations. + +### `skip-allocs` + +Indicates whether scanning on suspicious memory allocations is disabled. + + +### `skip-mmaps` + +Indicates whether scanning on suspicious mappings of sections is disabled. + +### `skip-registry` + +Indicates whether registry value scanning is disabled. + + +### `excluded-files` + +Contains the list of the comma-separated file paths that shouldn't be scanned. Wildcard matching is supported. For example: + +```python +excluded-files: + - ?:\\Windows\\System32\\kernel32.dll +``` + +### `excluded-procs` + +Contains the list of the comma-separated process executable paths that shouldn't be scanned. Wildcard matching is supported. diff --git a/docs/yara/alerts.md b/docs/yara/alerts.md deleted file mode 100644 index 8dd379e38..000000000 --- a/docs/yara/alerts.md +++ /dev/null @@ -1,39 +0,0 @@ -# Alerts - -Alerts on rule matches are automatically sent via all active alert senders. - -## Event metadata {docsify-ignore} - -When the event triggers a specific YARA rule, its metadata is automatically decorated with the rule matches. -The `yara.matches` tag contains the JSON array payload where each object represents the YARA rule match. For example: - -```json -[ - { - "rule": "AnglerEKredirector ", - "namespace": "EK", - "tags": null, - "metas": [ - { - "identifier": "description", - "value": "Angler Exploit Kit Redirector" - } - ], - "strings": "..." - }, - { - "rule": "angler_flash_uncompressed ", - "namespace": "EK", - "tags": [ - "exploitkit" - ], - "metas": [ - { - "identifier": "description", - "value": "Angler Exploit Kit Detection" - } - ], - "strings": "..." - } -] -``` diff --git a/docs/yara/introduction.md b/docs/yara/introduction.md deleted file mode 100644 index e8c0fe145..000000000 --- a/docs/yara/introduction.md +++ /dev/null @@ -1,15 +0,0 @@ -# Pattern Matching Swiss Knife - -[YARA](https://virustotal.github.io/yara/) is a prominent tool for binary pattern matching that aims to streamline and accelerate the classification of malware specimens. Fibratus interacts with the `libyara` through C bindings. The `libyara` dependency is statically linked, so no further software needs to be installed. - -**Fibratus/YARA** tandem aims to detect in-memory threats and malicious **PE** files by reacting on various signals including: - -- new process creation -- loading of an unsigned/untrusted executable/DLL or when the executable/DLL is loaded from the unbacked memory region -- creation of executable, DLL, or driver PE files -- creation of [ADS](https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/e2b19412-a925-4360-b009-86e3b8a020c8) (Alternate Data Streams) -- RWX memory allocations -- mapping of a suspicious view of section -- writing a binary registry value - -The YARA scanner is not enabled by default, but you can do that by modifying the `yara.enabled` key in the configuration file. diff --git a/docs/yara/scanning.md b/docs/yara/scanning.md deleted file mode 100644 index dcbe3d556..000000000 --- a/docs/yara/scanning.md +++ /dev/null @@ -1,86 +0,0 @@ -# Scanning Processes - -For the YARA scanner to operate correctly, the rules have to be compiled and loaded into the engine. This is accomplished by providing file system paths with YARA rule definitions in the `rule.paths` configuration keys. The directories are scanned recursively for any `.yar` file. Alternatively, it is possible to provide the rules as inline strings directly in the Fibratus configuration file. - -### Configuration {docsify-ignore} - -YARA scanner related options are located in the `yara` section of the configuration file. - -#### enabled - -Indicates if the YARA scanner is enabled. When enabled, each newly created process is scanned for pattern matches. - -**default**: `false` - -#### rule - -The `rule` key contains various nested keys. The `paths` key identifies directories that hold the YARA rule definitions. It is also possible to link each directory to YARA namespace. The `strings` key allows for defining inline YARA rules. Example: - -``` -rule: - paths: - - path: C:\\yara-rules - namespace: default - - path: C:\\pdf-rules - namespace: pdf - - strings: - - string: rule test : tag1 { meta: author = \"Hilko Bengen\" strings: $a = \"abc\" fullword condition: $a } - namespace: notepad -``` - -#### alert-template: - -Specifies templates for the alert text in Go [templating](https://golang.org/pkg/text/template) language. - - -#### fastscan - -Determines when multiple matches of the same string can be avoided when not necessary. - -**default**: `true` - -#### scan-timeout - -Specifies the timeout for the scanner. If the timeout is reached, the scan operation is cancelled. - -**default**: `20s` - -#### skip-files - -Indicates whether file scanning is enabled. This affects the scan triggered by the image loading, create file, and file mapping operations. - -**default**: `false` - -#### skip-allocs - -Indicates whether scanning on suspicious memory allocations is disabled. - -**default**: `false` - -#### skip-mmaps - -Indicates whether scanning on suspicious mappings of sections is disabled. - -**default**: `false` - - -#### skip-registry - -Indicates whether registry value scanning is disabled. - -**default**: `false` - - -#### excluded-files - -Contains the list of the comma-separated file paths that shouldn't be scanned. Wildcard matching is possible. For example: - -``` -excluded-files: - - ?:\\Windows\\System32\\kernel32.dll -``` - -#### excluded-procs - -Contains the list of the comma-separated process image paths that shouldn't be scanned. Wildcard matching is possible. diff --git a/logo.png b/logo.png deleted file mode 100644 index 715f8fa05..000000000 Binary files a/logo.png and /dev/null differ diff --git a/logo.svg b/logo.svg new file mode 100644 index 000000000..d2936f265 --- /dev/null +++ b/logo.svg @@ -0,0 +1 @@ +Fibratus Kernel Detection SensorIcon representing Fibratus, an open-source Windows kernel security and behavioral threat detection sensor \ No newline at end of file diff --git a/rules/README.md b/rules/README.md index f735475b4..ca73990d2 100644 --- a/rules/README.md +++ b/rules/README.md @@ -6,7 +6,7 @@

-This directory contains a catalog of detection rules modeled around the prominent [MITRE ATT&CK](https://attack.mitre.org/) framework. The goal is to provide a direct mapping of tactics, techniques, and sub-techniques for each rule. The following sections introduce the general structure, design guidelines, and best practices to keep in mind when creating new rules. +This directory contains a catalog of detection rules mapped to [MITRE ATT&CK](https://attack.mitre.org/) framework. The goal is to provide a direct mapping of tactics, techniques, and sub-techniques for each rule. The following sections introduce the general structure, design guidelines, and best practices to keep in mind when creating new rules. ## Structure @@ -28,7 +28,7 @@ required attributes such as rule identifier, name, and the minimum engine versio ### Read the docs -This should be your starting point. Before trying to write new rules, explore the [docs](https://www.fibratus.io/#/filters/introduction) to learn about [filter expressions](https://www.fibratus.io/#/filters/filtering) fundamentals, [operators](https://www.fibratus.io/#/filters/operators), [functions](https://www.fibratus.io/#/filters/functions), [filter fields](https://www.fibratus.io/#/filters/fields) reference, and [rule engine](https://www.fibratus.io/#/filters/rules) specifics. +This should be your starting point. Before trying to write new rules, explore the [docs](https://www.fibratus.io/docs/rules) to learn about [operators](https://www.fibratus.io/docs/rules/operators), [functions](https://www.fibratus.io/docs/rules/functions), [filter fields](https://www.fibratus.io/docs//rules/fields) reference. ### Stick to naming nomenclature @@ -55,4 +55,4 @@ Fibratus comes with a [macros](https://www.fibratus.io/#/filters/rules?id=macros ### Formatting styles -Pay attention to rule condition/action formatting style. If the rule consists of multiple or large expressions, it is desirable to split each spanning expression on a new line and properly indent the `and`, `or`, or `not` operators. By default, we use 1 space tab for indenting operators and rule actions. This notably improves readability and prevents formatting inconsistencies. +Pay attention to rule condition/action formatting style. If the rule consists of multiple or large expressions, it is desirable to split each spanning expression on a new line. This notably improves readability and prevents formatting inconsistencies.