Skip to content

[pull] master from kevoreilly:master#469

Merged
pull[bot] merged 37 commits into
threatcode:masterfrom
kevoreilly:master
May 15, 2026
Merged

[pull] master from kevoreilly:master#469
pull[bot] merged 37 commits into
threatcode:masterfrom
kevoreilly:master

Conversation

@pull
Copy link
Copy Markdown

@pull pull Bot commented May 15, 2026

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.4)

Can you help keep this open source service alive? 💖 Please sponsor : )

josh-feather and others added 30 commits May 12, 2026 13:42
Previously, a failure in save_script_to_storage caused download_file to
return ("error", ...) with a hardcoded message. The submission view routes
to error.html vs complete.html based solely on whether task IDs were
produced, so callers that received an error status would never set
task_ids — sending the user to error.html with the generic heading
"Error adding task(s) to CAPE's database." even though the tasks had
already been created in the database.

Now the exception is caught and its message appended to a warnings list,
which is merged into the 'errors' field of the final 'ok' response. Task
submission proceeds, task IDs are returned to the caller, and the user
lands on complete.html seeing the actual exception rather than a hardcoded
fallback string.

Also fix a pre-existing misleading label on complete.html: the error block
heading read "Submission Failed!" but that template is only ever rendered
when tasks were successfully queued. Rename it to "Errors" to accurately
reflect that it surfaces non-fatal warnings alongside a successful
submission.
Idle timeout:
- Add SessionTimeoutManager (web/guac/timeout_manager.py) which tracks
  last mouse/keyboard activity and signals analysis completion by creating
  the cape-<task_id> folder on the guest via the CAPE agent — the same
  mechanism used by the End Session button.
- Add guac_utils.py with is_user_activity(), which matches mouse/keyboard
  Guacamole protocol instructions using a regex against the wire format.
- Update consumers.py to initialise a SessionTimeoutManager per session,
  update activity on every inbound message, and run a monitor_timeout()
  background task that fires handle_timeout() when the idle threshold is
  exceeded. The timeout sends a Guacamole error frame (code 522) to the
  client before closing.
- Add _close_websocket() to prevent double-close race conditions that were
  producing stack traces when the reader task and disconnect() both tried
  to close simultaneously.
- Allow 'pending' tasks as well as 'running' in ACTIVE_GUAC_TASK_STATUSES
  so sessions can connect during the brief window before a task starts.
- Add idle_timeout_seconds and activity_check_interval config options to
  web.conf.default; both default to disabled (0).
- Add tests: test_guac_consumers.py, test_guac_timeout_manager.py,
  test_guacamole_activity_detection.py.

Guacamole 1.6 upgrade:
- The previously bundled 1.4.0 client was behind what the upstream
  CAPEv2 installer deploys; this aligns the bundled JS with 1.6.0.
- Add guacamole-1.6.0-all.min.js static asset.
- Update guac/index.html and analysis/overview/_playback.html to reference
  the 1.6.0 build.
- Refactor guac-main.js from a procedural function to an ES6 class
  (GuacSession). Extracts keyboard, mouse, clipboard, scaling and error
  handling into dedicated setup methods. Differentiates error codes 514,
  515, and 522 for cleaner user-facing messages. Uses sendMouseState with
  the Guacamole 1.6 fromServer flag.
… evasion)

When malware uses DCOM out-of-process activation (e.g. Excel embeds an
HTA object → Windows launches mshta.exe -Embedding via svchost broker),
the OS-level process parent is svchost — hiding the true originator.

Changes:
- behavior.py: NetworkMap now captures CoCreateInstance calls with
  CLSCTX_LOCAL_SERVER context (cat=com) and known out-of-process CLSIDs
  (htafile/mshta, InternetExplorer, ShellWindows, etc.) into a new
  `com_activations` list. After all instances run, _enrich_tree_com_parents()
  walks the processtree and annotates each node whose binary matches a COM
  activation record with `com_logical_parent_pid`, `com_logical_parent_name`,
  `com_progid`, and `com_clsid`.

- generic_tags.py: proctreetolist filter passes the four new COM fields
  through to the flattened output list.

- _tree.html: Processes with `com_logical_parent_pid` render a dashed
  annotation line "COM-activated by <parent> via <progid>" linking back
  to the logical originator, making the LethalHTA chain visible without
  requiring the analyst to correlate COM call logs manually.

Observed in tasks where Excel CoCreates CLSID_HtaFile (htafile) and
mshta.exe -Embedding is spawned through the DcomLaunch svchost broker.
- wait.html / error.html / index.html: Apply CAPE Bootstrap dark theme
  (bg-dark, navbar header, Bootstrap components) to match the rest of
  the UI. Previously these pages used bare unstyled HTML.

- index.html toolbar: Thin bg-dark bar with CAPE logo, task link, and
  End Session button; Guacamole canvas fills remaining viewport height.

- guac-main.js stopTask(): Add X-CSRFToken header so the POST succeeds
  under Django CSRF protection. Redirect to task status page on success.
  Show spinner on button while request is in flight.

- guac-main.js disconnect overlay: When the Guacamole connection drops,
  show the #launch_error overlay via CSS display:block instead of the
  jQuery UI .dialog() widget (which was never loaded). Normal session end
  shows green "Session Complete"; unexpected disconnect shows red
  "Session Error".

- guac-main.css: Dark overlay styling for the disconnect dialog using
  CAPE colors (#212529 bg, white text).

- apiv2/views.py tasks_status: Enable user_stop (was disabled) so
  End Session POST actually signals the guest VM to complete.
When multiple processes query the same hostname or connect to the same IP,
the network tables previously showed only one process badge. Renders one
badge per attributed process in both Network tab and Overview tab.

- _dns.html / _dns_not_ajax.html: iterate p.processes list for the
  Process Name (PID) cell; fall back to legacy single-field render.
- _hosts_not_ajax.html: sync multi-process badge iteration to match
  the AJAX version.
Adds a collapsible certificate chain card to the analysis overview tab,
surfacing Authenticode signing information that was previously buried in
the Static Analysis tab accordion.

Card features:
- Header badge: Valid (green), Untrusted Root (yellow), Invalid (red),
  Known Malicious Cert (red skull, when bad_certs signature fires)
- Signing timestamp and error description inline in header area
- Code Signing Chain accordion: each cert in the chain is a collapsible
  row showing Subject CN/O/OU/C/email, Issuer DN, Not Before/After,
  Serial, SHA1/MD5/SHA256 fingerprints. Self-signed certs get a red
  "Self-Signed" badge.
- Timestamp Chain accordion: DigiCert/TSA chain entries, collapsible.
- Falls back gracefully when only DigiSig.json data is available (SHA1
  only) vs. full digital_signers data.

Also fixes pkcs7 extraction (see perf PR): parse_pe.py was using
backend.load_der_pkcs7_certificates() which was removed in cryptography
≥ 40.x, causing digital_signers to be empty for all signed PEs.

New web.conf toggle: [display_authenticode] enabled = no (default off).

web.conf.default: Add missing display_* toggle defaults:
display_clamav, display_cape_yara, display_submitter, display_etw,
display_authenticode — all default off so existing deployments are
unaffected until explicitly enabled.
…earch parity

**index.html:**
- Task tags: render as clickable `tags_tasks:<tag>` search links (red badges)
  instead of static spans
- YARA column (`display_cape_yara`): shows count of CAPE-yara hits with
  tooltip listing rule names; default off in web.conf.default
- Submitter column (`display_submitter`): shows username of submitting user
  when WEB_AUTHENTICATION is enabled; default off
- Task tags column (`display_task_tags`): already existed, now properly
  gated and tags are clickable
- Detection badges use `bg-danger` (CAPE red) instead of blue Bootstrap default

**search.html:**
- Results table now mirrors the main analysis list exactly: same columns,
  same config gates (display_task_tags, display_submitter, display_cape_yara,
  display_clamav, expanded suricata), same badge styles
- Previously the search results table was missing YARA, tags, submitter, and
  the expanded Suricata columns

**overview/_info.html:**
- Submitter row in the analysis report info card, gated on
  `settings.WEB_AUTHENTICATION and config.display_submitter`

**header.html:**
- "API Keys" menu item only shown to users with `may_manage_apikeys` context
  (local users always; SSO users only when is_staff=True)
Adds a new processing module that correlates ETW (Event Tracing for Windows)
telemetry with network observations to provide per-process attribution for
DNS queries, TCP/UDP connections, TLS sessions, HTTP transactions, and
extracted files.

## Processing module (modules/processing/network_etw.py)

ProcessFlowIndex: a correlation engine built from multiple ETW sources:
- Sysmon EID 3 (NetworkConnect): direct TCP/UDP with PID
- Kernel-network ETW: per-flow (src_port, dst_ip, dst_port, pid)
- Sysmon EID 1 (ProcessCreate): PID→image mapping
- Sysmon EID 22 (DnsQuery) + DNS-Client ETW: hostname→PID
- behavior.network_map.dns_intents: hooked getaddrinfo/DnsQuery API calls
- UDP/53 fallback: correlates raw UDP flows with Suricata DNS src_port
- Suricata DNS/HTTP: adds resolutions and HTTP host attributions

Enriches existing network records (suricata.alerts/tls/http, network.dns,
network.hosts, network.http_ex, dropped files, sigma detections) with
per-process attribution without duplicating data.

## Analyzer auxiliary module

network_etw.py auxiliary: removed port 53 from the UDP filter so DNS-over-UDP
flows include the ephemeral source port needed for UDP/53 fallback attribution.

## ETW Telemetry tab (web/)

New tab on the analysis report (gated on `display_etw` and `analysis.has_etw`):
- DNS sub-tab: queries with process attribution and resolution data
- Network sub-tab: TCP/UDP connections with process, geo, ASN
- WMI sub-tab: WMI activity timeline
- ThreatIntel sub-tab: ETW-sourced threat intelligence events
- AMSI sub-tab: AMSI scan events

## Configuration

New web.conf toggle: `[display_etw] enabled = no` (default off).
…ple activations

- Restore process['calls'].reset() after event_apicall loop so downstream
  reporting modules can re-read the call log (critical regression fix).
- Fix _OOP_CLSIDS: 25336920 (HTMLDocument OOP) maps to mshta.exe not mshtml.dll.
- Use _safe_int() for clscontext parsing (handles hex and decimal).
- Remove per-(pid,clsid) deduplication to capture multiple COM server
  instances spawned by the same process (e.g. multiple mshta.exe -Embedding).
…k; O(1) UDP port index; add display_etw config
Refactor title attribute in badge span for better readability.
feat: Authenticode certificate card on analysis overview tab
…hment

feat: COM logical parent enrichment in process tree (LethalHTA / DCOM evasion)
fix: Guac UI dark theme, working End Session button, and themed overlay
…attribution

fix: DNS and Hosts network tables support multiple process attributions
…ements

feat: Analysis list view — YARA column, clickable tags, submitter column, search parity
kevoreilly and others added 7 commits May 15, 2026 12:51
@pull pull Bot locked and limited conversation to collaborators May 15, 2026
@pull pull Bot added the ⤵️ pull label May 15, 2026
@pull pull Bot merged commit 6767653 into threatcode:master May 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants