Runnable plugin workflows
Start with the possible malicious access walkthrough if you have an alert or a suspicious sign-in. It follows one investigation from intake through evidence review and export, with malicious, benign, and insufficient-evidence examples.
Use the table below to exercise a particular plugin. Each playback starts with local sample input, invokes the package’s tools, and checks the resulting artifacts and decisions. A failed expectation makes the command exit with a nonzero status. The examples use synthetic data or local fixtures; passing them establishes the documented local behavior, not the state of a production system or readiness of an isolated laboratory.
Run a playback
From the repository root, use Python 3.11 or later. Complete the
contributor setup for the full test suite. The
optional laboratory dependency is needed for signed laboratory receipts.
python3 -m cops list
python3 -m cops info soc-investigation-workbench
python3 -m cops demo soc-investigation-workbench
python3 -m cops check soc-investigation-workbench
demo runs the playback declared in the package manifest. check runs its
validation suite. The package playbook explains the individual commands, how to
retain output when supported, and how to supply your own sanitized evidence.
Commands that use temporary workspaces remove them when the run finishes.
Choose a workflow
| Plugin | What the playback exercises | Step-by-step instructions |
|---|---|---|
security-logging-advisor |
Collect repository evidence, disclose partial coverage, and validate an analyst-authored recommendation. | Logging review |
telemetry-proof-pack |
Validate telemetry mappings and compare supported, degraded, and missing evidence. | Telemetry proof |
soc-investigation-workbench |
Intake, handoff, hunt evidence, revise hypotheses, and export all three investigation outcomes. | SOC investigation |
sentinel-hunt-workbench |
Render the hunt library and evaluate malicious, benign, and invalid-input fixtures. | Sentinel hunting |
attack-path-workbench |
Import a graph, evaluate supported and candidate paths, quarantine invalid evidence, and verify report integrity. | Attack paths |
detection-quality-workbench |
Evaluate detection fixtures, benign controls, and mapping drift against expected results. | Detection regression |
threat-intelligence-enrichment |
Enrich observables with local provider responses and assess conflicts, stale data, and provider failures. | Intelligence enrichment |
entra-identity-workbench |
Build an identity graph, evaluate privilege hypotheses, and generate reports from local exports. | Entra identity |
ad-assessment-workbench |
Plan bounded directory queries, import observations, evaluate a candidate path, and report evidence gaps. | Active Directory |
exposure-triage-workbench |
Join vulnerability and asset evidence, prioritize supported exposure, and preserve unresolved cases. | Exposure triage |
exploit-validation-workbench |
Preflight exact-version reproduction metadata and local hashes, then refuse a mismatched version without execution. | Versioned reproduction preflight |
patch-security-review |
Review malicious and benign changes, validate findings, and produce an evidence-backed report. | Patch review |
attack-surface-planner |
Normalize passive exports, apply scope, quarantine exclusions, and propose bounded assessments. | Attack-surface planning |
foundry-agent-harness |
Evaluate local mock attacks, classify outcomes, and test authorization denials. | Foundry agent review |
offensive-engagement-workbench |
Validate engagement scope, build a plan and specialist handoffs, and record an inert local subprocess observation. | Engagement planning |
kubernetes-assessment-workbench |
Reconcile context identity, namespace access, incomplete coverage, and a rejected cluster change. | Kubernetes assessment |
host-assessment-workbench |
Import canary observations, distinguish crossings from blocked cases, and reject missing cleanup evidence. | Host assessment |
incident-response-sandbox |
Plan and rehearse response actions, check authorization, and verify rollback and expiry behavior. | Response rehearsal |
compliance-posture-workbench |
Map dated fixture evidence to controls while preserving supported, missing, stale, and unknown states. | Compliance posture |
cloud-posture-workbench |
Assess imported Terraform, cloud, Kubernetes, and IAM fixtures without claiming live tenant state. | Cloud posture |
supply-chain-cicd-workbench |
Assess imported lockfiles, SBOMs, CI workflows, and provenance fixtures without claiming registry or deployment reachability. | Supply chain and CI/CD |
secrets-remediation-workbench |
Build a revocation and rotation plan from redacted evidence without storing a secret or claiming provider-side completion. | Secrets remediation |
network-segmentation-workbench |
Assess imported policy, flow, DNS, packet-capture, and reachability fixtures without claiming deployed isolation. | Network segmentation |
tls-crypto-workbench |
Assess imported certificate, TLS, JWT, JWKS, and OpenSSL fixtures without probing an endpoint or proving enforcement. | TLS and cryptography |
web-api-assessment-workbench |
Import redacted HAR, OpenAPI, GraphQL, Postman, and gateway fixtures; preview requests without dispatching them. | Web and API assessment |
Understand the playback structure
Each package owns its workflow. Its README explains the purpose, design choices,
data contracts, and limits; docs/PLAYBOOK.md gives the runnable procedure;
docs/VERIFICATION.md, when present, explains qualification and maturity
evidence. The seven experimental import workflows exercise checked-in fixtures;
they do not establish live provider state, deployed enforcement, completed
remediation, production authorization behavior, or isolated-laboratory readiness.
The executable flow follows the same broad sequence:
- Prepare input. Load checked-in synthetic fixtures or create a private temporary fixture. Validate required fields and the allowed scope.
- Run the tools. Invoke public commands to build intermediate artifacts, such as a graph, query plan, normalized evidence, or a response plan.
- Evaluate evidence. Check expected findings and explicit gaps. Missing evidence must remain unresolved; it must not become a benign verdict.
- Inspect output. Check the report, decision records, and integrity or provenance fields supported by that package.
- Rerun and recover. Confirm the documented repeat behavior, reject invalid inputs, and remove temporary artifacts. Packages that refuse an existing workspace explain how to use a fresh one.
Local schemas describe different contracts: an imported observation, an analyst decision, and an execution receipt are not interchangeable. Use each README’s schema table to identify the producer, consumer, validation command, and example.
Run the catalog acceptance test
With contributor dependencies installed, this test discovers every catalog
entry and invokes its public cops demo command twice. It then exports and runs
each playback twice with an isolated COPS runtime, without source package files or
contributor scripts. Adding a plugin to the catalog automatically adds it to the
test. Exports use the Git index as their file inventory, so stage new package
files before testing an uncommitted change. See the
export inventory rules.
python3 -m pytest -q tests/step_defs/test_portable_operator_workflow.py
python3 scripts/agent/validate_docs.py
make check
Package tests check domain behavior in detail; the catalog test checks expected results through the complete public entry point. Documentation validation also requires a linked playbook, its demo command, and a schema contract table for each catalog entry. None of these checks substitutes for the environment-specific evidence required by plugin qualification.