sentinel-harness

🎯 North Star: Agent Builds Agents

A self-iterating security-operations platform where natural language flows in, and the platform auto-builds, tests, evaluates, iterates, and promotes agents — fully controllable (HITL gates) and observable.

NL request → Meta-Agent (Opus) → auto-build → evaluate → iterate → promote

✅ Live-validated: closed loop proven on real AWS (weak→improve→pass→HITL→promote)

Why sentinel-harness?

The thesis: a production SecOps agent should be configuration, not orchestration code.

⚡

Zero Orchestration Code

Declare model · prompt · tools · skills · memory · limits. AWS runs the entire agent loop in a per-session Firecracker microVM.

Live-validated

🔄

Self-Improving Loop

Build → evaluate → retry-with-reasoning → promote. The framework auto-improves its own agents via managed evaluation + regression guard + safety veto.

Proven on AWS

🛡️

Security-First

HITL gates everywhere. Sandbox hooks. Egress control. Guardrail injection defense. Supply-chain hardened (SBOM + SLSA + CodeQL + pip-audit).

100 defects fixed

🔌

Plug-and-Play

Backend-pluggable tools (Splunk / Elasticsearch / ServiceNow / Jira connectors), multi-account ops, a `*_LIVE` env to connect any real SIEM.

Offline-first

📊

Detection Engineering Suite

7 deterministic, LLM-free tools: lint → translate (YARA/Suricata/SPL/EQL) → dedup → coverage → audit → Navigator → baseline regression gate.

CI-gateable

🤖

MCP Server Mode

All 20 tools exposed as a standard MCP server. Any AI agent (Claude Code, Cursor, Windsurf) connects with zero integration code.

M15 — NEW

Detection Engineering Pipeline

7 deterministic tools, LLM-free, offline, CI-gateable — with an explicit "cannot analyze" honesty ledger.

sigma_yara_lint → detection_translate → detection_dedup → detection_coverage → detection_audit → detection_navigator → detection_baseline

    # One-command CI gate

    $ sentinel detection ci ./rules --min-score 70 --against baseline.json

    # Translate Sigma to Splunk SPL + Elastic EQL

    $ sentinel detection audit ./rules --targets splunk,elastic

    # Export ATT&CK Navigator layer

    $ sentinel detection ci ./rules --navigator-out layer.json

Live-Validated on AWS

Not mock — exercised against real AgentCore APIs on a non-production account, then torn down.

Proof	What ran	Result
Runtime A2A	CreateAgentRuntime → arm64 microVM → live A2A message/send → Bedrock Haiku → Log4Shell verdict	✓ HTTP 200
Closed Loop	Weak agent (0.0) → improve → re-score (1.0) → safety veto + regression guard → HITL → promote endpoint	✓ closed:true
CUSTOM_JWT Gateway	Cognito OIDC M2M → RS256 token accepted (200) + no/garbage token rejected (401)	✓ enforced
Memory Recall	4 sessions tenant-1 → SEMANTIC extraction → cross-session recall; tenant-2 = 0 (isolation)	✓ isolated
Managed Evaluator	SESSION LLM-judge ACTIVE + OnlineEvaluationConfig scoring 100% spans (Faithfulness/Harmfulness)	✓ ACTIVE
Guardrail	GUARDRAIL_INTERVENED masked a fake AWS key + sk- token in tool response	✓ masked
Egress Control	Private VPC — no IGW, no NAT, no 0.0.0.0/0 — PrivateLink only	✓ default-deny
Registry Governance	Registry + record: DRAFT → PENDING_APPROVAL (autoApproval=false dual-gate)	✓ governed
Drift Fix (live)	Underscore-named Registry created (pre-fix: ValidationException) → torn down, zero residue	✓ fixed

Milestones Delivered

15 milestones from baseline reproduction to world-class depth — all shipped.

Environment & baseline

2352 offline tests green + live scenario reproduction

Meta-agent self-iteration engine

NL request → Opus emits harness spec → really builds + invokes + tears down

Evaluation-driven self-improvement

Score → retry-with-reasoning → promote endpoint (LLM-judge, safety veto, regression guard)

Attack validation & simulation

BAS replay + Sigma matcher finds blind spots; detonation lifecycle (honest simulated)

Foundation: identity / gateway / egress / observability

CDK + Terraform dual-track; Guardrail/Identity/Observability live-deployed

Data planes + domain skills

DIY mock world, backend-pluggable tools, multi-account ops

Feedback-loop automation

Disposition auto-feeds strategy; whitelist optimization preserves TPs

One-command delivery

Makefile + sentinel export (Strands, no lock-in) + QUICKSTART

Quality CI enforcement

Coverage gate, Hypothesis property tests, pre-commit, py3.13

Security-product credibility

SECURITY.md, pip-audit, CodeQL, Scorecard, sandbox fuzzing, SSRF guard

M10

Adopter experience

INTEGRATIONS, COOKBOOK, API reference site, SLSA provenance + SBOM

M11

On-platform proof

Token metrics, adversarial-reviewer, Memory recall, JWT gateway, online eval, A2A Runtime

M12

North-star closed loop

End-to-end: weak→improve→pass→HITL→promote (runner-orchestrated, live-proven)

M13

World-class depth

Benchmark model, golden datasets, enterprise world, compliance mapping, connectors, autonomy controller

M14

Adversarial hardening + detection suite

8 audit rounds (100 fixes), service-model drift scan, 7-tool detection CLI, SPL/EQL

Get Started in 60 Seconds

    # Install

    $ pip install sentinel-harness

    # Run the full offline test suite

    $ make test

    # Demo: all 21 offline scenarios

    $ make demo

    # Detection engineering one-liner

    $ sentinel detection ci ./my-sigma-rules --min-score 70

    # Export to Strands (no lock-in)

    $ sentinel export harnesses/alert-triage/harness.yaml -o agent.py

    # Deploy to AWS (one command)

    $ make deploy

🎯 North Star: Agent Builds Agents

Why sentinel-harness?

Zero Orchestration Code

Self-Improving Loop

Security-First

Plug-and-Play

Detection Engineering Suite

MCP Server Mode

Detection Engineering Pipeline

Live-Validated on AWS

Milestones Delivered

Get Started in 60 Seconds

Architecture

Documentation

Quickstart

Architecture

API Reference

Cookbook

Integrations

Threat Model

Compliance

MCP Server

Changelog