ACSP — notes for dissertation text (thresholds, ethics, limitations)
===================================================================

Evaluation thresholds
-----------------------
- Binary positive class is ``malicious``. Defaults live in Settings / env:
  ``eval_rule_min_severity_malicious`` (default ``medium``): rule baseline severity
  ranks at or above this are counted malicious for ``rule_prediction``.
  ``eval_ai_score_malicious_min`` (default ``50``): AI ``severity_score`` in 0..100
  at or above this maps to ``ai_prediction`` = malicious.
- Sweeps without re-ingestion: use GET /api/v1/research/metrics with
  ``recompute_thresholds=true`` and optional ``ai_score_min`` / ``rule_min_severity``.

Latency methodology
---------------------
- ``ingest_through_ai_ms`` and ``ai_infer_ms`` are written to ``audit_entries``
  (stage ``timing`` / ``ai``). Report p50/p95 from ``compute_latency_summary`` on a
  fixed hardware profile (document CPU model, RAM, container CPU cap).

Feedback loop honesty
-----------------------
- ``acsp.research.feedback_loop`` adjusts a bounded global score bias from labeled
  mismatches; it is operator-feedback-informed calibration, not online ML training.

Automated response / ethics
----------------------------
- Mitigations are analyst guidance strings only (``analyst_guidance``); no autonomous
  enforcement is implemented. Discuss blast radius, human approval, and auditability
  if you extend to SOAR-style actions later.

Data limitations
----------------
- Simulator events are Falco-shaped but not real kernel noise; public IDS CSV rows
  (``ids_benchmark`` / ``model_supervised_ids_tabular_v1.joblib``) are a domain-shifted
  comparative baseline, not proof of Falco detection quality.

Ablation interpretation
-----------------------
- ``metrics_ablation.json`` from ``scripts/train_acsp_baselines.py`` compares rule-only,
  full-feature HGBDT, rule-ablated HGBDT, and a simple IsolationForest proxy on the same
  validation split. Runtime ``ai_engine=sklearn_gbdt_no_rules`` replays ML-only scoring
  in the live pipeline for qualitative comparison.
