Evaluating and securing AI

A benchmark of how AI models handle faith, across 13 models from 6 providers. Every question runs in repeated trials, with the answer order swapped in half of them. Five judge models from five vendors score the answers, with quorum and reliability checks and confidence intervals. No model judges its own output. More than 5 million model calls to date.

Document classification that runs entirely on local hardware, built on data-classification and text-mirror. A quantized open-weight model runs on llama.cpp, reachable only over loopback, with every model and binary hash-pinned. Tested against a labeled synthetic corpus of 4,584 documents.

Reports security-relevant defects to widely used open-source projects, in the memory-safety, denial-of-service and injection classes. Remediates access-control and credential-exposure issues in production AI systems.