Adam Noonan
when a model says 85%, does 85% happen? i work on calibrated uncertainty — models that know what they don’t know
Research
Calibration is a property of outputs: when you say 85%, does 85% happen? i work on earning that property honestly, auditing that it holds, and acting on it.
arXiv:2608.21262. Set a cutoff that 90% of a population falls below, estimated from a sample. When the observations come in groups — pupils in classrooms, sentences in articles, prefixes from one reasoning trace — the fraction below the cutoff varies more than independence implies, by a factor the paper proves and the usual design effect gets wrong. So a dataset has no single effective sample size: the same 1,000 rows carried about 217 independent observations’ worth at the median and about 621 at the 95th percentile. The number of rows belongs to the dataset. The effective sample size belongs to the analysis. Code and the neff estimator in exceedance-design-effect.
Open source, on PyPI as surestop with an Optuna pruner. Conformal risk control on the loss “good and killed”: you pick α, the rule keeps the false-kill rate at or below it. Measured on 400 real LoRA fine-tuning runs under a pre-registered protocol: the bound held on both splits and about half the evaluation compute was saved. The run also found that every kill fired at the 3rd of 50 evaluations, which is the part worth reading.
Paper submitted to arXiv. The consumer picks the coverage level; the oracle returns the band that empirically delivers it, with a receipt anyone can re-derive from public data. 12 years of weekend and overnight gaps. Reference implementation in soothsayer.
In progress. When a minibatch holds several examples from the same document or author, how many independent examples is it worth? Measured on the correlation between the examples’ gradients, not their inputs, and tracked as the run trains.
In draft. When calibration-set membership depends on the score being calibrated, the correction weighted conformal applies to covariate shift doesn’t exist. What replaces it is a sharp interval.
Fine-tuning a model to carry a person’s idiolect while it learns none of their private facts — measured with extraction canaries on one side and authorship embeddings on the other.
Models represent internally that their evidence doesn’t cover a question, and answer anyway. Reading that signal so every answer is grounded or escalated, under an error budget you set instead of hope for.
Writings
Occasional posts on what i’m building and why.
A joke ASR benchmark for fast rap — and the discovery that the models had memorized the answers.
Ditching a dictation subscription for Hex + Yapper — voice in, voice out, nothing leaves the machine.
About
Director of Data & AI at Digital Locke — individualized digital twins, and the calibration research that keeps one honest.
Co-founded two AI startups in Spain as CTO and wrote the production code for both: Samachi (acquired) and helloDoJo (bought out).
Before that, five years in enterprise data at NetApp, Celonis and Lenses.io, designing streaming architectures for Fortune 100 companies.
Taught AI engineering at Immune Technology Institute, Madrid.
Into meditation, centering prayer, kettlebells, ML & quant research, hiking, camping & SUP.
Walked the Camino de Santiago. Speaks English and Spanish.
Background
Education
UT Austin MS · ECU BBA
Side Projects
Experiments i’m building in public.
Conformal early-stopping for training runs — pip install surestop, with an Optuna ConformalPruner and a script that reproduces the 400-run measurement exactly.
Calibration-transparent fair-value oracle for tokenized RWAs on Solana, with per-regime audit receipts.
Manifest-driven Rust data platform feeding soothsayer — versioned, deduped parquet for crypto, oracle, and market-data research.
Open-source autonomous docking stack for single-engine sailboats — simulation, control, and retrofit hardware design.
Open Source
Codebases i maintain & put out in the open — libraries i needed that didn’t exist.
Pure-Rust, no-RPC swap math for Raydium’s concentrated-liquidity AMM — the missing Solana analogue of uniswap_v3_math.
Pure-Rust, no-RPC swap math for Meteora’s bin-based DLMM — closing the last math-crate gap among Solana’s major liquidity models.
Elixir SDK for Restate’s durable execution runtime, on protocol V5 and V6. Passes all 202 of Restate’s official SDK conformance tests.