Open-weight model safety
What actually survives fine-tuning, quantization, and open release, once the deployment-time guardrails are gone.
AI Safety Researcher · ML Engineer
Andreas Hermann
I'm an ML engineer and researcher moving into AI safety full-time. I did a PhD, then spent three years taking machine-learning systems into production. Now I work on two things: keeping open-weight models safe after they're released, and the misalignment that shows up when you wire models together into agents.
Now
I'm doing independent AI safety research, funded by a transition grant that runs through April 2027. Until early December I'm also a research fellow at the Singapore AI Safety Hub (SASH). My project there asks what makes open-weight AI agents break the rules they were given, for example by gaming a task they cannot solve or by taking actions nobody approved. I'm building a tool that turns real incidents into evaluations. It rebuilds the setting of an incident as a test environment, so that different models can be run through the same situation and compared. Alongside it, I test which deployment conditions (such as oversight or high stakes) change how often agents do this. I'm also finishing a write-up of an inoculation project with Safe AI Germany.
I'm open to research collaborations, especially where the question is whether safety holds up in deployment.
as of November 2026
The bet
I spent years keeping machine-learning systems running in production. That work shaped how I think about safety. I've watched which safety properties hold up once a model leaves the lab, and which ones quietly fall apart, and I want to work on that gap.
There's real research behind it too. I did a PhD, published 20+ peer-reviewed papers (490+ citations), and spent the last year properly upskilling through ARENA, the AI Alignment Research Fellowship, and BlueDot. Read the longer version →
Career
-
2026
AI Safety Research Fellow · Singapore AI Safety Hub (SASH)
Propensity evaluations of open-weight AI agents: which deployment conditions, such as oversight, permissions and irrelevant noise, make them take unsanctioned actions.
-
2026 – present
Independent AI Safety Researcher
Full-time research on open-weight model safety, compositional misalignment, and interpretability, backed by a BlueDot Impact transition grant.
-
2026
AI Safety Research Fellow · Safe AI Germany (SAIGE)
Inoculation against model poisoning: training a model so its safety properties can't just be fine-tuned away later.
-
2026
Facilitator · BlueDot Impact
Teaching technical AI safety cohorts and contributing to the alignment curriculum.
-
2023 – 2026
ML / AI Engineering Lead
Three years taking machine-learning systems into production at scale, across classification, data extraction, and data-quality models.
-
PhD
Doctorate · University of Münster
Information systems. REACH Award for the dissertation.
Research focus
Compositional misalignment
Why alignment tested on single models fails to compose once you chain them into multi-agent orchestrations, tool chains, and memory-augmented agents.
Interpretability & evaluation
The measurement tooling for safety: probes, evals, and mechanistic analysis that hold up in a real deployment.
Selected publications
-
2026
Differences in Alignment Behavior between Single-Agent and Multi-Agent LLM Systems
Recent writing
- 2026-08
- 2026-06
- 2026-05
Reading list
I curate and annotate papers, courses, and tools for AI safety researchers and engineers crossing over from adjacent fields.