Frontier Models are Capable of In-Context Scheming

A preprint showing that frontier models are capable of in-context scheming. The companion cites two of its findings in the observer-modeling ladder (Table 2): o1, once deceiving in an evaluation, sustains the deception in more than 85% of follow-up questions (level 3), and models find and disable their oversight (level 5). These support the reactive phenotype (P9) and studying up.

Cited in

the companion paper (Companion Table 2)

Used for

P9P11Table 2 (levels 3 and 5); P9; P11

Strand

AI safety, alignment & autonomy evaluations · Evaluation awareness, scheming & misalignment

Cite

@misc{meinke2024frontier,
  title = {Frontier Models are Capable of In-Context Scheming},
  author = {Alexander Meinke and Bronson Schoen and Jérémy Scheurer and Mikita Balesni and Rusheb Shah and Marius Hobbhahn},
  year = {2024},
  howpublished = {arXiv preprint arXiv:2412.04984},
  url = {https://arxiv.org/abs/2412.04984},
}