Large Language Models Often Know When They Are Being Evaluated

A preprint showing that large language models often know when they are being evaluated. The companion cites it for models that name an evaluation's purpose, the datasphere example for level 4 of the observer-modeling ladder (Table 2), inferring the observer's criterion or purpose. It supports the reactive phenotype (P9).

Cited in

the companion paper (Companion Table 2)

Used for

P9Prediction 6Table 2 (level 4); P9; prediction 6

Strand

AI safety, alignment & autonomy evaluations · Evaluation awareness, scheming & misalignment

Cite

@misc{needham2025large,
  title = {Large Language Models Often Know When They Are Being Evaluated},
  author = {Joe Needham and Giles Edkins and Govind Pimpale and Henning Bartsch and Marius Hobbhahn},
  year = {2025},
  url = {https://arxiv.org/abs/2505.23836},
}