OCR feature fragility

Enter a reference text and its OCR version. Each reference token is encoded by a historical NER model (BERT, en/fr/de) and a TopK sparse autoencoder (2,048 features, 32 active per token). A feature is lost when its activation on the aligned OCR token falls below half of the reference activation. Colour shows the share of a token's active features that were lost; unchanged tokens next to an edit can lose features too (collateral damage).

NER block the SAE reads

Tokens

Tokens

Most frequently lost features

Most frequently lost features
Examples (the last two are real chunks where a correctly transcribed entity still changes label)
Reference text OCR text NER block the SAE reads