End-to-end information extraction in handwritten documents: Understanding Paris marriage records from 1880 to 1940

Thomas Constum (1, 2) , Lucas Preel (1, 2) , Théo Larcher (1, 2) , Pierrick Tranouez (1, 2) , Thierry Paquet (1, 2) , Sandra Brée (3)

1 LITIS - Laboratoire d'Informatique, du Traitement de l'Information et des Systèmes
2 UNIROUEN - Université de Rouen Normandie
3 LARHRA - LAboratoire de Recherche Historique Rhône-Alpes - UMR5190

The EXO-POPP project aims to establish a comprehensive database comprising 300,000 marriage records from Paris and its suburbs, spanning the years 1880 to 1940, which are preserved in over 130,000 scans of double pages. Each marriage record may encompass up to 118 distinct types of information that require extraction from plain text. In this paper, we introduce the M-POPP dataset, a subset of the M-POPP database with annotations for full-page text recognition and information extraction in both handwritten and printed documents, and which is now publicly available. We present a fully end-to-end architecture adapted from the DAN, designed to perform both handwritten text recognition and information extraction directly from page images without the need for explicit segmentation. We showcase the information extraction capabilities of this architecture by achieving a new state of the art for full-page Information Extraction on Esposalles and we use this architecture as a baseline for the M-POPP dataset. We also assess and compare how different encoding strategies for named entities in the text affect the performance of jointly recognizing handwritten text and extracting information, from full pages.

Mots clés: handwriting recognition named entity recognition information extraction document understanding; handwriting recognition; named entity recognition; information extraction; document understanding

Domaines: Intelligence artificielle [cs.AI]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Thomas Constum, Lucas Preel, Théo Larcher, Pierrick Tranouez, Thierry Paquet, et al.. End-to-end information extraction in handwritten documents: Understanding Paris marriage records from 1880 to 1940. 2024. ⟨hal-04565198⟩