Document Layout Analysis and Structure Recognition

Document Layout Analysis (DLA) and Structure Recognition address the automated interpretation of document images by identifying and organising their constituent elements. At its core, DLA segments a page into zones such as text blocks, tables, figures and decorative graphics. Structure Recognition then infers the logical relationships between these zones, reconstructing reading order, hierarchical headings and semantic groupings. Traditional methods relied on hand-crafted features and rule-based heuristics, but the explosive growth of deep learning has led to a shift towards convolutional and transformer-based architectures. These models leverage large annotated corpora and pre-trained language models to perform end-to-end tasks, from block detection to cell-level content extraction. Benchmarks such as ICDAR, PubLayNet and TableBank have spurred innovation in table detection and structure parsing, while human-in-the-loop approaches have improved labelling efficiency and robustness. Emerging techniques include quickest change detection to monitor layout drift in dynamic document streams and hierarchical encoding schemes that capture nested layout groups. Advances in dataset curation—especially multilingual and irregular-layout collections—have broadened applicability to diverse domains, from financial reports and historical archives to government filings. Collectively, these developments are forging a more accessible and interoperable digital document ecosystem, with downstream impact on information retrieval, digital preservation and assistive technologies.

Research from Nature Portfolio

No recent Nature Portfolio content available.

Research from all publishers

Continuous document layout analysis has been advanced through a semi-automatic human-in-the-loop pipeline that generated a novel public affairs database comprising over 37 000 documents and 8 million labelled layout units. This work demonstrated near-perfect labelling accuracy and introduced quickest change detection techniques to monitor layout variability across sources in real time. A large-scale end-to-end table recognition dataset has addressed the scarcity of multilingual benchmarks by providing 38 000 tables in English and Chinese, complete with polygonal annotations for table bodies, cell boundaries and logical structure. This resource supports simultaneous table detection, structure recognition and content extraction in the wild. In scientific publishing, methods that incorporate explicit Visual Layout Groups (VILA) into transformer-based language models have yielded significant improvements in token classification accuracy and inference efficiency. By marking text-block and line boundaries and applying hierarchical encoding, these approaches deliver up to 1.9% macro-F1 gain and a 47% reduction in inference time without additional pretraining.

Document Layout Analysis and Structure Recognition publication trend

The graph below shows the total number of articles in document layout analysis and structure recognition across all publications each year (not limited to Nature Index journals).

Technical terms

Document Layout Analysis (DLA): The process of segmenting a document image into fundamental components such as text, tables, figures and graphics.

Structure Recognition: The task of inferring logical relationships and reading order among layout elements to reconstruct document structure.

Table Detection (TD): Locating the spatial region of a table within a document image.

Table Structure Recognition (TSR): Identifying the grid of rows, columns and cells and their logical relationships within a detected table.

Visual Layout Group (VILA): A grouping of tokens into coherent text lines or blocks used to embed hierarchical layout information into language models.

References

1. Continuous document layout analysis: Human-in-the-loop AI-based data curation, database, and evaluation in the domain of public affairs. Information Fusion (2024).
2. A large-scale dataset for end-to-end table recognition in the wild. Scientific Data (2023).
3. VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups. Transactions of the Association for Computational Linguistics (2022).