Understanding Document Processing

We wanted to investigate how good the current state of the art (M)LLMs are at solving the relatively simple problem of extracting revenue figures from publicly available financial reports. To test this, we created 3 different datasets, all based on the same selection of 1,156 randomly selected annual reports for the year 2023 of publicly listed US companies. The resulting datasets contain a combined total of 10,404 rows, 37,536,847 tokens and 1,156 images. For our study, we are evaluating 8 state-of-the-art (M)LLMs on a subset of 100 reports.

In the following we will be comparing the results of the Parsee Document Loader vs. the PyPDF Langchain Document Loader for various datasets. All datasets that are used here can be found on Huggingface (links below), so the results are all reproducible.

The core functionality of the Parsee Extraction Templates explained.

Parsee aims to be a simple, opinionated framework for easily structuring data from the most common sources of unstructured data. These are in our opinion: pdfs, HTML files and images.

Exploring the advantages of Parsee extraction templates over simple prompt templates.

Navigating the world of data extraction in 2024 can feel like a maze. But don't worry, we've got you covered! In this guide, we'll break down the seven top ai document processing tools out there. This article is your handy roadmap to mastering data extraction from PDFs, web pages, e-mails or images. Dive in and discover the tools that can supercharge your work!