Think Build Implement Repeat
London, UK +44 7367 067226
WhatsApp FOLLOW f in X
  1. Home
  2. Blog
  3. Turning Files Into Structured Data
Python & Django

Turning Files Into Structured Data

Extracting data from PDFs, spreadsheets and scanned images with Python, why validation matters more than extraction, and routing unclear results to staff.

Updated 2 min readBy SpiderHunts Technologies

Free estimateNo obligation

Get a free estimate

Tell us what you need. A senior engineer reads every enquiry.

Takes under a minute. We never share your details.

  • Free consultation
  • No commitment
  • NDA on request

Prefer to talk? Book a free 30-minute call →

Quick answer — TL;DR

Text-based PDFs and clean spreadsheets extract reliably. Scans and photographs need OCR and produce variable results, which is why validation matters more than extraction.

Know what you are dealing with

SourceDifficulty
PDF with real textStraightforward
Well-structured spreadsheetStraightforward
PDF with tablesModerate — layout matters
Scanned documentNeeds OCR, quality varies
Photograph of a documentHardest — lighting and angle

The first question in any document project is what proportion of the input falls into each category. That determines the achievable accuracy more than anything else.

Validation matters more than extraction

Extraction alone might reach ninety per cent on mixed documents. Extraction with validation reaches the same accuracy and knows which ten per cent it got wrong, which is what makes it usable.
  • Do the line items sum to the stated total?
  • Does the supplier exist in your records?
  • Is the date plausible?
  • Is this document a duplicate?
  • Are required fields present?

Route the uncertain to a person

Anything that fails validation, or where extraction confidence is low, should reach a review queue rather than being written as though it were certain.

That review interface determines whether the system saves time. If checking takes longer than keying, people stop using it.

Spreadsheets are messier than they look

  1. Merged cells that break row alignment
  2. Several tables on one sheet
  3. Headers that repeat partway down
  4. Numbers stored as text
  5. Formulas whose values matter, not the formulas

Each is common and each needs handling explicitly rather than assuming a clean rectangular table.

Preserve the source

Always keep the original document alongside the extracted data, linked. When a value is questioned six months later, the answer is in the source.

It is also what makes the audit conversation short, and it costs almost nothing in storage.

FAQ

Frequently asked questions

The questions readers ask us after this guide.

Still have a question?

Ask us directly — a senior engineer will get back to you.

Ask about your project

How accurate is document extraction?

Eighty-five to ninety-five per cent per field on text-based documents with good validation. Considerably lower on poor scans.

Does AI improve this?

Substantially for varied layouts, where rules-based extraction struggles. Validation is still what makes it reliable.

How many samples do you need?

A hundred representative documents including the awkward ones. Clean samples from one supplier tell you very little.

What about handwriting?

Poor results in general. Route handwritten documents to a person by default rather than attempting extraction.

Keep reading

More on Python & Django

Python & Django

An API Other Systems Can Depend On

Designing a Python API service others can depend on: validation at the boundary, consistent errors and status codes, early versioning and documentation.

Python & Django

Moving and Transforming Data Reliably

Building data pipelines in Python that cope with malformed input: restartable stages, quarantining failures, reconciling counts and alerting on absence.

Start here

Someone keying documents all day?

Send us thirty representative examples and we will tell you honestly what share could be automated.

  1. You tell us what you needTwo minutes on the form, or a message on WhatsApp.
  2. A senior engineer reviews itAnd comes back with questions, a realistic range and an honest view on fit.
  3. Free 30-minute scoping callWe talk through scope, options and a realistic estimate — with no obligation.
Free estimateNo obligation

Talk to someone who builds this

Send a short brief and we will come back with an honest view and a realistic range.

Takes under a minute. We never share your details.

  • Free consultation
  • No commitment
  • NDA on request

Prefer to talk? Book a free 30-minute call →