3

My question is primarily: is there any ML research paper about splitting a pdf containing a batch of scanned documents (eg bank statements) into individual documents?

I have searched for this but I have not found any relevant research paper or any application in general mentioned on the Internet.

I would be primarily interested in the feature engineering of these papers/applications but also in general in the whole approach.

Valentin Calomme
  • 5,396
  • 3
  • 20
  • 49
Outcast
  • 1,037
  • 2
  • 11
  • 27
  • You could try [tesseract](https://github.com/madmaze/pytesseract). You can parse image as pdf. This will give you exact locations of text, though you'll have to postprocess it later manually. – Piotr Rarus Dec 10 '19 at 10:03
  • Thank you for your comment @PiotrRarus-ReinstateMonica. However, if I am not wrong, this is only about reading the text not splitting it per document which is my actual question ;) – Outcast Dec 10 '19 at 13:43
  • Do you see the sheet edges on scans? – Piotr Rarus Dec 10 '19 at 13:45
  • Hm @PiotrRarus-ReinstateMonica no but I think then that I have to have a more thorough look :) – Outcast Dec 10 '19 at 13:48
  • Thank you. I am using tesseract, and am considering to build own classifier based on linguistic and format differences on subsequent pages. However, if something already existed, it would be preferable. Documents are highly heterogeneous. – tsttst Dec 10 '19 at 14:33
  • @tsttst, to be honest in the end any pre-trained/constructed classifier will be trained on too big variety of documents or too specific kind of documents and it may not work well with your own documents (which may be for example only financial documents). However, by coincidence something off the shelf could work for you. Personally, I created this post not necessarily to use a tool like that which may exist but primarily to explore its logic and give some ideas to myself about some useful features for my own classifier ;) – Outcast Dec 10 '19 at 15:42
  • @Poete As main question / title would be identical, I refrained from opening a separate question and placed the bounty. – tsttst Dec 10 '19 at 16:41
  • @tsttst, sure that's good, I agree :) – Outcast Dec 10 '19 at 16:48

2 Answers2

2

"Machine Learning for Digital Document Processing: from Layout Analysis to Metadata Extraction" by Esposito, Ferilli, Basile, and Mauro goes into detail about how to create a custom system for parsing digital documents, including pdfs. It proposes a generalized process to learn any structure within documents.

Brian Spiering
  • 20,142
  • 2
  • 25
  • 102
0

After Working a lot with pdf like documents bank statements and the sorts the 3 major conference/workshops you want to see is ICDAR, DocEng and the Document Intelligence workshop at NeuralIPS.

The chance of the paper you are looking for not being here is really low as these 3 are probably the biggest document research places. I have been going through the for a few months now and I cant find a mention of what you are looking for.

A simple model that classifies pages based on text on the page as ending page might work but no guarantee.

Such topics are an important part of Machine learning with documents research.

Shivam Patel
  • 26
  • 3