Is there a way to use the extractFileText() function (from Text Analytics Toolbox) for correctly extracting two-column text from PDF scientific articles? Or another function?

I want to use Matlab tools for analyzing a large collection of PDF scientific papers. I start with using

str=extractFileText(name_dirfile)

on a test PDF file named name_dirfile in my code, which shows me the text all in one object (so far, so good). Yet, the text is two-column, and a stricking problem is that extractFileText() does not read the text column by column, but rather line by line, namely first line of column 1, then first line of column 2, then second line of column 1, etc. The text being extracted this way, I see no simple manner of searching for N-grams, or even for single-word occurrenced (as end-of-line hyphen-split words are now difficult to recombine). Has anyone encountered this problem and found a solution ?

[dpb]

What is the output class from extractFileText? It appears, I gather, to be simply on big long string?

Is it displayed by the lines?

Attach a .mat file with the content of one for folks to poke at.

Would seem worthy of an enhancement request although that won't help in the short term...

[Patrick]

The code has simply four lines :

name_directory = "C:\Users\the_path_of_the_directory_where_the_PDF_document_is\";
name_file = "Author_2018.pdf";
name_dirfile=strcat(name_directory,name_file);
str = extractFileText(name_dirfile);

The output class of extractFileText is supposed to be a string (and I think it is, as printing "str" generates a long chain of characters in the Command Window). I also tried the function with an option

str = extractFileText(name_dirfile,'extractionMethod','article');

but the two-column aspect of the PDF remains undetected, or at least not considered.

[Walter Roberson]

Some pdf are just like that. At core, a pdf can consist of a whole series of "move to this position; draw this character vector", with barely any information about pages, and no information about paragraphs or columns.Some pdf do have information about columns; it is not immediately clear which kind of pdf you are dealing with.

[Patrick]

And yes, the output (here [str]) is simply "one big long string", starting with publication month, family name of the Author, page number, article title, etc. Things go relatively well until after the abstract, as items are well separated. The abstract itself is well read, except maybe for many missing space characters, because it is a one-column segment of the article. But as soon as the program starts reading in the two-column section (beginning with the title of the first section), things get problematic (the way I explained above).

[Patrick]

@Walter Roberson : Thanks for the info. If I infer correctly from your message, there would be nothing to do with some PDFs, because the meta-information is not available in the PDF for guiding extractFileText in doing its job. I'll use a couple of test PDF, to see if I can find some patterns (journals whose two-column structures are recognized, and journals for which it is not the case).

[Stephen23]

"but the two-column aspect of the PDF remains undetected, or at least not considered"

This task is much more complex than it appears (a small PDF joke for y'all). PDF is not a simple text container, but is actually a page-layout language that precisely define how content appears on a page. The implications of that are important.

If you're working with multi-column PDFs, simply using basic text extraction tools will likely result in:

• Text being pulled in a jumbled, non-linear order
• Columns mixing together randomly
• Loss of intended reading sequence
• Significant post-processing required to reconstruct the original layout

This is because the text is not stored in blocks/lines of contiguous text (like it might be in e.g. a markup-based file format). Those hyphens you mention might just be graphics elements with some x-y positions: how is the tool supposed to know that some graphic element at one location has anything to do with some text elements in some distant location? The point is, there is nothing that links all of the text into some order. There is nothing that requires PDF text to be easy to extract. PDFs are NOT a file format designed for data exchange!

You could try some specialized PDF Extraction Tools (and understand that these are complex tools which may still return data in ways you do not expect/want and require significant post-processing):

• use libraries like PyMuPDF, Apache PDFBox, or Tika
• these tools can preserve spatial awareness during text extraction
• they understand the coordinate-based nature of PDF layout

Or use computer vision techniques to understand page layout and content.

PDFs are complex. Treat them as intricate layout documents, not simple text files. Patience and the right tools will make your extraction process much smoother. Expecting one function to return a perfect output is unlikely to get you very far.

[Walter Roberson]

It is common for pdf extraction tools to essentially be pdf rendering engines that analyze the rendered page, rather than being tools that analyze the input pdf.

[Patrick]

@Stephen23 and @Walter Roberson : So, I now realize a PDF is less fit for automatic text extraction than what I thought. Maybe I'll have to simplify my methodology by focussing on the Abstracts, as these one-column segments are relatively well read and the required post-processing is probably doable (mainly a problem of missing space characters, in some PDFs). Thanks also for the mention of PyMuPDF and other tools, I'll look at that.

[dpb]

As other noted, it's messy trying to extract useful text from a pdf file; the rendering info is lost.

One may be better off with OCR; I've tried a lot of the so-called extraction tools and also had very little success in getting useful text documents by extracting from pdf files.

I thought just maybe there was more inside this function than those that it might actually try to process the file.

[Patrick]

@dpb : many thanks for this idea, I didn't know about this ocr() function and the Computer vision Toolbox. I'll store this idea for later, if I do not succeed in the extractFileText() way.

[dpb]

You could possibly combine the two approaches in using the OCR data to find column line-ending sequences with which to locate the column breaks in the continuous text.

[dpb]

I just saw <a promo for Swifdoo PDF Pro> $30 lifetime license if you're on Windows by any chance. I haven't used it, but it's supposed to be able to extract formatted text to Word documents via OCR...perhaps a 3rd party tool might help; it looks as though there is a free trial that could test with before committing...

Answer 1

[Aryan]

Hi Patrick,

I understand that you are trying to process two-column scientific PDFs with “extractFileText”, but are facing the issue that text is extracted in an interleaved way (line from column 1, then line from column 2, etc.). Unfortunately, this happens because PDF files do not store the “reading order” of text — they only store text objects with coordinates.

The most reliable option is to use OCR (Optical Character Recognition) with the Computer Vision Toolbox. The idea is to convert the page into an image, split it into its two columns, and run OCR separately on each column.

Steps for OCR-based extraction in MATLAB

1. Convert PDF to images

• OCR in MATLAB (”ocr” function) works on images, not directly on PDFs.

2. Preprocess the page image (grayscale, binarization, noise removal)

• Improves OCR accuracy by enhancing text contrast and removing background noise.

3. Detect the vertical “gutter” between columns

• Two-column layouts need separation; otherwise OCR reads across columns. Detecting and splitting at the gutter ensures each OCR pass processes text in the correct column order.

4. Crop the image into left and right column regions

• Running OCR separately on each cropped column guarantees the reading order is preserved (top-to-bottom in column 1, then top-to-bottom in column 2).

5. Apply OCR on each column image

6. Post-process the OCR text (fix hyphenation and line breaks)

• Cleaning these artifacts makes the output ready for search, n-gram analysis, or NLP tasks.

NOTE: Accuracy depends on image quality — scanned PDFs or low-resolution pages may need extra preprocessing (deskewing, denoising, upscaling).

This workflow can be extended to 3-column layouts, pages with mixed full-width + column sections, etc., just by adjusting the cropping logic.

Please refer to the following MATLAB R2025a documentation:

• https://www.mathworks.com/help/vision/ug/getting-started-with-ocr.html

Hope it helps!