OpenDataLoader PDF: Open-Source PDF Parser for RAG Pipelines (Local, No GPU)

[00:00:01]For production use cases, rag is still alive and kicking.

[00:00:06]If you have ever tried to feed a PDF into your rag pipeline and got back garbage, today's tool is going to fix that, maybe.

[00:00:17]This new tool, which is called as Open Data Loader PDF, is an open-source PDF parser designed for AI data extraction.

[00:00:26]In this video, we are going to install it and I will show you how you can use it without any GPU, without any API cost.

[00:00:34]This is Fahad Mirza and I welcome you to the channel. Please like the video and subscribe and consider become a member.

[00:00:42]Also, please follow me on X if you're looking for AI updates. So, what exactly is this tool? As I said, it's an open-source PDF parser designed for AI data extraction. It sits at number one in benchmarks with an overall accuracy of.907 and.928 on table extraction specifically. That's ahead of DocLing, Marker, and PyMuPDF for LLM, which we have covered on the channel, too.

[00:01:12]Now, when it comes to rag, that is quite an important component in any AI production environment. Rag, which stands for retrieval augmented generation, is where you pass your own documents and feed them into an LLM so it can answer questions grounded in that context. The thing is that all of these LLMs, no matter how much big they are, they really don't know about your own personal information.

[00:01:39]If you want to provide that to an LLM, you need to first convert your own data into numerical representation after chunking it, then get it stored somewhere, and whenever a user asks a query, you need to find that similar data in your own data and then append it to your prompt, give it to LLM as context. That is what the whole rag pipeline is at very high level.

[00:02:04]The quality of your rag system depends heavily on how well you pass those documents.

[00:02:11]Most parsers lose reading order on multi-column layouts such as this one.

[00:02:19]Mangle tables or give you no coordinates to trace answers back to the source.

[00:02:25]Open data loader is trying to fix all these three. It outputs structured markdown for chunking, JSON with bounding boxes for every element so you can cite exactly where an answer came from and it runs entirely on your own machine. You don't really need any GPU, no API cost. It has Python, Node.js and Java SDK, a LangChain integration. Where I'm a bit, you know, not really happy because LangChain could be slow anyway and it's Apache 2 so free for commercial usage. Let's get it installed and I will show you how exactly you can use it.

[00:03:01]This is my Ubuntu system. I'm creating a virtual environment with conda. As I said, you really don't need any GPU, but if you want to integrate it with any local model such as from Olama, you can easily rent a GPU from MassCompute on very good price. You can find the link to their website in video's description with a discount coupon code of 50% for a range of GPUs.

[00:03:24]Okay, so let's go back. Everything is installed.

[00:03:28]Now, one thing you would definitely need in order to get this thing installed is to have the JDK and Java installed.

[00:03:37]And you can see that my Ubuntu already has this JDK and Java installed or OpenJDK.

[00:03:43]If you don't have it, just run this command and it should take care of it.

[00:03:48]After that, you can install the base package by running this Open data loader PDF command. Shouldn't take too long.

[00:03:54]It's already done.

[00:03:57]After the base is done, what you can do, you can install it in hybrid mode. So, which means that it has got now support for complex tables and scanned PDF.

[00:04:10]Everything is installed and now let's test it out. First up, what I'm going to do, I'm just going to use this um some sample corporate report PDF where we have some of the multi-column layout.

[00:04:24]Some tables are also there and there are various other data around header and footer. So, I'm just going to run it in a in a local mode first and then we are going to output both markdown and JSON.

[00:04:37]So, let's test it out.

[00:04:40]And there you go. This is a simple code where we are importing that library. We are giving it the input path of that PDF and where it needs to put that markdown and JSON format. And that's it. So, let's go back and run it from my local system.

[00:04:56]And there you go. It was almost instant.

[00:04:59]As soon as I pressed enter, it has just processed uh all of the file. So, 12 pages processed as you can see. Both JSON and markdown have been created and it was under 1 second. No GPU used. And it also detected the title Tech Corp Industries annual report as you can see. Really really good stuff. Okay, and it has created the resultant files here in the output directory.

[00:05:22]This is where my files were. This is output directory. Let's first check the JSON.

[00:05:27]There you go. So, it has converted it in all the JSON formats. This is really good.

[00:05:33]And if I go back and see the markdown, there you go. The markdown is also there.

[00:05:38]Not only it was instant and looks pretty accurate to me. I don't see any errors at all.

[00:05:46]The previous example was with local mode. Now, let me show you the back [clears throat] end mode with another example. I'm going to use this spec sheet with lot of data, some images, some tabular format, and few other things.

[00:06:02]So, if you check out this new mode, local mode runs entirely in Java on your CPU, fast but basic.

[00:06:11]Now, hybrid mode splits the work. Simple pages stay local, but complex pages like tables and scanned content get routed to an AI back end running locally on your machine. And you can even put it on your network. Your application through API can connect to it on local server on port 5002.

[00:06:28]And your PDF parser talks to it automatically whenever it hits a complex page. So, you can just even mix and match that thing. So, let me run this so that it will start this back end.

[00:06:40]And there you go. So, the back end has started, and now it is running on our local host. I will just let it run, and I will show you the code in another terminal.

[00:06:50]And again, the code is very similar. You just provide it input file and output directory, and then give it the format, but you must specify hybrid mode, which is Docling Fast.

[00:07:01]And you can also specify your port, but I'm not doing it. Uh but if you're using it on any other port, you know, other than 5002, then you need to specify this. But as I am using the default one, so I'm not specifying it. Once you have done that, all you need to do is to just run this app.py.

[00:07:20]And again, very very quickly, you can see that it has processed the number of pages. It has identified those GPU cards, which I was showing in this spec sheet.

[00:07:30]And it has not only done that, so it automatically went to the Docling Fast back end, as you can see. This is a back end, Java one, but for the heavy pages, it went to the Docling fast. So, in the hybrid mode, it automatically decides which one to use for which.

[00:07:48]Okay, and it has already gone and created that directory. So, I'll just go into my output directory. This is output hybrid one. So, this is the JSON. I'll just quickly show you.

[00:08:02]Again, looks pretty good. I'll just scroll down.

[00:08:05]For images, it has given the path.

[00:08:08]And for markdown, this is a markdown. I'll make it bigger.

[00:08:14]Again, it has produced it wonderfully well. All the image paths are there. And separately, it has put the images here. And all these images, they really look uh absolutely accurate.

[00:08:29]These are the ones.

[00:08:31]There you go.

[00:08:34]So, this is open data loader PDF for you.

[00:08:38]The tool which simply works as you can see, fast, no GPU, no API. And when you need accuracy on complex tables, I think this could be a good one choice in order to create your own data sets and all that.

[00:08:51]But, one thing to notice that as the data volume grows and as complexity grows, I think it is going to struggle a bit because of the LangChain.

[00:09:01]Uh but, give it a try. Let me know how you go. I have tried it out on some of the PDFs around 1,000 pages or with some of the images, it was struggling a bit. I mean, it was not as instant as it sounds in the demo. So, make sure that you also test it if you're looking to put it in your production pipeline because the best test would be on your own data.

[00:09:22]That's it. Let me know what do you think. Please follow me on X. And thank you for all the support.