Build a RAG System in R | Easy Setup, No API Key, Free & Local

[00:00:00]I have a folder on my computer with many files that contain university policies.

[00:00:07]And suppose I ask this question for offering a course in a blended format, whom should the faculty member provide rational. Let's hit enter.

[00:00:18]So it looks at those documents and then I get this answer. The faculty member should submit the pedagogical rational to the department chair as part of the department's scheduling process and also share it with the dean.

[00:00:35]So if you look on the internet, every course, every tutorial, every blog post, everywhere you'll see that for creating a rack system, you need Python. But this is totally made in R. And also there's no API, no cost involved.

[00:00:55]So how this system works is very simple.

[00:00:57]So you may have some documents like PDFs on your computer maybe in a folder. So in the next step basically it creates chunks. It splits documents into smaller pieces and then converts chunks into dense vectors of some numbers and then that's what is stored. So stores embeddings for efficient retrieval. So that is done offline and then when you're online and you're asking a question it converts a question into vector and then searches for vector store for similar vectors and then combines stop chunks and the question and generate answer using context and question using LLMs. So that gives you the final answer. Basically, LLM doesn't know your documents which could be reports, it could be policies or it could be just notes and rag basically fixes that in three steps. First, it splits the document into chunks and then converts each chunk into embeddings which is basically a vector of numbers that captures the meaning or the essence. And then second step is when you ask a question, we embed the question as well. and also find chunks whose vectors are most similar. So that's the retrieval part of the rack system. In step three, we paste those chunks in the prompt and ask LLM to answer only using the context. So that's the generation part of the rack system.

[00:02:25]That's it. So retrieval plus generation and every step in between is just math plus API calls and R is really good at those.

[00:02:40]So first we need Olama which is a free app that helps to run LLMs on the local machine. I'm going to provide a link below and it will take you over some steps that help to install Olama. Very straightforward steps and once you do that you can run library or when you run test connection to check if everything is working. So you get a response status. It says okay. And one of the models that we'll use for LLM which is freely available is lama 3.2.

[00:03:14]So I'm going to pull that.

[00:03:18]And we also need nomic-bed.ext.

[00:03:22]Now if you run list models, it will also list you like what models are available.

[00:03:28]So these are the models that I have downloaded already. And llama 3.2 2 is a very small model just 2 GB and parameter size is about 3.2 billion parameters. So note that this runs on a normal laptop you don't really need any GPUs. So this nomic embed- text so basically turns text into vectors and lama 3.2 will basically generate answers based on the question we ask. We also need two other libraries HTTR2. So you need to install that. This package basically helps to talk to Olama's local web API and PDF tools helps to extract text from PDF files.

[00:04:15]So in my downloads folder I created this folder called docs. And for this example I have added three university policy related PDF files. So approved blended teaching policy, faculty instructional responsibilities and guidelines for faculty personal actions. So we are going to work with this data. So you can see that in this fourth window in files currently I'm in downloads folder where this docs folder exists.

[00:04:51]So next I'm going to use a function called source and within quotes I'm going to provide a link which is also going to be available below this video.

[00:05:00]So basically this is the file that has all the R codes. I have divided this code into four groups chunks embeddings retrieval and generation. So you'll notice that the docs folder appears here in this first line. So here we are reading the files that are available within docs folder and then we break down the text into different chunks. So note that LLM context usually is limited. So retrieval works best with smaller and more focused pieces. So we break each document into chunks of about 500 characters with little overlap so that we don't cut sentences into half.

[00:05:39]So in this part we create the embeddings that I talked about and then we have retrieval based on cosign similarity.

[00:05:45]Basically whatever context is retrieved that is handed over to LLMs in the generation phase so that a answer can be generated. So note that we also tell here that if the answer is not in the context say you don't know. So basically it stops the rack system from hallucinations and it is not forced to provide some wrong answer. What I'm going to do is uh I will click on raw and then copy this link paste it here and run.

[00:06:22]So it runs that part and now it's ready.

[00:06:29]So let's see what is the length of the chunks.

[00:06:38]So it says 61. So let's also see dimension of the embeddings. So 61 by 728 that's the matrix that we get based on text files that we have provided. Now if we use the retrieve function and say for promotion to full professor when it goes to FC to chair. So let's see what it gives. So it exactly pulls those paragraphs that are meaningful and related to this question. So there's no keyword matching here. We we did not provide any keyword and then it is looking for the keyword but it totally looks at the context. So basically it is a semantic kind of search.

[00:07:23]Now using ask_rag let's ask this question for offering a course in blended format to whom should a faculty member provide rational. So if I run that, it provides this answer. A faculty member should submit the rational to the department chair as part of the department's scheduling process and it shall be shared with the dean.

[00:07:50]Now let me also ask this question. What is our policy on office pets? Note that uh in the policy documents there is no such policy. But let's ask this question.

[00:08:04]So we ask this question and it says I don't know there is no mention of office pets in the provided context. So basically it refuses to make things up.

[00:08:15]So what we have got is a working rack system. What we just now created is a real architecture. Same pattern that's behind enterprise level rack systems.

[00:08:26]Now you can easily extend this from PDF files to maybe HTML or word documents.

[00:08:33]So doing modern AI in R is as easy as this. If you use this and create a rack system for example a document question answer tool for your team or maybe a research assistant. So please share that in the comments. If you find this useful, don't forget to like and subscribe to the channel so that I can continue to provide more modern AI related R videos. Thanks for watching.

[00:08:59]I'll see you soon.