RAG Chunking Strategies Explained (With Visual Examples)

[00:00:00]In our last video, we have learned how to build a rag architecture in detail step by step.

[00:00:07]So, in this video, we will talk about the major important step inside rag building which is called as chunking.

[00:00:15]Many companies are trying to build this rag architecture but when they build this architecture, at the end, their performance will not be good.

[00:00:24]And one of the main reason of performance not getting good in rag architecture or rag solutions is because of the chunking strategies.

[00:00:34]We have many types of chunking strategies.

[00:00:36]There may be 15 or 20 of them but not everything is important for us.

[00:00:41]And one more thing that I want to talk about is chunking.

[00:00:44]It should be purely based on your data.

[00:00:47]Depending on your data, you need to decide how to do chunking.

[00:00:51]But in this video, I will give you a high level overview on the top chunking strategies and how you can use them.

[00:00:58]And before that, let me quickly revise what is a rag architecture.

[00:01:02]So, we have PDFs, data, documents, text and lot of stuff.

[00:01:08]And after that, we load all of these documents into a place, anywhere in our code.

[00:01:13]And once we load all of these documents, because we cannot afford larger context windows in LLM models, what we do is we try to chunk the data into multiple parts.

[00:01:25]Such that every chunk can become an embedding vector.

[00:01:28]So, once we get all the chunks for our data, we will try to create an embedding model.

[00:01:34]What is this embedding model?

[00:01:35]It is going to convert chunks into vectors using any kind of models.

[00:01:44]We have OpenAI embedding models, sentence transformers, free hugging phase.

[00:01:48]There are so many embedding models.

[00:01:50]Once we use any embedding model, we can create these vector embeddings.

[00:01:55]And to store these vector embeddings, we will use a vector store or vector database.

[00:02:01]This is where we store what is the chunk information that we have in the third step.

[00:02:06]And along with the vector embedding that we generate using the vector embedding models.

[00:02:10]And we will talk about embedding models in detail in the upcoming video.

[00:02:14]So, let us not worry about that right now.

[00:02:16]Once we complete this process, you finally have a vector store in which we have chunks.

[00:02:22]Along with that, we have vector embeddings for those chunks.

[00:02:26]And whenever you want to retrieve the answer for a question, let us say a user asks us a question.

[00:02:31]And using the same embedding model, you will try to convert that question into vector embeddings.

[00:02:39]And once that vector embedding comes, you will try to go to the vector data store.

[00:02:44]And after seeing all the embeddings that we have stored, we will try to find the top similar chunks.

[00:02:49]Telling what are the similar chunks for our question.

[00:02:52]And taking those chunks, you will build the context.

[00:02:55]And taking this context, you will pass it to an LLM model with a prompt.

[00:02:58]Here you can use any LLM model.

[00:03:01]ChatGPT, Claude, Grok or anything.

[00:03:04]And once you are done with this, this LLM will generate you the answer.

[00:03:07]Which is grounded and which is actual data from the documents.

[00:03:11]It is not even hallucinating anywhere.

[00:03:13]But if we want this answer to be good, we need to have these chunks better quality.

[00:03:20]And how to do those chunks, we are going to talk about in this video.

[00:03:24]There are so many chunking strategies, but we will not be able to cover every single thing.

[00:03:28]Let us say there are these top chunking strategies.

[00:03:31]One is fixed size chunking.

[00:03:33]And second is fixed size chunking with overlap.

[00:03:37]And third is sentence based chunking.

[00:03:40]And fourth is paragraph based chunking.

[00:03:42]And then there is recursive chunking and semantic chunking.

[00:03:46]Usually 90 % of the cases, you are good with recursive chunking or the semantic chunking.

[00:03:52]Because they will try to capture as much meaningful information as possible when you are chunking.

[00:03:58]Let us go into each and every one of these processes and understand it more deeply.

[00:04:03]So first let us talk about fixed size chunking.

[00:04:06]So let us say this is all my data, which is a single page in multiple documents.

[00:04:10]I may have hundreds of documents.

[00:04:12]And let us understand the chunking strategies using this single page.

[00:04:18]What I will do is I will try to fix the chunk size.

[00:04:26]Maybe let us say I want to have a chunk size of 500 tokens.

[00:04:30]The meaning of this is whenever I try to split these sentences or split this text into multiple chunks.

[00:04:38]What I will do is I will try to chunk at a limit of 500 tokens.

[00:04:43]That means by mixing all the text in this, they should be less than or equal to 500 chunks.

[00:04:49]I don't want to go over that.

[00:04:51]And from where 500 chunks will end, I will create one more chunk.

[00:04:56]And even this chunk should also be around 500 tokens.

[00:05:02]And from where this will end, next we will create one more chunk.

[00:05:07]So like this, in every page, in every page, first 500 is one chunk.

[00:05:12]Next 500 tokens is another chunk.

[00:05:14]Next 500 tokens is another chunk.

[00:05:16]So like this, I will create multiple chunks.

[00:05:19]And I will do this process for all the documents.

[00:05:22]And this is what we call it as a fixed size chunking.

[00:05:26]The problem with this is that it is not able to capture the data more correctly.

[00:05:32]Because let us assume a scenario where all this is one paragraph.

[00:05:37]All these paragraphs will give a single meaning to our text.

[00:05:41]But depending on the limit of 500 tokens, you are going to break in between.

[00:05:46]Meaning, when you are creating vector embeddings, you are creating a vector 1 for this one.

[00:05:51]And you are creating a vector embedding for this one.

[00:05:54]So here, a single paragraph which is bringing some importance meaning, you split it.

[00:06:00]When you split it, the vector embeddings are going to also get split into multiple things.

[00:06:04]And when you are trying to retrieve, maybe you are only retrieving the vector 1, not the vector 2.

[00:06:09]So in such cases, the retrieval quality is not good.

[00:06:12]But there are some pros as well.

[00:06:14]Because it is a very easy one, you can do it as simple as possible.

[00:06:18]It is very simple.

[00:06:21]And it is easy to implement as well.

[00:06:27]Because of this fixed size chunking, you might lose context if the data is complex.

[00:06:39]So this is the problem with fixed size chunking.

[00:06:41]And next, let us talk about the other one, fixed size chunking with overlap.

[00:06:46]And why is it needed?

[00:06:49]Earlier we thought, I will break it like this under multiple chunks in fixed size.

[00:06:53]But when you are breaking it like this, you are abruptly cutting the paragraph here.

[00:07:00]Cutting the text here.

[00:07:04]So by cutting it like this, you are not going to gain more information.

[00:07:09]Sometimes, you might want the information from the previous chunk to be overlapped in the current chunk.

[00:07:16]Because you can have better semantic or retrieval quality.

[00:07:21]So for that, we have this technique which is called as fixed size chunking with overlap.

[00:07:26]What does this do?

[00:07:27]Whenever you are creating two different chunks, let us say chunk 1 and chunk 2.

[00:07:33]And whenever you are creating these kinds of chunks, I will have some chunk overlap.

[00:07:38]This part is in chunk 1 and chunk 2 as well.

[00:07:44]See, this is chunk 1 and this is chunk 2.

[00:07:47]There is an overlap.

[00:07:49]When you are creating vector embeddings for chunk 1, V1 and V2, they both can have some common vectors in between.

[00:07:57]Such that whenever you are retrieving, you might retrieve either from the chunk 1 or chunk 2.

[00:08:02]That can have overlap of some other techniques as well.

[00:08:06]So this is also a better option compared to the first one.

[00:08:09]Because you are trying to improve the context.

[00:08:14]Improve the context and it will improve the retrieval quality.

[00:08:26]But the cons is that because you are doing an overlap, you might need to create more chunks.

[00:08:32]And if you have more chunks, you are going to create more embeddings.

[00:08:36]And those embeddings need to be stored in a database.

[00:08:38]So you need a lot of vector embeddings to be created when you are doing this kind of chunking strategy.

[00:08:45]And because of this, you might need to pay for the storage and indexing as well.

[00:08:49]And the next one, let us talk about sentence based chunking.

[00:08:53]This is also fine. It is not too good.

[00:08:55]What this will do is, let us say we have a chunk limit.

[00:08:58]And in chunk limit, let us do it at 500 limit.

[00:09:02]Every chunk should have 500 tokens.

[00:09:03]So what it will do is, first it will break the chunk into 500 tokens approximately.

[00:09:12]And after that, internally the text that is present inside the chunk 1, it will break it at sentence level.

[00:09:21]Chunk 1 and this is second one, third one and fourth one.

[00:09:26]So this is a single chunk.

[00:09:28]And in that single chunk, you have multiple sentences.

[00:09:31]And all of these sentences, you will create vector embeddings.

[00:09:36]So what this will do is, it will preserve the complex sentence relations and it will provide the readability as well.

[00:09:46]And sometimes, because you are chunking sentence based, there is a lower chance of you cutting the data in between.

[00:09:53]The next one is basically the paragraph based chunking.

[00:09:56]This is a good one.

[00:09:57]So whenever you can use this, when you are working on novel or books, because when you are looking at story books or novels, a paragraph contains a more semantic meaning.

[00:10:09]Isn't it?

[00:10:10]So cutting the paragraph in between doesn't make any sense.

[00:10:14]So in that scenario, you take one paragraph and you create that as one chunk.

[00:10:19]Paragraph is small and big.

[00:10:21]You take one paragraph and you will keep it as a chunk.

[00:10:25]Like that paragraph 2 as another chunk.

[00:10:27]Paragraph 3 is another chunk.

[00:10:29]Paragraph 4 is another chunk.

[00:10:30]So you are creating chunks based on paragraphs, not on the chunk size and limits.

[00:10:41]So what happens is, it will maintain logical structure and context more clearly.

[00:10:56]Cons is, sometimes the paragraph can have multiple pages.

[00:11:00]In case, in such situations, the vector embeddings might not capture the entire details of the paragraph.

[00:11:08]If paragraph is large, then vector embeddings might not capture full details.

[00:11:24]What can you do in such situations?

[00:11:26]You can mix this with the sentence based chunking.

[00:11:28]So first split the paragraph.

[00:11:31]Then try to split the data into paragraphs and you create one more chunk.

[00:11:36]So there is no dedicated answer to how to do chunking.

[00:11:41]No.

[00:11:42]You have to understand the data and decide how to do chunking.

[00:11:48]Then you can think about all of these different techniques.

[00:11:52]I know chunking techniques.

[00:11:54]So I will go ahead and use these chunking techniques for my use case.

[00:11:57]What is your data?

[00:11:59]Think about how much chunking you need for that data and design your own chunking strategy.

[00:12:06]If all these work, but if it is not working, then I think designing your own chunking strategy is much better actually.

[00:12:12]And now let's talk about the other one which is called as recursive chunking.

[00:12:16]This is a popular chunking technique.

[00:12:20]Because it will look at the heading, there is a subheading and there is a text inside this subheading.

[00:12:29]So let me create a chunk for this one.

[00:12:32]And there are multiple parts inside another subheading.

[00:12:35]Let me create one more chunk for this one.

[00:12:37]There are multiple parts inside another subheading.

[00:12:39]So let me create one more chunk for this.

[00:12:42]What it will do is it will look like a tree.

[00:12:45]It will look at the hierarchy.

[00:12:47]This is my subheading, heading.

[00:12:49]What data is there inside the heading?

[00:12:51]What data is there inside the subheading?

[00:12:52]I will try to create chunks based on the data that I have.

[00:12:56]And usually recursive chunking works really good.

[00:13:00]And it will preserve document structure and context as well.

[00:13:11]And because when we do such things, when you convert vector embeddings, they capture more semantic meaning.

[00:13:21]More semantic meaning.

[00:13:25]And usually, Elantic chunking strategy works with any kind of documents.

[00:13:30]And let's talk about the most popular one which is called as semantic chunking.

[00:13:34]Which is also called as meaning -aware chunking.

[00:13:37]Why are we doing chunking?

[00:13:39]Because we are going to convert them into vector embeddings.

[00:13:42]And vector embeddings should carry the semantic meaning.

[00:13:45]In such cases, if we even do the chunking that can do semantically, so why not use that technique?

[00:13:52]Yes, 95 % of the cases you are good to go with semantic chunking.

[00:13:58]But as I said, based on your data, you need to decide what kind of chunking you want.

[00:14:03]What this semantic chunking will do is it will look at data.

[00:14:07]This paragraph has a meaning.

[00:14:10]Or this sentence has a meaning.

[00:14:11]Or maybe this sentence has a meaning.

[00:14:15]So it will find that data and it will try to create a chunk.

[00:14:20]And second one, maybe all of this is for solar energy.

[00:14:24]It will create one more chunk for solar energy.

[00:14:26]And the third one is for wind energy.

[00:14:28]It will create for wind energy chunk.

[00:14:30]So looking at the semantic meaning of the data, it is going to create the chunks.

[00:14:35]And that chunks will carry more semantic differences.

[00:14:39]The advantage of this is that you are able to capture or retrieve the information more reliably.

[00:14:45]It is going to do time taking chunking.

[00:14:49]Because you need to understand the data first and then split it.

[00:14:52]If I have like million documents, then it is going to take multiple hours if not days for you to complete this type of chunking.

[00:15:00]Because it is computationally expensive.

[00:15:04]And the boundaries, chunks, everything is random.

[00:15:07]Not random actually based on the semantic meaning.

[00:15:10]But there is no fixed stuff.

[00:15:12]So you can do this one.

[00:15:14]But it is going to be computationally expensive.

[00:15:17]And you need to wait a lot of time for it to complete.

[00:15:20]So these are all the top chunking strategies that I want to talk about.

[00:15:23]You can ask Mr. Balaji now that we know these chunking strategies, how to implement these things.

[00:15:29]Yes, that is an interesting question.

[00:15:30]So for that, we have a framework which is called as LangChain.

[00:15:35]What this LangChain does is, it will be able to provide us all the existing packages.

[00:15:43]Where you can just call the chunking strategy and it will take care of it.

[00:15:49]If you want, you can build it from scratch.

[00:15:51]And that is what exactly we are going to do on this Sunday live on YouTube.

[00:15:56]We are doing 2 hour master class on building a rag from scratch.

[00:16:01]We will not be using LangChain guys.

[00:16:03]We will build it from scratch.

[00:16:05]And then maybe later on, we will try to use LangChain to solve the same problem as well.

[00:16:10]What happens when you build it from scratch is, you will understand the details more properly.

[00:16:15]Step by step, what to do, how to do it.

[00:16:18]And once you understand this, using LangChain is just a piece of cake.

[00:16:21]So if you want to learn how to build the entire rag from start to end.

[00:16:28]So this is the one.

[00:16:29]From start to end, if you want to understand how it is being built.

[00:16:34]Make sure that you are joining or attending the Sunday live.

[00:16:37]I have scheduled it on my channel actually.

[00:16:39]You can click on this and you can wait for this one.

[00:16:42]And once the live starts, YouTube will give you notification.

[00:16:47]We are doing custom live coding.

[00:16:50]We will not do the code and we will explain the code now.

[00:16:53]We are doing the live coding actually.

[00:16:55]So yeah, that is all about the chunking strategy, Sanamata.

[00:16:58]And next, we will talk about embedding strategies.

[00:17:00]We will talk about every single part in detail.

[00:17:03]So let's not worry about that.

[00:17:04]And if you like the content, please subscribe.

[00:17:07]And if you know someone who wants to listen to this kind of content, please share the channel with them.

[00:17:11]And that's all for this one.

[00:17:13]Bye bye.