mediumRAGChunking
Chunk Text with Overlap
Split a document into word chunks of a fixed size with an overlap between neighbours, the first step of every RAG indexing pipeline.
JavaScript or Python
Before documents are embedded for retrieval-augmented generation, they are split into chunks, and neighbouring chunks share a few words so that an idea on a boundary isn't cut in half.
Given a text, a chunk size (in words) and an overlap (in words), split the text on whitespace and return the list of chunks, each joined back with single spaces. The first chunk starts at word 0; each next chunk starts size - overlap words after the previous one. Stop after the chunk that reaches the last word. An empty or all-whitespace text gives an empty list.
Examples
Example 1
Input: text = "a b c d e", size = 2, overlap = 0 Output: ["a b", "c d", "e"] Why: No overlap: consecutive pairs, the last chunk is shorter
Example 2
Input: text = "a b c d e", size = 3, overlap = 1 Output: ["a b c", "c d e"] Why: Each chunk repeats the last word of the previous one
Constraints
0 <= number of words <= 10^51 <= size <= 10^40 <= overlap < size
Hints
Hint 1
Split on any whitespace, so repeated spaces and newlines don't create empty words.
Hint 2
Advance the start by size - overlap, and stop as soon as a chunk reaches the end.
Solution
Show the solution (try it yourself first)
Split the text into words, then slide a window of size words with a step of size - overlap. The important detail is the stopping rule: once a chunk reaches the last word, stop, otherwise you emit a final chunk that is entirely overlap and adds nothing.
def chunk_words(text, size, overlap):
words = text.split()
chunks = []
start = 0
while start < len(words):
chunks.append(" ".join(words[start:start + size]))
if start + size >= len(words):
break
start += size - overlap
return chunksComplexity
Time O(n × size / step) to build the strings, which is O(n) for a fixed overlap ratio; space for the output. Real pipelines usually chunk by tokens rather than words and prefer to break at sentence or heading boundaries, but the window-and-step logic is the same. Interviewers often ask how you would pick the size and overlap: test retrieval quality on real questions rather than guessing.
Found a mistake in this problem, its tests or the solution?