learn deep. ship safe.Practice testsFollow ByteMunSubscribe

mediumRAGChunking

Chunk Text with Overlap

Split a document into word chunks of a fixed size with an overlap between neighbours, the first step of every RAG indexing pipeline.

JavaScript or Python

Before documents are embedded for retrieval-augmented generation, they are split into chunks, and neighbouring chunks share a few words so that an idea on a boundary isn't cut in half.

Given a text, a chunk size (in words) and an overlap (in words), split the text on whitespace and return the list of chunks, each joined back with single spaces. The first chunk starts at word 0; each next chunk starts size - overlap words after the previous one. Stop after the chunk that reaches the last word. An empty or all-whitespace text gives an empty list.

Examples

Example 1

Input: text = "a b c d e", size = 2, overlap = 0
Output: ["a b", "c d", "e"]
Why: No overlap: consecutive pairs, the last chunk is shorter

Example 2

Input: text = "a b c d e", size = 3, overlap = 1
Output: ["a b c", "c d e"]
Why: Each chunk repeats the last word of the previous one

Constraints

  • 0 <= number of words <= 10^5
  • 1 <= size <= 10^4
  • 0 <= overlap < size

Hints

Hint 1

Split on any whitespace, so repeated spaces and newlines don't create empty words.

Hint 2

Advance the start by size - overlap, and stop as soon as a chunk reaches the end.

Solution

Show the solution (try it yourself first)

Split the text into words, then slide a window of size words with a step of size - overlap. The important detail is the stopping rule: once a chunk reaches the last word, stop, otherwise you emit a final chunk that is entirely overlap and adds nothing.

def chunk_words(text, size, overlap):
    words = text.split()
    chunks = []
    start = 0
    while start < len(words):
        chunks.append(" ".join(words[start:start + size]))
        if start + size >= len(words):
            break
        start += size - overlap
    return chunks

Complexity

Time O(n × size / step) to build the strings, which is O(n) for a fixed overlap ratio; space for the output. Real pipelines usually chunk by tokens rather than words and prefer to break at sentence or heading boundaries, but the window-and-step logic is the same. Interviewers often ask how you would pick the size and overlap: test retrieval quality on real questions rather than guessing.

Found a mistake in this problem, its tests or the solution?

Written by Published Tests checked by Deepak
Case 1Input: "a b c d e", 2, 0Expected: ["a b","c d","e"]
Case 2Input: "a b c d e", 3, 1Expected: ["a b c","c d e"]

Run checks these 2 samples. Submit checks all 7 tests, including hidden ones.

  1. Top-K Similar DocumentsRAG · Retrieval · Embeddings
    medium
  2. Maximum Subarray SumArrays · Dynamic programming
    medium
  3. Longest Substring Without RepeatsStrings · Sliding window · Hash map
    medium
  4. Group AnagramsStrings · Hash map · Sorting
    medium