Audio to Markdown for RAG: Podcast & Meeting Knowledge Base
RAG over audio is hard because raw transcripts are flat. Chunking by character count breaks mid-thought, embeddings average across unrelated passages, and retrieval surfaces fragments that lose context. Convert to Markdown first and chunking by section becomes one line of code: chunks become coherent topics, embeddings cluster on meaning, retrieval surfaces complete exchanges.
Where audio RAG pipelines fall apart
Two failure modes show up immediately. First, fixed-size chunking on flat transcript text routinely splits a single explanation into two chunks, while joining the end of one subject to the start of the next. Embeddings then encode noise, because half a question plus half an answer reads as nothing in particular. Second, retrieval over those chunks surfaces fragments that the LLM can't synthesise from, because the surrounding topic context is gone.
Markdown with ## Topic [HH:MM:SS] headings solves both. Header-aware chunking respects topic boundaries. Each chunk is one subject discussed once. Embeddings encode that subject cleanly. Retrieval surfaces complete exchanges instead of orphan fragments.
The pipeline
Convert each audio file on Audio to Markdown, save the .md, then chunk and embed locally. Building a multi-source pipeline? Also convert your PDFs (PDF for RAG) and web pages (URL for RAG) so every modality reaches the vector DB through the same structured-Markdown path.
Recommended chunking
Split first by ## (topic section), then sub-split anything still over your token budget. Target 600-1000 tokens per chunk, 50-100 overlap. Keep the section heading and timestamp as chunk metadata, so your retrieval can filter by topic, by time range, or both.
Code example
# Local pipeline: load the .md you downloaded from mdisbetter.com,
# chunk by topic section, embed, upsert.
# Install: pip install langchain-text-splitters
from langchain_text_splitters import MarkdownHeaderTextSplitter, RecursiveCharacterTextSplitter
# 1. Load the converted transcript
with open("meeting-2026-01-15.md", "r", encoding="utf-8") as f:
md_text = f.read()
# 2. Split by section headings, each chunk is one topic section
md_splitter = MarkdownHeaderTextSplitter(headers_to_split_on=[
("##", "section"), # ## Pricing objections [00:14:22]
("###", "topic"),
])
turn_chunks = md_splitter.split_text(md_text)
# 3. Sub-split any over-budget sections, preserving section metadata
char_splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=80)
final_chunks = char_splitter.split_documents(turn_chunks)
# Each chunk now has section metadata you can filter on at query time.
# Upsert to your vector DB of choice (Pinecone, Chroma, Weaviate, Qdrant).
Frequently asked questions
Why is a flat-text transcript bad for RAG?
Because chunking by character count slices through topic boundaries, joining half of one answer to half of the next question. Embeddings encode that noise; retrieval surfaces fragments that lose conversational context. Structured Markdown lets you chunk on section headings instead, which produces coherent embeddings.
How should I chunk audio Markdown for retrieval?
Split on ## headings (each topic section becomes one chunk), then sub-split any over-budget sections with a recursive character splitter. Keep the section heading and timestamp as chunk metadata, so retrieval can filter by topic, time range, or both.
Can I build a podcast knowledge base this way?
Yes. Convert each episode to Markdown, chunk by topic section, embed, store with episode title and timestamp metadata. Queries like "find all episodes that discussed X" become trivial, and every hit comes back with the timestamp so you can jump to the moment in the audio. The transcript carries no speaker labels, so "find where the host pushed back" is not a query it can answer.
What about meeting recordings across a whole quarter?
Same pattern, scaled. Convert each meeting, tag chunks with meeting date and project metadata, embed all of them in one vector DB. Cross-meeting queries ("how has our position on pricing evolved this quarter?") return time-ordered chunks tagged with their source talk. This is the highest-value use of RAG for ops and product teams.
Pinecone, Chroma, Weaviate, Qdrant: does the vector DB matter?
Not much, given clean structured input. All four handle the section-chunk-with-metadata pattern equally well. Pick on operational concerns (managed vs self-hosted, query language preferences, cost). Pinecone for managed simplicity, Chroma for local development, Weaviate for hybrid retrieval, Qdrant for filtering performance: all work.