PDF to Markdown for Researchers — Papers to Knowledge Base
A literature review is hundreds of papers, none searchable, none cross-referenced, all locked in PDF. Convert each paper to Markdown and you have a knowledge base: searchable across abstracts, cross-linkable in Obsidian, summarisable by an LLM, citation-extractable.
Why this is hard without the right tool
- Reading 200 papers means opening 200 PDFs
- Cross-references between papers require manual tracking
- Quotes for citations are copy-pasted with formatting artefacts
- AI summarisation on raw PDFs is shallow because of layout noise
- Sharing notes with collaborators means sending more PDFs
Recommended workflow
- Convert PDFs one at a time via the web UI, or script your own pipeline locally with an OSS extractor (Marker, Docling, PyMuPDF)
- Save to an Obsidian vault, one note per paper, with YAML metadata
- Use Obsidian's graph view to track citations and themes
- Feed converted Markdown to Claude Projects for chapter-style synthesis
- Export annotated notes as Markdown for collaboration — no PDF round-trip
Frequently asked questions
How do I batch convert 200 papers from arXiv?
MDisBetter is a web tool today (no public API or CLI), so the right path for true batch is local OSS: <a href="https://github.com/VikParuchuri/marker">Marker</a> in a Python loop, <a href="https://github.com/DS4SD/docling">Docling</a>, or PyMuPDF if you only need raw text. Drop the PDFs in a folder, run the script, save .md output back — typical run is 10–30 minutes for 200 papers depending on hardware. For the small handful you can't script (paywalled, oddly-formatted), use the MDisBetter web tool one at a time.
Best Obsidian setup for a literature vault?
One folder per topic, one note per paper, YAML front matter for metadata (title, authors, year, tags, DOI). Add the Citations plugin for BibTeX integration and the Smart Connections plugin for semantic search across the vault. The graph view becomes a real Zettelkasten.
Can AI synthesise across many converted papers?
Yes — paste 5–20 papers as Markdown into Claude Projects (200k context window) and ask for thematic synthesis, contradicting findings, or methodology comparisons. Same workflow works in Gemini 2.5 (1M tokens, fits 50+ papers).
How do I extract just the methodology sections?
After conversion, the methodology section has a predictable heading (<code>## Methodology</code> or similar). A 3-line script or even <code>grep</code> can pull just those sections across all your papers — useful for methodology surveys.
How do I handle citation extraction?
The bibliography survives conversion as a final <code>## References</code> section. For BibTeX, run the converted bibliography through a parser (anystyle.io, GROBID) or paste into Claude with "format as BibTeX" — both work for ~95% of standard citation styles.