Convert GitHub Documentation to Local Markdown Files

Find the documentation source first. A README may already be Markdown, a GitHub wiki has its own Git repository, and a rendered site may need HTML extraction. Use original files for fidelity and URL conversion for public rendered text. Private repositories require your authenticated local Git workflow.

README and repository docs: download the source

Use GitHub's Raw control or clone the repository and keep its documentation folder. Preserve Markdown, code fences and source diagrams without reconverting HTML. Record the repository, branch and commit so you know which version you saved.

Retain assets and relative paths: README.md alone may contain broken image or document links. Rewrite a copy if the destination needs different links. The Obsidian workflow covers captured notes in a vault.

Wiki: clone its separate repository

GitHub's wiki documentation explains local wiki cloning and supports several markup formats. Use git clone with the wiki clone URL shown by GitHub (the repository name ends in .wiki.git). Check extensions: not every page is Markdown.

For a private wiki, use Git credentials authorized for that repository. The URL converter does not inherit your browser session. For one public wiki page, URL to Markdown captures fetched text; inspect links and assets afterward.

Rendered docs: convert public HTML

1. Open URL to Markdown and sign in or create a free account. Conversions use credits.
2. Paste a public URL, or one per line for several pages.
3. Convert and inspect before downloading.
4. Compare a heading, code sample, table and final paragraph with the source.

Crawl mode follows eligible links within configured limits. Record pages actually returned: success does not prove complete site capture. Fetching does not execute JavaScript. Empty shells and access walls require another source path.

Local script for a public URL list

This example uses Requests and Trafilatura to fetch HTML without executing JavaScript. Install both packages. A new output directory prevents overwriting snapshots; URL digests reduce collisions between pages sharing a final path segment.

from pathlib import Path
from hashlib import sha256
import requests
import trafilatura

urls = ['https://docs.github.com/en/rest/repos/repos']
out = Path('docs-snapshot')
out.mkdir(exist_ok=False)
for url in urls:
    response = requests.get(url, timeout=30)
    response.raise_for_status()
    md = trafilatura.extract(response.text, output_format='markdown',
                            include_links=True, include_tables=True)
    if not md:
        raise RuntimeError(f'No extracted text: {url}')
    name = sha256(url.encode('utf-8')).hexdigest()[:16] + '.md'
    (out / name).write_text(f'<!-- source: {url} -->\n\n{md}',
                           encoding='utf-8')

Keep a URL-to-file manifest. A login page can return HTTP 200; check expected content too. For browser-rendered pages, capture the browser content or accessible source before extraction. See website to Markdown for RAG.

Check code, diagrams and cross-links

Rendered Mermaid SVG does not guarantee recovery of its source. Prefer original fences. Inspect language labels, table columns, collapsible sections and relative links. Markdown permits HTML, so support can vary by renderer.

GitHub Pages may originate in Markdown, MDX, reStructuredText or another format. Check source first. Convert rendered pages when that is the useful accessible representation.

Prepare a useful corpus

• Keep source URLs and snapshot identifiers.
• Separate repository, wiki and web sources.
• Retain assets for offline reading.
• Check page counts and missing sections.
• Rebuild snapshots separately from curated notes.

Upload release PDFs to PDF to Markdown and compare with the original. See PDF converter options for mixed collections and URL-to-Markdown for RAG for indexing.