HTML Is Killing Your LLM Token Budget (Here's Why)

You're feeding webpages to an LLM and the bill is bigger than you expected. The model is reading more than the article: it's reading the entire navigation, the footer, the cookie banner, twelve embedded JSON-LD blocks, and the inline scripts that load three different analytics vendors. By raw token count, the article you actually care about is a small part of the input. The rest is noise you're paying for, that's diluting the model's attention, and that's almost trivially removable. Here's where the tokens go, how to measure it on your own pages, and how to fix it.

Anatomy of a typical webpage: where the bytes go

Open any modern webpage's source code and the article body is a needle in a haystack. The page is built around the article, but the article is a small part of what's actually shipped to the browser. A typical news or blog page contains:

• The visible article. Headings, paragraphs, lists, occasionally a table or image: the thing you came to read.
• Navigation chrome. Top nav, side nav, breadcrumbs, mega-menus, footer nav. Repeated on every page of the site.
• Inline scripts. Analytics tags, A/B testing scripts, consent management platforms, pixel trackers, ad-network glue.
• Inline styles. Either critical CSS injected at the top, or per-element style attributes from the CMS.
• Hidden metadata. Open Graph tags, Twitter card tags, schema.org JSON-LD blocks (sometimes duplicated for different consumers), canonical URLs, hreflang sets.
• Recommendation widgets. Pre-rendered "related articles" sections that ship dozens of titles, summaries, thumbnails, and click trackers.
• Comment sections. Either inline or as a script template that hydrates later.
• Newsletter modals, social share buttons, cookie banners, exit-intent popups. All shipped in the HTML even when not visible.

None of this matters to the LLM you're going to feed the page to. None of it answers your question. All of it costs tokens. On a typical webpage, the article is a minority of what ships: most of the bytes you're sending to the model are wasted.

Measure it on your own pages

You don't need to take anyone's word for this. Pick a few pages you actually feed to a model. For each one, fetch the raw HTML, convert the same URL to Markdown, and count both with a tokenizer such as tiktoken, using the encoding that matches your model (o200k_base for GPT-4o). Divide the HTML count by the Markdown count to get the ratio for that page.

Token counts vary with the tokenizer version and with page-specific markup, so treat each page's figure as an estimate. The pattern to look for: the bigger and more interactive the page (a long forum thread, a heavily templated article), the more chrome, widgets and inline scripts it carries, and the larger the gap.

The cost math at GPT-4o pricing

GPT-4o input pricing at the time of writing is roughly $2.50 per million input tokens (output is $10/M but we're focused on input here, since the page goes in as input). The math for any workload is the same:

• Monthly input cost = pages per month × tokens per page × $2.50 / 1,000,000
• Run it once with your HTML count and once with your Markdown count. The difference is what the markup costs you.

Because the cost scales linearly with volume, a gap that's pocket change at a few pages a month becomes a real line item at 10,000 pages a month (say a content moderation system, a research aggregator, or a competitive intelligence tool). The longest, most script-heavy pages are where it hurts most.

And these are just the input savings. The output side benefits too, because cleaner input means tighter, more accurate outputs: the model doesn't waste its response budget summarizing irrelevant chrome.

The hidden cost: context window and quality

The dollars matter, but the bigger problem is often the context window. Check the context and file limits of the model and account you are actually using. A single long, script-heavy page in raw HTML can eat a large share of that, leaving little room for a system prompt, conversation history, or follow-up questions. Convert to Markdown and the same content takes a fraction of the space, leaving the rest for everything else.

Quality also degrades with bloat. The "lost in the middle" effect (where models pay less attention to content in the middle of a long context) is well documented. When most of the context is noise, the relevant part is more likely to land in the dead zone of model attention. You're not just paying more, you're getting worse answers for the higher price.

The same problem exists with PDFs; see why PDF wastes your AI tokens and how to measure PDF vs Markdown token counts for the document-side equivalent. Different format, same fundamental waste pattern.

Why every webpage is this bad

It's not the developers' fault. Modern websites are this heavy because they have to be: SEO needs the structured metadata, advertising needs the script tags, regulators need the consent banners, marketing needs the recommendation widgets, A/B testing needs the experiment scaffolding. The page is correctly engineered for a browser visit by a human; it's catastrophically wrong as input to an LLM.

The fix isn't to ask websites to ship lighter HTML: they won't, and shouldn't. The fix is to do the cleanup yourself, once, before the page hits the model. Extract the article, drop everything else, hand the LLM the clean Markdown.

The fix: convert before you feed

Open /convert/url-to-markdown. Paste the URL. Hit convert. Download the .md file or copy the Markdown text. Feed that to the LLM instead of the raw page.

That's the entire fix. The work happens once, you get a stable clean artifact, and every subsequent token spent on the file goes to actual content rather than navigation chrome.

For an LLM-pipeline-focused setup, see URL to Markdown for LLM. For LangChain pipelines specifically, see URL to Markdown for LangChain. For ChatGPT-specific workflows, see URL to Markdown for ChatGPT.

Doing it at scale

If you have a pipeline that processes many URLs per day, you have two reasonable patterns:

1. Use the web tool ad-hoc for one-off pages and research workflows. Paste, convert, copy. Thirty seconds per URL. Fine for human-driven research.
2. Run an open-source extractor locally for high-volume automation. Trafilatura (Python) is probably the best open-source HTML-to-clean-text extractor available: it's fast, accurate, handles JavaScript-rendered content via a Playwright integration, and produces Markdown output. html2text, Mozilla's readability.js, and Newspaper3k are also reasonable options depending on language and use case.

For self-hosted automation, point Trafilatura or Readability at your URL list, write the resulting .md files to disk or a vector store, and feed them to your LLM pipeline. The conversion runs entirely on your machines: there's nothing to integrate with mdisbetter for this case.

What if you really need the HTML structure?

Edge case worth addressing: sometimes you need to know about the original HTML, for instance, if you're building a tool that analyzes the page's actual structure (DOM analysis, accessibility audit, ad-tech inspection). In those cases, raw HTML is the right input.

But for content-oriented tasks (summarization, Q&A, claim extraction, classification, RAG indexing, knowledge base ingestion), you want the article text, not the markup. Convert first, feed second.

Combining with PDF workflow

Most real-world pipelines deal with both webpages and PDF documents: research papers, company reports, regulatory filings. The same principle applies to both. Use PDF to Markdown for documents, URL to Markdown for webpages, then your downstream pipeline operates on a single uniform Markdown corpus regardless of source format. This dramatically simplifies chunking, indexing, and retrieval logic.

For background on why structure matters as much as size, see how to save a webpage so AI can actually read it.

The summary

HTML is several times larger than the equivalent Markdown for the same content. At GPT-4o input pricing, that multiple applies directly to your input bill, and the gap grows with every page you process. The quality of LLM responses also degrades when input is bloated, both because of attention dilution and because chunking and retrieval downstream become noisier.

The fix takes thirty seconds: convert the URL to Markdown before you feed it to the model. The savings are immediate and compound across every interaction. There's no scenario where feeding raw HTML is the right call for content-oriented LLM work: the format is wrong for the job, and a one-step conversion fixes it.

One final calibration

If your AI-powered product feels expensive to run and you haven't audited what you're actually feeding the models, this is probably the highest-leverage place to look. Moving from raw-HTML to Markdown ingestion cuts the input bill by the markup share you measured, and answer quality tends to improve in parallel. Two wins from one cheap fix. Worth an afternoon.