ChiselForge

Overview

ChiselForge extracts structured data from web pages without writing a scraper. You give it a URL and describe the fields you want, and it returns validated JSON, along with a record of exactly how it got there.

Its principle: use the cheapest, most deterministic method that can reliably answer the request, and only escalate when necessary.

The problem

Most AI extraction tools send every page to an LLM. That’s convenient, but slow, expensive and non-reproducible on the large share of pages that already expose their data in structured form. Traditional scrapers are the opposite: deterministic, but brittle and expensive to write and maintain.

Constraints

• Cost and latency. Don’t pay for a model call a page didn’t need.
• Reproducibility. Where a deterministic answer exists, use it.
• Honest failure. When a page doesn’t contain the requested data, return nothing rather than a plausible guess.
• Light install. Browser rendering and proxy support shouldn’t be installed unless a call needs them.

My role

I designed the extraction strategy and the tiered architecture, and I run the benchmark and the evaluations. The code is written with Claude.

Reasoning and architecture

1. Rendering checkDoes this page need a browser at all?
2. Tier 1 · JSON-LDStructured data already on the page. Deterministic, no LLM.
3. Page propertiesTitle, description, headings and similar fields read from markup. Deterministic.
4. Tier 2 · Hydration stateFramework app state (for example Next.js) mapped to the schema by an LLM.
5. Tier 3 · Raw textLast resort. An LLM interprets the page text.
6. Schema validationEvery result is validated before it is returned.

Every result reports which strategy ran, whether an LLM or a browser was used, and a confidence score. --verbose prints the decision trail as it happens.

Implementation

A Node.js engine with lazy-loaded transports (HTTP and Playwright), a job queue, rate limiting, a proxy pool, checkpointing for long crawls, and site discovery from robots.txt, sitemaps and links. A CLI covers single-page extraction and whole-site crawls, including a bulk mode that periodically restarts the browser.

Evidence and outcomes

Benchmark (8 live cases): all 8 succeeded. On the same site, the JSON-LD tier answered in 918 ms with no LLM, against about 33 s for the LLM-backed hydration tier. The benchmark also checks that irrelevant JSON-LD correctly falls through to the next tier.

A real audit run found real bugs. Crawling the Society of Intellectuals website surfaced three faults, and all three were fixed with regression tests:

• Duplicate crawling. HTTP sitemap URLs and HTTPS page links were counted as different pages, so 16 real pages became 32 crawl jobs. Discovery now takes the canonical origin from where robots.txt redirects, and treats www and the apex domain as the same site.
• No-key failure. With no LLM key set, every page failed, even for fields readable straight from the markup. A deterministic page-properties step was added between JSON-LD and the LLM tiers.
• Pointless retries. A missing key was retried with 20–40 second backoffs on every page, so a 16-page crawl was still running after eight minutes. Configuration errors are now marked non-retryable and fail fast.

Verdict

• Keep: deterministic first, LLM last, and every result reporting which tier ran. That makes the tool cheaper, faster and auditable, and it’s now the default in everything I build.
• Change: live-site runs belong in the release gate, not after it. All three audit bugs were invisible to unit tests and obvious within minutes of a real crawl.