Reveal
The problem: content you can’t see in one fetch
Section titled “The problem: content you can’t see in one fetch”Most crawlers fetch a page once and parse whatever HTML shipped. But a lot of content isn’t in that first response — it appears only when you do something:
- scroll an infinite feed
- click “Load more”
- open an accordion or expand a collapsed section
- page through numbered / “Next” results
Fetch-once crawlers miss all of it. You either get the first screenful, or you hand-write a script of clicks and scrolls for every site.
How Reveal works
Section titled “How Reveal works”Reveal is Marrow’s automatic interaction layer. Before extracting, it drives the page to surface the hidden content, then chunks what’s now visible — with no steps to script.
Each round it:
- Perceives the page’s operable surface (its buttons, links and disclosures).
- Chooses one tactic deterministically from that surface — no LLM in the loop.
- Acts (expands, clicks, or scrolls), waits for new content, and re-extracts.
It repeats until a round surfaces nothing new, then stops.
Tactics
Section titled “Tactics”| Tactic | What it handles | What it does |
|---|---|---|
| expand | accordions, collapsed <details>, disclosure widgets | opens them in place |
| load-more | ”Load more” / “Show more” buttons (multilingual) | clicks, content appends |
| next | ”Next” / numbered pagination | clicks, navigates, accumulates across pages |
| scroll | infinite-scroll feeds | scrolls to the bottom |
Reveal picks the applicable tactic from what’s actually on the page, so one setting handles feeds, grids, docs and listings alike.
Bridging the vocabulary
Section titled “Bridging the vocabulary”If you know these terms, Reveal is a familiar idea assembled for you automatically:
| In plain terms | Industry equivalent |
|---|---|
| Run the page’s JavaScript in a real browser | rendering |
| Interact to load more content | actions — but auto-detected, not hand-written |
| Handle feeds that load on scroll | infinite scroll / lazy loading |
| Open collapsed sections | disclosure (aria-expanded) |
vs. Firecrawl actions
Section titled “vs. Firecrawl actions”Firecrawl’s actions make you write the click and scroll steps yourself, per site. Reveal detects the controls (“Load more”, “Next”, accordions) and drives them for you — call it auto-actions.
It’s the same philosophy as Marrow’s caching: the hard part is standard equipment. Cache is standard. Auto-reveal is standard.
When it stops
Section titled “When it stops”Reveal is bounded by design. Each round advances one tactic and re-extracts; when a round produces no new content, the loop ends. There are no infinite loops on endless feeds and no fetches spent on pages that add nothing — you get everything that’s reachable, and not a request more.
Where it’s headed
Section titled “Where it’s headed”Reveal runs today as a validated prototype across all four tactics. Next it graduates into the main scrape path as an option — so a single scrape call can surface and return content behind interactions, with the same clean extraction and change tracking as a static page.