Get the part that's behind a click

The companion page, Turn one page into a catalog, works because a Wikipedia table is already in the HTML. Plenty of pages are not like that. A course schedule, a seat map, a ticket queue, an admin panel’s “show details” — the summary is server-rendered and the rows you actually want appear only after a click.

For those, ask for a deep read. The difference is not subtle, and it does not look the way you would guess.

The cheap path returns more text and none of the data

University of Maryland’s Schedule of Classes lists courses in the HTML and hides every section — meeting times, instructor, seats — behind Show Sections. Measured on 2026-08-06, same URL, same hour:

soc/202501/…	fast path	deep read
ENGL	162,680 chars, 0 Seats	106,665 chars, 484 Seats
CMSC	117,977 chars, 0 Seats	73,428 chars, 174 Seats
HIST	—	55,798 chars, 92 Seats

Read that top row again: the fast path returned 56,000 more characters and not one seat count. If you had measured your extraction by “did we get a lot of text”, it would have looked like the better result.

Length is not coverage. The two paths do not differ in how much they return; they return different documents.

That cuts both ways, and links are where you will notice it. A deep read keeps the links inside the prose it selects — footnote markers, inline references, links in table cells — and drops the navigation and card grids the fast path hands back with everything else. On /wiki/Web_scraping that is 346 links from the fast path and 288 from a deep read: the same article, minus the chrome. On a marketing page whose links are all cards rather than prose, a deep read can return none at all. If your parser needs every outbound link on a page, read it shallow; if it needs the links in the text, read it deep.

Asking for it

curl -X POST https://api.marrow.navii.online/v1/scrape \
  -H "Authorization: Bearer $MARROW_API_KEY" -H "Content-Type: application/json" \
  -d '{"url":"https://app.testudo.umd.edu/soc/202501/ENGL",
       "formats":["markdown"],
       "depth":"deep",
       "maxAge":0}'

Two parameters carry the weight:

• depth: "deep" — render the page in a real browser, open what can be opened, then extract. Without it you get the server’s HTML.
• maxAge: 0 — skip the caches and capture live. Leave it out and a stored capture may answer instead; see below, because this one has teeth.

depth also changes which extractor reads the page

It is not only “the same reader, applied to a rendered page”. The two depths are two different extractors, so the body you get back differs even when rendering adds nothing at all.

Measured 2026-08-11 on en.wikipedia.org/wiki/Robots.txt — a server-rendered article where JavaScript reveals nothing — both ways, minutes apart, cache skipped. The exact numbers move whenever either reader changes or the article is edited; the disagreement is the point, not the digits:

	shallow	deep
body	47,009 chars	46,711 chars
links	420	384
headings	24	18
first line of the body	**Checked**	- Martijn Koster (original author)

121 lines appear only in the shallow read and 75 only in the deep one. Nothing on that page needs a browser; the difference is entirely who did the reading.

What to do with that: pick one depth per parser and stay on it, rather than treating deep as a superset of shallow you can fall back from. If you need to know which one answered, provenance.track says fast-path or deep-path.

We do not think this is right, and it is being reworked — one reader, two inputs. Until then it is documented rather than hidden.

Cost is 1 credit, the same as any scrape — the browser is not billed extra. Time is not the same: 13–19 seconds against 0.4–5 for the fast path. Budget for it.

Freshness has a default, and it is 7 days

With no maxAge, a read may be answered from a capture up to a week old — in milliseconds, at cache-hit price. That is the right default for a page you are re-reading in a loop and the wrong one while you are developing an extractor. Send maxAge: 0 while you iterate, then drop it once you know what you are parsing. The response tells you which happened: provenance.track is store-cache for a stored answer and fast-path / deep-path for a live one.

Verify by what you needed, not by what you got

A deep read is not byte-stable. The browser scrolls before it starts expanding, and how much it scrolls decides how many expanders fall inside the viewport — so two runs of the same page can open a different number of sections. We measured one department returning 17 seat counts on one run and 45 on another.

So do not assert on sizes:

body = read(url, deep=True)
seats = body.count("Seats")
if seats < expected_courses:          # you know roughly how many rows should exist
    raise SystemExit(f"partial capture: {seats} sections, expected ~{expected_courses}")

This matters because the API does not tell you the expansion was partial. The worker knows — it counts the expanders it could not open — but that count does not ride in the response today. Until it does, the check above is yours to write, and it is the difference between “we got a page” and “we got the page”.

Be a decent guest

This is someone’s university server, and one department page carries dozens of expanders — a deep read is dozens of clicks and a full render. Go serially, keep N small, put a few seconds between requests, and cache what you got (that is what the 7-day default is for, once you are past development). Nothing here needs to be fast; it needs to run once a term.

Which page is which

	the data is…	ask for
Wikipedia-style tables	already in the HTML	a plain read — cheap, seconds
This page	behind an expander	depth:"deep" — 13–19 s, still 1 credit

The tell is quick: fetch it once without depth, then search the body for a string only the hidden part would contain (Seats, Instructor, a price, a status). Zero hits on a page that visibly shows them in a browser means the data is behind a click.