Remix.run Logo
everyos_ 2 hours ago

The page requires JS to load its content - user agents without JS support just get a blank page.

I'm not sure if the bots that scrape data to train LLMs are capable of loading that type of page, or if they only work on pages that have the content inside the HTML itself?

replygirl 2 hours ago | parent | next [-]

any serious scraping service these days will fail over to a headless browser when it fetches an asset referencing a js bundle that isn't verifiably a vendor script

bstrama 2 hours ago | parent | prev | next [-]

I'm aware and will implement SSR soon ;)

m3047 2 hours ago | parent | prev [-]

It's entirely possible they simply ingest the JS as-is.