What Firecrawl does
Firecrawl is the crawl engine that fetches your site and the competitor's, rendering JavaScript, parsing meta tags, extracting schema, and producing structured page data.
Integration · Web scraping
Rivalist + Firecrawl
JavaScript-aware site crawling — Firecrawl extracts what curl can't see.
Why it matters for your audit
Most SEO audit tools use curl-style fetching that misses JS-injected content — including schema markup added by Yoast, RankMath, AIOSEO, and other plugins. Firecrawl renders pages like a real browser.
Integration depth
How Rivalist uses Firecrawl
-
Render JavaScript-injected schema (JSON-LD added by SEO plugins).
-
Extract meta tags, headings, alt text, internal links per page.
-
Pull canonical tags, hreflang, robots directives.
-
Capture page content for word count + readability analysis.
In your report
What you actually see
-
Accurate schema detection even on JS-heavy sites.
-
Per-page meta description coverage including dynamically-rendered ones.
-
Real internal link graph counts (not just HTML-parsed).
BYOK note
BYOK plan: bring your own Firecrawl API key. Hosted plan: we supply the key, included in subscription.
FAQ
Common questions
Why not just use curl or wget?
Modern sites render most content via JavaScript. Curl sees only the initial HTML which often lacks the schema tags, meta descriptions, and structured data the page actually serves to browsers and search engines.
Does Firecrawl respect robots.txt?
Yes, by default. Rivalist crawls only public pages and respects rate limits.
How much does Firecrawl cost?
Firecrawl pricing: roughly $0.10-0.30 per full site crawl on the BYOK plan, included in Hosted.
Other integrations