Libraries & frameworks
crawlee
apify/crawlee
A Node.js web scraping and browser automation library for building reliable crawlers with proxies and storage.
Topics
Related projects
crawlee-python9.7kA Python web-scraping and browser-automation library for building fast, reliable crawlers at scale.
spider2.8kA concurrency-first Rust crawling engine that streams pages locally or at scale for search and RAG feeds.
puppeteer95.8kA JavaScript library for automating headless Chrome and Firefox through the DevTools protocol.
ferrum2.1kA clean high-level Ruby API for controlling headless Chrome over CDP, built for automation and testing.
browserless1.8kA production-ready headless Chrome driver built on Puppeteer that exposes screenshot, PDF, HTML and text capture APIs to Node.js developers.
Scrapegraph-ai32.7kA Python web-scraping library that uses LLMs and graph pipelines to turn websites and documents into structured data.