Libraries & frameworks
crawlee-python
apify/crawlee-python
A Python web-scraping and browser-automation library for building fast, reliable crawlers at scale.
Topics
Related projects
crawlee26.4kA Node.js web scraping and browser automation library for building reliable crawlers with proxies and storage.
spider2.8kA concurrency-first Rust crawling engine that streams pages locally or at scale for search and RAG feeds.
ferrum2.1kA clean high-level Ruby API for controlling headless Chrome over CDP, built for automation and testing.
Scrapling100.4kAn adaptive Python web-scraping framework with stealth fetching, spiders, proxy rotation and scaling for developers.
puppeteer95.8kA JavaScript library for automating headless Chrome and Firefox through the DevTools protocol.
browserless1.8kA production-ready headless Chrome driver built on Puppeteer that exposes screenshot, PDF, HTML and text capture APIs to Node.js developers.