Libraries & frameworks
Scrapling
D4Vinci/Scrapling
An adaptive Python web-scraping framework with stealth fetching, spiders, proxy rotation and scaling for developers.
Topics
Related projects
creatorhub2.9kSelf-hosted web panel for creators monitoring, collecting, reposting, and pushing short-video content across Douyin, Xiaohongshu, and Kuaishou from one centralized dashboard interface.
fortress799A stealth Chromium engine that lets scrapers and browser agents bypass bot detection via CDP endpoints and MCP-powered browsing tools.
crawlee-python9.7kA Python web-scraping and browser-automation library for building fast, reliable crawlers at scale.
spider2.8kA concurrency-first Rust crawling engine that streams pages locally or at scale for search and RAG feeds.
thepipe1.5kthepipe is a Python package that uses vision-language models to scrape clean Markdown, tables, images and structured data from PDFs, Office files, videos, audio and other tricky documents.
firecrawl199.5kA developer API service that gives AI agents real-time web search, scraping, and crawling with LLM-ready structured output.