crawler module · apify/crawlee-python

PlaywrightCrawler

Browser-based crawler for JavaScript-rendered pages and interactive automation.

Type
crawler module
Repository
apify/crawlee-python
Readiness
Production-oriented library with documented examples, type hints, tests, CI badges, packaged releases, and optional integrations, although deployment-specific validation remains necessary.
Keywords
5

Location

Repository path

README.md

Invocation

`from crawlee.crawlers import PlaywrightCrawler` then `await crawler.run([...])`

Setup

Installation / activation

Install the `playwright` extra and Playwright dependencies, then run the crawler.

Keywords

playwrightbrowserjavascriptheadlesscrawler

Repository context

apify/crawlee-python

Apache-2.0 Python 3.10+ library and CLI for building asynchronous HTTP and browser-based crawlers with routing, retries, proxy and session management, persistent queues, pluggable storage, templates, and optional parsing, Playwright, database, Redis, observability, and AI-related integrations; no native Claude Code integration is evidenced.

web scrapingweb crawlingbrowser automationdata extractiondata collectionHTTP automationAI data preparationRAG ingestiondeveloper tooling

Open full repository research