What is Crawlee?

Crawlee is an intuitive, customizable open-source library for web scraping and browser automation. Quickly scrape data, store it, and avoid getting blocked with auto-generated human-like fingerprints, headless browsers, and smart proxy rotation.

Do you use Crawlee?

Recent Launches

Crawlee for Python Build reliable scrapers in Python

Launched on July 9th, 2024

Crawlee Build reliable web scrapers and robots, fast!

Launched on August 23rd, 2022

Alternatives

12 reviews

34 reviews

8 reviews

Review Crawlee?

4.64/5 based on 14 reviews

View all reviews

Reviews

Matthias Stephens

•1 review

Compared to competing libraries, Crawlee is easier to use, yet also more feature rich. In just like 20 lines of code, you can have a crawler set up and ready to go that uses proxy rotation, automatic session management, header generation, fingerprint generation, and request management. When building a crawler with Crawlee, you have more time to focus on the job at hand rather than reinventing the wheel and creating custom implementations for proxy-management or request-management etc.

Share

2yr ago

Steven Smith

•1 review

A convenient tool. It allows you to quickly set up and start gathering information, even if you don't have much programming experience.

Share

5mo ago

Xuejiang Song

•6 reviews

Crawlee is super usable for us.

Share

1yr ago

Antonis Blackjim

•1 review

not that great onboarding. Weak examples and poor support in the discord channel. Almost no support I'd say.

Share

2yr ago

Alexey Udovydchenko

•2 reviews

I used Apify SDK for over 3 years in various projects, for evaluation created two actors under Crawlee, outcome is better since less resource consuming and it was easy and fun to learn and use.

Share

2yr ago

View all