Refine by Language

Refine by Category

Framework Projects


scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

Python     21217   4 days ago


binux / pyspider

A Powerful Spider(Web Crawler) System in Python.

Python     9036   7 days ago


bda-research / node-crawler

Web Crawler/Spider for NodeJS + server-side jQuery

JavaScript     2786   26 days ago


chriskite / anemone

Anemone web-spider framework

Ruby     1588   19 days ago


propublica / upton

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

Ruby     1547   %d years ago


cgiffard / node-simplecrawler

Flexible event driven crawler for node.

JavaScript     1474   19 days ago


lorien / grab

Web Scraping Framework

Python     1449   5 days ago


chineking / cola

A high-level distributed crawling framework.

Python     1137   2 months ago


felipecsl / wombat

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Ruby     960   8 months ago