Refine by Language

Refine by Category

Web-Scraping Projects

scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

Python     22204   today

binux / pyspider

A Powerful Spider(Web Crawler) System in Python.

Python     9557   17 days ago

scrapinghub / portia

Visual scraping for Scrapy

Python     5367   2 months ago

bda-research / node-crawler

Web Crawler/Spider for NodeJS + server-side jQuery

JavaScript     2926   10 days ago

hickford / mechanicalsoup

A Python library for automating interaction with websites.

Python     1615   2 days ago

chriskite / anemone

Anemone web-spider framework

Ruby     1590   2 months ago

propublica / upton

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

Ruby     1556   %d years ago

cgiffard / node-simplecrawler

Flexible event driven crawler for node.

JavaScript     1529   today

lorien / grab

Web Scraping Framework

Python     1489   18 days ago

chineking / cola

A high-level distributed crawling framework.

Python     1164   4 months ago

felipecsl / wombat

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Ruby     982   2 months ago

jjlee / mechanize

Stateful programmatic web browsing in Python, after Andy Lester's Perl module WWW::Mechanize .

Python     640   5 months ago

er-rousy / sss


PHP     0   2 months ago