Scraping with Node.js | Codementor

Johnny B. (Ionică Bizău)

Follow

Full Stack Developer, *nix user, Pianist & Organist, Vegetarian, Jesus follower

[Headless] Browsers
Crawlers
Scrapers

Headless browsers and cute libraries are useful when you have to scrape some content from the internet.

[Headless] Browsers

https://github.com/GoogleChrome/puppeteer

Headless Chrome Node API

https://github.com/amir20/phantomjs-node

PhantomJS integration module for NodeJS.

https://github.com/segmentio/nightmare

A high-level browser automation library. It uses Electron from GitHub.

https://github.com/johntitus/node-horseman

A friendlier way to communicate with PhantomJS.

https://github.com/totorojs/totoro

A simple and stable cross-browser testing tool.

Crawlers

https://github.com/bda-research/node-crawler

Web Crawler/Spider for NodeJS + server-side jQuery

https://github.com/antivanov/js-crawler

Web crawler for Node.JS

https://github.com/simplecrawler/simplecrawler

Flexible event driven crawler for node.

https://github.com/remy/snap

Creates a static snapshot of a website. Sort of like wget's mirror mode, but with nice urls.

Scrapers

https://github.com/IonicaBizau/scrape-it

A Node.js scraper for humans.

https://github.com/rchipka/node-osmosis

Web scraper for NodeJS

https://github.com/matthewmueller/x-ray

The next web scraper. See through the `` noise.

https://github.com/Narzerus/yakuza

Highly scalable Node.js scraping framework for mobsters

https://github.com/inikulin/ineed

Web scraping and HTML-reprocessing. The easy way.

Johnny B. (Ionică Bizău)

Full Stack Developer, *nix user, Pianist & Organist, Vegetarian, Jesus follower

Hey! I am Johnny B. 🙏 (Actually, my Romanian friends call me Ionică, which in English is basically: Johnny). ...and I 💖 emojis! I started coding around 2010, starting with simple static websites, WordPress applications, and th...

Follow

Like this collection? Save it to read later.

2