brian
/
fullscrape
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Update Readme

This commit is contained in:
Brian Buller 2015-08-15 10:38:37 -05:00
parent 14d105740a
commit f70a1b01cf
1 changed files with 12 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -1,4 +1,15 @@
fullscrape
==========
A Full Website Scraper
A Full Website Scraper
Usage:
> fullscrape <url> <out-dir> [-d=depth] [-x]
Arguments:
> url: The url that you want to start the scrape at.
> out-dir: The output directory. If it doesn't exist, it will be created.
> d: The depth of the scrape. Defaults to no limit.
> x: Cross Domain scraping. This could get dangerous, the internet's a big place.
Sometime I may add the ability for fullscrape to automatically make non-relative links relative. Because that seems useful.