You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
7 years ago | |
---|---|---|
.gitignore | 7 years ago | |
LICENSE | 7 years ago | |
README.md | 8 years ago | |
fullscrape.go | 7 years ago |
README.md
fullscrape
A Full Website Scraper
Usage:
fullscrape url out-dir [-d=depth] [-x]
Arguments:
url: The url that you want to start the scrape at. out-dir: The output directory. If it doesn't exist, it will be created. d: The depth of the scrape. Defaults to no limit. x: Cross Domain scraping. This could get dangerous, the internet's a big place.
Sometime I may add the ability for fullscrape to automatically make non-relative links relative. Because that seems useful.