H

heroshi

Settings | Report Duplicate

0

I Use This!

Activity Not Available

Commits : Listings

Analyzed about 1 year ago. based on code collected about 1 year ago.

Commit Message	Contributor	Files Modified	Lines Added	Lines Removed	Code Location	Date
Jan 20, 2023 — Jan 20, 2024 Showing page 3 of 9 Search / Filter on:
setup.py: accept eventlet 0.9.9 and higher	Sergey Shepelev	More...				about 14 years ago
Crawler: removed queuing, read urls from stdin.	Sergey Shepelev	More...				about 14 years ago
worker.io: strip lines read from io-worker	Sergey Shepelev	More...				about 14 years ago
io-worker: extracted encodeResult function, try to recover from json encoding errors	Sergey Shepelev	More...				about 14 years ago
manager: fixed dumb typo	Sergey Shepelev	More...				about 14 years ago
heroshi-report: script for sending \n-separated JSON crawl reports	Sergey Shepelev	More...				about 14 years ago
storage: calculate urlhash	Sergey Shepelev	More...				about 14 years ago
worker.Crawler: use utcnow for visited timestamp	Sergey Shepelev	More...				about 14 years ago
io-worker: added fetch_time to report (in milliseconds)	Sergey Shepelev	More...				about 14 years ago
io-worker: added visited timestamp (in UTC)	Sergey Shepelev	More...				about 14 years ago
io-worker: use os.Stdout.Write instead of println	Sergey Shepelev	More...				about 14 years ago
heroshi-get-jobs: script for fetching pack of URLs for crawling.	Sergey Shepelev	More...				about 14 years ago
io-worker report format: use lowercase keys, closer to Heroshi report API	Sergey Shepelev	More...				about 14 years ago
io-worker: sync with new Go Make.inc and http package	Sergey Shepelev	More...				about 14 years ago
Drop custom DNS resolving subsystem.	Sergey Shepelev	More...				about 14 years ago
Drop CouchDB support.	Sergey Shepelev	More...				about 14 years ago
fixup	Sergey Shepelev	More...				over 14 years ago
doc: fix style	Sergey Shepelev	More...				over 14 years ago
added gitignore	Sergey Shepelev	More...				over 14 years ago
doc: added Download info	Sergey Shepelev	More...				over 14 years ago
removed size calculation before sending item to manager	Sergey Shepelev	More...				over 14 years ago
removed excessive debug log (fetch)	Sergey Shepelev	More...				over 14 years ago
cosmetic	Sergey Shepelev	More...				over 14 years ago
worker.io: inlined _read and _write, handle json decode error	Sergey Shepelev	More...				over 14 years ago
cosmetic	Sergey Shepelev	More...				over 14 years ago
handle robotparser KeyError	Sergey Shepelev	More...				over 14 years ago
detailed death log (datetime)	Sergey Shepelev	More...				over 14 years ago
run-dev-worker shortcut for ever restarting crawler	Sergey Shepelev	More...				over 14 years ago
cosmetic: removed unused code	Sergey Shepelev	More...				over 14 years ago
extracted io-worker interaction to separate module, also, io-worker process is now not restarted automatically	Sergey Shepelev	More...				over 14 years ago

←
1
2
3
4
5
6
7
8
9
→