Added
- Add [HCCrawler.defaultArgs()] method.
- Add
requestretriedevent.
changed
- Use
cacheoption not only for remembering already requested URLs but for request queue for distributed environments. - Moved
onSuccess,onErrorandmaxDepthoptions from HCCrawler.connect() and HCCrawler.launch() to crawler.queue().