cn.edu.hfut.dmic.webcollector.example

类 DemoDepthCrawler

java.lang.Object

cn.edu.hfut.dmic.webcollector.crawler.Crawler

cn.edu.hfut.dmic.webcollector.crawler.AutoParseCrawler

cn.edu.hfut.dmic.webcollector.plugin.berkeley.BreadthCrawler

cn.edu.hfut.dmic.webcollector.example.DemoDepthCrawler

所有已实现的接口:

Executor, Visitor, Requester

public class DemoDepthCrawler
extends BreadthCrawler

本教程和深度遍历没有任何关系 一些爬取需求希望加入深度信息，即遍历树中网页的层 利用2.20版本中的新特性MetaData可以轻松实现这个功能

作者:

hu

字段概要

从类继承的字段 cn.edu.hfut.dmic.webcollector.crawler.AutoParseCrawler

autoParse, LOG, parseImg, regexRule, requester, visitor

从类继承的字段 cn.edu.hfut.dmic.webcollector.crawler.Crawler

dbManager, executeInterval, executor, fetcher, forcedSeeds, maxExecuteCount, nextFilter, resumable, RUNNING, seeds, status, STOPED, threads, topN

构造器概要

构造器

构造器和说明
DemoDepthCrawler(String crawlPath, boolean autoParse)

方法概要

限定符和类型	方法和说明
protected void	afterParse(Page page, CrawlDatums next)
static void	main(String[] args)
void	visit(Page page, CrawlDatums next)

从类继承的方法 cn.edu.hfut.dmic.webcollector.crawler.AutoParseCrawler

addRegex, execute, getRegexRule, getRequester, getResponse, getVisitor, isAutoParse, isParseImg, parseLink, setAutoParse, setParseImg, setRegexRule, setRequester, setVisitor

从类继承的方法 cn.edu.hfut.dmic.webcollector.crawler.Crawler

addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, addSeed, getDBManager, getExecuteInterval, getExecutor, getMaxExecuteCount, getNextFilter, getThreads, getTopN, inject, injectForcedSeeds, isResumable, setDBManager, setExecuteInterval, setExecutor, setMaxExecuteCount, setNextFilter, setResumable, setThreads, setTopN, start, stop, toString

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

构造器详细资料

DemoDepthCrawler

public DemoDepthCrawler(String crawlPath,
                        boolean autoParse)

方法详细资料

visit

public void visit(Page page,
                  CrawlDatums next)

afterParse

protected void afterParse(Page page,
                          CrawlDatums next)

覆盖:

afterParse 在类中 AutoParseCrawler

main

public static void main(String[] args)
                 throws Exception

抛出:

Exception