cn.edu.hfut.dmic.webcollector.fetcher

类 Fetcher

java.lang.Object

cn.edu.hfut.dmic.webcollector.fetcher.Fetcher

public class Fetcher
extends Object

抓取器

作者:

hu

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	Fetcher.FetchItem
static class	Fetcher.FetchQueue
static class	Fetcher.QueueFeeder

字段概要

字段

限定符和类型	字段和说明
DBManager	dbManager
Executor	executor
static int	FETCH_FAILED
static int	FETCH_SUCCESS
static org.slf4j.Logger	LOG
NextFilter	nextFilter

构造器概要

构造器

构造器和说明
Fetcher()

方法概要

限定符和类型	方法和说明
void	fetchAll(Generator generator) 抓取当前所有任务，会阻塞到爬取完成
DBManager	getDBManager()
long	getExecuteInterval()
Executor	getExecutor()
NextFilter	getNextFilter()
int	getThreads() 返回爬虫的线程数
void	setDBManager(DBManager dbManager)
void	setExecuteInterval(long executeInterval)
void	setExecutor(Executor executor)
void	setNextFilter(NextFilter nextFilter)
void	setThreads(int threads) 设置爬虫的线程数
void	stop() 停止爬取

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

public static final org.slf4j.Logger LOG

dbManager

public DBManager dbManager

executor

public Executor executor

nextFilter

public NextFilter nextFilter

FETCH_SUCCESS

public static final int FETCH_SUCCESS

另请参阅:

常量字段值

FETCH_FAILED

public static final int FETCH_FAILED

另请参阅:

常量字段值

构造器详细资料

Fetcher

public Fetcher()

方法详细资料

getExecutor

public Executor getExecutor()

setExecutor

public void setExecutor(Executor executor)

fetchAll

public void fetchAll(Generator generator)
              throws Exception

抓取当前所有任务，会阻塞到爬取完成

参数:

generator - 给抓取提供任务的Generator(抓取任务生成器)

抛出:

IOException - 异常

Exception

stop

public void stop()

停止爬取

getThreads

public int getThreads()

返回爬虫的线程数

返回:

爬虫的线程数

setThreads

public void setThreads(int threads)

设置爬虫的线程数

参数:

threads - 爬虫的线程数

getDBManager

public DBManager getDBManager()

setDBManager

public void setDBManager(DBManager dbManager)

getExecuteInterval

public long getExecuteInterval()

setExecuteInterval

public void setExecuteInterval(long executeInterval)

getNextFilter

public NextFilter getNextFilter()

setNextFilter

public void setNextFilter(NextFilter nextFilter)