|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use RobotsTxt | |
---|---|
com.norconex.collector.http.crawler | |
com.norconex.collector.http.handler | |
com.norconex.collector.http.handler.impl |
Uses of RobotsTxt in com.norconex.collector.http.crawler |
---|
Methods in com.norconex.collector.http.crawler with parameters of type RobotsTxt | |
---|---|
void |
IHttpCrawlerEventListener.documentRobotsTxtRejected(HttpCrawler crawler,
String url,
IURLFilter filter,
RobotsTxt robotsTxt)
|
void |
HttpCrawlerEventAdapter.documentRobotsTxtRejected(HttpCrawler crawler,
String url,
IURLFilter filter,
RobotsTxt robotsTxt)
|
Uses of RobotsTxt in com.norconex.collector.http.handler |
---|
Methods in com.norconex.collector.http.handler that return RobotsTxt | |
---|---|
RobotsTxt |
IRobotsTxtProvider.getRobotsTxt(org.apache.http.impl.client.DefaultHttpClient httpClient,
String url)
|
Methods in com.norconex.collector.http.handler with parameters of type RobotsTxt | |
---|---|
void |
IDelayResolver.delay(RobotsTxt robotsTxt,
String url)
|
Uses of RobotsTxt in com.norconex.collector.http.handler.impl |
---|
Methods in com.norconex.collector.http.handler.impl that return RobotsTxt | |
---|---|
RobotsTxt |
DefaultRobotsTxtProvider.getRobotsTxt(org.apache.http.impl.client.DefaultHttpClient httpClient,
String url)
|
Methods in com.norconex.collector.http.handler.impl with parameters of type RobotsTxt | |
---|---|
void |
DefaultDelayResolver.delay(RobotsTxt robotsTxt,
String url)
|
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |