com.norconex.collector.http.handler.impl
Class DefaultRobotsTxtProvider
java.lang.Object
com.norconex.collector.http.handler.impl.DefaultRobotsTxtProvider
- All Implemented Interfaces:
- IRobotsTxtProvider, Serializable
public class DefaultRobotsTxtProvider
- extends Object
- implements IRobotsTxtProvider
Default implementation of IRobotsTxtProvider
.
XML configuration usage (not required since default):
<robotsTxt ignore="false"
class="com.norconex.collector.http.handler.DefaultRobotsTxtProvider"/>
- Author:
- Pascal Essiembre
- See Also:
- Serialized Form
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
DefaultRobotsTxtProvider
public DefaultRobotsTxtProvider()
getRobotsTxt
public RobotsTxt getRobotsTxt(org.apache.http.impl.client.DefaultHttpClient httpClient,
String url)
- Specified by:
getRobotsTxt
in interface IRobotsTxtProvider
Copyright © 2009-2013 Norconex Inc.. All Rights Reserved.