Skip

Name

Skip -- skips visiting the documents with URL matching the given pattern

indexer.conf

Synopsis

Skip [Match | NoMatch] [Case | NoCase] [String | Regexp] {pattern...}

Description

indexer skips downloading and parsing the documents with URL matching the given patterns.

Every time a matching document expires and appears in the crawler queue, indexer just marks the document as fresh again by modifying its next_index_time value according to Period settings. Word and section information about the document remains untouched.

The meaning of the first three optional parameters is exactly the same as in the Allow command.

Note: Skip can be useful for excluding sites from revisiting, for example, when the sites are temporarily not available.

It's possible to use multiple patterns in the same Skip command.

Skip takes global effect for the entire configuration file and can be used multiple times.

Examples


Skip http://site1/* http://site2/*
      

See also

Allow, CheckMP3, CheckMP3Only, Disallow, HrefOnly, Period, SkipIf.