The Disallow command disallows adding documents with URL matching the given pattern to the database.
The meaning of the first three optional parameters is exactly the same as in Allow command.
You can use multiple patterns in the same Disallow command.
# Disallow URLs not in udm.net domains using "String" match:
Disallow NoMatch *.udm.net/*
# Disallow any except known extensions and directory index using "regex" match:
Disallow NoMatch Regex \/$|\.htm$|\.html$|\.shtml$|\.phtml$|\.php$|\.txt$
# Exclude cgi-bin and non-parsed-headers using "string" match:
Disallow */cgi-bin/* *.cgi */nph-*
# Exclude anything with '?' sign in URL. Note that '?' sign has a
# special meaning in "string" match, so we have to use "regex" match here:
Disallow Regex \?