Help > Indexes > Crawler Routes

Crawler Routes

Routes are a new concept introduced in Perceptive Enterprise Search 10.5, that apply to collections.

Routes allow you to partition data for large systems into several indexes. There are many benefits to doing this; it increases performance as the indexes build and update in parallel. The member indexes may have different permissions attached, making departmental/group security (i.e. role based security) simplier. And can make sub-searching faster and easier.

Routing rules are broken down into two parts, the expression and the destination. The expression is more complex than the indexing rules patterns, as the routing engine has access to content source metadata, and may contain boolean operators.

Expressions

The expression syntax uses a format similar to that of a format expression in a spreadsheet. It is made up of a field lookup followed by an operator and a value.

Field lookup can be any meta data value returned by the content source, or one of the builtin values below. To specify a field, use the following format: $(FIELDNAME)

To get the file's size would be: $(SIZE), or to get a metadata field of Drawer, use $(DRAWER)

Literal values may be:

Built-in Fields

FILENAMEThe filename of the file, not including the path
SIZEThe size of the file, in bytes
FULLNAMEThe fully qualified name of the file, including the path
PATHThe path/directory where the file is stored
DATECREATEDThe creation date of the file, in UTC
DATEMODIFIEDThe modified date of the file, in UTC
NOWThe current local date/time
UTCNOWThe current date/time, in UTC
EXTENSIONThe extension of the file
DIRECTORYThe directory containing the file
TOP_LEVEL_DIRECTORYThe top-level directory containing the file

Operators

begins-withtext begins with$(FILENAME) begins-with 'TEST'
ends-withtext ends with$(FILENAME) ends-with '.PDF'
containstext contains$(PATH) contains 'DOCS'
!=not equals$(DRAWER) != ''
<>not equals$(DRAWER) <> ''
==equals$(DRAWER) == 'Finance'
=equals$(DRAWER) = 'Finance'
<=less than or equals$(SIZE) <= 10000000
<less than$(SIZE) < 10000000
>=greater than or equals$(SIZE) >= 10000000
>greater than$(SIZE) > 10000000
||logical OR$(DRAWER) = 'Finance' || $(DRAWER) = 'Sales'
&&logical AND$(DRAWER) = 'Finance' && $(DOCUMENTTYPE) = 'Invoice'
*multiply$(SIZE) > 10 * 1000000
+addition$(DATEMODIFIED) + 7 < $(UTCNOW)
-subtraction$(DATEMODIFIED) > $(UTCNOW) - 7
/divide
!negation!( $(DRAWER) = 'Finance' || $(DRAWER) = 'Sales' )

Destinations

The destination may be an exact value or a lookup field value. Expressions are not evaluated for the destination, they are used as-is.

To route to a fixed location of documents, simply set the destination to 'documents' (without the quotes).

To route to a location based on metadata, use $(FIELDNAME). For example, to route to an index based on the document's Drawer metadata field, use '$(DRAWER)' (without quotes).

As with rules, routes are processed in-order. The first match will be used.

There is a special destination of '*discard*' that tells the crawler to ignore the file. You may use this option to exclude content from being indexed.

The following collections will be pre-configured with routes by default: