In the world of corporate networks, there will be scenarios where the same document will be inadvertently stored in multiple locations. While these duplicates may not be easily detectable due to the sheer volume of data across a network, Perceptive Search has the ability to de-duplicate them to ensure that only a single copy is returned in the search results. Perceptive Search achieves this by storing the text of each document in a checksum which is used at query time to determine which documents have been duplicated. This prevents the need for users to waste their time filtering through the clutter of duplicate documents and more time working with relevant results.
Document de-duplication is enabled by default for all indexes that were originally created in version 9, however, if you are upgrading your indexes from a previous version (8 or below) you will need to enable it via the index options.