Document Level Security

This section describes the document level security model introduced in Perceptive Enterprise Search 9.6.

Previous generations of the Perceptive Enterprise Search security model were modeled on an optimized hybrid late binding model where the system would filter the result list POST search. Although the system was optimized, it did have some drawbacks such as:

• Variable response times
• No ability to get accurate result counts without filtering the entire result list
• Faceted navigation counts may be inaccurate (categories, entities)

This system was optimized to recognize repeating patterns in the security information, so could significantly reduce the number of actual security checks, however it only processed enough results to render the first page.

1 Introduction

Enter Early Bound Security; Perceptive Enterprise Search 9.6 implements an Early Binding security model that allows secure search across NTFS file systems and proprietary content sources. The security model is extensible and can be used to implement document-level security filtering from any content source.

Early Binding is a process whereby security information is recorded in the index for each document at indexing time. When a user executes a search, their security profile is determined and the document set returned is restricted to only the documents they are allowed to see.

The benefits of Early Binding are many:

• Users only see documents that they have access to
• No post-filtering of results is required and search times are not dependent on how much or how little the user can see
• The number of results reported is 100% accurate, not approximate
• Refinement navigators such as categories, entities and metadata are 100% accurate, not approximate

The early bound model is designed to be extensible to any document repository type, but is initially implemented for Windows File System, with MS SharePoint and HP TRIM to follow in Q3 2010.

2 How Does Early Bound Security Work?

One of the main differences between early binding and late binding is the need to "understand" the security information about each file (Security Descriptor or ACL).

At  indexing time, Perceptive Enterprise Search requests the Security Descriptor for each file being indexed (in the case of NTFS, GetNamedSecurityInfo).

The descriptor is then decomposed into its component parts (Access Control Entries or ACEs), being:

• User/Group ID (SID)
• Access Mask (Allow or Deny)

Each ACE and it's access mask are stored against each document that refers to it.

SID S-1-5-21-606747145-813497703-839522115-1553 S-1-5-18 S-1-5-21-606747145-813497703-839522115-1552 S-1-5-21-606747145-813497703-839522115-1554

Read Allow Allow Deny Allow

At query time, the system determines the list of SIDs that represent the searching user. This includes groups that the user is a member of, both direct groups (e.g. user mentioned explicitly as a member) and indirect groups (e.g. user is a member of a sub-group), Everyone and Authenticated Users.

The system can use this information alongside the user supplied query to ensure that:

• One or more SIDs are in the allow list
• AND no SIDs are in the deny list

2.1 Consider the following user

Figure 1 - Group membership for Joe Smith (direct and indirect)

When Joe executes a search, the system will automatically include permission checks on the following:

3 Deployment Considerations

The system relies on one security driver (Windows, TRIM, SharePoint, etc) per index. The best practice is to create one index per system, these can easily be "chained" together and searched as one. There is no difference to the user searching as a single result list will be returned.

To enable early binding security, the index must be configured with "Cache File Security Information". If enabling this on an existing index, you will need to perform a "Refresh Security".

Figure 2 - Cache file security information index option in the Enterprise Server Administrator

3.1 Keeping the security information up to date

As security information is cached in the index, it is possible that the current security information may differ from the stored security information. Perceptive Enterprise Search provides tools that allow you update the stored security information both manually and on a scheduled timer. An index rebuild is not required to update the security cache.

To manually refresh from the Web Administration Site, select:

Indexes > Index Name  > Refresh Security

Figure 3 - Refresh Security action in Enterprise Server Administrator

3.2 Searching with Early and Late Bound Security

The Perceptive Enterprise Search security model will automatically perform early binding security where possible. If you have the situation of searching two indexes where one does not contain early binding (i.e. not file system), the system will enter hybrid mode.

When in hybrid mode, the system will perform early binding on the indexes that support it. For the other indexes, it will fall back into the previous optimized late bound security model.

3.3 Local Groups

The first revision of early binding security does not recognize local groups on remote servers even when the searching user is explicitly or implicitly a member of that local group.

For this reason, it's recommend that only Active Directory groups are used to enforce security on the indexed content.

This will be addressed in future versions.

3.4 Security Implications

3.4.1 Navigation prohibited access

On Windows, it is possible to have permission to a file, but not have permission to browse or navigate into a folder. This essentially means that the user will never be able to "access" the document as they cannot navigate or find the file.

However as Perceptive Enterprise Search is storing the file's security, it is bypassing any potential folder navigation blockage. Giving the user permission through the search, which they may not have through Windows Explorer.

3.4.2 Out-of-date security information

See "Keeping the security information up to date".

4 Custom Early Binding Connectors

The Perceptive Enterprise Search Security Model is extensible allowing you to map non-Windows based security systems into the early binding module.

The system currently supports extension via Script, however this will be extended to support .NET, DLL, COM and Java modules in the near future.

There are several use-cases to consider when implementing the custom connectors:

• The system uses Windows to authenticate the user (by default), with:
• the content source using Windows (Active Directory) groups not stored in NTFS
• the content source using custom groups, independent of those stored in Windows/Active Directory
• The system does not use Windows to authenticate the user (Lotus Notes for example)

4.1 Windows User, Built-in Windows Groups

This scenario covers systems that use Windows to both authenticate the user and Windows users/groups to control permissions to resources, but the security information is not stored in the NTFS file system. An example of this maybe an SQL database.

For this option, all that is needed to be implemented is the RequestSecurity function that returns the permissions for the requested resources. This will be called once for each document at indexing time.

The sample below demonstrates programmatically setting permissions based on the folder, overriding the NTFS permissions.

function RequestSecurity(Request, Response)

     if InStr(Request.Filename, "/top-secret/") > 0 then

          Response.AddSID "GroupTopSecret", True

     elseif InStr(Request.Filename, "/classified/") > 0 then

          Response.AddSID "GroupClassified", True

     else

          Response.AddSID "Everyone", True

     end if

end function

Figure 4 - Pseudo code demonstrating RequestSecurity

4.2 Windows User, Custom Groups

This scenario covers systems that use Windows to authenticate the user, but use a different user/group mechanism than standard Windows/Active Directory. An example of this is an ASP.NET website that uses a database stored, role based security scheme.

For this option, we need to implement both RequestSecurity to return the permissions of the resource; and RequestUserGroups to return the list of groups that the user is a member of.

Consider the following database schema:

Figure 5 - sample database schema for role based security

In the above schema:

• the USER_NAME column contains the user's Windows username.
• a user can belong to zero or more roles.
• a resource can contain zero or more permissions including or excluding specific roles.
• this sample assumes the RESOURCE table has been indexed.

function RequestSecurity(Request, Response)

     Set RecordSet = ExecuteSQL("

         SELECT

              ROLE.ROLE_NAME,

              RESOURCE_PERMISSION.ALLOW_OR_DENY

         FROM          

              RESOURCE_PERMISSION

              JOIN ROLE ON ROLE.ROLE_ID = RESOURCE_PERMISSION.ROLE_ID

              JOIN RESOURCE ON RESOURCE.RES_ID = RESOURCE_PERMISSION.RES_ID

         WHERE

              RESOURCE.FILENAME = ?", Request.Filename)

     for each Record in RecordSet

         Response.Add Record("ROLE_NAME"), Record("ALLOW_OR_DENY")

     next         

end function

function RequestUserGroups(UserName, Response)

     Set RecordSet = ExecuteSQL("

         SELECT

              ROLE.ROLE_NAME

         FROM

              ROLE

              JOIN ROLE_MEMBER ON ROLE_MEMBER.ROLE_ID = ROLE.ROLE_ID

              JOIN USER ON USER.USER_ID = ROLE_MEMBER.USER_ID

         WHERE

              USER.USER_NAME = ?", Username)

     for each Record in RecordSet

         Response.Add Record("ROLE_NAME")

     Next         

end function

Figure 6 - Pseudo code demonstrating a role based security implementation

4.3 Non Windows User based Security System

At this time, Perceptive Enterprise Search supports one user authentication system to be installed on a website at a time. A lot of systems use Windows as their user authentication system (File System, SharePoint, TRIM etc), so there is no special requirement to search these systems together.

For other systems, it's recommended that a mapping be applied between the Windows username and the remote system username. This becomes easier if the organization has a username policy that enforces that the same username be is used for both Windows and the other systems (i.e. logon to Windows as Joe.Smith and Lotus as Joe.Smith).

As with "Windows User, Custom Groups", you will need to implement both RequestSecurity and RequestUserGroups.

4.4 Sequence of Events during Indexing

4.5 Sequence of Events during Searching