Bulk metadata indexing - Bulk metadata indexing - Alfresco - Alfresco Search Enterprise - Alfresco/Alfresco-Search-Enterprise/3.2/Alfresco-Search-Enterprise/Alfresco-Search-Enterprise/Overview/Bulk-metadata-indexing - 3.2

You can customize Search Enterprise by having the index ready with just the metadata of uploaded files or with the content of the files as well. If you have the content of your files indexed there are time and cost implications you must consider and it is only recommended when necessary. This example describes how to set up Search Enterprise to show the speed achievable when processing one billion files. Amazon Web Services have been used as the host but your setup will be specific to your requirements.

The configuration used in this example:

Search Enterprise Data Node

• AWS Elasticsearch 7.10 with Availability Zone: 1-AZ.
• Number of Data Nodes: 3.
• Instance Type: r6g.2xlarge.search.
• Storage type: EBS.
• EBS volume type: Provisioned IOPS (SSD).
• EBS volume size: 1000GB per node.
• Fielddata cache allocation: 20.
• Max clause count: 1024.

Indexing Instance

• Amazon EC2 Indexing instance to run alfresco-elasticsearch-connector-distribution-3.2.1.
• Indexing Instance type: t2.2xlarge (8vCPUs, 32GB RAM).
• Number of Amazon EC2 Instances for Indexing: 3.
• Number of threads running on instance 1 and 2 is 7 each with 6 threads on instance 3. Total threads running in parallel is 20.
• Maximum Heap allocated to each thread is 4GB (-Xmx4G).

Search Enterprise Master Node

• Number of Master Nodes: 3.
• Master Node instance type: m5.large.search.
• Master node is added for resilience, and can be avoided without having significant impact.

Search Enterprise Settings

• Number of Primary shards: 32.
• Number of Replica shards: 0.
• Refresh time: Disabled.
• Translog flush threshold: 2GB

Other Components

• Active MQ: mq.m4.large.
• RDS is used as the Database with db.r5.2xlarge running PostgreSQL.
• Amazon EC2 Instance running Content Services and the Transform Service: m5a.xlarge
• Content Services: 7.2.0.

The deployment architecture of the system: