You must configure the Live Ingester to enable it to listen to real-time events from Alfresco Content Services and send document metadata and content to Content Intelligence for indexing.
To configure the Live Ingester:
- Open your docker-compose file in a text editor, such as Visual Studio Code.
- Navigate to the live-ingester service section.
-
Under the environment section of the
live-ingester service, configure the following required
settings, replacing each placeholder with the value for your environment:
environment: SPRING_ACTIVEMQ_BROKERURL: nio://activemq:61616 ALFRESCO_REPOSITORY_BASEURL: http://alfresco:8080/alfresco ALFRESCO_TRANSFORM_SHAREDFILESTORE_BASEURL: http://shared-file-store:8099 HYLANDEXPERIENCE_INSIGHT_INGESTION_BASEURL: https://ingestion.insight.dev.experience.hyland.com AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTID: [add-your-client-id] AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTSECRET: [add-your-client-secret] AUTH_PROVIDERS_HYLANDEXPERIENCE_ENVIRONMENTKEY: [add-your-environment-key] AUTH_PROVIDERS_HYLANDEXPERIENCE_TOKENURI: [add-nucleus-iam-origin]/idp/connect/token AUTH_PROVIDERS_HYLANDEXPERIENCE_GRANTTYPE: client_credentials AUTH_PROVIDERS_ALFRESCO_USERNAME: admin AUTH_PROVIDERS_ALFRESCO_PASSWORD: admin APPLICATION_SOURCEID: [add-your-system-id]The placeholders represent the following:
Placeholder Description [add-your-client-id] The client ID of the Content Intelligence environment. For example: sc-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
[add-your-client-secret] The client secret of the Content Intelligence environment. [add-your-environment-key] The Content Intelligence environment key. For example: alfresco-kd-ci-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
[add-nucleus-iam-origin] The URL token for the Content Intelligence environment. For example: https://auth.staging.app.hyland.com/idp/connect/token
[add-your-system-id] The Universally Unique Identifier (UUID) identifying which systems can use this agent. For example: nio://activemq:61616
The following table describes all available settings in detail.
Setting Description SPRING_ACTIVEMQ_BROKERURL The ActiveMQ broker basic URL. For example: http://alfresco:8080/alfresco.
ALFRESCO_REPOSITORY_BASEURL The Alfresco Repository basic URL, including the /alfresco suffix. For example: http://alfresco:8080/alfresco.
ALFRESCO_TRANSFORM_SHAREDFILESTORE_BASEURL The base URL of the Alfresco Shared File Store. For example: http://shared-file-store:8099 .
HYLANDEXPERIENCE_INSIGHT_INGESTION_BASEURL The Content Intelligence ingestion service base URL. The correct value depends on the geographic region of your Content Intelligence environment. Contact your Hyland representative to confirm which URL to use. The available values are:
Note: The URL must not include the /v1 suffix. This is a common migration issue for customers moving from the legacy Knowledge Discovery connector for ACS, where the ingestion URL required /v1.- US (Production): https://ingestion.insight.experience.hyland.com
- EU (Production): https://ingestion.insight.app.hyland.eu
AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTID The client ID of the Content Intelligence environment. For example: sc-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTSECRET The client secret of the Content Intelligence environment. AUTH_PROVIDERS_HYLANDEXPERIENCE_ENVIRONMENTKEY The Content Intelligence environment key. For example: alfresco-kd-ci-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx
AUTH_PROVIDERS_HYLANDEXPERIENCE_TOKENURI The URL token for the Content Intelligence environment. For example: https://auth.staging.app.hyland.com/idp/connect/token
AUTH_PROVIDERS_HYLANDEXPERIENCE_GRANTTYPE The OAuth 2.0 grant type used to obtain an access token. Always set to client_credentials. Do not change this value.
AUTH_PROVIDERS_ALFRESCO_USERNAME The username of an Alfresco user with read access to the REST API. AUTH_PROVIDERS_ALFRESCO_PASSWORD The password for the Alfresco user. APPLICATION_SOURCEID The Universally Unique Identifier (UUID) identifying which systems can use this agent. For example: nio://activemq:61616
-
Optional: Under the environment section of the
live-ingester service, configure the following optional
node filtering settings to control which nodes are sent to Knowledge
Discovery:
environment: # Optional: Node filtering # ALFRESCO_FILTER_TYPE_ALLOW: # ALFRESCO_FILTER_TYPE_DENY: # ALFRESCO_FILTER_ASPECT_ALLOW: # ALFRESCO_FILTER_ASPECT_DENY: # ALFRESCO_FILTER_PATH_ALLOW: # ALFRESCO_FILTER_PATH_DENY:
Allow and deny lists can be used to filter nodes based on their type, aspect, or location in the repository. A node is sent to Knowledge Discovery if it matches an entry in any allow list and does not match an entry in a corresponding deny list. If an allow list is empty, all nodes are considered a match for that filter type.
Note: Lists can be passed using the YAML configuration. When specifying properties, the indexes of each item in the list must be specified (starting from n=0).PATH filters use node IDs without the leading space reference. For example: dad275aa-affc-487d-a7ed-92cf8e6ce351.
Types and aspects must be specified with short namespaces. For example: cm:folder.
-
Optional: Under the environment section of the
live-ingester service, configure the following optional
properties to override any of the default settings:
# Optional: Alfresco Repository # ALFRESCO_REPOSITORY_EVENTSENDPOINT: activemq:topic:alfresco.repo.event2 # ALFRESCO_REPOSITORY_RESPONSETIMEOUTMS: 30000 # ALFRESCO_REPOSITORY_DISCOVERYENDPOINT: http://alfresco:8080/alfresco/api/discovery # ALFRESCO_REPOSITORY_VERSIONOVERRIDE: 23.2.0 # ALFRESCO_REPOSITORY_HEALTHPROBE_ENDPOINT: http://alfresco:8080/alfresco # ALFRESCO_REPOSITORY_HEALTHPROBE_TIMEOUTSECONDS: 1800 # ALFRESCO_REPOSITORY_HEALTHPROBE_INTERVALSECONDS: 30 # Optional: Repository event subscription # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DURABLE: true # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_NAME: LiveIngesterSubscription # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DEADLETTERUNSUPPORTEDTYPES: true # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_OBSERVECONTENTDROPS: false # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DEADLETTERENABLED: true # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DEADLETTERURI: activemq:queue:ActiveMQ.DLQ # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_MAXIMUMREDELIVERIES: 6 # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_REDELIVERYDELAYMS: 1000 # ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_USEEXPONENTIALBACKOFF: false # Optional: Bulk Ingester integration # ALFRESCO_BULKINGESTER_ENDPOINT: activemq:queue:bulk-ingester-events # ALFRESCO_BULKINGESTER_OBSERVECONTENTDROPS: false # Optional: Transform service # ALFRESCO_TRANSFORM_REQUEST_ENDPOINT: activemq:queue:acs-repo-transform-request # ALFRESCO_TRANSFORM_REQUEST_TIMEOUT: 20000 # ALFRESCO_TRANSFORM_RESPONSE_QUEUENAME: org.alfresco.cic-connector.transform.response # ALFRESCO_TRANSFORM_RESPONSE_RETRYINGESTION_ATTEMPTS: 6 # ALFRESCO_TRANSFORM_RESPONSE_DEADLETTERENABLED: true # ALFRESCO_TRANSFORM_RESPONSE_DEADLETTERURI: activemq:queue:ActiveMQ.DLQ # ALFRESCO_TRANSFORM_RESPONSE_MAXIMUMREDELIVERIES: 6 # ALFRESCO_TRANSFORM_RESPONSE_REDELIVERYDELAYMS: 1000 # ALFRESCO_TRANSFORM_RESPONSE_THROWFAILEDTRANSFORMS: true # Optional: ActiveMQ redelivery # ALFRESCO_ACTIVEMQ_REDELIVERY_MAXIMUMREDELIVERIES: 6 # ALFRESCO_ACTIVEMQ_REDELIVERY_INITIALREDELIVERYDELAYMS: 1000 # ALFRESCO_ACTIVEMQ_REDELIVERY_USEEXPONENTIALBACKOFF: false # ALFRESCO_ACTIVEMQ_REDELIVERY_BACKOFFMULTIPLIER: 5.0 # Optional: Content Intelligence ingestion # HYLANDEXPERIENCE_INSIGHT_INGESTION_PRESIGNEDURLSCOUNT: 1 # HYLANDEXPERIENCE_INSIGHT_INGESTION_RESPONSETIMEOUTMS: 0 # HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_ATTEMPTS: 10 # HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_INITIALDELAY: 500 # HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_DELAYMULTIPLIER: 2 # Optional: Authentication # AUTH_PROVIDERS_HYLANDEXPERIENCE_TYPE: oauth2 # AUTH_PROVIDERS_HYLANDEXPERIENCE_SCOPE[0]: hxp.integrations # AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTNAME: acs-insight-connector # AUTH_PROVIDERS_ALFRESCO_TYPE: basic # AUTH_RETRY_ATTEMPTS: 10 # AUTH_RETRY_INITIALDELAY: 500 # AUTH_RETRY_DELAYMULTIPLIER: 2
The following tables list all available settings, grouped by function.Alfresco Repository
Setting Description Default ALFRESCO_REPOSITORY_EVENTSENDPOINT The ActiveMQ topic to listen on for repository events. activemq:topic:alfresco.repo.event2 ALFRESCO_REPOSITORY_RESPONSETIMEOUTMS The per-request response timeout in ms for ACS REST content downloads. Set to 0 to fall back to the HTTP client's built-in timeout.
30000 ALFRESCO_REPOSITORY_DISCOVERYENDPOINT The Alfresco Discovery API URL, used to detect the ACS version at startup. Derived from base-url ALFRESCO_REPOSITORY_VERSIONOVERRIDE The ACS version string to use instead of calling the Discovery API, such as 26.2.0. (empty) ALFRESCO_REPOSITORY_HEALTHPROBE_ENDPOINT The health check endpoint for Alfresco. Derived from base-url ALFRESCO_REPOSITORY_HEALTHPROBE_TIMEOUTSECONDS The maximum time to wait for Alfresco to become healthy, in seconds. 1800 ALFRESCO_REPOSITORY_HEALTHPROBE_INTERVALSECONDS The interval between health checks, in seconds. 30 Repository event subscription
Setting Description Default ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DURABLE True/false indicator of whether the Live Ingester subscribes as a durable topic consumer. When true, the Live Ingester subscribes as a durable topic consumer, so events published while it is disconnected are retained by the broker and replayed on reconnect. When false, events published while the Live Ingester is disconnected are silently dropped by the broker.
true ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_NAME The subscription identifier and JMS connection client ID. The value must be unique per deployment and stable across restarts.
LiveIngesterSubscription ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DEADLETTERUNSUPPORTEDTYPES When true, an unrecognised eventType is routed to the dead-letter queue. When false, it is only counted and acknowledged.
true ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_OBSERVECONTENTDROPS When true, every content event skipped due to an empty MIME mapping emits an INFO log and increments a counter. false ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DEADLETTERENABLED When true (default), failed repository events are retried and, if all retries are exhausted, moved to the dead-letter queue. When false, failed events are silently discarded after retry exhaustion.
true ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_DEADLETTERURI The Camel endpoint URI used as the dead-letter destination for repository events. Only applies when dead-letter-enabled=true.
activemq:queue:ActiveMQ.DLQ ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_MAXIMUMREDELIVERIES Maximum redelivery attempts before dead-lettering. Set to 0 to disable retries.
6 ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_REDELIVERYDELAYMS The delay between redelivery attempts, in ms. 1000 ALFRESCO_REPOSITORY_EVENTSSUBSCRIPTION_USEEXPONENTIALBACKOFF When true, the delay between retries increases with each attempt. When false (default), a fixed delay is used between every retry.
false Bulk-ingester (consumed by the Live Ingester)
Setting Description Default ALFRESCO_BULKINGESTER_ENDPOINT The ActiveMQ queue to receive bulk ingestion events. The value must match the Bulk Ingester's ALFRESCO_BULK_INGEST_PUBLISHER_ENDPOINT.
activemq:queue:bulk-ingester-events ALFRESCO_BULKINGESTER_OBSERVECONTENTDROPS When true, every document skipped during bulk ingestion because its file type is not configured for ingestion emits an INFO log entry and increments a counter, making it easier to detect unintended content gaps. When false (default), skipped documents are recorded only at DEBUG level and no counter is incremented.
false Transform Service
Setting Description Default ALFRESCO_TRANSFORM_REQUEST_ENDPOINT The ActiveMQ queue to send transform requests to ATS. activemq:queue:acs-repo-transform-request ALFRESCO_TRANSFORM_REQUEST_TIMEOUT The maximum time to wait for a transform to complete, in milliseconds. 20000 ALFRESCO_TRANSFORM_RESPONSE_QUEUENAME The queue name for receiving transform responses. org.alfresco.cic-connector.transform.response ALFRESCO_TRANSFORM_RESPONSE_ENDPOINT The Camel endpoint for the transform-response queue. Derived from queue-name ALFRESCO_TRANSFORM_RESPONSE_RETRYINGESTION_ATTEMPTS The retry attempts before dead-lettering. Set the value to -1 for unlimited retries (dead-letter channel is never reached).
6 ALFRESCO_TRANSFORM_RESPONSE_DEADLETTERENABLED When true (default), failed transform responses are retried and, if all retries are exhausted, moved to the dead-letter queue so no content is silently lost. When false, failed transform responses are discarded after retry exhaustion with no dead-letter entry.
true ALFRESCO_TRANSFORM_RESPONSE_DEADLETTERURI The dead-letter destination URI. Only applies when dead-letter-enabled=true.
activemq:queue:ActiveMQ.DLQ ALFRESCO_TRANSFORM_RESPONSE_MAXIMUMREDELIVERIES The maximum redelivery attempts before dead-lettering. 6 ALFRESCO_TRANSFORM_RESPONSE_REDELIVERYDELAYMS The delay between redelivery attempts, in milliseconds. 1000 ALFRESCO_TRANSFORM_RESPONSE_THROWFAILEDTRANSFORMS When true, an ATS-reported failure is routed to the dead-letter queue. When false, it is logged and acknowledged.
true ALFRESCO_TRANSFORM_SHAREDFILESTORE_FILEENDPOINT The Shared File Store file endpoint used to download renditions. Derived from base-url ActiveMQ redelivery
Setting Description Default ALFRESCO_ACTIVEMQ_REDELIVERY_MAXIMUMREDELIVERIES The maximum number of times the ActiveMQ broker retries a failed message across all consumers before moving it to the dead-letter queue. Set to 0 to disable broker-side retries and move failed messages to the dead-letter queue immediately.
6 ALFRESCO_ACTIVEMQ_REDELIVERY_INITIALREDELIVERYDELAYMS The time to wait before the first redelivery attempt, in milliseconds. When ALFRESCO_ACTIVEMQ_REDELIVERY_USEEXPONENTIALBACKOFF is false (default), this fixed delay is used between every retry attempt. When exponential backoff is enabled, this value is used as the starting delay and multiplied on each subsequent attempt.
1000 ALFRESCO_ACTIVEMQ_REDELIVERY_USEEXPONENTIALBACKOFF When false (default), a fixed delay is used between every retry attempt, as set by ALFRESCO_ACTIVEMQ_REDELIVERY_INITIALREDELIVERYDELAYMS. When true, the delay increases with each attempt by multiplying the previous delay by the value of ALFRESCO_ACTIVEMQ_REDELIVERY_BACKOFFMULTIPLIER, reducing pressure on a recovering downstream system.
false ALFRESCO_ACTIVEMQ_REDELIVERY_BACKOFFMULTIPLIER The multiplier used to increase the delay between retry attempts when ALFRESCO_ACTIVEMQ_REDELIVERY_USEEXPONENTIALBACKOFF is true. For example, with a default multiplier of 5.0 and an initial delay of 1000 ms, the delays are 1 s, 5 s, 25 s, and so on. It has no effect when exponential backoff is disabled.
5.0 Content Intelligence ingestion
Setting Description Default HYLANDEXPERIENCE_INSIGHT_INGESTION_PRESIGNEDURLSCOUNT The number of presigned URLs requested from Content Intelligence per ingestion event. A presigned URL is a temporary upload address used to send document content securely to Content Intelligence storage. Increase this value if you are ingesting documents with multiple content streams. For most deployments, the default value of 1 is sufficient.
1 HYLANDEXPERIENCE_INSIGHT_INGESTION_RESPONSETIMEOUTMS The maximum time to wait for a response from the Content Intelligence ingestion endpoint (/ingestion-events), in milliseconds. Set to 0 (default) to wait indefinitely, which risks blocking the Live Ingester if Content Intelligence becomes unresponsive. For production deployments, set a finite value such as 30000 (30 seconds).
0 HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_ATTEMPTS The maximum number of retry attempts for a failed Content Intelligence ingestion request. Retries are triggered by transient errors such as a 5xx response or a network timeout. If all attempts are exhausted, the message is moved to the dead-letter queue. Reduce this value to fail faster in environments with tight processing SLAs, or increase it for deployments where Content Intelligence may experience longer recovery windows.
10 HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_INITIALDELAY The time to wait before the first retry attempt, in milliseconds. When HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_DELAYMULTIPLIER is greater than 1, this value is used as the starting delay and multiplied on each subsequent attempt.
500 HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_DELAYMULTIPLIER The multiplier applied to the delay between each retry attempt. For example, with the default multiplier of 2 and an initial delay of 500 ms, the delays between attempts are 500 ms, 1 s, 2 s, 4 s, and so on.
Set to 1 to use a fixed delay between every retry attempt, as set by HYLANDEXPERIENCE_INSIGHT_INGESTION_RETRY_INITIALDELAY.
2 Authentication
Setting Description Default AUTH_PROVIDERS_HYLANDEXPERIENCE_TYPE The authentication method used to obtain an access token for Content Intelligence. The default value of oauth2 uses the OAuth 2.0 client credentials flow, where the connector authenticates using the client ID and secret configured in AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTID and AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTSECRET.
oauth2 AUTH_PROVIDERS_HYLANDEXPERIENCE_SCOPE[n] The OAuth 2.0 scopes requested when obtaining an access token for Content Intelligence. hxp.integrations AUTH_PROVIDERS_HYLANDEXPERIENCE_CLIENTNAME An internal identifier for the Content Intelligence authentication provider, used by the connector to reference the correct credentials at runtime. The default value of acs-insight-connector is sufficient for all standard deployments.
acs-insight-connector AUTH_PROVIDERS_ALFRESCO_TYPE The authentication method used when making REST API calls to Alfresco Content Services. The default value of basic uses the username and password configured in AUTH_PROVIDERS_ALFRESCO_USERNAME and AUTH_PROVIDERS_ALFRESCO_PASSWORD.
basic AUTH_RETRY_ATTEMPTS The maximum number of retry attempts when an authentication or token request fails, for example due to a network error or a temporarily unavailable identity provider. Set to -1 for unlimited retries, which prevents startup failure during prolonged identity provider outages but may delay the detection of persistent authentication misconfigurations.
10 AUTH_RETRY_INITIALDELAY The time to wait before the first authentication retry attempt, in milliseconds. When AUTH_RETRY_DELAYMULTIPLIER is greater than 1, this value is used as the starting delay and multiplied on each subsequent attempt. When the multiplier is 1, this fixed delay is used between every retry.
500 AUTH_RETRY_DELAYMULTIPLIER The multiplier applied to the delay between each authentication retry attempt. For example, with the default multiplier of 2 and an initial delay of 500 ms, the delays between attempts are 500 ms, 1 s, 2 s, 4 s, and so on. Set to 1 to use a fixed delay between every retry attempt, as set by AUTH_RETRY_INITIALDELAY.
2 -
Save the docker-compose.yml file.
The Live Ingester is configured to receive Alfresco repository events through ActiveMQ and send document metadata and content to Content Intelligence in real time.
- Configure the Bulk Ingester. See Configuring the Bulk Ingester.