As part of the step by step series, I outlined the steps to install Alfresco Content Services 7.x and Solr6 (Alfresco Search Services 2.x) using distribution package in my previous posts.
You can take a look at the steps here:
I am here with another part of step by step series on ACS installation. We will try to setup Alfresco Content Services-7.2.x Enterprise with Elasticsearch (Search Enterprise-3.x) and all other components. This post will set the background for those who are looking to upgrade from Solr6 (Alfresco search service 2.x) to Elasticsearch (Search Enterprise-3.x) and evaluating.
Let me know in the comments as to why should you upgrade to Elasticsearch from Solr 6.
*** Updated this post to install ACS 7.2.1.3 that contains the patch for a memory leak issue. You can find more details here ***
What we need before we start doing setup?
- ACS-7.2.x package (alfresco-content-services-distribution-7.2.1.3)
- ASE-3.x package (alfresco-elasticsearch-connector-distribution-3.1.1)
- Alfresco Transform 1.5.x package (alfresco-transform-service-distribution-1.5.3)
- Java: Oracle jdk-11.0.13 or later/Open JDK 11.0.13 or later (I will use Oracle jdk-11.0.15 for this post)
- Elasticsearch 7.10
- Tomcat: Tomcat 9.0.62
- ActiveMQ: ActiveMQ v5.16.2.
- DB: PostgreSQL 13
- ImageMagick: ImageMagick v7.1.0
- Libreoffice: LibreOffice v7.0.6
Checkout the documentation for additional details on ACS Supported Platforms and Search Enterprise Supported Platforms
Platform:
- Windows 10 x64
- ACS, Share, Elasticsearch (Search Enterprise-3.x), and Transformation Service on same machine
Let’s download all the required packages that we need for the setup.
- Download Alfresco Content Services Enterprise 7.2.x package from Support Portal. If you have an enterprise license, you can also download the package from alfresco artifacts repository.
- Download Alfresco Transform Service 1.5 package from Support Portal. If you have an enterprise license, you can also download the package from alfresco artifacts repository.
Checkout this GitHub repo for more info on transform core all-in-one project
- Download Alfresco Search Enterprise 3.1.0 package from Support Portal. If you have an enterprise license, you can also download the package from alfresco artifacts repository.
- Download Alfresco model namespace map generator add-on, it can be used for setting up elastic search indexing app.
https://www.oracle.com/java/technologies/javase/jdk11-archive-downloads.html
Note: Make sure you set the JAVA_HOME environment variable (on windows). It is the installation path of jdk. E.g. JAVA_HOME=C:\Program Files\Java\jdk-11.0.15
https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.62/bin/apache-tomcat-9.0.62-windows-x64.zip
Note: Make sure ports 8005, 8080, 8443, AJP port 8009 are open and not in use already. These are default ports used for tomcat. If you have these ports already in use, make sure you change the ports accordingly in <TOMCAT_INSTALLATION>/conf/server.xml.
Download ActiveMQ v5.16.2 binary package (used for transformation service):
Alternative options (ImageMagick notoriously removes the specific versions and shows only latest versions):
- All other alternatives: https://downloadarchive.documentfoundation.org/libreoffice/old/
Download and Install PostgreSQL 13.x:
Alternatively you can also download the binary package and extract it. No installation needed. It is useful if you have trouble doing installation on Windows 10.
Download PostgreSQL 13.x binary
Note: Make sure port 5432 is open and not already in use. Port 5432 is default for postgres to get db connection. If you have this port already in use, make sure you select a different port and use the same while configuring alfresco-global.properties.
Optional Alfresco module packages (amps)-Useful for admins/developers:
*** Updated this post to provide prerequisites to install ACS 7.3.x and other related components. All the below given steps should be pretty much the same. You can find more details here ***
Now, we have completed all the prerequisites. Let’s start with the setup.
Assuming you have already installed Postgres, launch “pgAdmin4” from your postgresql install location <PostgresInstallLocation>\PostgreSQL\13\pgAdmin 4\bin\pgAdmin4.exe
In my case: C:\PostgreSQL\13\pgAdmin 4\bin\pgAdmin4.exe
- Create a new user named “alfresco” and set a password which will be used by alfresco content service to connect to database. Here we will be using username as “alfresco” and password “alfresco”.
- Create a new database named : “alfresco”, make sure you select default encoding which is “utf8”.
- Grant all permissions for user “alfresco” on database “alfresco”.
- Alternatively, you can open the query tool and execute following SQL queries:
create role alfresco LOGIN password 'alfresco'; create database alfresco encoding 'utf8'; grant all on database alfresco to alfresco;
Create directory structure as shown below on your windows drive (e.g. C:\):
You can create the same structure anywhere else, make sure all the config paths are mapped accordingly. For this post all the setup and configurations will be done in C:\ drive. You can also clone this git repository and copy the pre-existing structure if that helps.
Configure ACS repository:
- Install Tomcat, get the tomcat binary package (apache-tomcat-9.0.62-windows-x64.zip) downloaded as per the steps given above and extract its contents directly under tomcat directory. For example: C:\alfresco72-enterprise\tomcat directory.
Note: Make sure ports 8005, 8080, 8443, AJP port 8009 are open and not in use already. These are default ports used for tomcat. If you have these ports already in use, make sure you change the ports accordingly in <TOMCAT_INSTALLATION>/conf/server.xml.
Assuming you have already downloaded the distribution packages from download locations provided above. Use the below given steps to setup and configure alfresco-content-services-distribution-7.2.1.3
- Set ALF_HOME environment variable, this will be helpful for alfresco scripts such as apply_amps.bat, clean_tomcat.bat etc. to work properly. You always have option to manually change the paths in these files as needed. (Optional step)
- ALF_HOME=C:\alfresco72-enterprise
- Copy contents from “C:\Downloads\alfresco-content-services-distribution-7.2.1.3\amps” into “C:\alfresco72-enterprise\amps”. We will be installing alfresco module packages (amps) at later stage.
- alfresco-share-services.amp -> Alfresco share service module containing extensions for search, datalist, sample sites patch etc.
Note: alfresco-share-services.amp installation is mandatory otherwise several functionalities including share search may not work. And you will also see following warning when you login to share: Alfresco is running without Share Services. See your System Administrator for more details.
- If you have downloaded the additional amps (optional) given below, then copy them to amps/amps_share directories as appropriate. For this post we will be using these amps:
- javascript-console-repo-0.7.amp (Updated with ACS7 Compatibility bugfix)
- Copy 'javascript-console-repo' amp to "C:\alfresco72-enterprise\amps" directory
- javascript-console-share-0.7.amp (Updated with ACS7 Compatibility bugfix)
- Copy 'javascript-console-share' amp to "C:\alfresco72-enterprise\amps_share" directory
- Copy the contents from “C:\Downloads\alfresco-content-services-distribution-7.2.1.3\bin” into “C:\alfresco72-enterprise\bin” directory
- Following files will be copied:
- alfresco-mmt.jar -> Alfresco Module Management Tool (MMT), A java library which supports alfresco module package installation/uninstallation/listing and preview operations etc.
- alfresco-spring-encryptor.jar -> Useful for Encrypting configuration properties. For more details visit: https://docs.alfresco.com/content-services/latest/admin/security/#encrypting-configuration-properties
- apply_amps.bat -> Utility to install amps kept in “amps” and “amps_share” directory. It will install amps related to alfresco on alfresco.war (keeps the backup of original war file) and amps related to share on share.war (keeps the backup of original war file) For more details visit: https://docs.alfresco.com/content-services/latest/install/zip/amp/
- clean_tomcat.bat -> Shell script for cleaning out temporary application server files from previous installations.
- Win32NetBIOS.dll -> DLL handles the connection between the native CIFS server and Alfresco Content Services.
- Win32NetBIOSx64.dll -> DLL handles the connection between the native CIFS server and Alfresco Content Services.
- Win32Utils.dll -> DLL handles the connection between the native CIFS server and Alfresco Content Services.
- Win32Utilsx64.dll -> DLL handles the connection between the native CIFS server and Alfresco Content Services.
Copy licenses:
- Copy the contents from “C:\Downloads\alfresco-content-services-distribution-7.2.1.3\licenses” into “C:\alfresco72-enterprise\licenses” directory. It contains files that have information about license terms used by alfresco including all third party licenses.
- Copy the contents from “C:\Downloads\alfresco-content-services-distribution-7.2.1.3\web-server\conf” into “C:\alfresco72-enterprise\tomcat\conf” folder. 'conf' directory contains application specific context files for alfresco and share.
- Copy the contents from “C:\Downloads\alfresco-content-services-distribution-7.2.1.3\web-server\lib\” into “C:\alfresco72-enterprise\tomcat\lib\” folder. 'lib' directory contains the PostgreSQL JDBC jar file (postgresql-42.3.2.jar).
- Copy the contents from “C:\Downloads\alfresco-content-services-distribution-7.2.1.3\web-server\shared\classes” into “C:\alfresco72-enterprise\tomcat\shared\classes\” folder. 'shared' directory includes the Alfresco Content Services configuration files:
- \classes\alfresco-global.properties.sample -> Sample alfresco global properties file, which is used for configuration properties.
- \classes\alfresco-encrypted.properties -> For Encrypting properties. For more details visit: https://docs.alfresco.com/content-services/latest/admin/security/#encryptconfigprops
- \classes\alfresco -> Contains the directory structure for the configuration override files, including the extension, and web-extension directories.
- Create a new directory named 'keystore' under 'C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension' directory. We need to copy the 'metadata-keystore' related configs in this newly created directory.
- Copy the 'metadata-keystore' directory from 'C:\Downloads\alfresco-content-services-distribution-7.2.1.3\keystore\metadata-keystore' to 'C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\keystore' directory.
Note: The alfresco's keystore directory (dir.keystore) config location has changed from previous version. 'dir.keystore' property in alfresco-global.properties will now be something like:
'dir.keystore=C:/alfresco72-enterprise/tomcat/shared/classes/alfresco/extension/keystore'
Refer alfresco-global.properties
Metadata encryption keys needs to be now set using 'JAVA_TOOL_OPTIONS' java environment variable. For example:
JAVA_TOOL_OPTIONS="-Dencryption.keystore.type=JCEKS -Dencryption.cipherAlgorithm=DESede/CBC/PKCS5Padding -Dencryption.keyAlgorithm=DESede -Dencryption.keystore.location=C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\keystore\metadata-keystore\keystore -Dmetadata-keystore.password=mp6yc0UD9e -Dmetadata-keystore.aliases=metadata -Dmetadata-keystore.metadata.password=oKIWzVdEdA -Dmetadata-keystore.metadata.algorithm=DESede"
We will take a look at it later in this post. For more info visit: https://docs.alfresco.com/content-services/latest/admin/security/#alfresco-keystore-configuration
Delete all files/folders from “C:\alfresco72-enterprise\tomcat\webapps”.
- Following files will be copied:
- alfresco.war -> Alfresco repository web application.
- share.war -> Share interface web application.
- ROOT.war -> Application for the server root, also contains additional code/setup for handling alfresco office services module (alfresco-office-services). Alfresco Office Services (AOS) allow you to access Alfresco Content Services directly from all your Microsoft Office applications. We will not be installing this module since SSL is a mandatory enablement for this module.
- _vti_bin.war -> App to help and support AOS module.
- Delete all files/folders from “C:\alfresco72-enterprise\tomcat\work\Catalina\localhost” if there are any.
- Open and edit server.xml file which can be found under: “C:\alfresco72-enterprise\tomcat\conf” directory [Full path: C:\alfresco72-enterprise\tomcat\conf\server.xml]:
- Find Connector with port "8080”.
- Add the URIEncoding and maxHttpHeaderSize attributes.
- Add the enableLookups, xpoweredBy and server attributes. (Optional Attributes for Tomcat 8.x and later)
- Set xpoweredBy attribute to true to cause Tomcat to advertise support for the Servlet specification using the header recommended in the specification. The default value is false.
- Set enableLookups to true if you want calls to request.getRemoteHost() to perform DNS lookups in order to return the actual host name of the remote client. Set to false to skip the DNS lookup and return the IP address in String form instead (thereby improving performance). By default, DNS lookups are disabled.
- Setting server to true, overrides the server header for the http response. If set, the value for this attribute overrides any Server header set by a web application. If not set, any value specified by the application is used. If the application does not specify a value then no Server header is set.
<Connector port="8080" protocol="HTTP/1.1" URIEncoding="UTF-8" connectionTimeout="20000" maxHttpHeaderSize="32768" redirectPort="8443" enableLookups="false" xpoweredBy="false" server="AlfrescoECM"/>
Tomcat uses ISO-8859-1 character encoding when decoding URLs that are received from a browser. This can cause problems when creating, uploading, and renaming files with international characters. By default, Tomcat uses an 8 KB header buffer size, which might not be large enough for the Kerberos authentication protocol. We need to increase this buffer size.
Update AJP Connector config you are planning to use it. By default it is commented in tomcat server.xml, make sure you add URIEncoding attribute to the config if you are planning to use it. This port is used when configuring external authentication.
See here for more details:
<Connector port="8009" address="::1" URIEncoding="UTF-8" protocol="AJP/1.3" redirectPort="8443" xpoweredBy="false"/>
- Open and edit the C:\alfresco72-enterprise\tomcat\conf\catalina.properties file:
- Update the value of the shared.loader= property to the following:
shared.loader=${catalina.base}/shared/classes,${catalina.base}/shared/lib/*.jar
Note: If you have used alternative names for the directories, you must specify these names in the shared.loader property.
- We need to add JAVA_TOOL_OPTIONS java environment variable for metadata encryption, we had copied the keystore under 'C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\keystore\metadata-keystore' directory already. Check the steps above. Open and edit C:\alfresco72-enterprise\tomcat\bin\catalina.bat . Add the variable after JAVA_OPTS in catalina.bat file. If you add this variable at the end of the file, it is not getting recognized for some reason. So make sure you add the variavle before JAVA_OPTS as mentioned above. See the example screenshot below for reference or check here.
rem ACS72 Custom changes [Start] ##############set "JAVA_TOOL_OPTIONS=-Dencryption.keystore.type=JCEKS -Dencryption.cipherAlgorithm=DESede/CBC/PKCS5Padding -Dencryption.keyAlgorithm=DESede -Dencryption.keystore.location=C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\keystore\metadata-keystore\keystore -Dmetadata-keystore.password=mp6yc0UD9e -Dmetadata-keystore.aliases=metadata -Dmetadata-keystore.metadata.password=oKIWzVdEdA -Dmetadata-keystore.metadata.algorithm=DESede" rem ACS72 Custom changes [End] ##############
- Rename “C:\alfresco72-enterprise\tomcat\shared\classes\alfresco-global.properties.sample” to “alfresco-global.properties”
- Open and edit “C:\alfresco72-enterprise\tomcat\shared\classes\alfresco-global.properties” and add the following configuration properties:
- Add the “dir.root” property as path to alf_data folder and add 'dir.keystore' property to the above copied encryption keystore path (see here)
dir.root=C:/alfresco72-enterprise/alf_data dir.keystore=C:/alfresco72-enterprise/tomcat/shared/classes/alfresco/extension/keystore
- Add alfresco and share host, port, context and protocol specific properties. For more details see: https://docs.alfresco.com/content-services/latest/config/#using-alfresco-globalproperties
alfresco.context=alfresco
alfresco.host=${localname}
alfresco.port=8080
alfresco.protocol=http
share.context=share
share.host=${localname}
share.port=8080
share.protocol=http- Add the database connection properties which alfresco will use to create a JDBC connection with db. Use the db name and credentials when we prepared the database.
db.driver=org.postgresql.Driver
db.username=alfresco
db.password=alfresco
db.name=alfresco
db.url=jdbc:postgresql://localhost:5432/${db.name}
db.pool.max=275
db.pool.validate.query=SELECT 1- Add the server mode property, leave it default to ‘UNKNOWN’.
system.serverMode=UNKNOWN
- Add the alfresco rmi services port and host properties.
alfresco.rmi.services.port=50500
alfresco.rmi.services.host=0.0.0.0
To setup Transformation service, we need to install Imagemagick, Libreoffice and AlfrescoPDFRenderer locally and executables will be used by transformation service spring boot application.
- Install imagemagick in the directory setup as per the structure given above. E.g. install ImageMagick-7.1.0 in C:\alfresco72-enterprise\imagemagick directory (use custom installation option to select the install directory). Alternatively you can download the binary package and extract the image magic binaries in the folder given above as per your choice.
- Install libreoffice in the directory setup as per the structure given above. E.g. install LibreOffice_7.0.6.2 in C:\alfresco72-enterprise\libreoffice directory (use custom installation option to select the install directory).
- Install ActiveMQ, get the activemq binary package (apache-activemq-5.16.2-bin.zip) downloaded as per the steps given above and extract its contents directly under activemq directory. For example: C:\alfresco72-enterprise\activemq directory.
Note: Make sure ports 61616, 5672, 61613, 1883, 61614 and 8161 are open and not in use already. These are default ports used for activemq. If you have these ports already in use, make sure you change the ports accordingly in <AMQ_INSTALLATION>/conf/activemq.xml and jetty.xml as needed.
Alfresco PDF Renderer Setup:
Assuming you have already downloaded the alfresco-pdf-renderer package from download locations provided above. Use the below given steps to install alfresco-pdf-renderer.
Extract “C:\Downloads\alfresco-pdf-renderer-1.1-win64.tgz", then again extract from "alfresco-pdf-renderer-1.1-win64.tar" and the copy the executables to "C:\alfresco72-enterprise\alfresco-pdf-renderer"
Exiftool Setup:
Seems like TIKA part of Local transformation service app is looking for exiftool to be available locally. It is used by TIKA for extracting image metadata if TIKAAuto metadata extraction is enabled. I noticed an error when 'exiftool-parser' was invoked by TIKA. So we will download and copy the exiftool under alfresco installation and export its path in environment PATH variable. Assuming you have already downloaded the exiftool package from download locations provided above. Use the below given steps to install exiftool
- Create a new directory named 'exiftool' under "C:\alfresco72-enterprise\"
- Extract “C:\Downloads\image-exiftool-12.25.tgz", then again extract from "Image-ExifTool-12.25.tar" and the copy the executables to "C:\alfresco72-enterprise\exiftool"
- Set the exiftool directory to Windows PATH Environment Variable:
%ALF_HOME%\exiftool
Open and edit “C:\alfresco72-enterprise\tomcat\shared\classes\alfresco-global.properties”
- Enable transformation service:
local.transform.service.enabled=true
# Transform Core properties: localTransform.core-aio.url=http://localhost:8090 # Shared File Store properties: sfs.url=http://localhost:8099 sfs.endpoint=${sfs.url}/alfresco/api/-default-/private/sfs/versions/1/file # Transform Router properties: transform.service.enabled=true transform.service.url=http://localhost:8095
- Add the activemq url property and enabled messaging subsystem. It will be used when you setup transformation service.
messaging.broker.url=tcp://localhost:61616
#This property is default true, here it it for information purpose.
messaging.subsystem.autoStart=true
#If you have setup username and password for AMQ, then set the below properties. In my case I have kept default admin/adminmessaging.broker.username=admin messaging.broker.password=admin
Note (Only applicable to Enterprise Edition, For community edition local transform service is only available option and must be enabled):
You can optionally disable local.transform service in alfresco Docker Container and enable transform service (asynchronous). Local Transform Service or Transform Service (supports only asynchronous requests) can be enabled or disabled independently of each other. Please keep in mind that when your deployment has Share and SOLR (think of full text indexing), or both then you'll need to have local.transform and transform service (asynchronous) enabled and running. The Repository will try to transform content using the Transform Service via the T-Router if possible and fall back to direct Local Transform Service. Share makes use of both, so functionality such as preview will be unavailable if local.transform service is disabled.
Setup transformation core aio:
Assuming you have downloaded the transform service package (alfresco-transform-service-distribution-1.5.3) already as per instruction given above.
- Extract the package alfresco-transform-service-distribution-1.5.3.zip
- Copy all the jar files to ''C:\alfresco72-enterprise\bin\" directory.
- Following jars (spring boot apps) will be copied:
- alfresco-shared-file-store-controller-1.5.3.jar
- alfresco-transform-core-aio-boot-2.6.0.jar
- alfresco-transform-router-1.5.3.jar
- Create a new directory named 'shared-file-store' under ''C:\alfresco72-enterprise" directory. This directory will be used for transform service shared file store. The Shared File Store allows components such as the repository, and the Transform Service to share a common place to store and retrieve files, for example, to enable transforms from an input source file to an output target file. You can choose to create this directory anywhere else as well, but make sure you map the path to the startup script accordingly. In this case path would be: C:\alfresco72-enterprise\shared-file-store
Post setup of activemq, imagemagick, libreoffice, alfresco-pdf-renderer and transform servive spring boot apps, you can start transform-core-aio-boot, transform-router and shared-file-store-controller apps with following java command with given params:
To start shared-file-store-controller, use the below given command:
java -Dscheduler.contract.path=</path/to/your/alfrescoInstallation>/scheduler.json ^
-DfileStorePath=</path/to/your/AlfrescoFileStore> ^
-Dscheduler.content.age.millis=86400000 ^
-Dscheduler.cleanup.interval=86400000 ^-jar <path_to_sfs-controller-jar>/alfresco-shared-file-store-controller-x.y.z.jar
java -DfileStorePath=C:\alfresco72-enterprise\shared-file-store ^
-Dscheduler.contract.path=C:\alfresco72-enterprise\shared-file-store-scheduler-location\scheduler.json ^ -Dscheduler.content.age.millis=86400000 ^
-Dscheduler.cleanup.interval=86400000 ^-jar C:\alfresco72-enterprise\bin\alfresco-shared-file-store-controller-1.5.3.jar
- Check the output to ensure that it starts successfully. By default, files are stored in fileStorePath=/tmp/Alfresco. This can be modified using the "fileStorePath" parameter as shown in the above example.
- Make sure you also pass the "scheduler.contract.path" parameter which tells the app where should it setup scheduler for cleanup. By default the path is: scheduler.contract.path=/tmp/scheduler.json. When the "shared-file-store-controller" started , the scheduler.json will be created automatically (you don't need to create it).
- Pass the following parameters if you want to control the cleanup schedule (values are presented in millis). See the above startup command example as well:
scheduler.content.age.millis scheduler.cleanup.interval
- Create a directory named "shared-file-store-scheduler-location" under your alfresco installation, example: C:\alfresco72-enterprise and use this "C:\alfresco72-enterprise\shared-file-store-scheduler-location" path for the "scheduler.contract.path" param.
Example: "-Dscheduler.contract.path=C:\alfresco72-enterprise\shared-file-store-scheduler-location\scheduler.json"
To start transform-core-aio-boot, use the below given command:
java -DPDFRENDERER_EXE="<alfresco-pdf-renderer_installation_dir>/alfresco-pdf-renderer" -DLIBREOFFICE_HOME="<libreoffice_installation_dir>" -DIMAGEMAGICK_ROOT="<imagemagick_installation_dir>" -DIMAGEMAGICK_DYN="<imagemagick_installation_dir>/lib" -DIMAGEMAGICK_EXE="<imagemagick_installation_dir>/convert" -DIMAGEMAGICK_CODERS="<imagemagick_installation_dir>/modules/coders" -DIMAGEMAGICK_CONFIG="<imagemagick_installation_dir>" -DACTIVEMQ_URL=tcp://server:61616 -jar <path_to_tranform-core-aio-jar>/alfresco-transform-core-aio-boot-x.y.z.jar
java -DPDFRENDERER_EXE="C:\alfresco72-enterprise\alfresco-pdf-renderer\alfresco-pdf-renderer.exe" -DLIBREOFFICE_HOME="C:\alfresco72-enterprise\libreoffice" -DIMAGEMAGICK_ROOT="C:\alfresco72-enterprise\imagemagick" -DIMAGEMAGICK_DYN="C:\alfresco72-enterprise\imagemagick\lib" -DIMAGEMAGICK_CODERS="C:\alfresco72-enterprise\imagemagick\modules\coders" -DIMAGEMAGICK_CONFIG="C:\alfresco72-enterprise\imagemagick" -DIMAGEMAGICK_EXE="C:\alfresco72-enterprise\imagemagick\convert.exe" -DACTIVEMQ_URL=tcp://localhost:61616 -jar C:\alfresco72-enterprise\bin\alfresco-transform-core-aio-boot-2.6.0.jarCheck the output to ensure that it starts successfully. The all-in-one core T-Engine combines the five T-Engines (i.e. LibreOffice, ImageMagick, Alfresco PDF Renderer, Tika, and Misc.) into one single engine. All functionality that’s available in the five T-Engines is available in the all-in-one core T-Engine. The command-line options provide the paths to the installation locations and the URL of the messaging broker.
If local transformation service is correctly setup as per above instructions and starts up successfully, then you can also access the local transformation service at http://localhost:8090 and play around with samples.
To start transform-router, use the below given command:
java -DCORE_AIO_URL=http://localhost:8090 ^
-DCORE_AIO_QUEUE=org.alfresco.transform.engine.aio.acs ^
-DACTIVEMQ_URL=tcp://localhost:61616 ^
-DFILE_STORE_URL=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file ^
-jar <path_to_transform-router-jar>\bin\alfresco-transform-router-x.y.z.jarjava -DCORE_AIO_URL=http://localhost:8090 ^
-DCORE_AIO_QUEUE=org.alfresco.transform.engine.aio.acs ^
-DACTIVEMQ_URL=tcp://localhost:61616 ^
-DFILE_STORE_URL=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file ^-jar C:\alfresco72-enterprise\bin\alfresco-transform-router-1.5.3.jar
Check the output to ensure that it starts successfully. The Transform Router allows simple (single-step) and pipeline (multi-step) transforms that are passed to the Transform Engines. The command-line options provide the router with the required data for T-Engines, queuing, and file-store URL.
The transform-core-aio-boot is with OOTB functionalities. As many of you know that most transformation service related classes are moved to alfresco-transform-core project.
Check here what's in repo and what is moved: https://docs.alfresco.com/content-services/latest/develop/repo-ext-points/metadata-extractors/#ootbextractors
Take an example of metadata extractors, earlier when you had to override metadata extractor configuration such as for PdfBoxMetadataExtracter, TikaAutoMetadataExtractor, PoiMetadataExtracter etc., you can define spring beans and override them by providing custom <metadata Extractor>.properties file on the repository side itself as legacy transformation service was still within repository. See example here up to ACS6.x.
Now, there is no more legacy transformation service available, you have to build custom docker image with your customization (if you are using containers) or build jar file with your custom changes to launch Local transformation service (if setup is using distribution package or ansible).
Documentation here indicates that, you can directly update the properties here and build customized jar/docker images and use it. I haven't tried myself yet, but do checkout the pointers given above.
Some useful posts pertaining to transformation service for you to refer:
Alfresco Transform Core Project
Alfresco Transform Core AIO Boot App
How to use local transformation service
Local vs Legacy Transformation Service
Update alfresco-global.properties to add other misc. properties:
- Add property for email notification on invite, it is by default disabled.
notification.email.siteinvite=false
- Add the license location property.
### License location ### dir.license.external=C:/alfresco72-enterprise
- Add following properties related to security, smart folder and JMX. All values are default out-of-the-box
security.anyDenyDenies=false smart.folders.enabled=false alfresco.jmx.connector.enabled=false
- Enable logging so that you can configure any custom loggers or change/update log levels (Optional)
- Go to C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\ folder and rename “custom-log4j.properties.sample” to “custom-log4j.properties”
- You can change the log levels as per your need.
- Mandatory Amps:
- “alfresco-share-services.amp” amp is mandatory for alfresco.war
Note: This amp installation is mandatory otherwise several functionalities including share search may not work. And you will also see following warning when you login to share: Alfresco is running without Share Services. See your System Administrator for more details
- Optional Amps: As part of this installation, we will be installing some optional amps which will be useful for admins/developers that can help in debugging and administration. In a previous step above we downloaded the amps and copied to appropriate directories.
- javascript-console-repo-0.7.3.amp -> A module that provides ability for developers and administrators to execute JavaScript code.
- javascript-console-share-0.7.3.amp -> A module that provides ability for developers and administrators to execute JavaScript code.
https://github.com/abhinavmishra14/js-console
- Execute 'apply_amps.bat' script to apply amps, it will install amps copied under C:\alfresco72-enterprise\amps and C:\alfresco72-enterprise\amps_share directories to alfresco.war and share.war.
- Open windows command prompt
- Go to C:\alfresco72-enterprise\bin path
- Execute the 'apply_amps.bat'
- After amps are install, you can choose to remove the backup files created by apply_amps.bat tool.
- You would see following output as result of executing 'apply_amps.bat':
This script will apply all the AMPs in C:\alfresco72-enterprise\amps to the alfresco.war and share.war files in C:\alfresco72-enterprise\tomcat\webapps Press control-c to stop this script . . . Press any key to continue . . . Module 'de.fme.alfresco.JavascriptConsole-repo' installed in 'C:\alfresco72-enterprise\tomcat\webapps\alfresco.war' - Title: fme Javascript Console Repository Extension - Version: 0.7.3 - Install Date: Thu Jun 02 00:45:14 EDT 2022 - Description: Adminstration console module to execute arbitrary javascript code. Module 'alfresco-share-services' installed in 'C:\alfresco72-enterprise\tomcat\webapps\alfresco.war' - Title: Alfresco Share Services AMP - Version: 14.148 - Install Date: Thu Jun 02 00:45:13 EDT 2022 - Description: Module to be applied to alfresco.war, containing APIs for Alfresco Share Module 'de.fme.alfresco.JavascriptConsole-share' installed in 'C:\alfresco72-enterprise\tomcat\webapps\share.war' - Title: fme Javascript Console Share Extension - Version: 0.7.3 - Install Date: Thu Jun 02 00:45:18 EDT 2022 - Description: Adminstration console module to execute arbitrary javascript code. . About to clean out tomcat/webapps/alfresco directory and temporary files... Press any key to continue . . .
- Make sure you update the "repository-url" to point to the alfresco host and port, in case you are setting up alfresco and share on different servers. Default values are "localhost" and "8080". We are setting up both ACS and Share on same host so we will leave it default.
<!-- If set, will present a WebDAV link for the current item on the Document and Folder details pages. Also used to generate the "View in Alfresco Explorer" action for folders. --> <repository-url>http://localhost:8080/alfresco</repository-url>
- Make sure you update the share "Remote" configuration to point to the alfresco host and port, in case you are setting up alfresco and share on different servers. Default values are "localhost" and "8080". We are setting up both ACS and Share on same host so we will leave it default.
<config evaluator="string-compare" condition="Remote"> <remote> <endpoint> <id>alfresco-noauth</id> <name>Alfresco - unauthenticated access</name> <description>Access to Alfresco Repository WebScripts that do not require authentication</description> <connector-id>alfresco</connector-id> <endpoint-url>http://localhost:8080/alfresco/s</endpoint-url> <identity>none</identity> </endpoint> <endpoint> <id>alfresco</id> <name>Alfresco - user access</name> <description>Access to Alfresco Repository WebScripts that require user authentication</description> <connector-id>alfresco</connector-id> <endpoint-url>http://localhost:8080/alfresco/s</endpoint-url> <identity>user</identity> </endpoint> <endpoint> <id>alfresco-feed</id> <name>Alfresco Feed</name> <description>Alfresco Feed - supports basic HTTP authentication via the EndPointProxyServlet</description> <connector-id>http</connector-id> <endpoint-url>http://localhost:8080/alfresco/s</endpoint-url> <basic-auth>true</basic-auth> <identity>user</identity> </endpoint> <endpoint> <id>alfresco-api</id> <parent-id>alfresco</parent-id> <name>Alfresco Public API - user access</name> <description>Access to Alfresco Repository Public API that require user authentication. This makes use of the authentication that is provided by parent 'alfresco' endpoint.</description> <connector-id>alfresco</connector-id> <endpoint-url>http://localhost:8080/alfresco/api</endpoint-url> <identity>user</identity> </endpoint> </remote> </config>
For more details see:
Configure Subsystem in Repository
- Open and edit “C:\alfresco72-enterprise\tomcat\shared\classes\alfresco-global.properties”
################## Set the Elasticsearch subsystem [Start] ##################### #For more details check here: https://docs.alfresco.com/search-enterprise/latest/config/
index.subsystem.name=elasticsearch
# Elasticsearch index properties elasticsearch.indexName=alfresco elasticsearch.createIndexIfNotExists=true
# Elasticsearch server properties elasticsearch.host=localhost elasticsearch.port=9200 elasticsearch.baseUrl=/
#default none, https and none possible elasticsearch.secureComms=none ################## Set the Elasticsearch subsystem [End] #####################
These configuration properties are used by Alfresco Content Services to communicate with the Elasticsearch server. Here we are configuring plain HTTP connection, but Alfresco Repository also supports communication with Elasticsearch server using Basic Authentication and the HTTPs protocol. For more details visit: https://docs.alfresco.com/search-services/latest/config/
Assuming you have already downloaded the alfresco-elasticsearch-connector-distribution as part of pre-requisite steps. We need to configure the elastic search re-indexing app.
Elasticsearch connector live indexing app listens to messages from ActiveMQ. This means some initial information from Alfresco Repository must be indexed using the Re-indexing component. The Re-indexing app can also be used to index a pre-populated Alfresco Repository.
- Create a new directory/folder named 'alfresco-elastic-search-services' in installation drive e.g. C:\ drive
- Unzip the “alfresco-elasticsearch-connector-distribution-3.1.1.zip” package which we downloaded initially
- Copy the extracted jar files, license etc. to “C:\alfresco-elastic-search-services” Following files/folders needs to be copied:
- alfresco-elasticsearch-live-indexing-3.1.1-app.jar (This is AIO app that contains alfresco-elasticsearch-live-indexing-content-3.1.1-app, alfresco-elasticsearch-live-indexing-mediation-3.1.1-app, alfresco-elasticsearch-live-indexing-metadata-3.1.1-app and alfresco-elasticsearch-live-indexing-path-3.1.1-app built-in). No need to copy the additional jars.
- alfresco-elasticsearch-reindexing-3.1.1-app.jar
- README.md
- third-party-indexing.txt
- third-party-reindexing.txt
- licenses (folder)
For more details visit: https://docs.alfresco.com/search-enterprise/latest/install/#install-using-jar-files
- Generate a JSON mapping of namespace to prefix, for your deployed content models. This is required so that all ootb and custom models are known to the indexer. Sample JSON mapping looks like (with ootb models):
{
"prefixUriMap": {
"":"",
"http://www.alfresco.org/model/workflow/invite/nominated/1.0":"inwf",
"http://www.alfresco.org/model/solrfacetcustomproperty/1.0":"srftcustom",
"http://www.alfresco.org/model/datalist/1.0":"dl",
"http://www.alfresco.org/model/webdav/1.0":"webdav",
"http://www.alfresco.org/model/cmis/1.0/cs01ext":"cmisext",
"http://www.alfresco.org/model/distributionpolicies/1.0/model":"dp",
"http://www.alfresco.org/view/repository/1.0":"view",
"http://www.alfresco.org/model/download/1.0":"download",
"http://www.alfresco.org/model/publishing/twitter/1.0":"twitter",
"http://www.alfresco.org/model/action/1.0":"act",
"http://www.alfresco.org/system/registry/1.0":"reg",
"http://www.alfresco.org/model/user/1.0":"usr",
"http://www.alfresco.org/model/calendar":"ia",
"http://www.alfresco.org":"alf",
"http://www.alfresco.org/model/content/metadata/IPTCXMP/1.0":"iptcxmp",
"http://www.alfresco.org/model/application/1.0":"app",
"http://www.alfresco.org/model/surf/1.0":"surf",
"http://www.alfresco.org/model/versionstore/1.0":"ver",
"http://www.alfresco.org/system/modules/1.0":"module",
"http://www.alfresco.org/model/linksmodel/1.0":"lnk",
"http://iptc.org/std/Iptc4xmpExt/2008-02-29/":"Iptc4xmpExt",
"http://ns.adobe.com/photoshop/1.0/":"photoshop",
"http://www.alfresco.org/model/sync/1.0":"sync",
"http://ns.useplus.org/ldf/xmp/1.0/":"plus",
"http://www.alfresco.org/model/remotecredentials/1.0":"rc",
"http://www.alfresco.org/model/emailserver/1.0":"emailserver",
"http://www.alfresco.org/model/hybridworkflow/1.0":"hwf",
"http://www.alfresco.org/model/sitecustomproperty/1.0":"stcp",
"http://www.alfresco.org/model/cmis/1.0/cs01":"cmis",
"http://www.alfresco.org/model/transfer/1.0":"trx",
"http://www.fme.de/alfresco/models/jsconsole":"jsc",
"http://www.alfresco.org/model/rendition/1.0":"rn",
"http://www.alfresco.org/model/exif/1.0":"exif",
"http://www.alfresco.org/model/publishing/youtube/1.0":"youtube",
"http://ns.adobe.com/xap/1.0/rights/":"xmpRights",
"http://www.alfresco.org/model/custommodelmanagement/1.0":"cmm",
"http://www.alfresco.org/model/workflow/invite/moderated/1.0":"imwf",
"http://www.alfresco.org/model/forum/1.0":"fm",
"http://www.alfresco.org/model/rule/1.0":"rule",
"http://www.alfresco.org/model/publishing/linkedin/1.0":"linkedin",
"http://www.alfresco.org/model/publishing/slideshare/1.0":"slideshare",
"http://www.alfresco.org/model/system/1.0":"sys",
"http://www.alfresco.org/model/content/smartfolder/1.0":"smf",
"http://www.alfresco.org/model/workflow/1.0":"wf",
"http://www.alfresco.org/model/qshare/1.0":"qshare",
"http://www.alfresco.org/model/versionstore/2.0":"ver2",
"http://www.alfresco.org/model/solrfacet/1.0":"srft",
"http://www.alfresco.org/model/audio/1.0":"audio",
"http://www.alfresco.org/model/blogintegration/1.0":"blg",
"http://www.alfresco.org/model/bpm/1.0":"bpm",
"http://www.alfresco.org/model/site/1.0":"st",
"http://www.alfresco.org/model/imap/1.0":"imap",
"http://www.alfresco.org/model/dictionary/1.0":"d",
"custom.model":"custom",
"http://www.alfresco.org/model/publishing/facebook/1.0":"facebook",
"http://www.alfresco.org/model/content/1.0":"cm",
"http://www.alfresco.org/model/cmis/custom":"cmiscustom",
"http://www.alfresco.com/model/activiti-bpm-suite/1.0":"abs",
"http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/":"Iptc4xmpCore",
"http://www.alfresco.org/model/publishing/flickr/1.0":"flickr",
"http://www.alfresco.org/model/workflow/resetpassword/1.0":"resetpasswordwf",
"http://www.alfresco.org/model/cmis/1.0/alfcmis":"alfcmis",
"http://www.alfresco.org/model/publishing/1.0":"pub",
"http://purl.org/dc/elements/1.1/":"dc",
"":""
}
}- To help you build the JSON file you can use Alfresco Model Namespace-Prefix Mapping. Assuming you have downloaded module named 'model-ns-prefix-mapping' as stated above in pre-requisites steps.
- Copy the 'model-ns-prefix-mapping-1.0.0.jar' to 'C:\alfresco72-enterprise\modules\platform'
- You can get above json by calling this endpoint if needed: http://localhost:8080/alfresco/s/model/ns-prefix-map
- Use the above sample json mapping and save it into a file named "reindex.prefixes-file.json" under path "C:\alfresco-elastic-search-services". Full path will be: "C:\alfresco-elastic-search-services\reindex.prefixes-file.json"
- Exact term search config (Optional):
- The Exact Term search feature that allows searching using the equals operator =, is disabled by default to save index space. It’s possible to enable it for specific properties and property types using the "/alfresco/search/elasticsearch/config/exactTermSearch.properties" configuration file located in the Alfresco Repository.
- Following are the configurations:
- alfresco.cross.locale.datatype.0 : A new cross locale field is added for any property of this data-type to enable exact term search. For example, {http://www.alfresco.org/model/dictionary/1.0}text. The Exact Term search is disabled by default.
- alfresco.cross.locale.property.0 : A new cross locale field is added for the property to enable exact term search. For example, {http://www.alfresco.org/model/content/1.0}content. The Exact Term search is disabled by default.
- Create a directory named "search" under "C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension"
- Create another directory named "elasticsearch" under "C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\search"
- Create another directory named "config" under "C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\search\elasticsearch"
- Create a file "exactTermSearch.properties" in "C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\search\elasticsearch\config".
- Add the following configs in "C:\alfresco72-enterprise\tomcat\shared\classes\alfresco\extension\search\elasticsearch\config\exactTermSearch.properties":
# Exact Term search is switched off by default as it introduces index size overhead.
# to enable it, please uncomment the following on a datatype or property name basis
alfresco.cross.locale.datatype.0={http://www.alfresco.org/model/dictionary/1.0}text
alfresco.cross.locale.datatype.1={http://www.alfresco.org/model/dictionary/1.0}content
alfresco.cross.locale.datatype.2={http://www.alfresco.org/model/dictionary/1.0}mltext
alfresco.cross.locale.property.0={http://www.alfresco.org/model/content/1.0}contentFor more details visit: https://docs.alfresco.com/search-enterprise/latest/admin/
Assuming you have already download elasticsearch server package as given above.
- Unzip the “elasticsearch-7.10.1” package which we downloaded initially.
- Copy the extracted folder named "elasticsearch-7.10.1" etc. to “C:\"
For more details visit: https://www.elastic.co/guide/en/elasticsearch/reference/current/zip-windows.html
- Elastic search Indices are stored in "data" directory. In current installation for example: "C:\elasticsearch-7.10.1\data"
- "elasticsearch.bat" (C:\elasticsearch-7.10.1\bin\elasticsearch.bat) file can be used to start elastic search server.
- To get all indices, delete specific indices or delete all indices you can use following commands (Just FYI):
Query the indices that are available: curl -GET http://localhost:9200/_cat/indices You can delete them by name (Do not try to delete 'alfresco' indices unless you are sure what you doing. Here it is just an example): curl -X DELETE http://localhost:9200/alfresco If you wanted to delete all indices: curl -X DELETE http://localhost:9200/_allIf you wanted to create an indices, e.g. 'alfresco':curl -X PUT http://localhost:9200/alfresco
Alfresco re-indexing app is used to run re-indexing and requires a working Alfresco Repository and Elastic Search server. To learn more on what all properties supported, visit here.
Steps to start/run Alfresco Re-indexing app:
- To start Alfresco Re-indexing app, use the following command (Use the appropriate file path for reindex.prefixes-file.json as per your setup):
java -jar alfresco-elasticsearch-reindexing-x.y.z-app.jar^
--alfresco.reindex.jobName=reindexByIds ^
--spring.elasticsearch.rest.uris=http://localhost:9200 ^
--spring.datasource.url=jdbc:postgresql://localhost:5432/alfresco ^
--spring.datasource.username=alfresco ^
--spring.datasource.password=alfresco ^
--spring.activemq.broker-url=tcp://localhost:61616?jms.useAsyncSend=true ^
--alfresco.reindex.prefixes-file=file:reindex.prefixes-file.json
java -jar C:\alfresco-elastic-search-services\alfresco-elasticsearch-reindexing-3.1.1-app.jar ^--alfresco.reindex.jobName=reindexByIds ^--spring.elasticsearch.rest.uris=http://localhost:9200 ^--spring.datasource.url=jdbc:postgresql://localhost:5432/alfresco ^--spring.datasource.username=alfresco ^--spring.datasource.password=alfresco ^--spring.activemq.broker-url=tcp://localhost:61616?jms.useAsyncSend=true ^--alfresco.reindex.prefixes-file=file:C:\alfresco-elastic-search-services\reindex.prefixes-file.json
- Upon successful start you should see following sort of log, after re-indexing completes, the app will shutdown automatically. It can be restarted on-demand using above given command:
2022-06-03 23:00:47.051 INFO 16972 --- [main] o.s.batch.core.step.AbstractStep : Step: [reindexByIdsStep] executed in 5s781ms
2022-06-03 23:00:47.067 INFO 16972 --- [main] o.a.r.w.ElasticsearchRepoEventItemWriter : Total number of indexed nodes (includes retried items) 815
2022-06-03 23:00:47.067 INFO 16972 --- [main] o.a.r.w.ElasticsearchRepoEventItemWriter : Total number of failed nodes (includes retried items) 0
2022-06-03 23:00:47.083 INFO 16972 --- [main] o.a.r.writers.ElasticsearchPathWriter : Total number of updated nodes with Path (includes retried items) 814
2022-06-03 23:00:47.083 INFO 16972 --- [main] o.a.r.writers.ElasticsearchPathWriter : Total number of nodes with failed Path update (includes retried items) 0
2022-06-03 23:00:47.098 INFO 16972 --- [main] o.a.r.listeners.JobLifecycleListener : Job 0, current status: COMPLETED
2022-06-03 23:00:47.114 INFO 16972 --- [main] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=reindexByIds]] completed with the following parameters: [{run.id=1}] and the following status: [COMPLETED] in 6s406ms
2022-06-03 23:00:47.176 INFO 16972 --- [main] org.alfresco.reindexing.ReindexingApp : JobExecution: id=0, version=2, startTime=2022-06-03 23:00:40.692, endTime=2022-06-03 23:00:47.098, lastUpdated=2022-06-03 23:00:47.098, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, Job=[reindexByIds]], jobParameters=[{run.id=1}]
2022-06-03 23:00:47.192 INFO 16972 --- [main] o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.15.0 (camel-1) shutting down (timeout:45s)
2022-06-03 23:00:47.286 INFO 16972 --- [main] o.a.c.impl.engine.AbstractCamelContext : Routes stopped (total:0 stopped:0)
2022-06-03 23:00:47.317 INFO 16972 --- [main] o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.15.0 (camel-1) shutdown in 125ms (uptime:8s656ms)
2022-06-03 23:00:48.176 INFO 16972 --- [main] o.s.i.endpoint.EventDrivenConsumer : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2022-06-03 23:00:48.176 INFO 16972 --- [main] o.s.i.channel.PublishSubscribeChannel : Channel 'application.errorChannel' has 0 subscriber(s).
2022-06-03 23:00:48.192 INFO 16972 --- [main] o.s.i.endpoint.EventDrivenConsumer : stopped bean '_org.springframework.integration.errorLogger'
2022-06-03 23:00:50.239 INFO 16972 --- [main] o.a.r.w.ElasticsearchRepoEventItemWriter : Total number of indexed nodes (includes retried items) 815
2022-06-03 23:00:50.239 INFO 16972 --- [main] o.a.r.w.ElasticsearchRepoEventItemWriter : Total number of failed nodes (includes retried items) 0
2022-06-03 23:00:50.255 INFO 16972 --- [main] o.a.r.writers.ElasticsearchPathWriter : Total number of updated nodes with Path (includes retried items) 0
2022-06-03 23:00:50.255 INFO 16972 --- [main] o.a.r.writers.ElasticsearchPathWriter : Total number of nodes with failed Path update (includes retried items) 0
2022-06-03 23:00:50.270 WARN 16972 --- [main] o.s.b.f.support.DisposableBeanAdapter : Custom destroy method 'close' on bean with name 'reindexByDateItemReader' threw an exception: org.springframework.batch.item.ItemStreamException: Error while closing item reader
2022-06-03 23:00:50.286 INFO 16972 --- [main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-06-03 23:00:50.286 INFO 16972 --- [main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2022-06-03 23:00:50.333 INFO 16972 --- [main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.Within the Elastic Search Connector there are a subset of components that work on indexing data. There is a component called Mediation (alfresco-elasticsearch-live-indexing-mediation) which subscribes to the alfresco.event.topic (activemq:topic:alfresco.repo.event2) and processes the incoming node events. The configuration of this component allows you to declare four blacklist sets for filtering out nodes or attributes to be indexed. These blacklists can be specified in the file using the alfresco.mediation.filter-file property. The default file is called mediation-filter.yml that must be in the module classpath.
Note: Default mediation-filter config resides in 'alfresco-elasticsearch-live-indexing-shared-xxx.jar' which is a dependency of alfresco-elasticsearch-live-indexing-xxx-app.
mediation-filter.yml out of the box (showing blacklisted aspects and fields):
mediation: nodeTypes: contentNodeTypes: nodeAspects: - sys:hidden fields: - cmis:changeToken - alfcmis:nodeRef - cmis:isImmutable - cmis:isLatestVersion - cmis:isMajorVersion - cmis:isLatestMajorVersion - cmis:isVersionSeriesCheckedOut - cmis:versionSeriesCheckedOutBy - cmis:versionSeriesCheckedOutId - cmis:checkinComment - cmis:contentStreamId - cmis:isPrivateWorkingCopy - cmis:allowedChildObjectTypeIds - cmis:sourceId - cmis:targetId - cmis:policyText - trx:password - pub:publishingEventPayload
Where:
nodeTypes: if the node wrapped in the incoming event has a type which is included in this set, the node processing is skipped.
contentNodeTypes: if the node wrapped in the incoming event has a content change associated with it and it has a type which is included in this set, then the corresponding content processing won’t be executed. This means nodes belonging to one of the node types in this set, won’t have any content indexed in Elasticsearch.
nodeAspects: if the node wrapped in the incoming event has an aspect which is included in this set, the node processing is skipped.
fields: fields listed in this set are removed from the incoming nodes metadata. This means fields in this set won’t be sent to Elasticsearch for indexing, and therefore they won’t be searchable.
Steps to start/run Alfresco Live-indexing app:
- To start Alfresco Live Indexing app, use the following command:
java -jar <path_to_jar_file>\alfresco-elasticsearch-live-indexing-x.y.z-app.jar
- Following parameters can be passed:
server.port
spring.activemq.broker-url
spring.elasticsearch.rest.uris
alfresco.sharedFileStore.baseUrl
alfresco.accepted-content-media-types-cache.base-url
elasticsearch.indexName
alfresco.mediation.filter-filejava -jar C:\alfresco-elastic-search-services\alfresco-elasticsearch-live-indexing-3.1.1-app.jar ^
--spring.activemq.broker-url=nio://activemq.local:61616 ^ --spring.elasticsearch.rest.uris=http://localhost:9200 ^ --alfresco.sharedFileStore.baseUrl=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/ ^ --alfresco.accepted-content-media-types-cache.base-url=http://localhost:8090/transform/config ^ --elasticsearch.indexName=alfresco
- If required additional memory may be assigned to these services using the default JVM options. For instance, to start the Elasticsearch connector with 4 GB of RAM.
java -Xmx4G -jar C:\alfresco-elastic-search-services\alfresco-elasticsearch-live-indexing-3.1.1-app.jar ^
--spring.activemq.broker-url=nio://activemq.local:61616 ^ --spring.elasticsearch.rest.uris=http://localhost:9200 ^ --alfresco.sharedFileStore.baseUrl=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/ ^ --alfresco.accepted-content-media-types-cache.base-url=http://localhost:8090/transform/config ^ --elasticsearch.indexName=alfresco
- Elasticsearch connector is starts using port 8080 by default. This port can be changed using the default Spring Boot command line parameter server.port , For example, to start the Elasticsearch Connector using port 8083.
java -Xmx4G -jar C:\alfresco-elastic-search-services\alfresco-elasticsearch-live-indexing-3.1.1-app.jar ^
--server.port=8083 ^--spring.activemq.broker-url=nio://activemq.local:61616 ^ --spring.elasticsearch.rest.uris=http://localhost:9200 ^ --alfresco.sharedFileStore.baseUrl=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/ ^ --alfresco.accepted-content-media-types-cache.base-url=http://localhost:8090/transform/config ^ --elasticsearch.indexName=alfresco
- If you wish to pass the mediation-filter.yml with your custom changes, you can use following (Use the appropriate file path for mediation-filter.yml as per your setup):
java -Xmx4G -jar C:\alfresco-elastic-search-services\alfresco-elasticsearch-live-indexing-3.1.1-app.jar ^
--server.port=8083 ^
--spring.activemq.broker-url=nio://activemq.local:61616 ^
--spring.elasticsearch.rest.uris=http://localhost:9200 ^
--alfresco.sharedFileStore.baseUrl=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/ ^
--alfresco.accepted-content-media-types-cache.base-url=http://localhost:8090/transform/config ^
--elasticsearch.indexName=alfresco ^
--alfresco.mediation.filter-file=file:mediation-filter.yml- If your services are deployed on a different server or port, the following parameters can be used. For example:
java -Xmx4G -jar C:\alfresco-elastic-search-services\alfresco-elasticsearch-live-indexing-3.1.1-app.jar ^
--server.port=8083 ^--spring.activemq.broker-url=nio://activemq.acs.dev:61616 ^ --spring.elasticsearch.rest.uris=http://indexer.acs.dev:9200 ^
--alfresco.sharedFileStore.baseUrl=http://indexer.acs.dev:8099/alfresco/api/-default-/private/sfs/versions/1/file/ ^
--alfresco.accepted-content-media-types-cache.base-url=http://transform-service.acs.dev:8090/transform/config ^
--elasticsearch.indexName=alfresco ^
--alfresco.mediation.filter-file=file:mediation-filter.yml
Additional administrative info can be found here:
Learn what is unsupported: https://docs.alfresco.com/search-enterprise/latest/using/unsupported/
Some unsupported features:
- Indexing of nodes bootstrapped which is done during content repository bootstrap.
- Fingerprinting (Similarity & duplicate detection). This was available/enabled with SOLR (ASS) by default until Alfresco Search Services 2.0.1 and it is disabled by default from Alfresco Search Services 2.0.2 onwards. Please note that Elastic search do support fingerprinting however, it is not an option available with Alfresco at the moment.
- Statistics
- Aspect queries
- Resource limiting searches
- Scoped searches
- Fuzzy matching
- Field facets tags exclusion and field facets pagination
- Field queries: https://docs.alfresco.com/search-enterprise/latest/using/unsupported/#field-queries
- PATHWITHREPEATS
- PNAME
- ANAME
- NPATH
- QNAME
- PRIMARYASSOCQNAME
- PRIMARYASSOCTYPEQNAME
- FTSSTATUS
- ISROOT
- ISCONTAINER
- CASCADETX
- DBID
- TX
- TXID
- INTXID
- TXCOMMITTIME
- ACLID
- INACLTXID
- ACLTXID
- ACLTXCOMMITTIME
- TENANT
- OWNERSET
- READERSET
- AUTHSET
- DENYSET
- Behaviors of unsupported fields: Checkout the options and differences here
Now we are done with all the setup and config changes. Here are full “alfresco-global.properties”, “custom-log4j.properties”, “reindex.prefixes-file.json” and "exactTermSearch.properties" files for reference:
###############################
## Common Alfresco Properties #
###############################
dir.root=C:/alfresco72-enterprise/alf_data
dir.contentstore=${dir.root}/contentstore
dir.contentstore.deleted=${dir.root}/contentstore.deleted
#
# Encryption properties
#
# default keystores location is now this, it was changed from previous versions
# for more details see: https://docs.alfresco.com/content-services/latest/admin/security/#alfresco-keystore-configuration
dir.keystore=C:/alfresco72-enterprise/tomcat/shared/classes/alfresco/extension/keystore
#
# URL Generation Parameters (The ${localname} token is replaced by the local server name)
#-------------
alfresco.context=alfresco
alfresco.host=${localname}
alfresco.port=8080
alfresco.protocol=http
share.context=share
share.host=${localname}
share.port=8080
share.protocol=http
### database connection properties ###
db.driver=org.postgresql.Driver
db.username=alfresco
db.password=alfresco
db.name=alfresco
db.url=jdbc:postgresql://localhost:5432/${db.name}
# Note: your database must also be able to accept at least this many connections. Please see your database documentation for instructions on how to configure this.
db.pool.max=275
db.pool.validate.query=SELECT 1
# The server mode. Set value here
# UNKNOWN | TEST | BACKUP | PRODUCTION
system.serverMode=UNKNOWN
### RMI registry port for JMX ###
alfresco.rmi.services.port=50500
# Default value of alfresco.rmi.services.host is 0.0.0.0 which means 'listen on all adapters'.
# This allows connections to JMX both remotely and locally.
alfresco.rmi.services.host=0.0.0.0
#
#
# Assign individual ports for each service for best performance
# or run several services on the same port. You can even run everything on 50500 if needed.
# Select 0 to use a random unused port.
#monitor.rmi.service.port=50508
### E-mail site invitation setting ###
notification.email.siteinvite=false
### License location ###
dir.license.external=C:/alfresco72-enterprise
### Allow extended ResultSet processing
security.anyDenyDenies=false
### Smart Folders Config Properties ###
smart.folders.enabled=false
### Remote JMX (Default: disabled) ###
alfresco.jmx.connector.enabled=false
################## AMQ And Transformation services [Start] #####################
messaging.broker.url=tcp://localhost:61616
messaging.subsystem.autoStart=true
#If you have setup username and password for AMQ, then set the below properties. In my case i have kept default admin/admin
messaging.broker.username=admin
messaging.broker.password=admin
#local.transform.service.enabled=true
# Transform Core properties:
localTransform.core-aio.url=http://localhost:8090
# Shared File Store properties:
sfs.url=http://localhost:8099
sfs.endpoint=${sfs.url}/alfresco/api/-default-/private/sfs/versions/1/file
# Transform Router properties:
transform.service.enabled=true
transform.service.url=http://localhost:8095
################## AMQ And Transformation services [End] #######################
################## Set the Elasticsearch subsystem [Start] #####################
#For more details check here: https://docs.alfresco.com/search-enterprise/latest/config/
index.subsystem.name=elasticsearch
# Elasticsearch index properties
elasticsearch.indexName=alfresco
elasticsearch.createIndexIfNotExists=true
# Elasticsearch server properties
elasticsearch.host=localhost
elasticsearch.port=9200
elasticsearch.baseUrl=/
#default none, https and none possible
elasticsearch.secureComms=none
################## Set the Elasticsearch subsystem [End] #####################
#Tranformer specific logs
log4j.logger.org.alfresco.repo.content.transform.TransformerDebug=info
log4j.logger.org.alfresco.util.exec.RuntimeExecBootstrapBean=info
log4j.logger.org.alfresco.util.exec.RuntimeExec=info
#ScriptLogger
log4j.logger.org.alfresco.repo.jscript.ScriptLogger=debug
#Log for email executer
log4j.logger.org.alfresco.repo.action.executer.MailActionExecuter=info
#Transaction specific logs
log4j.logger.org.alfresco.repo.transaction.RetryingTransactionHelper=info
#Metadata extractor
log4j.logger.org.alfresco.repo.content.metadata.AbstractMappingMetadataExtracter=info
log4j.logger.org.alfresco.repo.content.metadata.MetadataExtracterRegistry=info
#Thumbnail logs
log4j.logger.org.alfresco.repo.thumbnail=info
{
"prefixUriMap": {
"": "",
"http://www.alfresco.org/model/workflow/invite/nominated/1.0": "inwf",
"http://www.alfresco.org/model/solrfacetcustomproperty/1.0": "srftcustom",
"http://www.alfresco.org/model/datalist/1.0": "dl",
"http://www.alfresco.org/model/webdav/1.0": "webdav",
"http://www.alfresco.org/model/cmis/1.0/cs01ext": "cmisext",
"http://www.alfresco.org/model/distributionpolicies/1.0/model": "dp",
"http://www.alfresco.org/view/repository/1.0": "view",
"http://www.alfresco.org/model/download/1.0": "download",
"http://www.alfresco.org/model/publishing/twitter/1.0": "twitter",
"http://www.alfresco.org/model/action/1.0": "act",
"http://www.alfresco.org/system/registry/1.0": "reg",
"http://www.alfresco.org/model/user/1.0": "usr",
"http://www.alfresco.org/model/calendar": "ia",
"http://www.alfresco.org": "alf",
"http://www.alfresco.org/model/content/metadata/IPTCXMP/1.0": "iptcxmp",
"http://www.alfresco.org/model/application/1.0": "app",
"http://www.alfresco.org/model/surf/1.0": "surf",
"http://www.alfresco.org/model/versionstore/1.0": "ver",
"http://www.alfresco.org/system/modules/1.0": "module",
"http://www.alfresco.org/model/linksmodel/1.0": "lnk",
"http://iptc.org/std/Iptc4xmpExt/2008-02-29/": "Iptc4xmpExt",
"http://ns.adobe.com/photoshop/1.0/": "photoshop",
"http://www.alfresco.org/model/sync/1.0": "sync",
"http://ns.useplus.org/ldf/xmp/1.0/": "plus",
"http://www.alfresco.org/model/remotecredentials/1.0": "rc",
"http://www.alfresco.org/model/emailserver/1.0": "emailserver",
"http://www.alfresco.org/model/hybridworkflow/1.0": "hwf",
"http://www.alfresco.org/model/sitecustomproperty/1.0": "stcp",
"http://www.alfresco.org/model/cmis/1.0/cs01": "cmis",
"http://www.alfresco.org/model/transfer/1.0": "trx",
"http://www.fme.de/alfresco/models/jsconsole": "jsc",
"http://www.alfresco.org/model/rendition/1.0": "rn",
"http://www.alfresco.org/model/exif/1.0": "exif",
"http://www.alfresco.org/model/publishing/youtube/1.0": "youtube",
"http://ns.adobe.com/xap/1.0/rights/": "xmpRights",
"http://www.alfresco.org/model/custommodelmanagement/1.0": "cmm",
"http://www.alfresco.org/model/workflow/invite/moderated/1.0": "imwf",
"http://www.alfresco.org/model/forum/1.0": "fm",
"http://www.alfresco.org/model/rule/1.0": "rule",
"http://www.alfresco.org/model/publishing/linkedin/1.0": "linkedin",
"http://www.alfresco.org/model/publishing/slideshare/1.0": "slideshare",
"http://www.alfresco.org/model/system/1.0": "sys",
"http://www.alfresco.org/model/content/smartfolder/1.0": "smf",
"http://www.alfresco.org/model/workflow/1.0": "wf",
"http://www.alfresco.org/model/qshare/1.0": "qshare",
"http://www.alfresco.org/model/versionstore/2.0": "ver2",
"http://www.alfresco.org/model/solrfacet/1.0": "srft",
"http://www.alfresco.org/model/audio/1.0": "audio",
"http://www.alfresco.org/model/blogintegration/1.0": "blg",
"http://www.alfresco.org/model/bpm/1.0": "bpm",
"http://www.alfresco.org/model/site/1.0": "st",
"http://www.alfresco.org/model/imap/1.0": "imap",
"http://www.alfresco.org/model/dictionary/1.0": "d",
"custom.model": "custom",
"http://www.alfresco.org/model/publishing/facebook/1.0": "facebook",
"http://www.alfresco.org/model/content/1.0": "cm",
"http://www.alfresco.org/model/cmis/custom": "cmiscustom",
"http://www.alfresco.com/model/activiti-bpm-suite/1.0": "abs",
"http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/": "Iptc4xmpCore",
"http://www.alfresco.org/model/publishing/flickr/1.0": "flickr",
"http://www.alfresco.org/model/workflow/resetpassword/1.0": "resetpasswordwf",
"http://www.alfresco.org/model/cmis/1.0/alfcmis": "alfcmis",
"http://www.alfresco.org/model/publishing/1.0": "pub",
"http://purl.org/dc/elements/1.1/": "dc",
"": ""
}
}# Exact Term search is switched off by default as it introduces index size overhead.
# to enable it, please uncomment the following on a datatype or property name basis
alfresco.cross.locale.datatype.0={http://www.alfresco.org/model/dictionary/1.0}text
alfresco.cross.locale.datatype.1={http://www.alfresco.org/model/dictionary/1.0}content
alfresco.cross.locale.datatype.2={http://www.alfresco.org/model/dictionary/1.0}mltext
alfresco.cross.locale.property.0={http://www.alfresco.org/model/content/1.0}content
There are three services that we need to start one by one in order:
- Start ActiveMQ
- To start ActiveMQ, you can execute the following:
- Open command prompt and execute this command: C:\alfresco72-enterprise\activemq\bin\win64\activemq.bat start
- Open webconsole and validate: http://127.0.0.1:8161/admin/queues.jsp
- Start core transformation service
- To start Local transformation service, you can execute the following:
- Open command prompt and execute this command: java -DPDFRENDERER_EXE="C:\\alfresco72-enterprise\\alfresco-pdf-renderer\\alfresco-pdf-renderer.exe" -DLIBREOFFICE_HOME="C:\\alfresco72-enterprise\\libreoffice" -DIMAGEMAGICK_ROOT="C:\\alfresco72-enterprise\\imagemagick" -DIMAGEMAGICK_DYN="C:\\alfresco72-enterprise\\imagemagick\\lib" -DIMAGEMAGICK_CODERS="C:\\alfresco72-enterprise\\imagemagick\\modules\\coders" -DIMAGEMAGICK_CONFIG="C:\\alfresco72-enterprise\\imagemagick" -DIMAGEMAGICK_EXE="C:\\alfresco72-enterprise\\imagemagick\\convert.exe" -DACTIVEMQ_URL=tcp://localhost:61616 -jar C:\\alfresco72-enterprise\\bin\\alfresco-transform-core-aio-boot-2.6.0.jar
- Open Test transformer console and validate: http://127.0.0.1:8090/
- Start transform router service
- To start transform router service, you can execute the following:
- Open command prompt and execute this command: java -DCORE_AIO_URL=http://localhost:8090 -DCORE_AIO_QUEUE=org.alfresco.transform.engine.aio.acs -DACTIVEMQ_URL=tcp://localhost:61616 -DFILE_STORE_URL=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file -jar C:\alfresco72-enterprise\bin\alfresco-transform-router-1.5.3.jar
- Start shared file store controller service
- To start shared file store controller service, you can execute the following:
- Open command prompt and execute this command: java -DfileStorePath=C:\alfresco72-enterprise\shared-file-store -Dscheduler.contract.path=C:\alfresco72-enterprise\shared-file-store-scheduler-location\scheduler.json -Dscheduler.content.age.millis=86400000 -Dscheduler.cleanup.interval=86400000 -jar C:\alfresco72-enterprise\bin\alfresco-shared-file-store-controller-1.5.3.jar
- Start DB
- To start DB, you can do either of the following:
- Use windows services, and start “postgresql-x64-13” service.
- Or Open command prompt and execute this command: C:\PostgreSQL\13\bin\pg_ctl.exe start -D “C:\PostgreSQL\13\data"
- Start Elastic Server
- To start Elastic Server, you can execute the following:
- Open command prompt and execute this command: C:\elasticsearch-7.10.1\bin\elasticsearch.bat
- Start Alfresco
- To start Alfresco, you can do either of the following:
- Open command prompt and navigate to “C:\alfresco72-enterprise\tomcat\bin” and execute following command: catalina.bat start
- Or navigate to “C:\alfresco72-enterprise\tomcat\bin” folder and execute “startup.bat”
- Start alfresco live indexing app
- To start alfresco live indexing app, you can execute the following: java -jar C:\\alfresco-elastic-search-services\\alfresco-elasticsearch-live-indexing-3.1.1-app.jar --server.port=8083 --spring.activemq.broker-url=nio://localhost:61616 --spring.elasticsearch.rest.uris=http://localhost:9200 --alfresco.sharedFileStore.baseUrl=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/ --alfresco.accepted-content-media-types-cache.base-url=http://localhost:8090/transform/config --elasticsearch.indexName=alfresco --alfresco.mediation.filter-file=file:C:\\alfresco-elastic-search-services\\mediation-filter.yml
- Start alfresco re-indexing app (Can be started on-demand, app shuts down after re-indexing automatically)
- To start alfresco re-indexing app , you can execute the following: java -jar C:\\alfresco-elastic-search-services\\alfresco-elasticsearch-reindexing-3.1.1-app.jar --alfresco.reindex.jobName=reindexByIds --spring.elasticsearch.rest.uris=http://localhost:9200 --spring.datasource.url=jdbc:postgresql://localhost:5432/alfresco --spring.datasource.username=alfresco --spring.datasource.password=alfresco --spring.activemq.broker-url=tcp://localhost:61616?jms.useAsyncSend=true --alfresco.reindex.prefixes-file=file:C:\\alfresco-elastic-search-services\\reindex.prefixes-file.json
I have created this bat file “start.bat”, you can use this to start all services instead of doing one by one.
@echo off ECHO ################ Starting ACS, DB, Transformation Service and Elastic Search Services ############## ECHO. SET ALF_INSTALL_PATH=%1 SET ALF_SE_INSTALL_PATH=%2 SET POSTGRES_INSTALL_PATH=%3 SET ELASTIC_INSTALL_PATH=%4 :init IF "%~1" == "" ( SET ALF_INSTALL_PATH=C:\\alfresco72-enterprise ) IF "%~2" == "" ( SET ALF_SE_INSTALL_PATH=C:\\alfresco-elastic-search-services ) IF "%~3" == "" ( SET POSTGRES_INSTALL_PATH=C:\\PostgreSQL\\13 ) IF "%~4" == "" ( SET ELASTIC_INSTALL_PATH=C:\\elasticsearch-7.10.1 ) goto startAMQLocal :startAMQLocal echo. echo Starting Active MQ service... start "ActiveMQ" /MIN cmd /c %ALF_INSTALL_PATH%\activemq\bin\win64\activemq.bat start timeout 10 if errorlevel 1 (goto end) else (goto startTrServ) :startTrServ echo. echo Starting core transformation service... start "transformationService" java -DPDFRENDERER_EXE="%ALF_INSTALL_PATH%\\alfresco-pdf-renderer\\alfresco-pdf-renderer.exe"^ -DLIBREOFFICE_HOME="%ALF_INSTALL_PATH%\\libreoffice"^ -DIMAGEMAGICK_ROOT="%ALF_INSTALL_PATH%\\imagemagick"^ -DIMAGEMAGICK_DYN="%ALF_INSTALL_PATH%\\imagemagick\\lib"^ -DIMAGEMAGICK_CODERS="%ALF_INSTALL_PATH%\\imagemagick\\modules\\coders"^ -DIMAGEMAGICK_CONFIG="%ALF_INSTALL_PATH%\\imagemagick"^ -DIMAGEMAGICK_EXE="%ALF_INSTALL_PATH%\\imagemagick\\convert.exe"^ -DACTIVEMQ_URL=tcp://localhost:61616^ -jar %ALF_INSTALL_PATH%\\bin\\alfresco-transform-core-aio-boot-2.6.0.jar timeout 50 if errorlevel 1 (goto end) else (goto startTrRouter) :startTrRouter echo. echo Starting transform router service... start "trRouter" java -DCORE_AIO_URL=http://localhost:8090 ^ -DCORE_AIO_QUEUE=org.alfresco.transform.engine.aio.acs ^ -DACTIVEMQ_URL=tcp://localhost:61616 ^ -DFILE_STORE_URL=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file ^ -jar %ALF_INSTALL_PATH%\\bin\\alfresco-transform-router-1.5.3.jar timeout 50 if errorlevel 1 (goto end) else (goto startSharedFileStore) :startSharedFileStore echo. echo Starting shared file store controller service... start "sfsController" java -DfileStorePath=%ALF_INSTALL_PATH%\shared-file-store ^ -Dscheduler.contract.path=%ALF_INSTALL_PATH%\shared-file-store-scheduler-location\scheduler.json ^ -Dscheduler.content.age.millis=86400000 ^ -Dscheduler.cleanup.interval=86400000 ^ -jar %ALF_INSTALL_PATH%\bin\alfresco-shared-file-store-controller-1.5.3.jar timeout 10 if errorlevel 1 (goto end) else (goto startDB) :startDB echo. echo Starting DB... :: Using the windows service to start the db. :: net start postgresql-x64-13 REM You can also use this command, if there is any issue with permission elevation on windows %POSTGRES_INSTALL_PATH%\bin\pg_ctl.exe restart -D "%POSTGRES_INSTALL_PATH%\data" timeout 20 if errorlevel 1 (goto end) else (goto startElasticServer) :startElasticServer echo. echo Starting elastic search server... start "ElasticServer" /MIN cmd /c %ELASTIC_INSTALL_PATH%\bin\elasticsearch.bat timeout 60 if errorlevel 1 (goto end) else (goto startACS) :startACS echo. echo Starting ACS... SET CATALINA_HOME=%ALF_INSTALL_PATH%\tomcat start "Tomcat" /MIN /WAIT cmd /c %ALF_INSTALL_PATH%\tomcat\bin\catalina.bat start timeout 250 if errorlevel 1 (goto end) else (goto startElasticliveIndexingApp) :startElasticliveIndexingApp echo. echo Starting alfresco live indexing app... start "liveIdexingService" java -jar %ALF_SE_INSTALL_PATH%\\alfresco-elasticsearch-live-indexing-3.1.1-app.jar ^ --server.port=8083 ^ --spring.activemq.broker-url=nio://localhost:61616 ^ --spring.elasticsearch.rest.uris=http://localhost:9200 ^ --alfresco.sharedFileStore.baseUrl=http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file/ ^ --alfresco.accepted-content-media-types-cache.base-url=http://localhost:8090/transform/config ^ --elasticsearch.indexName=alfresco ^ --alfresco.mediation.filter-file=file:C:\\alfresco-elastic-search-services\\mediation-filter.yml timeout 60 if errorlevel 1 (goto end) else (goto startElasticReindexingApp) :startElasticReindexingApp echo. echo Starting alfresco re-indexing app... start "reindexingService" java -jar %ALF_SE_INSTALL_PATH%\\alfresco-elasticsearch-reindexing-3.1.1-app.jar ^ --alfresco.reindex.jobName=reindexByIds ^ --spring.elasticsearch.rest.uris=http://localhost:9200 ^ --spring.datasource.url=jdbc:postgresql://localhost:5432/alfresco ^ --spring.datasource.username=alfresco ^ --spring.datasource.password=alfresco ^ --spring.activemq.broker-url=tcp://localhost:61616?jms.useAsyncSend=true ^ --alfresco.reindex.prefixes-file=file:%ALF_SE_INSTALL_PATH%\\reindex.prefixes-file.json timeout 10 if errorlevel 1 (goto end) :end echo. echo Exiting.. timeout 10
Validate Alfresco and Share:
Validate Elastic search subsystem:
Test Elastic search connection from Alfresco:
Try some searches:
Note: If you see any warning in logs such as "org.apache.catalina.webresources.Cache.getResource Unable to add the resource at [/WEB-INF/classes/alfresco/templates/org/alfresco/dashboard.ftl] to the cache for web application [/share] because there was insufficient free space available after evicting expired cache entries - consider increasing the maximum size of the cache" or "org.apache.catalina.webresources.Cache.backgroundProcess The background cache eviction process was unable to free [10] percent of the cache for Context [/share] - consider increasing the maximum size of the cache. After eviction approximately [9,411] KB of data remained in the cache." then you can increase the resource case of disable resource caching. Follow the below given steps if you see these warnings:
- By default value for resource caching is 10240 kbytes, even when not added to [ALF_HOME]\tomcat\conf\context.xml. so either increase it until you stop seeing the warning or disable it completely.
- Make sure you clean the tomcat 'work' (C:\alfresco72-enterprise\tomcat\work\Catalina) directory before restarting the server.
<Resources cacheMaxSize="100000" cachingAllowed="true"/>Or disable the caching<Resources cachingAllowed="false" />
- The above configuration is default for tomcat but since for 'alfresco' and 'share' webapps we have cross contexts under '[ALF_HOME]\tomcat\conf\Catalina\localhost\' directory.
- Make the above given configuration changes in "alfresco.xml" and "share.xml" and clean the tomcat 'work' (C:\alfresco72-enterprise\tomcat\work\Catalina) directory before restarting the server
<Context crossContext="true">
<Resources cachingAllowed="true" cacheMaxSize="100000">
<PostResources base="${catalina.base}/../modules/platform"
className="org.apache.catalina.webresources.DirResourceSet"
webAppMount="/WEB-INF/lib"/>
</Resources>
</Context><Context crossContext="true">
<Resources cachingAllowed="true" cacheMaxSize="100000">
<PostResources base="${catalina.base}/../modules/share"
className="org.apache.catalina.webresources.DirResourceSet"
webAppMount="/WEB-INF/lib"/>
</Resources>
</Context>Stopping:
There are three services that we need to stop one by one in order:
- Stop shared file store controller service
- Kill the process running the service on windows command processor.
- Stop transform router service
- Kill the process running the service on windows command processor.
- Stop core transformation service
- Kill the process running the service on windows command processor.
- Stop Alfresco
- To stop Alfresco, you can do either of the following:
- Open command prompt and navigate to “C:\alfresco72-enterprise\tomcat\bin” and execute following command: catalina.bat stop
- Or navigate to “C:\alfresco72-enterprise\tomcat\bin” folder and execute “shutdown.bat”
- Stop DB
- To stop DB, you can do either of the following:
- Use windows services, and stop “postgresql-x64-11” service.
- Or Open command prompt and execute this command: C:\PostgreSQL\13\bin\pg_ctl.exe stop -D “C:\PostgreSQL\13\data"
- Stop alfresco live indexing app
- Kill the process running the service on windows command processor.
- Stop alfresco re-indexing app (if running, usually it shutdown after re-index automatically).
- Kill the process running the service on windows command processor.
- Stop ActiveMQ
- To start ActiveMQ, you can execute the following:
- Open command prompt and execute this command: C:\alfresco72-enterprise\activemq\bin\win64\activemq.bat stop
- or Kill the process running the service on windows command processor.
- Stop ElasticServer
- Kill the process running the service on windows command processor.
I have created this bat file “stop.bat”, you can use this to stop all services instead of doing one by one.
@echo off
ECHO ################ Stopping ACS, DB, Transformation Service and Elastic Search Services ##############
ECHO.
SET ALF_INSTALL_PATH=%1
SET ALF_SE_INSTALL_PATH=%2
SET POSTGRES_INSTALL_PATH=%3
SET ELASTIC_INSTALL_PATH=%4
:init
IF "%~1" == "" (
SET ALF_INSTALL_PATH=C:\\alfresco72-enterprise
)
IF "%~2" == "" (
SET ALF_SE_INSTALL_PATH=C:\\alfresco-elastic-search-services
)
IF "%~3" == "" (
SET POSTGRES_INSTALL_PATH=C:\\PostgreSQL\\13
)
IF "%~4" == "" (
SET ELASTIC_INSTALL_PATH=C:\\elasticsearch-7.10.1
)
goto stopTrServ
:stopTrServ
echo.
echo Stopping transformation service ...
taskkill /fi "WINDOWTITLE eq sfsController"
taskkill /fi "WINDOWTITLE eq transformationService"
taskkill /fi "WINDOWTITLE eq trRouter"
if errorlevel 1 (goto end) else (goto stopACS)
:stopACS
echo.
echo Stopping ACS from %ALF_INSTALL_PATH% ...
SET CATALINA_HOME=%ALF_INSTALL_PATH%\tomcat
start /MIN /WAIT cmd /c %ALF_INSTALL_PATH%\tomcat\bin\catalina.bat stop
taskkill /fi "WINDOWTITLE eq Tomcat"
taskkill /F /IM soffice.bin
if errorlevel 1 (goto end) else (goto stopDB)
:stopDB
echo.
echo Stopping DB from %POSTGRES_INSTALL_PATH% ...
:: Using the windows service to stop the db.
:: net stop postgresql-x64-13
REM You can also use this command, if there is any issue with permission elevation on windows
%POSTGRES_INSTALL_PATH%\bin\pg_ctl.exe stop -D "%POSTGRES_INSTALL_PATH%\data"
if errorlevel 1 (goto end) else (goto stopElasticConnectorApps)
:stopElasticConnectorApps
echo.
echo Stopping alfresco re-indexing and live indexing apps...
taskkill /fi "WINDOWTITLE eq reindexingService"
taskkill /fi "WINDOWTITLE eq liveIdexingService"
if errorlevel 1 (goto end) else (goto stopAMQLocal)
:stopAMQLocal
echo.
echo Stopping Active MQ service ...
taskkill /F /IM wrapper.exe
taskkill /fi "WINDOWTITLE eq ActiveMQ"
if errorlevel 1 (goto end) else (goto stopElasticServer)
:stopElasticServer
echo.
taskkill /fi "WINDOWTITLE eq ElasticServer"
if errorlevel 1 (goto end)
:end
echo.
echo Exiting..
timeout 10Some Issues noticed:
There may be more known issues since elastic search with alfresco is still not fully mature. Currently, Alfresco Search Enterprise is only supported with the Postgres database and supports only English language queries. For more details visit here.
- Image Preview Dashlet search query fails to parse, you will see following exception every time you go to Dashboard if you have Image Preview dashlet setup.
- Exception: 05030005 Wrapped Exception (with status template): 05030374 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/images.get.js': 05030373 Failed to execute search: +PATH:"/app:company_home/st:sites/cm:swsdp/cm:documentLibrary//*" +TYPE:"cm:content" +@cm\:content.mimetype:image/*
- Workaround: "Remove Image Preview Dashlet from the dashboard"
All the structure and updated files are available here for reference:
https://github.com/abhinavmishra14/alfresco72-elasticsearch-transform-dist-setup
A memory leak vulnerability has been found and fixed in ****Alfresco Content Services 6.2 through 7.2****
Here is the description from Alfresco and I am quoting: "The issue allows a user to run potentially malicious JavaScript code in the system that consumes increasing amounts of memory on each execution. This could lead to a denial of service (DoS). The fix involved the introduction of some configurable settings to limit the resources that scripts could use and to put controls in place to reclaim memory used by previous script execution."
Note that, This vulnerability requires administrator privileges for exploitation. Ensure that only trusted users are added to the ALFRESCO_ADMINISTRATORS group. Non-administrator users cannot upload scripts to the repository.
The cause of the memory leak has been fixed.
The configurations that needs to be considered and adjusted as per your system needs as a second line of defense (properties with default values):
## Script execution configs for alfresco-global.properties ## # Rhino optimization level # When set to -1, the interpretive mode is used. # When set to 0, no optimizations are performed. Default value is 0 # When set to 1-9, optimizations are performed. scripts.execution.optimizationLevel=0 # Max seconds a script is allowed to run # The number of seconds a script is allowed to run. If script execution exceeds the configured seconds, it will be stopped. # To enable this limit, set the property with a value bigger than zero. The default value is -1 (disabled). scripts.execution.maxScriptExecutionSeconds=-1 # Max call stack depth # The maximum stack depth (call frames) allowed in a single invocation of the interpreter. # This configuration only works for scripts compiled with interpretive mode, which means the optimization level will always be -1, overriding the value from the scripts.execution.optimizationLevel property. # To enable this limit, set the property with a value bigger than zero. The default value is -1 (disabled). scripts.execution.maxStackDepth=-1 # Max memory (bytes) a script can use # The maximum memory (in bytes) a script is allowed to use. If script execution exceeds the configured memory, it will be stopped. # To enable this limit, set the property with a value bigger than zero. The default value is -1 (disabled). # This configuration only works with the supported JVM. scripts.execution.maxMemoryUsedInBytes=-1 # Number of instructions that will trigger the observer # The number of instructions that will trigger the observer that applies the described limits: seconds, memory, and call stack depth. # The value may vary depending on the optimization level. # This configuration allows you to monitor the script execution and needs to be set to a value bigger than zero so that the described limits work. The default value is -1 (disabled). scripts.execution.observerInstructionCount=-1
Note: These properties won't take effect if you try to put in any previous versions other than the version mentioned below. You must upgrade to below given versions in order to make use of above properties.
Be aware that adding these properties and setting random value may result in issues. You will have to adjust the property values based on error-and-trial depending on your system needs.
Referenced from Official Repository Configuration Docs
If you are using any of the version from 6.2 through 7.2, consider upgrading it to following versions that includes the fix:
Ensure to take a backup of your Alfresco
Repository and the Alfresco Database before upgrade to the versions mentioned above.
-----------------------------------------------------------------------------------------------------------------
If you want to setup ACS-7.3.x, complete the prerequisites given below and follow the installation steps. Checkout the release notes here.
Download following packages required to setup ACS-7.3 and follow the steps outlined above. All steps should be pretty much the same.
1.
ACS-7.3.x package (alfresco-content-services-distribution-7.3.1)
2.
ASE-3.x package
(alfresco-elasticsearch-connector-distribution-3.2.0)
3.
Alfresco Transform 2.x package
(alfresco-transform-service-distribution-2.0.2)
If you would like to upgrade to Transform Core 3.x, check
this post: https://hub.alfresco.com/t5/alfresco-content-services-blog/alfresco-7-3-upgrading-to-transform-core-3-0-0/ba-p/315364
4.
Java: Oracle jdk-17.0.4 or
later/Open JDK 17.0.4 or later
Download OracleJDK 17.0.x: https://download.oracle.com/java/17/archive/jdk-17.0.4.1_windows-x64_bin.msi
5.
Elasticsearch 7.17.x
Download: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.0-windows-x86_64.zip
6.
Tomcat 9: Tomcat 9.0.72
Download: https://dlcdn.apache.org/tomcat/tomcat-9/v9.0.74/bin/apache-tomcat-9.0.74-windows-x64.zip
7.
ActiveMQ v5.17: ActiveMQ v5.17.4.
Download: https://archive.apache.org/dist/activemq/5.17.4/apache-activemq-5.17.4-bin.zip
8.
DB: PostgreSQL 14.7
Download: https://get.enterprisedb.com/postgresql/postgresql-14.7-2-windows-x64.exe
9.
ImageMagick v7.1.x: ImageMagick v7.1.1-8-Q16
Download: https://download.imagemagick.org/archive/binaries/ImageMagick-7.1.1-8-Q16-HDRI-x64-dll.exe
Alternative options (ImageMagick notoriously removes the
specific versions and shows only latest versions):
https://imagemagick.org/script/download.php#windows
https://download.imagemagick.org/ImageMagick/download/binaries/
10. Libreoffice:
LibreOffice v7.2.5
All alternatives: https://downloadarchive.documentfoundation.org/libreoffice/old/
11. PDF Renderer: alfresco-pdf-renderer v1.1
12. Exif tool v12.x: image-exiftool 12.5
Check the ACS-7.X Supported Platform and Search Enterprise Supported Platform for more details.
References:
Great article, thanks for this. I have been struggling with the setup, this is really help. Really appreciate it.
ReplyDeleteI was trying to setup with MySQL, then i noticed your note that elastic search will work only with Postgresql. Does that mean MySQL can't be used and if someone is using MySQL can't upgrade if they want to?
ReplyDeleteThat's correct, only postgresql is support claims alfresco. Checkout more info here: https://docs.alfresco.com/search-enterprise/latest/using/unsupported/
DeleteReally great article by the way
ReplyDeleteThanks , i am glad it is helpful.
DeleteGreat Article Abhinav. Much appreciated.
ReplyDeleteI would like to know if Elastic Search would work without Alfresco Transform service? My requirement is to index only Metadata properties, not content. Could you please check and let me know?
Yes, the minimal install stack is alfresco content services and database. You can disable the messaging services and use alfresco with solr or elastic search without activemq and transform service.
DeleteNote that, to index only metadata and not the content requires you to either update the cm:indexControl aspect on each node to set cm:isContentIndexed as false. or update following "alfresco.index.transformContent=false" property in solrcore.properties to disable content indexing.
To disable content indexing via elastic search, you need to exclude cm:content type from contentNodeTypes in live-indexing-mediation service. See here for more details: https://docs.alfresco.com/search-enterprise/latest/config/#alfresco-live-indexing-app
Take a look at this post as well which may be helpful : https://hub.alfresco.com/t5/alfresco-content-services-blog/coding-with-the-hub-4-deploying-alfresco-repository/ba-p/312406
Great article.
ReplyDeleteCan you please let me know if this Search service will work without Alfresco Transform Service?
Yes, the minimal install stack is alfresco content services and database. You can disable the messaging services and use alfresco with solr or elastic search without activemq and transform service.
DeleteTake a look at this post as well which may be helpful : https://hub.alfresco.com/t5/alfresco-content-services-blog/coding-with-the-hub-4-deploying-alfresco-repository/ba-p/312406
Thanks Abhinav,
ReplyDeleteI am referring this blog in Nov-2022, I am not able to download that enterprise search 3.1.1 jar file. If you have offline please provide.
It is still available here: https://artifacts.alfresco.com/nexus/service/local/repositories/enterprise-releases/content/org/alfresco/alfresco-transform-service-distribution/1.5.3/alfresco-transform-service-distribution-1.5.3.zip
DeleteMentioned at the very beginning of the post that you need to connect with Alfresco support or you can use Alfresco Nexus repo credentials (provided by alfresco if you have enterprise license) to download the packages.
You can't download if you don't have enterprise license. And i can't share as it is proprietary
Dear Abhinav,
ReplyDeleteWhy alfresco 7.x installation is very complex (with different components) rather than simple steps, which can be automated?
Alfresco provides following options to install:
Delete1- Containers via docker-compose.yml or helm for K8s
2- Ansible
3- Distribution package
Earlier before ACS 6.x, alfresco used to provide installers. Installers are no longer provided. If option 1 and 2 are not viable for you then you have to setup manually via option 3. Alfresco doesn't provide automated scripts or so.
You can create yourself if you feel like you an automate. The purpose of this post is to outline steps
are properties for transformation services like below not required to be added in global properties post 7.x version?
ReplyDeleteooo.*
jobconvertor.*
img.*
No, from ACS 6.2 onwards these options are no longer required. They are deprecated.
DeleteGreat article. Very informative.
ReplyDeletethanks
DeleteDear Abhinav,
ReplyDeleteCan you please tell below things, I see it is very very complex to follow all (couldn't get still step by step)
1) If I am installing on my VM (centos7) instance which user you have used? is that all done with root user?
2) do we need to create tomcat (seperate user) or e.g If I have alfresco user, all runs fine.?
3) for Centos7 still I faces issues running transformation services, any simplification instructions for centos7 installing alfresco-7.3.1 with elasticsearch?
Alfresco user and groups are used. Not required to setup as root. Check this post: https://javaworld-abhinav.blogspot.com/2021/06/setup-acs70-ass201-and-transformation-service.html#setup-usr-grp
DeleteWith alfresco user everything runs just fine.
Please check: https://javaworld-abhinav.blogspot.com/2021/06/setup-acs70-ass201-and-transformation-service.html#setup-trserv-local
Step should be pretty much same even if you use elastic search.
Detail steps which helped to setup. Can you please share the mTLS(SSL) configuration steps??
ReplyDeleteThanks Srikanth. It is in my todo list.
DeleteThanks for this post. Can we've similar post for Alfresco ACS 23.1 setup and configuration in linux environment
ReplyDeleteIt is in my todo list. And may put up something after i have some spare time. However steps should be pretty much similar. So you can try to setup using the same steps making sure you use the latest compatible version of components.
Delete