1. Overview

This article describes the recommended process for upgrading the full platform in a live environment. For a full process overview, click ps:here.

2. The environment before the upgrade

The image below shows the default setup for a Fredhopper environment, with one indexer instance and two query instances distributed across 3 servers. Each of the servers operates one instance through a Deployment Agent and the query instances retrieve their configuration and data from the indexer instance.

Additional Data Manager instances Generally speaking, you may likely have additional Data Manager instances (e.g. etl, deployment, etc.) installed on your query servers. These are not necessary for the following operations and will therefore be omitted from the diagrams - in most cases you can expect them to be installed correctly during the platform update. Please check with your Fredhopper Consultant in case you have a custom setup.

Multiple Environments Sharing Servers The Fredhopper default setup has a single instance on every server, but your setup may also have servers that are shared by multiple instances. The upgrade procedure works on a server-by-server base, so all environments need to be upgraded at the same time, or alternatively need to be moved to distinct servers before the upgrade. Please check with your Fredhopper Consultant in such complex situations.

3. Conditions for upgrading

The following conditions must be met before attempting an upgrade of the full platform:

• You have a good understanding of the changes introduced in the new Fredhopper platform (see Release Notes and Upgrade Notes).
• You have access to all machines that run Fredhopper instances scheduled for the upgrade.
• You have a version of the Fredhopper platform including the required Fredhopper version.
• The front end is able to handle the responses of both Fredhopper versions at the same time (as at specific moments both will be accessible through the load balancer).
• You have confirmed which of the following files need to be ported, verified that they are compatible and backed up:
  • Custom Fredhopper extensions (e.g. in ./data/instances/<INSTANCE>/custom)
  • Custom Data Manager libraries (e.g. in ./data/installations/DM/libext or ./data/installations/DM/plugins)
  • Custom Data Manager transformations (including enrichments)
• You have verified that the version of the Data Manager that you will be using, correctly generates the required Fredhopper input data.
• You have verified that the environment (e.g. Java version) is able to support the new platform (see Install Java).
• You have frozen any business configuration changes and verified that the team is not changing any configuration within Fredhopper anymore.
• You have published the configuration you would like to use initially in the new Fredhopper version and have made a capture of it.
• You have backed up the content of the platform configuration directory (copy the content of ./config to a safe location)
• If you are using standard data transformations, you have verified if your data and enrichments will still function correctly in the newest version.
• You have paused all automatic reindex and incremental update jobs for the time of the update.

Query capacity during the upgrade During the upgrade your query capacity will be reduced by one instance. Ensure that this capacity is sufficient for the time of the upgrade. If you're unsure you should consider adding additional capacity for the time of the upgrade.

4. Upgrading the indexer

The upgrade will first focus on the indexer instance. The query instances will be upgraded after the work on the indexer has been completed, to allow testing the updated indexer instance without impacting the query instances.

4.1. Switch off replication

In order to ensure that the data and configuration of the new Fredhopper version will not leak accidentally to the query instances, you must shut down all processes involved in replication. This means that you need to stop the syncserver process on the indexer instance and the syncclient processes on all query instances.

The image below shows the state of the environment after this action.

No updates to query instances (e.g. incremental updates, configuration publishes) will be possible from this point on until the environment has been migrated.

.

4.2. Shut down old platform

As the link between the indexer and the query instances is now disconnected you will be able to shut down the platform on the indexer server without issues. Ensure that no Data Manager process is running anymore by checking the status of each Data Manager instance.

Execute the following command to stop the platform.

This will stop all active processes on the local platform and shut down the Deployment Agent.

4.3. Install new platform

At this point, no Java processes should be active anymore in the Fredhopper platform directory. In order to avoid changes to any scripts or monitoring settings you might have set up around the Fredhopper platform, we recommend to install the new version as follows:

1. Rename the existing Fredhopper platform directory: We recommend you keep your current Fredhopper platform installation at hand in the case of an emergency rollback. The best way to do this is to rename the current directory to an easy identifiable name, e.g. assuming that your platform directory is /home/fredhopper or C:\software\fredhopper you move it to /home/fredhopper_backup or C:\software\fredhopper_backup respectively.
2. Unpack the new platform in the place of the previous platform: After moving the previous Fredhopper platform to a new directory, install the new release in the same place the old one was operated from before.

   Please be aware that the archive already contains the standard platform directory ./fredhopper with all contents contained in that directory

4.4. Start the new platform and create the indexer instance

After installing the platform you will first need to copy the content of the previous platform configuration directory into the new platform ./config directory, most importantly the files ./config/fasrc and ./config/topology.txt.

Please ensure that the fasrc file points to the correct Java version.

You can now start the Deployment Agent on the new platform:

Please verify in the agent.log (located in ./data/log in the platform) that the Deployment Agent started successfully without errors.
Next you will need to install your indexer instances - as this process requires the generation of additional folders and changes to the settings of the instances we recommend that you adjust your ./config/topology.txt to only name the indexer instances and execute that setup-cluster command.

The easiest way to deactivate non-indexer entries is to comment them out.

# instance name | host | preset | indexer |  comment
indexer|server1|1|-|Indexer instance on server1
#query_instance_1|server2|1|-|Query instance on server2
#query_instance_2|server3|1|-|Query instance on server3

The query instances are not yet ready to be set up and therefore should not be created - they will be manually introduced later in the process.
After these changes have been made, you can now run the setup-cluster command to create the indexer instances (in case you have more than one).

This process assumes that you have your indexers on the same server. If this is not the case, you will first need to install the platform on all servers that have indexers before running setup-cluster

4.5. Copy required data from old to new platform

If you are using standard data transformations you should have tested the validity of your data generated with the latest version of the standard transformations. If the result was positive, you should only need to copy your existing enrichments to the suitable location in the new transformations. If your transformations are not standard or are not supported by the latest release of the standard transformations, you will need to copy and most likely replace your current set of "load-data" transformations, located in <INDEXER-INSTANCE>/transformations/load-data.

Please contact your Fredhopper consultant if you are not sure about the placement of the transformations.

In a next step, make sure that your input data is provided in the new instance folder. Usually, the input data is provided in ./<INSTANCE>/data/baseline. If you are using incremental updates please also make sure that the incremental update data is provided in the correct folder.

The correct versions of the custom Fredhopper libraries must be placed in the custom directory of the instance (./data/instances/<INSTANCE>/custom) and the custom Data Manager libraries should be correctly installed in the Data Manager installation (e.g. ./data/installations/DM).

4.6. Generate the data

After the non-standard files in your environment have been placed accordingly in the new Fredhopper platform you will need to generate your Fredhopper XML input data. By default, you should use the following command unless your environment has been set up otherwise.

Please verify that this process has finished successfully and that the relevant log files and/or data report reports contain no unexpected errors or warnings.

4.7. Import the configuration and reindex the data

In order to operate your Fredhopper instance you will need to provide it with the necessary configuration. This will be done by importing the Fredhopper configuration capture that contains the configuration you would like to use as base for the migration.

Next you will need to reindex your Fredhopper instance. This will start the indexing process which will result in an operational indexer instance with a switched off syncserver process. The command is as follows:

Please verify that your data has been loaded successfully and correctly and that there are no errors in the relevant log files.

Publish your configuration You will need to publish at this point to ensure that the query instances will find the correct configuration once they are started.

4.8. Start the syncserver process

The query instances in the next steps will require a replication endpoint to connect to. Therefore, you need to start the syncserver process on the indexer instance.

The image below shows the environment at this stage:

5. Upgrading the query instances

At this point, the upgrade of the Fredhopper platform is complete for the indexer instance. You should have a fully working Business Manager and the Preview Pages also operate as expected. For the query instances, you will operate on exactly one instance at a time. This is done to avoid any downtime, as the other instances will be responsible for serving the live query during the time of the instance update.

5.1. Shut down the platform of the first query instance

As a first measure, you will need to shut down the platform used to operate the first live instance. Typically, you will have the following processes still operating at this time:

1. The qserver process of the query instance
2. The Deployment Agent of the platform

After logging in on the server hosting the first query instance, start by shutting down the qserver process of the query instance by executing the following command:

Next, you will need to shut down the Deployment Agent itself. You can achieve this through the following command:

This will leave your environment in the state that is depicted in the image below. As you can see, you will only have one active live server at this moment.

5.2. Update platform on first query server.

Follow the instructions as described in ps:the installation of the new platform on the indexer server.
Please also start the Deployment Agent:

5.3. Create and start query instance manually

After you have restarted the Deployment Agent, you will need to add and configure the Fredhopper query instance manually:

1. Create the instance
2. Configure the role of the instance
3. Configure the syncclient server
4. Configure the syncclient server port

and then start the instance by first starting the syncclient process followed by the qserver process.

You can follow the steps in Add a new query instance to an installation to accomplish these points.

Depending on you setup, you might also need to make changes to additional settings.

After this has been accomplished, your environment will look similar to the image below.

At this point, you will receive responses from both Fredhopper versions, the old and the new, in your front end.

5.4. Shutdown the second query instance

Follow the steps for ps:shutting down the platform from above on the second query server to shut down the existing platform on your second query server.
After finishing the tasks you will have an environment in the state below.

From this point on, you will only receive responses from the new Fredhopper version.

5.5. Install the new platform and update and restart the second query instance

Repeat the steps in ps:installing the new platform and Add a new query instance to an installation on the second query server to install the new platform on the second query server and add and configure the final instance.
Once accomplished, you will now have a fully upgraded Fredhopper environment with the restored query fail-safe.

5.6. Finalizing steps

If you have not done so yet, you will need to uncomment the two query instances in the topology.txt:

# instance name | host | preset | indexer |  comment
indexer|server1|1|-|Indexer instance on server1
query_instance_1|server2|1|-|Query instance on server2
query_instance_2|server3|1|-|Query instance on server3

You should run fresh-index-to-live afterwards to ensure that all the correct links have been made and all upgraded instances act in the expected manner.

6. Process overview

The image below provides an overview over the full process.

7. Rollback

If at any point in time you experience a situation that requires an immediate rollback, you should adhere to the following procedures.

7.1. Indexer instance rollback

1. Shut down the new Deployment Agent and all attached processes

   OS	Command
   Unix	bin/deployment-agent-client stop-agent --force
   Windows	bin\deployment-agent-client stop-agent --force

1. Rename the directory of the platform (e.g. to fredhopper-upgrade and then rename the directory of the previous platform to put it back in the original position
2. Start the Deployment Agent in the old platform

   OS	Command
   Unix	bin/deployment-agent
   Windows	bin\deployment-agent

1. Generate new input data, if necessary
2. Reindex the indexer instance

   OS	Command
   Unix	bin/reindex <INDEXER-INSTANCE>
   Windows	bin\reindex <INDEXER-INSTANCE>

1. Ensure that all upgraded query instances are switched off.

7.2. Query instance rollback

Only attempt this after the indexer has been rolled back. At this point all upgraded query instances are presumed to be shut down.

1. Shut down the new Deployment Agent and all attached processes

   OS	Command
   Unix	bin/deployment-agent-client stop-agent --force
   Windows	bin\deployment-agent-client stop-agent --force

2. Rename the directory of the platform (e.g. to fredhopper-upgrade and then rename the directory of the previous platform to put it back in the original position
3. Start the Deployment Agent in the old platform

   OS	Command
   Unix	bin/deployment-agent
   Windows	bin\deployment-agent

After all query instances have been rolled back, run the fresh-index-to-live command on the indexer instance to restart the old Fredhopper environment.