Step 2 - Install Fredhopper Suggest

This page explains all essential steps to run Fredhopper Suggest on your website.

Step 2.0: Review Hardware Requirements

For a typical website Fredhopper Suggest will receive about three times as many queries per second as FAS. For example, a website with 25 FAS queries per second requires hardware that supports 75 Suggest queries per second.

For 75 "Suggest QPS" the following hardware is required:

• 2 GB RAM;
• 1x single core 2GHz+ CPU;
• 5GB hard disk (incl. log files);
• Sun JDK 1.6+.

Index Generation The Fredhopper Suggest data index is generated in the Fredhopper Managed Service environment. As such only the Fredhopper Suggest query server has to be installed locally.

Step 2.1: Deploy the Fredhopper Suggest query server

There are two different approaches to deploying the Fredhopper Suggest query server:

1. Install the Fredhopper Suggest query server using the deployment package;
2. Install the Fredhopper Suggest query server as additional module into FAS. (Not recommended and only possible with Fredhopper Suggest 2.0.x and below.)

Legacy Support The option to install the Fredhopper Suggest query server as a module inside FAS is no longer the preferred installation option. This feature has been removed as of Fredhopper Suggest 2.1. Installation of the Fredhopper Suggest query server inside FAS leads to various problems due to the difference in the two systems with regards to indexing and query behavior. Such problems are mitigated by installing Fredhopper Suggest using the Deployment Package.

There are four versions of Fredhopper Suggest available:

• Fredhopper Suggest 1.0.x is intended for installation with versions of FAS up to FAS 6.3. This version only supports installation as a module inside FAS.
• Fredhopper Suggest 2.0.x is a version of Fredhopper Suggest with a new architecture. This version supports installation using the Fredhopper Deployment Agent as well as installation as a module inside FAS (up to FAS 7.1).
• Fredhopper Suggest 2.1.x comes with vastly reduced memory requirements. It can only be installed using the Fredhopper Deployment Agent.
• Fredhopper Suggest 2.2.x allows on-premise users to better monitor the service. New customers are urged to use this version.

Please make sure to use the correct version of Fredhopper Suggest.

Deploying the Fredhopper Suggest query server using the deployment package

1. Download the Fredhopper Suggest query server from the Tools and Downloads section.
2. Create a new Suggest installation (called 'SUGGEST' in this example).

   $ bin/deployment-agent-client --location HOST create-installation --name SUGGEST fredhopper-releases/suggest-distribution-2.1.zip
   

3. Create a new Suggest instance.

   $ bin/deployment-agent-client --location HOST create-instance suggest SUGGEST
   

4. Configure the Suggest instance port (optional). By default Suggest is configured to use port 8380.

   $ bin/deployment-agent-client --location HOST set-option suggest suggest_serverport=8380
   

Configuration in config/topology.txt Fredhopper Suggest cannot be configured in config/topology.txt yet. Please do not add entries for Suggest to this file. Fredhopper recommends one includes information about Suggest in the project integration documentation.

Deploying the Fredhopper Suggest query server as a module

Supported Versions This installation approach is only supported with FAS 6.3 (Fredhopper Suggest 1.0.x), or FAS 7.1 (Fredhopper Suggest 2.0.x).

Follow these steps to install Suggest into the Fredhopper Query Server:

1. Remove all previous versions of the Fredhopper Suggest query server.

   cd FAS
   cd j2ee/jboss/server/fas/deploy
   

   Make sure that you delete all *suggest*.war files (for example fredhopper_suggest-1.0.0.war or 9suggest.war).

2. Download the Fredhopper Suggest query server from the Tools and Downloads section.
3. Unzip the downloaded Fredhopper Suggest zip file within the FAS directory.

   cd FAS
   unzip fredhopper-suggest.zip
   

4. Restart FAS.

   bin/qserver
   

After a successful installation, http://HOST:PORT/suggest/json can be queried. After loading the index, the suggestions will be available.

Step 2.2: Updating the Suggest index

Uploading data

Please use the Fredhopper Services API to upload data.

Depending on the type of suggestions you want to use, different files are required. Choose the type that you would like to have from the following list and include the required files in the data.zip upload:

• Searchterms: Analytics data containing searchterms and the frequency of their usage;
• Products: fredhopper-*.xml.gz;
• Categories: fredhopper-*.xml.gz;
• config/suggest-include.csv (optional)
• config/suggest-exclusions.csv (optional)

Example:

curl -v -k -u user:password -X PUT -H "Content-Type: application/zip" --data-binary @data.zip https://my.eu1.fredhopperservices.com/suggest:live1/data/input/data.zip?checksum=731b0140b43a830255cc2dd28c4923fd

Store the data ID found in the response (bolded):

Triggering index generation

You can trigger the generation of the Suggest index generation whenever you like using the trigger suggest:generate using the REST API. Example (Using the ID from the previous step):

curl -v -k -u user:password -X PUT -H "Content-Type: text/plain" --data-binary "data-id=2010-10-01_15-15-53" https://my.eu1.fredhopperservices.com/suggest:live1/trigger/generate

Again, you can find the ID of the trigger in the response (bolded):

The Suggest index should be updated on a daily basis, but depending on the nature of the data a less frequent cycle may also be sufficient. For example, one may opt for a weekly update of authors and brands. For shorter transfer times one can upload the data frequently, e.g. hourly.

Monitoring the generation

Using the trigger ID obtained in the previous step one can monitor the progress of the Suggest generation using the Fredhopper REST API. Example:

curl -k -u user:password https://my.eu1.fredhopperservices.com/suggest:live1/trigger/generate/2010-10-01_15-41-44/status

Downloading an index

Execute the following steps to download the index for your local Suggest Query Server after receiving a SUCCESS status message for the trigger:

1. Download the new index via the Fredhopper Services API using the trigger ID.
   Example:

• wget

  wget --no-check-certificate --user=user --password=password https://my.eu1.fredhopperservices.com/suggest:live1/data/output/2011-10-01_15-41-44/output.zip
  

• curl

  curl -k -u user:password https://my.eu1.fredhopperservices.com/suggest:live1/data/output/2011-10-01_15-41-44/output.zip -o output.zip
  

1. Extract the index files (*.idx) from the output.zip file into FAS/data/suggest.
2. Restart the Fredhopper query server.

Downloading the latest output It is also possible to download the "latest processed output" rather than the output for a specific trigger ID.

To verify the correctness of the index, request the suggestions. For example, a search for the letter s should contain suggestions in JSON format: http://SERVER:PORT/suggest/json?search=s&scope=//UNIVERSE/LOCALE.

Step 2.3: Continuous synchronization of the Number of Results

The Number of Results for each suggestion can become inconsistent with responses from FAS, because Fredhopper Suggest does not use the same index structures as FAS. Fredhopper Suggest can query FAS regularly to retrieve the latest number of search results for each suggestion. To not interfere with FAS, the process sends only one query every several seconds. When querying FAS, suggestions for which Fredhopper Suggest does not know the number of results take precedence, followed by the most heavily used suggestions.

To update the number of results, your Suggest index must to contain at least one of two fields: either searchterm for keyword suggestions or fhLocation for category suggestions. Please check with your Fredhopper contact person whether your indices meet this criterion. If you want to influence the interval and speed with which these updates are performed, please consult the Suggest configuration page.

Deploying Suggest Query Server behind a firewall Web browsers require that the Ajax code connects to the same domain and port for security reasons, e.g. the web site http://www.my-shop.com cannot make Ajax requests to http://www.your-shop.com. Since the Suggest Query Server is typically not installed on the front-end server, a tunneling proxy or gateway is required to invisibly forward the requests (i.e. the content is forwarded without sending a 301/302 HTTP code to the web browser).