Skip to main content

FHR Search: How to create a standard search configuration

Ernst Vorndran
  • Updated

    This article will explain how to set up a best-practice search configuration for Fredhopper.

    A good search configuration will help you and your customers to find what they are looking for. Also, it will help you to understand better why a specific product is in your result set or why it is not in the result set. You will learn how to set up your search and spell index and set up your multi-pass configuration. We will use a general best-practice approach that covers a wide range of use cases. 

    Step 1: Setting up your search index

    The search index determines which data points will be included (indexed) for search. We will call these data points the search fields. Before configuring your search index, you should have a clear idea about which fields you want to put in the index. Therefore you should know about your customers' search behaviours and your data. Try using only highly descriptive fields in your search index such as:

    • Brand
    • Product name
    • Short description
    • Productid
    • Categories (depending on the quality of your categorization, you might not want to index the whole category tree)
    • Colour
    • Material

    Avoid indexing the long description of your products and all other fields that contain too much text as they contain too much noise. For example, the long description of most T-shirts states that they can be washed in a washing machine. If you sell both T-shirts and washing machines, a search for a "washing machine" will return hundreds of T-shirts in your result.

    Please note that you cannot add int and float type attributes to the search index.

    In our example, we will work with the demo data provided by the data manager. The search fields we will use are:

    • Brand
    • Categories
    • Product name
    • ProductID
    • Gift finder
    • Colour
    • Size

    You can access the search index configuration page in the Merchandising Studio > System manager > Search > Search index.

    Set up for each search field, except productids and other ids, a stemmed and non-stemmed search field. Assuming your search index is empty, here are the steps you have to follow:

    1. Click on "Add".
    2.  Select a source field (i.e. Brand).
    3. Select a stemmed analyzer depending on the language of the shop (i.e. English Stemmed).
    4. Name your search field "brand" (choose a human understandable name for your search field).
    5. Click on "Add".
    6. Select same source field (Brand).
    7. Select a default analyzer depending on the language of the shop (i.e. English Default).
    8. Name your search field "brand_spell" (the fields using the default analyzers will be used in the spell
      index).
    9. Repeat step 1 to 8 for the other search fields (except ProductID which only needs to be indexed with the default analyzer).
    10. Click "Apply" to save your changes.

    A note on Search Fields and naming conventions: Although this page recommends naming the fields according to their intended use, for example, brand_spell for the default brand search field, you may see different naming conventions in the Merchandising Studio. What is important is not the name of the field but the linguistic analyser incorporated in the creation of the search field. Always check that every search field that is used in the Spell Index utilizes a default linguistic analyser (not to be confused with the Spell Check Pass, which should use exactly the same search fields as the EXACT pass and use only search fields employing a stemmed linguistic analyser). Your search field may be named “name_s”. Do not assume that “_s” refers to “_spell” when activating this in your Spell Index or deactivating it elsewhere. It may refer to a
    stemmed field. Always check your search index and name fields clearly where possible.

    In our example the search index looks as follows:

    mceclip2.png

    Your search index is now set up.

    Please note that there is a difference between the Spell Check Pass and the Spell Index.
    The Spell Check Pass should not be confused with the Spell Index. The Spell Index specifies the fields used to enable spelling corrections and provide an alternative. The SPELL CHECK pass specifies the fields
    that are used to:

         a) check if the spelling alternative will return any results, and
         b) to return the results the spelling alternative is matched to.


    It looks something like this: when the search query is not matched in the ID or EXACT pass and ‘arrives’ at the SPELL CHECK pass. In the SPELL CHECK the SpellCheckPass Component first sends the query to the Spell Index to find a viable alternative and, when found, to send it back. When the alternative arrives at the SPELL CHECK pass, it is then treated as it would be in an EXACT pass, hence the need for exactly the same active search fields in the EXACT pass and the SPELL CHECK pass. Changes in the search index require a reindex to take effect, but you can continue configuring the next steps before reindexing your data.

    Step 2: Set up your SPELL index

    The spell index determines which search fields are used to create your spell correction index. You can access the spell index in the Business Manager > System manager > Search > Spell index.

    The fields used for the spell index need to be selected by clicking on the red light next to it so it turns green and by deactivating green fields that should not be in the spell index.

    Only use the non-stemmed fields for the spell index! In other words, only use search fields that employ the Default linguistic analyser, for example, the English Default.

    Your spell index should look as follows:

    mceclip1.png

    Your spell index is now set up.

    Step 3: Configure your SEARCH passes

    Your search pass configuration is the heart of the search. It determines which search fields are used at what time. Our best practice approach consists of 4 search passes:

    • ID Pass
    • EXACT Pass
    • SPELL CHECK Pass
    • OR Pass

    The passes are executed from top to bottom, so first Fredhopper will check if the user is searching for an ID (ProductID, ISBN, etc...). If not, an EXACT match will be executed looking for products that exactly
    match the users' search query.

    Please note: Synonyms, lower/upper case differences, plural/singular differences etc, are all handled as exact matches

    If, however, there is no exact match to the users query, Fredhopper will perform a spell check (i.e. addidas -> adidas) and provide a spell check feedback (your search for "addidas" gave 0 results, we also searched for "adidas"). If the spell check does not find anything, Fredhopper will perform an OR search (i.e. "red adidas shoes" becomes "red shoes" or "adidas shoes" or "adidas red"). As you can see these passes are going from strict (ID, EXACT) to loose SPELL CHECK, OR) like a pyramid, you should therefore configure the SPELL CHECK and the OR pass in such a way that they are only executed if there was no result in the strict passes! This is done by setting the display conditions of the SPELL CHECK and OR pass to "Don't show if other results > 0". Your overall search pass configuration should look like this:

    mceclip3.png

    Changes in your search pass configuration are immediately applied and do not need a reindex!

    ID Pass

    This pass only holds your ID fields (productID, ISBN, etc.). The pass component used is the UniqueIdPassComponent. In our example, the ID Pass looks as follows:

    mceclip4.png

    Be sure to set the maximum token length to a value big enough to cope with your IDs.

    EXACT Pass

    This pass holds all your stemmed search fields. Do not put long descriptions in this pass!
    The pass component used is the PrefixPassComponent. In our example the EXACT pass looks as follows:

    mceclip5.png

    The settings for this pass are always:

    • Minimum token length = 50.
    • Maximum query tokens = 1.

    SPELL CHECK pass

    This pass holds exactly the same search fields as your EXACT pass. The pass component used is the SpellCheckPassComponent.


    Please note that if you use different fields in the spell check pass than in the exact pass, the result sizes and the sorting of products will differ from the exact match.


    In our example the SPELL CHECK Pass looks as follows:

    mceclip0.png

    mceclip1.png

    Optimise spelling corrections:

    Fredhopper automatically corrects mis-spelled search terms, e.g. televisino is corrected to television and jaens is corrected to jeans. Learn more on how to optimise spelling corrections here.

    OR PASS

    This pass holds exactly the same search fields as your EXACT and SPELL CHECK PASS. The pass component used is the MaxTokenPassComponent.

    In our example the OR Pass looks as follows:

    It may seem redundant to employ exactly the same search fields for the EXACT, SPELL CHECK and OR passes, but, although the search fields are identical the components and relevant settings differ.
    Learn more about search components in Fredhopper Search Passes here.

    mceclip2.png

    mceclip3.png

    In short, the search fields for the EXACT, SPELL CHECK and OR passes are exactly the same but the components and relevant settings change for each pass:

    Pass  Component/Algorithm  Suggested Fields
    ID Unique ID Component Identification fields such as Product
    EXACT Prefix Pass Component All your stemmed search fields. Do not include long descriptions
    SPELL CHECK  Spell Check Pass Component Same search fields as the EXACT pass
    OR Max Token Pass Component Same search fields as the EXACT pass and SPELL CHECK pass


    Congratulations! You have successfully configured your search.

    Related articles

    Comments

    0 comments

    Please sign in to leave a comment.