Tuning Fredhopper Garbage Collection

Generally you will not need to tune Fredhopper's Garbage Collection processes because it already uses the most efficient algorithm, (G1GC), but included below are some of the options you might want to try tuning to improve the performance of your instance. Before you begin making any changes to the GC configuration of your instance(s), please ensure you have read and understood our knowledge base article on Garbage Collection and Memory Management: https://www.fredhopper.com/learningcenter/x/6CJwAQ

Monitoring Fredhopper Process Activity

It is vitally important that you analyse the results of the changes you make to the GC configuration in order to know whether you are making improvements or not, it is advisable to first measure the memory consumption of both the overall Heap and the various regions. You can do this by following the instructions in the following knowledge base articles:

In order to effectively analyse the impact of your tuning, you should monitor from all the points suggested below and thoroughly test the actual performance of Fredhopper under the given conditions!

Garbage Collection Monitoring: https://www.fredhopper.com/learningcenter/x/ryNwAQ
Monitoring Fredhopper JVM: https://www.fredhopper.com/learningcenter/x/XBxwAQ
Application Server Monitoring: : https://www.fredhopper.com/learningcenter/x/URpwAQ
Monitoring Fredhopper Logging: https://www.fredhopper.com/learningcenter/x/XAgnAQ

Tuning Fredhopper Processes

Fredhopper runs various processes, each of which perform a specific task and have very different behavioural characteristics. Therefore, each process needs to be tuned individually because the only "one-size-fits-all" tuning is the default configuration shipped with Fredhopper. To understand the various Fredhopper processes and find their configuration files please see our Fredhopper Processes knowledge base article: https://www.fredhopper.com/learningcenter/x/GCNwAQ

Environmental Tuning

The first and most important is to ensure that the servers running Fredhopper are correctly configured and suitable for the task. Tuning Garbage Collection processes is a pointless task if the application is running in an unstable/unsuitable environment, so if you have not yet configured your server environment then please first review our advice on optimal server configuration before proceeding to make any of the tuning steps detailed in the next section. You can read our Optimal Server Environment Configuration knowledge base here: https://www.fredhopper.com/learningcenter/x/GSRwAQ

GC Tuning Options

Once you are ready to start making changes to the GC configuration(s) of your instance.

It is vitally important that you are able to revert any changes you make so always back up configuration files before making changes!

All changes to the GC configuration should be thoroughly tested before being deployed to your production environment!

Below is a list of common tuning options, but if you are serious about tuning your instance(s) you should also see the Addendum and review some of the external tutorials provided here: https://www.fredhopper.com/learningcenter/x/ZBxwAQ

Heap Size

The most common GC tuning step is to configure the size of the Heap memory in order to ensure that there is sufficient space for objects, (including dead and uncollected objects), in order to prevent the GC process running too often. However, this requires a careful balance because making the Heap too large will increase STW pause times and waste system resources. You can find some insights into the advisable Heap size of each of Fredhopper's processes you can review our knowledge base article on the subject: https://www.fredhopper.com/learningcenter/x/tZX1
..

By default Fredhopper does not specify a minimum Heap size, but in production instances it is advisable to configure the Minimum and Maximum Heap sizes to be the same value to save resources on Heap resizing

When configuring Heap Size using the official JVM parameters, (included below), the value should be added to the end of the argument and suffixed with "k" for Kilobytes, "m" for Megabytes or "g" for Gigabytes, as illustrated in the provided examples

Maximum Heap Size

The most important argument for GC tuning is the Maximum Heap Size argument, which tells the JVM to reserve the maximum amount of memory that will be allowed for the Heap. The JVM will then monitor usage and resize the Heap according to its usage, (using an algorithm based on heap usage and minimum configured Heap size). To set the maximum heap size you can use the following Java option:

-Xmx2g

Minimum Heap Size

The minimum Heap size argument tells the JVM how much memory to allocate to the Heap at startup, and ensures that the Heap size never decreases below this allocated value. When using large heap sizes, it is advisable to configure a minimum Heap size in order to ensure that the Heap is able to exploit the maximum allowable memory and discourage the JVM from decreasing Heap size. To set the minimum heap size you can use the following Java option:

-Xms512m

Eden/Young/New Generation

As explained in our Garbage Collection knowledge base article, most "objects die young", (especially in burst process applications such as the XmlLoader process), so in these cases increasing the size of the New Generation, (also referred to as the Young or Eden generation), can give big performance boosts by ensuring more garbage is collected during Minor Collections and thus reducing the need for Major Collections to occur.

New Generation Size

The default size of the New Generation is 5%, which the JVM will resize as it thinks is appropriate, but this can often be too conservative so values upto 60% can significantly improve the throughput and performance of the GC process.

-XX:G1NewSizePercent=5

Maximum New Generation Size

Sometimes it is preferable to have a very small New Generation to speed up GC cycles, (especially where the portion of long lived objects is high), and therefore the JVM's default maximum of 60% can be too large. In these cases you should specify to the JVM that it should never use more than 10-30% for the New Generation.

-XX:G1MaxNewSizePercent=30

GC Pause Time

The "Max Pause" is a soft limit that the GC process will try to stick to, ensuring that any Full, (Stop The World), Collections don't lead to a pause time higher than the explicit limit. Fredhopper ships with a default value of 500(ms), but this is a soft limit and in periods of excessive GC activity this limit can be exceeded, causing longer pauses. Reducing the value below 500(ms) can lead to increased GC activity, so it is recommended to only change this value when GC activity is excessive.

-XX:MaxGCPauseMillis=n

Parallel Threads

Each Parallel Thread will consume one CPU processor to process the GC, so the more threads the more efficient the GC process will be. The Parallel Threads argument is not shipped with Fredhopper, so instead we rely on the JVM's default setting of N <= 8 parallel GC will use as many cores as available, i.e. 4 GC threads in a Quad Core server. For N > 8 available processors, the number of GC threads will be computed as 3+5N/8, which can lead to less GC threads being used than are available to the server. Applying the maths, we can see that a server with 12 available cores will only actually use 8 Parallel GC Threads, even though it could potentially use 12.

However, it may be preferable to ensure that the JVM doesn't consume all the processing power of the server for the GC process, as this removes CPU from other processes, so judicious use of the Parallel Threads argument can help avoid the server having a seizure. Often leaving some processors available to the server during the GC process can be beneficial, but this should be tested thoroughly as less processors may reduce the effectiveness of the GC process.

-XX:ParallelGCThreads=n

Concurrent Threads

Concurrent Threads are generally managed well by the JVM, and therefore it is best not to change it as it can increase the synchronisation overhead of the GC process, but the formula used to calculate Concurrent GC Threads is ConcGCThreads = (ParallelGCThreads + 3)/4 and the number of threads can be configured using the following argument.

-XX:ConcGCThreads=n

G1 Heap Region Size

The Heap Region Size Sets the size of a G1 region. The value will be a power of two and can range from 1MB to 32MB, with the goal being to have around 2048 regions based on the minimum Java heap size. For G1 GC, any object that is more than half a region size is considered a "Humongous object". Such an object is allocated directly in the old generation and is collected less often, which can lead to Heap memory wastage, so it is recommended to set the G1 Heap Region Size to either 2MB or 4MB to ensure that "humongous" objects are kept in the Eden Generation and collected more quickly.

-XX:G1HeapRegionSize=n

Non-Uniform Memory Addresses

Non-Uniform Memory Addresses can be used to improve performance of the Java heap allocation by exploiting optimisations in server technology, (using NUMA), to allow memory allocations for threads local to the CPU socket. You should check whether your server uses NUMA architecture, although most modern servers do.

Non-Uniform Memory Addresses will conflict with the Explicit Huge Page configuration discussed in the following knowledge base article, so you should use one or the other but not both: https://www.fredhopper.com/learningcenter/x/TRxwAQ

XX:UseNUMA

String Deduplication

Introduced in Java 8 u20, so not compatible with earlier JVM versions

The actual expected benefit ends up at around 10% heap reduction. Note that this number is a calculated average based on a wide range of applications. The heap reduction for a specific application could vary significantly both up and down.

-XX:+UseStringDeduplication

Addendum

This knowledge base article is only a brief explanation of Garbage Collection Tuning, but for a deeper review of the subject you can review our collection of external guides and tutorials: https://www.fredhopper.com/learningcenter/x/ZBxwAQ