Clean-up Fredhopper Directories and Reduce Disk Space Usage – Rezolve Ai Support Portal

This articles explains what files to cleanup on a daily base to prevent disks filling up completely. Typically we advise to archive files after N days, where N is 7.

Files should always be archived to a different disk partition to the one where Fredhopper is located in order to avoid filling the Fredhopper partition!

Cleanup directories and files

In the table below INSTANCE_DIRECTORY refers to the root directory of the instance. In a typical installation this is the directory data/instances/INSTANCE_NAME.

The deployment-agent-client can be used to determine the directory:

$ bin/deployment-agent-client --location INSTANCE_HOST describe-instance INSTANCE_NAME

This command returns a list of key=value pairs, the instance directory is returned as value of the directory key.

Component	Directory	Files	Comment
FAS Indexer + Business Manager	`INSTANCE_DIRECTORY/log`	all rotated logfiles (*.log) with yyyymmdd in their file names and older than N days	Don't touch files ending on pid, wrapper, instance or socket.
	`INSTANCE_DIRECTORY/data/xml/processed`	all files older than N days	only if there is no need to ever look into the processed files again.
	`INSTANCE_DIRECTORY/tmp`	all files older than N days
	`INSTANCE_DIRECTORY/jboss-server/log`	all rotated server.log.yyyy-mm-dd and accesslog.yyyy-mm-dd older than N days
FAS Live Query Server	`INSTANCE_DIRECTORY/log`	all rotated logfiles (*.log) with yyyymmdd in their file names, except fredhopper.biz.log files**, and older than N days	Don't touch files ending on pid, wrapper, instance or socket.
	`INSTANCE_DIRECTORY/tmp`	all files older than N days
	`INSTANCE_DIRECTORY/jboss-server/log`	all rotated server.log.yyyymmdd and accesslog.yyymmdd older than N days	Remove access logs only if you don't need them anymore for statistics.
Data Manager	`INSTANCE_DIRECTORY/log`	all log files older than N days	A typical setup has two data manager instances: 'etl' and 'deployment'. Both instances require periodic cleaning.

Automate cleanup

The deployment package is shipped with a Unix tool called: ?bin/cleanup that can be run periodically (for example on cron) on each node.
This tools takes as argument the number of days and defaults to 5:

?bin/cleanup <days>
# Or on cron:
5 4 * * * (cd ~fredhopper ; bin/cleanup >> log/cleanup.log 2>&1 )

It removes the following files older than <days> days in FAS home directory:

data-manager/log
tmp
log
data/log
data/instances/deployment/log
<INSTANCE>/backup
<INSTANCE>/data/xml/processed
<INSTANCE>/log
<INSTANCE>/jboss-server/log