Skip to main content

Directories and Files with Deployment Package

Vasil Kanev
  • Updated
    This page is an overview of the Fredhopper deployment directories and files using the Fredhopper Deployment Package.

    Directories

    <FREDHOPPER HOME DIRECTORY>
|-- bin // Executable scripts
|-- config // Deployment Package Configuration files
|   |-- topology.txt
|   `-- fasrc
|
|-- data // Data for the deployment agent
|   |-- installations // Home directories for installed software
|   |   |-- DM
|   |   |-- FAS
|   |   `-- ...
|   |
|   |-- instances // Home directory for instances
|   |   |-- etl
|   |   |-- deployment
|   |   |   `-- template // Templates used when setting up
|   |   `-- <INSTANCE>
|   |       +-- log      // all FAS logs: qserver.log, fredhopper.log, xmlloader.log, etc.
|   |       +-- config   // place for business.xml, localization files.
|   |       +-- custom   // extensions for this FAS instance (optional)
|   |       +-- data
|   |       |   +-- xml
|   |       |       +-- incoming
|   |       |       |   +-- batch  // Input FAS XML for XMLloader for full reindex
|   |       |       |              // Pushed here by bin/reindex from
|   |       |       |   +-- rt     // Incremental FAS XML
|   |       |       +-- processed  // Once processed by XMLloader, the files are archived here
|   |       |           +-- batch
|   |       |           +-- rt
|   |       +-- ...
|   |
|   `-- deployment.properties // Configuration of the deployment agent
|
|-- data-manager // Data Manager
|-- deployment-agent // Deployment agent home directory
|-- log // Global log files
|-- tmp // Temporary files
`-- <INSTANCE DIRECTORIES>

    Additionally, the following directory structure is created for each instance defined in the config/topology.txt file.

    <INSTANCE>
|-- backup // FAS capture backups, created by re-index before it wipes the current index
|   `-- hostname-cix-201004073531.zip
|
|-- data
|   |-- backup // Backup of 3rd-party data
|   |-- baseline // Standardized fredhopper source data for full re-index
|   |   |-- categories.csv
|   |   |-- custom_attributes_meta.csv
|   |   |-- custom_attributes_values.csv
|   |   |-- custom_variant_attributes.csv
|   |   |-- products.csv
|   |   `-- variants.csv
|   |
|   |-- fas-xml // XML made by ETL (data manager) from CSV, this data is pushed to
|   |              XMLloader for a full re-index by bin/reindex
|   |-- fas-xml-incremental // XML made by data manager out of CSV, this data is
|   |              picked up by the indexer for an incremental update
|   |-- incremental // Incremental source data
|   |   `-- ...
|   |
|   `-- tmp // Temp data
|
|-- transformations  // instance specific transformations
|   `-- load-data // Standard instance transformations for data generation for re-indexing or updating the instance
|
|-- logs // logs directory for instance specific logs
|   |-- yyyy-mm-dd_HH-mm-ss // per ETL job run a new directory is created
|   `-- ...
|
`-- config // demo configuration files

    Log Files

    • data/log/agent.log, output of the deployment agent
    • data/instances/deployment/log/kitchen.log, output of jobs
    • data/instances/etl/log/kitchen.log, output of data manager transformations
    • data/instances/<INSTANCE>/log/*.log, these are the FAS logs of an instance
      • Indexer logs:
        • xmlloader.log
        • treebuilder.<universe>.log
        • searchindexer.log
        • syncserver.log
        • qserver.log
        • fredhopper.log
        • fredhopper.biz.log
        • access.log
        • *.wrapper.log - startup logs
        • *.gc.log - Grabage collection logs
      • Live qserver logs:
        • qserver.log
        • qserver.gc.log
        • fredhopper.log
        • syncclient.log
        • syncclient-wrapper.log
        • fredhopper.biz.log
        • access.log
    • <INSTANCE>/logs/*, these are the ETL process logs of an instance

    Configuration Files

    The following configuration files configure the entire FAS cluster

    • config/topology.txt
    • config/fasrc

    config/topology.txt

     

    config/topology.txt is a '|'-separated text file with fixed columns.

    Column Name Description
    1 Instance Name Name of the instance (must be unique)
    2 Host Name The host of the instance
    3 Preset Reference to one set of pre-defined configuration settings that control memory and port assignments
    4 Indexer Name Name of the instance that provides the index for this live instance, or '-' if this instance is an indexer
    5 Comments Additional comments can be empty

    An example config/topology.txt might look like this:

    #
# This is a comment.
#
demo|indexer.example.com|1|-|Indexer and Preview environment for demo setup
demolive1|live1.example.com|2|demo|Live Query Server for demo setup
demolive2|live2.example.com|2|demo|Live Query Server for demo setup

     

    You can find more information about config/topology.txt at Working with the topology file.

    config/fasrc

    config/fasrc contains additional configuration variables for the Fredhopper Deployment Package.

    Not all configuration variables apply for all installations. 
    # Paths
export JAVA_HOME=/usr/java/default
export PATH=$JAVA_HOME/bin:$HOME/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin

# Fill in the minimum number of items that should be in an index at any point in time. (Take 60% of normal catalogue size)
export MINITEMS=22

# set to true if you want cleanup to remove also 10\+ days processed input xml.
export WIPE_PROCESSED_XML=false

# Time in seconds to wait between marking an instance unavailable and actually stopping it
export LB_WAITTIME=0

# Number of live servers fresh-index-to-live should work on in parallel
export BATCH_SIZE=1
     

    Comments

    0 comments

    Please sign in to leave a comment.