Addendum for Manual for Administrators: Apache HTTP Server as Front-End or Reverse Proxy with Jetty or Tomcat as backend

Author: isenberg
Publication Date: 10/21/2014 11:06

This addendum replaces:

PDF: FirstSpirit Manual for Administrators

Chapters:
4.5.1 Apache HTTP Server with the Jetty servlet engine

4.5.2 Apache HTTP Server with the Tomcat servlet engine

FirstSpirit Version: 5.0, 5.1, 5.2

Summary

Apache httpd is used as front-end web server, or also named reverse proxy, to receive all http requests by web browser clients and the FirstSpirit Java based clients SiteArchitect and ServerManager. All requests are forwarded for processing to either the embedded Jetty web server of the FirstSpirit backend server or one or more Tomcat servers.

This updated configuration, compared to the version contained in the FirstSpirit Administrator Manual, is especially important in long-latency network environments (WAN), where for instance the FirstSpirit SiteArchitect is used in a location on another continent than the FirstSpirit Server. The reason is, the default setting for parameter KeepAlive, which is set to "off" on Red Hat Enterprise Linux 6 and newer, but must be enabled to KeepAlive on for FirstSpirit. The FirstSpirit clients SiteArchitect and ServerManager require http/1.0 keepalive or http/1.1 persistent connections for optimal response time. Otherwise, the overall processing time for client operations doubles, as without KeepAlive, a new TCP connection is created for each HTTP request with the SiteArchitect using up to 100 requests for a single client function and establishing a TCP connections requires one additional packet roundtrip.

Software requirements

Apache httpd 2.2 or 2.4
mod_proxy
mod_proxy_balancer
mod_rewrite
mod_proxy_ajp (Tomcat) or mod_proxy_http (Jetty)

Those modules are usually included as standard in Linux distributions. They are all released by the Apache Software Foundation. Replacing mod_proxy_ajp by mod_jk 1.2 is possible but not described in this document, as it is not needed for FirstSpirit and mod_jk is not released by the Apache Software Foundation themself.

Due to a bug in mod_proxy_ajp, which was fixed in version 2.2.22, it is recommended that Apache httpd version 2.2.22, 2.4 or later be used with FirstSpirit. If a faulty version of mod_proxy_ajp is used, a single HTTP request whose processing time exceeds the timeout value specified in httpd.conf will cause disconnection of the entire Tomcat worker. A workaround for most of the problems in older versions of mod_proxy_ajp is to use a value of sufficient size for the "timeout" parameter, e.g. 1200s, as in the following configuration text.

Since the Jetty web server is not fully compatible with AJP 1.3, HTTP must be used in this case for connection Apache and Jetty, i.e. mod_proxy_http. For Tomcat AJP with mod_proxy_ajp is preferred as it supports load balancing and is more transparent for the client-backend connections, with forwarding the client addresses to the backend.

Configuration

The Apache httpd configuration environment depends on the operating system and is usually distributed across multiple configuration files. A standard convention is to use the file /etc/apache2/httpd.conf for general parameters, to use the directory /etc/apache2/mods-available for the module configuration, and to use a file for each virtual web server under /etc/apache2/sites-available. FirstSpirit is using a single virtual web server, i.e. one combination of protocol, hostname and port.

If file types like PHP have to be used by the FirstSpirit project, those will be processed and rendered into HTML by the httpd itself. For that to happen, the lines marked with PHP must be enabled in the configuration settings.

After the Apache server is configured and restarted, the FirstSpirit start page (example: http://firstspirit.mydomain.com) can be accessed via Apache.

If the start page is to be displayed directly via Jetty under the Apache httpd environment for testing or for administration purposes, http://fs5server.localdomain:8000 needs to be entered as the start page in the browser. Port 8000 in this case corresponds to the Jetty server port for HTTP_PORT entered in fs-server.conf.

# Installation on Red Hat:

# copy this file content to /etc/httpd/conf.d/firstspirit.conf

# mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf_

# mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf_

# service httpd restart

# Installation on Debian

# copy this file content to /etc/apache2/sites-available/firstspirit.conf

# disable all lines with "LoadModule" in the file above

# a2enmod rewrite

# a2enmod proxy

# a2enmod proxy_balancer

# Apache 2.4 in addition: a2enmod lbmethod_byrequests

# Tomcat only: a2enmod proxy_ajp

# Jetty only: a2enmod proxy_http

# a2ensite firstspirit

##################################

# common configuration for all VirtualHosts

# enable PHP if required by FirstSpirit project

#LoadModule php5_module modules/libphp5.so

#AddType application/x-httpd-php .php .phtml

#AddType application/x-httpd-php-source .phps

# common modules

LoadModule proxy_module modules/mod_proxy.so

LoadModule rewrite_module modules/mod_rewrite.so

LoadModule proxy_balancer_module modules/mod_proxy_balancer.so

# for Jetty only

#LoadModule proxy_http_module modules/mod_proxy_http.so

# for Tomcat only

LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

ProxyRequests Off

AddDefaultCharset off

Order allow,deny

deny from all

</Proxy>

# keep persistent connection to minimize latency of FirstSpirit clients

# SiteArchitect and ServerManager

KeepAlive On

MaxKeepAliveRequests 0

KeepAliveTimeout 15

Timeout 120

###########

# VirtualHosts

ServerName firstspirit.mydomain.com

LogLevel warn

# debugging:

#LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{COOKIE}i\" %{BALANCER_WORKER_ROUTE}e %D" fslog

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{BALANCER_WORKER_ROUTE}e %D" fslog

CustomLog /var/log/httpd/fs5_access.log fslog

ServerSignature off

UseCanonicalName off

AddDefaultCharset off

ProxyRequests off

RewriteEngine on

ProxyPreserveHost on

LimitRequestLine 65536

# if PHP is required by FirstSpirit project:

#DocumentRoot /opt/firstspirit5/web

#DirectoryIndex index.html index.jsp index.php

# with PHP and Apache 2.4 only

#<Directory /opt/firstspirit5/web>

# Require all granted

#</Directory>

# protect configuration files.

order deny,allow

deny from all

</LocationMatch>

# protect FirstSpirit previews, to be accessible

# from servlet engine only, not from browser clients

order deny,allow

deny from all

# All LAN addresses where servlet engine Tomcat or Jetty is connecting from:

allow from 127.0.0.1

#allow from 1.2.3.4

</LocationMatch>

# If PHP combined with Tomcat and Reverse Proxy with different URL path ("Junction path") is used:

#RewriteRule ^/myjunction/fs5preview(_[0-9]+)?/preview_cache/(.*) /myjunction#fs5preview$1/preview_cache/$2 [last]

#RewriteRule ^/myjunction/fs5webedit(_[0-9]+)?/preview_cache/(.*) /myjunction#fs5webedit$1/preview_cache/$2 [last]

# status monitor for mod_proxy and balancer

SetHandler balancer-manager

order deny,allow

deny from all

# allow access from management network only

allow from 1.2.3.4/24

</Location>

# if using Jetty, set to hostname of FirstSpirit Server (Jetty)

# and to port given by HTTP_PORT in fs-server.conf

# and enable the following line:

# BalancerMember http://localhost:8000 retry=10

# if using Tomcat, set to AJP port given in tomcat/conf/server.xml and enable the following line:

BalancerMember ajp://localhost:8009 retry=10 connectiontimeout=10 ping=5 ttl=1800 timeout=1200

Order Allow,Deny

Allow from all

</Proxy>

# forward almost all requests to FirstSpirit WebApps

RewriteCond %{REQUEST_URI} !^/balancer-manager

RewriteCond %{REQUEST_URI} !^/server-status

RewriteCond %{REQUEST_URI} !^/manager

RewriteCond %{REQUEST_URI} !^/fs5preview(_[0-9]+)?/preview_cache

RewriteCond %{REQUEST_URI} !^/fs5webedit(_[0-9]+)?/preview_cache

RewriteRule ^/(.*) balancer://firstspirit/$1 [proxy,last]

</VirtualHost>

Add the URL and hostname of the frontend server via FirstSpirit ServerMonitor to the file firstspirit5/conf/fs-server.conf. The URL and hostname must be set with the parameters URL, fs.url.* and allowedRedirectHosts according to https://community.e-spirit.com/docs/DOC-1901. If not set, you will get an HTTP ERROR 403 with error message "Forbidden request host:" when accessing the FirstSpirit start page over the frontend server.

In this example:

URL=http://firstspirit.mydomain.com

fs.url.hostname=firstspirit.mydomain.com

fs.url.httpport=80

fs.url.usehttps=false

allowedRedirectHosts=firstspirit.mydomain.com,fs5server.localdomain

Verifying Configuration

If the FirstSpirit start page is available via the URL http://firstspirit.mydomain.com with "firstspirit.mydomain.com" set to the same hostname as given in ServerName above in httpd.conf, then the basic configuration is correct. The next step is to check, if the SiteArchitect is usable, including the display of a preview page within its embedded browser.

Additionally, verification of the availability for http keepalive or persistent connection is required. To check this, use "curl" or any other command line http client. With curl the check continues with using the following command on the client system:

curl -Iv http://firstspirit.mydomain.com

If keepalive or persistent connections are enabled, the output contains the following lines:

[...]

* Connection #0 to host firstspirit.mydomain.com left intact

* Closing connection #0

With an erroneous configuration the test results in those lines without any message of "Connection ... left intact":

[...]

< Connection: close

* Closing connection 0

[...]

Connection: close

https and Jetty backend

To use HTTPS in the Apache httpd and Jetty with http as backend, the following modifications are necessary: In the Apache configuration, additional information must be added to the following parameters:

SSLEngine on

# disable insecure SSLv3 on httpd 2.2.23 and newer:

SSLProtocol ALL -SSLv2 -SSLv3

# disable insecure SSLv3 on httpd 2.2.22 and older:

# SSLProtocol TLSv1

SSLCertificateFile /etc/ssl/certs/myhostnamecert.pem

SSLCertificateKeyFile /etc/ssl/certs/myhostnamekey.pem

#SSLCACertificateFile /etc/ssl/certs/cacerts.pem

# protocol information for backend

RequestHeader set X-Forwarded-Proto "https"

The file paths specified must point to a valid TLS/SSL certificate.

As protocol between Frontend Apache httpd and backend Jetty, the embedded AppServer in the FirstSpirit Server, http is used. Because of the protocol switch from https to http on the backend, a RewriteHandler for setting the protocol in WebApp generated URLs must be added in file firstspirit5/conf/fs-webapp.xml. Otherwise, internal links created by the Firstspirit WebApps point to a wrong URL with protocol http instead of https and launch of the SiteArchitect fails. The complete file with modified sections marked with "X-Forwarded-Proto":

firstspirit5/conf/fs-webapp.xml for FirstSpirit 5.1:

<?xml version="1.0"?>

<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">

<!--

Configuration of FirstSpirit Web-Server.

For parameter details see http://wiki.eclipse.org/Jetty/ and FirstSpirit Administration Manual.

This file is only used when INTERNAL_SERVLET_ENGINE=1 is set in conf/fs-server.conf.

-->

</New>

</Set>

<Arg>

<Set name="statsOn">false</Set>

<Set name="forwarded">true</Set>

</New>

</Arg>

</Call>

<!--

<Set name="KeyStore"><SystemProperty name="cmsroot" />/conf/fs-keystore.jks</Set>

<Set name="KeyStorePassword">changeit</Set>

<Set name="KeyManagerPassword">changeit</Set>

</New>

<Arg>

</New>

</Arg>

</Call>

-->

<New id="RewriteHandler" class="org.eclipse.jetty.rewrite.handler.RewriteHandler">

<Set name="rules">

<Array type="org.eclipse.jetty.rewrite.handler.Rule">

<Item>

<New id="forwardedHttps" class="org.eclipse.jetty.rewrite.handler.ForwardedSchemeHeaderRule">

<Set name="header">X-Forwarded-Proto</Set>

<Set name="headerValue">https</Set>

<Set name="scheme">https</Set>

</New>

</Item>

</Array>

</Set>

</New>

<Item>

</Item>

<Item>

</Item>

<Item>

</Item>

</Array>

</Set>

</New>

</Set>

<!--

<Arg>

<Set name="filename"><Property name="jetty.logs" default="./log"/>/access_yyyy_mm_dd.log</Set>

<Set name="extended">false</Set>

<Set name="logCookies">false</Set>

</New>

</Set>

</New>

</Arg>

</Call>

</Ref>

-->

</New>

</New>

</New>

</New>

</New>

</Configure>

firstspirit5/conf/fs-webapp.xml for FirstSpirit 5.2:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_0.dtd">

<!--

Configuration of FirstSpirit Web-Server.

For parameter details see http://wiki.eclipse.org/Jetty/ and FirstSpirit Administration Manual.

This file is only used when INTERNAL_SERVLET_ENGINE=1 is set in conf/fs-server.conf.

-->

<Set name="detailedDump">false</Set>

</Get>

<Arg>

</New>

</Arg>

</Call>

</Call>

</New>

<Arg>

</Arg>

<Item>

</New>

</Item>

</Array>

</Arg>

</New>

</Arg>

</Call>

<!--

<Arg>

</Arg>

</Call>

</New>

<Set name="KeyStorePath"><SystemProperty name="cmsroot" />/conf/fs-keystore.jks</Set>

<Set name="KeyStorePassword">changeit</Set>

<Set name="KeyManagerPassword">changeit</Set>

</New>

<Arg>

<Item>

</Arg>

</New>

</Item>

<Item>

</Arg>

</New>

</Item>

</Array>

</Arg>

</New>

</Arg>

</Call>

-->

<Item>

</Item>

<Item>

</Item>

</Array>

</Set>

</New>

</Set>

<Arg>

<Set name="filename"><Property name="jetty.base" default="."/>/log/fs-request.log</Set>

<Set name="extended">false</Set>

<Set name="logCookies">false</Set>

</New>

</Set>

</New>

</Arg>

</Call>

</Ref>

</New>

</New>

</New>

</New>

</New>

</Configure>

https and Tomcat backend

To use HTTPS in the Apache httpd and Tomcat as backend, the following modifications are necessary: In the Apache configuration (/etc/httpd/conf.d/firstspirit.conf) the following two lines must be added to the global section before the line <VirtualHost...:

LoadModule ssl_module modules/mod_ssl.so

Listen 443

Then, <VirtualHost *:80> must be changed to <VirtualHost *:443> and the following lines added:

SSLEngine on

# disable insecure SSLv3 on httpd 2.2.23 and newer:

SSLProtocol ALL -SSLv2 -SSLv3

# disable insecure SSLv3 on httpd 2.2.22 and older:

# SSLProtocol TLSv1

SSLCertificateFile /etc/ssl/certs/myhostnamecert.pem

SSLCertificateKeyFile /etc/ssl/certs/myhostnamekey.pem

#SSLCACertificateFile /etc/ssl/certs/cacerts.pem

# protocol information for backend

RequestHeader set X-Forwarded-Proto "https"

The file paths specified must point to a valid TLS/SSL certificate.

If HTTP is used as protocol between Apache httpd and Tomcat instead of the preferred AJP, modify the Tomcat HTTP connector in tomcat/conf/server.xml as follows:

<Connector port="8080" protocol="HTTP/1.1"

URIEncoding="UTF-8"

secure="true"

scheme="https"

proxyPort="443"

maxKeepAliveRequests="-1"

maxHttpHeaderSize="65536"
connectionTimeout="300000"

<!--

Tomcat Manager for automatic deploying of FirstSpirit-Webapps only,

as using the http connector with scheme="https" fails.

URL for FirstSpirit ServerManager, section Webserver: http://tomcathost:8081/manager/text

-->

<Connector port="8081" protocol="HTTP/1.1"

URIEncoding="UTF-8"

packetSize="65536"
connectionTimeout="300000"

Documentation

Tags: admindoc, ajp, front-end, proxy, reverse, tomcat

This addendum replaces:

Summary

Software requirements

Configuration

Verifying Configuration

https and Jetty backend

https and Tomcat backend

Documentation

Related articles