introduction

The installation and configuration of Scarab is intended to be as simple as possible. This document outlines the directory structure of the distribution (currently it mirrors the CVS tree), the requirements for running Scarab, the instructions for building and running an installation as well as requirements for setting up the database.

If you are doing development work with Scarab, please make sure to read

http://www.solitone.org/scarab_doc/
http://www.solitone.org/scarab_doc/howto/build-howto.html

In this document we use the following logical locations:

• $SCARAB_ROOT
  
  This is the root of your Scarab source installation
• $SCARAB_WEBAPP
  
  This is the root of your webapplication

We appreciate your deciding to try Scarab out and appreciate constructive feedback on your success (or failure...we hope not!) in getting the system running.

requirements

• SDK 1.3.1 or higher available from

  http://java.sun.com/

  Make sure that you download the SDK and not the JRE! On some operating systems (OS X, for instance), the SDK is included.

  When compiling Scarab with Jikes, version 1.18 or higher is required. Please do not try with Jikes 1.17, as it is buggy.

• Maven-1.0 or higher available from

  http://maven.apache.org/

  Note: If you prefer to use Ant (see below), you don't need to install Maven. If you want to contribute to Scarab Source code, or you need to rebuild all bits and pieces of Scarab, you will be more happy with Maven though.

• Ant 1.5 or higher available from

  http://ant.apache.org/

  If you prefer to use Maven (see above), you don't need to install Ant. If you only want to install, configure and use Scarab, Ant is your best choice.

• Tomcat 4.0.4 or higher available from

  http://jakarta.apache.org/tomcat/

  Tomcat 4.1.x is included with Scarab.

• An RDBMS preferably one out of the list below
  • MySQL 3.23.x/4.0.x available from

    http://www.mysql.com/

    If you want to use the faster/newer JDBC driver with MySQL, you can download it from the MySQL website and use them instead. We do not distribute it because it is GPL. To use it, just copy the .jar file into your scarab/lib directory and put this in your build.properties file:

    scarab.database.jdbc.driver.jar=mysql-connector*.jar

    see: http://www.mysql.com/downloads/api-jdbc-dev.html

    Scarab will eventually require the transactional support present only in MySQL version 4 and higher, and will drop support for the transaction-less versions 3.23.x and lower. If getting started using MySQL, it's recommended that you install version 4 or higher.

  • Postgresql 7.3.x available from

    http://www.postgresql.org/

  • hypersonic 1.7.1 available from

    http://hsqldb.sourceforge.net

  NOTE: More detailed instructions for setting up the database on different database vendors are available on our website.

  There are also several other databases with partial support: MS SQL Server, Oracle and DB2. At least oracle and DB2 have known problems, but we do encourage people to try these databases and submit fixes for them.

Environmental settings

JAVA_HOME

You must have the JAVA_HOME environment variable properly set to be the location of your SDK installation directory.

Note for MacOSX: On MacOSX this path is: /System/Library/Frameworks/JavaVM.framework/Home

Database settings (for MYSQL)

The database is assumed to be installed and running with appropriately configured access control setup (see below for more detail). You must have the database binaries in your PATH (ie: $MYSQL_HOME/bin).

Database settings for other RDBM's

With the Scarab communities help, we will be supporting a wide range of databases in the released version of Scarab, however, the Scarab developers are currently doing development primarily on MySQL and thus do not guarantee that Scarab will work on other databases.

Example scripts

The following scripts can be used as templates for your own environmental setup. The templates have been created for use wih MySQL. You must modify them according to your environment:

UNIX/LINUX sh/zsh/bash

export MYSQL_HOME=/path/to/mysql-install
export JAVA_HOME=/path/to/jdk-install
export PATH=${PATH}:${ANT_HOME}/bin:${MYSQL_HOME}/bin:${JAVA_HOME}/bin

Put these settings into your ~/.bashrc or ~/.login file

UNIX/LINUX csh/tcsh

setenv MYSQL_HOME /path/to/mysql-install
setenv JAVA_HOME /path/to/jdk-install
setenv PATH ${PATH}:${ANT_HOME}/bin:${MYSQL_HOME}/bin:${JAVA_HOME}/bin

Put these statements into your ~/.cshrc file

WINDOWS

set MYSQL_HOME=\path\to\mysql-install
set JAVA_HOME=\path\to\jdk-install
set PATH=%PATH%;%ANT_HOME%\bin;%MYSQL_HOME\bin;%JAVA_HOME%\bin

Add these settings to your Environment ( start settings system )

.jar files

All of the necessary .jar files for building and running Scarab are included in the /www/repository/* directories and the build system is setup to include these into your classpath for you. Please do not add any jar files to your CLASSPATH as it may cause compile errors.

Port settings of the application server

If you already have an existing webserver or service running on ports 8080 and 8005, and you are using Scarab's version of Tomcat, you will need to change the port number to another unused port number by editing

File permission settings

By default, the file permissions of the web application directory tree WEB-INF/... must allow write access for the userid which runs the application server.

XML-environment

NOTE: Make sure to use the copy of Xerces which is included with the Scarab distribution in /www/repository/xerces/jars and make sure that no other copies are in your JAVA_HOME or in your CLASSPATH. Otherwise, you may get build errors.

Directory structure

Here is a description of the Scarab directory tree. In the remainder of this document we will refer to the root diretory of your distribution as $SCARAB_ROOT.

scarab
  +- build          <-- Ant scripts for building the sample webapp
  |                     and creating/loading the sql scripts.
  +- extensions
  |   +- usermods   <-- place your modifications to the standard
  |   |                    distribution here. These modifications will be
  |   |                    incorporated into the build. This simplifies
  |   |                    the process of replacing *.vm files,
  |   |                    among other things.
  |   +- scripts    <-- Helper shell scripts
  |   +- dtd        <-- DTD's for validating XML files
  |   +- bugzilla   <-- Scripts for converting from Bugzilla to Scarab
  +- lib
  +- src            <-- The source files.
  +- tomcat         <-- Included Tomcat configured ready to run with Scarab.
  +- www            <-- Scarab Website
      +- repository <-- The .jar repository for jars used by Scarab.
  +- xdocs          <-- xml versions of manuals (guides, howtos etc)

Within the /src directory you find following subdirectories...

src
  +- conf
  |    +- conf      <-- Various configuration files for Scarab.
  |    |                TurbineResources.properties and Scarab.properties
  |    |                live here.  Copied to WEB-INF/conf
  |    +- classes   <-- Various configuration files copied directly 
  |                     into the classpath.
  +- java           <-- The Java source code for Scarab.
  +- migration      <-- Java source for migration ant tasks
  +- schema         <-- database schema files
  +- sql            <-- SQL files for defining the database.
  +- templates      <-- ??? [FIXME] what is tis for ???
  +--test           <-- Test suite code.
  +--webapp         <-- All the web resources required for Scarab like 
  |                     images, css files, javascript files, html files,
  |                     velocity templates.

Build properties

The Scarab build process has been designed for maximum flexibility. Most of the aspects in Scarab can be controlled via build-properties. We provide a set of "default properties" which will be used by the build process to build an "out of the box" installation. These default properties will be used if not overwritten otherwise (see below). They are all defined in the file

Note that currently the out of the box installation won't work, because some properties MUST be configured for every individual site.

Now you will want to modify some of the default settings to your individual settings. For this purpose you must create the file

This file does not exist by default and you must create it manually. To make life a bit easier, we have provided a sample file, which contains only the most common properties which are very likely to be changed by you:

You can simply copy this file to your build.properties and then apply all your modifications to build.properties. If you want to have full control over all available properties, you can also copy project.properties to your build.properties instead of minimal.properties

Please note that the properties modified in build.properties only affect the build. Changing them and then immediately restarting your appliation server has no effect at all - see the following section on runtime properties for more information.

For advanced users who have several development projects on their system, or even multiple Scarab distributions, the Scarab build-system also reads properties from the following locations:

where '~' is your home directory. Any values set in these files override the ones in build.properties. The former is designed to hold properties that should be shared between different Scarab builds, while the latter is for common settings that are available in more than one project.

You will find more information about how to build Scarab further below in the chapter "Building the sandbox"

Runtime properties general notes

Scarab supports runtime properties which will be loaded each time the server is restarted. So for example, if you want to change the address of the mail server to use, all you would have to do is modify the appropriate runtime property and restart your application server.

Where you change your runtime properties is dependent on how you install Scarab:

Developer's choice

If you are a developer and you build Scarab on your own, your best choice is to always modify build.properties. After deployment there is no further need for modifications and your runtime properties are ready available in the file

Every time you rebuild Scarab from sratch, your runtime property settings will be regenerated from the settings in build.properties.

Administrator's choice

If you got a binary distribution of Scarab (i.e. a .war file or a .ear file), either from you developer team or from the Scarab website (currently under preparation), your best choice is probably to create/modify the file

Your best approach in this case would be to copy the file $SCARAB_WEBAPP/WEB-INF/conf/defaultCustom.properties to custom.properties in the same directory and then modify your settings in custom.properties. Note: The file custom.properties exists and is empty by default.

Master of quality assurance choice

If you need to often redeploy Scarab AND you want to frequently change your runtime settings, you will need to modify custom.properties and build.properties. Your best strategy would be to keep these files always identical.

In addition to the build.properties and custom.properties file, you can also define environment properties in your servlet's JNDI tree. You can do this either via the Tomcat Admin application or by editing the appropriate files. To add a property that won't be overriden by each new .WAR file you install, edit

and add entries to your <context/> entry for Scarab. For example, to set the system.mail.host property you would add these line:

<Environment description="Mail Host to Use"
                name="system/mail/host"
                override="true"
                type="java.lang.String"
                value="127.0.0.1"
/>

To set which database adapter to use you would add a line like this:

<Environment description="Database Adapter"
                name="torque/database/scarab/adapter"
                override="true"
                type="java.lang.String"
                value="oracle"
/>

For more information about other sources of Configuration data, read about Commons-Configuration:

In the remainder of this text we will use the following notations:

• build.properties
  
  for the file build.properties in the Scarab Root installation directory.
• custom.properties
  
  for the file WEB-INF/conf/custom.properties in the webapplication directory tree.

Mailserver settings

Scarab is a "email-centric" application. Hence you need to set the relay outgoing mail server so that email can be sent from Scarab. This is important for many different aspects of Scarab, such as the confirmation email sent when a user registers with the system. By default, the mail server is defined as "localhost". Therefore if you don't change the settings, you must have an SMTP server running on the same server where Scarab runs.

You can set the location of your email server by modifying the property

in custom.properties or by using the JNDI tree as discussed above.

You will need to stop and restart Scarab/Tomcat for the changes to take effect.

Upload Directory setting

When users attach files to an issue, by default, the file is saved to disk in the WEB-INF/attachments directory. This may or may not be an optimal place to store attachments potentially because of disk size issues. Therefore you have two options:

1. On Unix you can move the WEB-INF/attachments directory to another location and then create a symlink from that location to WEB-INF/attachments.
2. On Unix and other platforms (Win32) you can set the property

   scarab.attachments.path

   to point at another directory. By default, this path is relative to the webapp directory and is set to WEB-INF/attachments. You can also define an absolute path such as

   /bigdisk/scarab/attachments.

It is recommended to also set the property

in your custom.properties. This property is used as a temporary location for the uploaded data during the upload process. By default the temporary upload directory is WEB-INF.

Indexes Directory settings

Scarab uses Lucene to create searchable indexes for the issue data. Lucene needs to be able to store its indexes somewhere on disk. If the disk is low on space, it is recommended to put the indexes on another disk. This can be done by setting the property

in your custom.properties. This path defaults to WEB-INF/index and can be defined as either a path relative to the webapp root directory or an absolute path anywhere in the file system (watch out for file write permissions!).

Running with Servlet-2.2

Non-current versions of some commonly used application servers may not support the servlet-2.3 specification. E.g. Websphere 4.0 only supports the servlet-2.2 specification.

While the Scarab project does not intend to officially maintain compatibility with the servlet-2.2 specification, there currently is only a single problem area, involving the HttpRequest.setCharacterEncoding() call in DetermineCharsetValve. Removing this call weakens Scarab's support for character set encoding, but it allows running Scarab with the Websphere 4.0 application server.

To enable this feature, add the property

to your custom.properties

Building the sandbox

The Scarab sandbox contains everything you need in order to get started with Scarab. It includes a stripped down version of the Java Servlet Engine (Tomcat-4.1.*) pre-configured to run Scarab.

To build the sandbox on your machine, you ensure, that the requirements in are fullfilled. You must have a Scarab source distribution unpacked to any convenient place in your file system. Now go to the Scarab root directory and start the Scarab build system as follows:

with ant:

    cd build
    ant

with Maven:

    maven java:compile
    maven war:webapp -Dmaven.test.skip
    

After the build system finishes, the directory target/scarab contains the Scarab webapplication. We provide additional informations about the Ant build system and the Maven build system

NOTE: Make sure that your TOMCAT_HOME environment variable is defined correctly. If you are using the Tomcat that comes with Scarab, you can safely undefine this value and follow the directions below for running the sandbox.

NOTE: If you already have an existing Tomcat installation and prefer to run Scarab from there, first build Scarab and then copy the directory

into your own Tomcat installation. Alternatively, add a new <Context> entry to tomcat's server.xml config file, pointing at the Scarab webapp (usually <scarab install dir>/target/scarab).

NOTE: If you already have an existing webserver running on port 8080, and you are using Scarab's version of Tomcat, you will need to change the port number to another unused port by modifying the /tomcat/conf/server.xml file and changing the 'port' attribute on the <Connector> element.

NOTE: There may be problems building and running Scarab with Tomcat 3.x. We have not done testing with this version of Tomcat.

NOTE: You should use the copy of Xerces that is included with Scarab and make sure that no other copies of Xerces are in your JAVA_HOME, ANT_HOME or your CLASSPATH. Otherwise, you may get build errors.

Installing the database

During the build of Scarab a set of database creation files is generated. To install a fresh database from these files you need first to install one of the supported RDBM's on your system and put the path to the database binaries into your PATH environment variable.

Simply put, the idea is that the database creation scripts and the Java database driver (JDBC) need the ability to connect to the database. In order to do this, the code needs to be given a host machine, database name, database-username and password.

By default, the scripts assume a database called 'scarab' and no username/password to connect to the database on localhost. For example, if you have MySQL up and running with no username/password for localhost access, you can simply execute the following:

with ant:

    cd build
    ant create-db

with Maven:

    maven scarab:create-db

If you need to specify a host/username/password/databasename, you will need to edit your build.properties. Please modify at least the follwing set of properties:

scarab.database.type
scarab.database.jdbc.driver
scarab.database.url
scarab.database.username 
scarab.database.password

You find more informations about the settings in project.properties

NOTE: PostgreSQL users should follow a different procedure for creating the database. See PostgreSQL.txt.

NOTE: If you would like to only load the required database data and not the sample/default data, you can do so by passing the property

ant create-db -Dskip.seed.data=true

to your JVM upon startup of the build processs or by adding the same property to your build.properties:

NOTE: More detailed instructions for setting up the database on different database vendors is available on our website.

NOTE: The create scripts will attempt to first drop your scarab database and then recreate it. If you execute "ant create-db", all of your previous data in that database will be lost without warning!

NOTE (for MySQL only): If you get a 'Server configuration denies access to data source' or 'access denied' or 'Invalid authorization' or 'java.lang.NullPointerException: Connection object was null.' error from MySQL in the log files, please read the MySQL documentation on how to fix this error. We will not provide support for this since it is really a MySQL configuration issue.

NOTE: On some operating systems, there seems to be an unexpected interation between the JVM, DNS resolution and the MySQL driver where a JDBC url pointing to 'localhost' will resolve as 'localhost.localdomain' and will prevent the connection to MySQL from authenticating correctly because most people configure MySQL for 'localhost'. One way to get around this is to use IP addresses in both the MySQL ACL as well as in the JDBC url.

At least one person has reported that using '127.0.0.1' instead of 'localhost' resolved a 'Server configuration denies access to data source' connection issue.

In order to setup the right permissions in MySQL, you may wish to try executing this command on a Unix command line (it has been reported to work for one person):

        echo "GRANT ALL ON scarab.* to ''@localhost" | mysql mysql

Running the sandbox

The Tomcat bundled with Scarab is preconfigured to run scarab out of the /target/scarab directory.

cd tomcat/bin
startup.sh    <-- Unix
startup.bat   <-- Win32

Then, in your web browser, go to:

<http://localhost:8080/scarab>

NOTE: Make sure that your TOMCAT_HOME is defined correctly. If you are using the Tomcat that comes with Scarab, you can safely undefine this environment variable.

NOTE: Substitute 'localhost' for the DNS name that the server is running on for remote access.

NOTE: You can define your own URL by editing the WEB-INF/web.xml and defining a different servlet mapping.

Customizing Scarab

At times it may be useful to make minor modifications to Scarab without applying those changes directly to the Scarab sources (since this will cause CVS to report those differences, and even worse, the custom modifications might get unintentionally checked into the Scarab CVS repository by a Scarab developer).

The extensions/usermod directory structure allows this to be done fairly simply.

It contains three directories, lib, conf, and templates. Any files in the lib directory will be copied into the appropriate target directory _after_ any such files in the standard distribution, and will replace the standard distribution files. Any files in the template directory will be copied to the target template directory _ONLY_IF_ the property "scarab.copy.templates" is set, again after the files from the standard distribution.

NOTE: It is Your responsibility to ensure that any subsequent changes to modified Velocity macros are transferred to these replacements.

Questions/problems

If you have problems or questions, please join the Scarab user mailing list and post a detailed message describing your issue. :-)