tBench - A benchmark framework for temporal XML and PSM

Release v0.2 (March 16, 2013)


===============================================================================
CONTACT
===============================================================================

For questions and comments:

- Stephen W. Thomas (Chief Programmer)
  Affiliation: Queen's University
  Email: sthomas@cs.queensu.ca

- Richard T. Snodgrass 
  Affiliation: University of Arizona
  Email: rts@cs.arizona.edu

- Rui Zhang 
  Affiliation: University of Arizona
  Email: ruizhang@email.arizona.edu


===============================================================================
INTRODUCTION
===============================================================================

This package contains the tBench benchmark framework, and supporting tools. For a
complete description of tBench, see the TimeCenter technical report
and our SP&E 2013 paper in the docs directory.

Briefly, the tBench benchmark framework (currently) contains 10 benchmarks. The
foundation benchmark is "DC-SD", borrowed from the XBench family of
benchmarks. The remaining benchmarks are temporal and PSM exensions to "DC-SD".
All of the components (i.e., data, schema, and workloads) of the remaining 9
benchmarks are derived from the components of "DC-SD", by applying one or more
(manual or automatic) transformations. These transformations are contained (if
automatic) or described (if manual) in the Makefile of each benchmark; see
below for further details. 


===============================================================================
DIRECTORY STRUCTURE
===============================================================================

- benchmarks 
  This directory holds the benchmarks: (possible symlinks) to data, schemas,
  and workloads. It will also hold a Makefile (described below) for creating
  the benchmark, and a benchmark.xml file that describes the metadata of the
  benchmark.

- tools 
  This directory holds the supporting tool suite to help create and
  validate the individual benchmarks (described further below)

- docs 
  Contains published documentation about tBench and benchmark frameworks.


===============================================================================
USAGE
===============================================================================

To iteratively build all the benchmarks, run the command:

./makeAll.sh

This command will descend into each benchmark, in the proper order, and call its 
Makefile. 

Each benchmark (located in the benchmarks/ directory) contains a Makefile that
will create the benchmark. In some cases, the Makefile can also be used to
perform validation checks and execute the benchmark. Each Makefile contains
the following eight targets:

- clean
  Removes all generated files

- data
  Creates/generates/transforms (as appropriate) the data

- schema 
  Creates/generates/transforms (as appropriate) the schema

- workload 
  Creates/generates/transforms (as appropriate) the workload

- vdata
  Validates the data (e.g., for consistency, against the schema)
  
- vschema
  Validates the schema (e.g., for consistency and wellformedness)

- vworkload
  Validates the workload (e.g., for wellformedness)

- execute
  Executes the workload against the data

If no target is specified, then all targets are performed. 

Note that some components can not yet be automatically created; these already
exist in the benchmark and are unaffected by the Makefile. The Makefile will
output a message if this is the case.



===============================================================================
SUPPORTING TOOLS
===============================================================================

Some additional tools are needed to run the tBench benchmarks (called within
the Makefiles):

- tXSchema tool suite v2.1 (included). 
  http://cgi.cs.arizona.edu/apps/tauXSchema/index.php

- tGenerator v0.2 (included). 

- xmllint (standard on most *nix distributions). 
  http://xmlsoft.org/

- XBench v1.0 (included).
  https://cs.uwaterloo.ca/~tozsu/ddbms/projects/xbench/Publications.html

- Nux v1.6 (included). 
  http://acs.lbl.gov/software/nux/


The paths to these tools are defined in top level Makefile.inc. If you need 
to change the path to one of the tools, change it in this file.



===============================================================================
Operating System Support
===============================================================================

Currently, tBench has been developed for and tested on Linux (specifically,
Ubuntu 12.04). Support for Windows remains as future work. The main
difficulties for Windows support are (1) tool support (i.e., xmllint, Nux),
and (2) symbolic links from children benchmarks to their parents' directories.
Both of these issues can be resolved by installing Cygwin on Windows, which is a
fully-featured *nix emulation environment. Although untested, tBench should
function properly using Cygwin.