AZDBLab is effectively an electronic laboratory notebook. Within this
environment, one can design and specify experiments, schedule these
experiments, monitor them when they are running, record both independent
variables (those controlled by the experimenter) and dependent variables
(those resulting from the experiment), and do
analyses on the data.
AZDBLab runs on a hardware lab of dedicated machines, one each for
each subject DBMS and one to host the DBMS that stores the lab
notebooks. Having dedicated hardware allows us to worry less about
other processes running on the machines that could dirty the results
(these machines are only for experiments), and allows us to run
extensive experiments taking days or weeks. We continue to use this
software and hardware lab to perform ergalic experiments on DBMSes.
Within the AZDBLab GUI, there are two panes, the one on the left
providing a hierarchical view of all the information and the one on the
right providing more detail about the specific item selected on the
left. Proceeding down the left hand side, at the first level, there are
multiple Users. Under each User, several
Notebooks have been created. For a Notebook, the user
can load Experiments. Each Experiment node contains
the source of the Tests specified by the Experiment.
Each Test defines the Data Definition and the
Query Generation Definition.
After an Experiment is loaded, it can be scheduled for
execution on an executor process
running on another machine. AZDBLab is configured so that multiple users
can view the experimental results concurrently, while multiple executors are each interacting with a
dedicated DBMS. Once an
Experiment is scheduled to run, it will be added to the
\textsf{Pending Run} node for the Executor to execute. All the
executors, once started, are shown
under the Executor node. After an executor discovers a Pending Run,
the discovered run
will be upgraded to a Running Run node, indicating the
experiment is currently running. During the execution of each
experiment, the user can pause or abort the experiment. The paused
running experiment can be later resumed.
At the end, each Experiment has all the Completed
Runs listed, each identified by their execution start time, and
each containing has all the experimental results, including the
visualized plan tree and textual details such as the cardinality of the
varying table for each plan and the optimal plan. The details of this
query, or of the enclosing test, run, or of the enclosing, experiment
can be viewed by clicking the desired node on the left-hand side.
At both the User and Notebook levels,
Aspects and Analytics can be defined. An
Aspect is a property of a query, such as whether the query is
suboptimal. An Analytic is statistical information computed on
one or more aspects, such as the number of suboptimal plans. The
definitions of the Aspects and Analytics are provided
by users via GUIs.
AZDBLab is extensible via plug-ins, each provided as a .jar file placed in a designated direction.
(These were inspired by the Eclipse plug-in architecture.) AZDBLab
automatically recognizes the plug-ins and loads them. The current system
supports five types of plug-ins: the Experiment Subject plug-in
(four instances), the Lab Notebook plug-in (two instances, in
order to achieve fail-over if our central repository DBMS or the
underlying server machine stops), the Aspect plug-in (one
instance), the Analytic plug-in (one instance), and the
Executor plug-in (a very simple plug-in that just provides
connection information to an executor
coupled to a particular DBMS on a particular machine).
