Stork Tutorial
For a simple walkthrough of using Stork to manage package installation on a PlanetLab
system, use the following instructions:
1. Install Stork and Storkutil
2. Create a package
3. Create an identity
4. Modify the Stork configuration
5. Modify your Trusted Packages File
6. Set packages for installation using Pacman, or install manually
7. Upload files to the Stork repository
To access a print-version of this tutorial, hit the following link:
www.cs.arizona.edu/stork/cleantut.html.
1. Installing Stork and Storkutil
To use Stork on your slice, you must run the Stork initscript from PlanetLab. Log into
PlanetLab, choose a slice, and select Add New Attribute. Select
pl_initscript, and then select arizona_stork2.
After finishing installation, download the Stork Enduser tarball and
unpack it to obtain the Stork enduser utilities. This tarball also includes a sample stork
configuration file, named sample-stork.conf which will need to be modified in order
to successfully use the stork utilities.
2. Creating a Package
Stork currently supports two types of packages: RPMs and tarballs.
Stork will install RPM packages according to the package scripts. RPMs
are generally suggested for complex installations, or where the package needs
to be installed outside of /root. However, for simple packages, it can be more
efficient to use a simple tarball package.
Creating a tarball package can be a great way to distribute your own
applications. With Stork, you can easily distribute your software to every
node that needs it. Stork also allows you to execute scripts before and
after you install or uninstall, making tarballs even more powerful.
Including scripts in your tarball is very simple. Just package
'.preinstall', '.postinstall', '.preremove', and/or '.postremove' scripts
along with your own files. Stork will automatically execute the script when
the user decides to install or uninstall your package. The scripts will run
from the same directory the tarball is unpacked (typically '/root/' or what
ever is defined to be the home directory). Note that these scripts are
completely optional.
For example, consider a simple directory called directory full
of dummy files, and also four scripts, .preinstall,
.postinstall, .preremove, .postremove. We can create a
simple zipped tarball by running the command:
tar -czf mypackage.tar.gz directory ...
Where "mypackage" is the name of the tarball to be created, and the
directory is the directory we wish to make into a package. Instead
directories, files can also be used.
Upon installing this package with
Stork, Stork will automatically run .preinstall, and will run
.postinstall after installation. This allows the package creator to
include scripts to confirm that installation is going to occur, and confirm
that it worked. Likewise, when the package is uninstalled, the preremove and
postremove scripts work in the same way.
3. Creating an Identity
To start using Stork's enduser utilities, you will first need to
create an identity. You can do this using the 'genpair' function. This
will create a private key and a public key, which will identify you and the
software you wish to distribute. The privatekey is used to sign the files
that Stork uses to install your software, so be sure to keep it safe. The
public key is distributed to all your nodes (and all others that wish to use
your software). Note that Stork will generate an error if you attempt to use
it before creating a pair of keys and setting it as your default identity.
To create your keys, just run the command:
./storkutil.py genpair USERNAME
where USERNAME is any username you desire. The username exsits
so that users can recognize each other without having to base interaction
solely on public and private keys. After running the genpair command,
Stork will ask you if you want this to be your default identity; if this is
your first time using Stork, it is best to set this key as your default. If
you've already created keys and would like to use those, you can use the
'storkutil.py setdefault USERNAME' command.
./storkutil.py genpair foo
Generated foo.privatekey...
Extracted foo.publickey...
Would you like to make 'foo' your default identity? (y/n) > y
'foo' is now your default identity
./storkutil.py genpair foo
Error: Key files for 'foo' already exist
|
or, if you have already created foo.privatekey and foo.publickey:
./storkutil.py setdefault foo
Previous Identity: someguy
'foo' is now your default identity
|
Be sure that the privatekey and publickey are in your current directory
while using storkutil. The publickey can be distributed to all your nodes
through the upload website.
Also, note that your public key is embedded in your TPFile filename, such that
the filename should be *publickey*Username.tpfile, without the *.
4. Configuration File
You will need to change the stork configuration file included in the
Storkutil tarball to match the
identity you created. Simply open up the file which should be named
sample-stork.conf and change the 'username = ' line
to match the username you chose for your keys. You will also need to modify the
'publickeyfile = ' line, to match the location of your publickey. By default, this should be
"publickeyfile = /usr/local/stork/var/keys/.publickey", where is the
name you generated your keys for.
You will need to distribute this
configuration file to all your
nodes through the upload website.
Note! The stork configuration file is different from the storkutil
configuration file. The setdefault command will appropriately change
the storkutil configuration file, but you will still need to change and distribute
the Stork configuration file manually.
5. Trusted Packages File
The Trusted Packages File (TPFILE) defines which files you trust to be autmoatically
downloaded and install. This helps your nodes stay secure and autonomous, as files will be
verified through a set of hash codes. Even if
two files share the same name, your nodes will only recieve the one that
matches the hash obtained when the Trusted Packages File was created
(preventing your nodes from downloading maliciously modified packages in the
repository). You are also allowed to trust other users, allowing your nodes
to accept packages from those users exclusively. This will let your nodes
download and install applications created by other users. Note that adding packages to
your TPFILE will not download the packages; it just notifies Stork which packages
you trust to be downloaded.
Adding a package to your tpfile:
To add a package to your tpfile, use the command
./storkutil.py
addfile FILE [FILE]...
where FILE is the location of the package you would
like to trust. If the tpfile does not exist, storkutil will create one for
you. Storkutil will look at the file, create a hash, and write it to your
tpfile. If you upload
the package to the repository and run
stork to install the package, your node will select your specific file and
install it.
For example, we will add packages 'package1' and 'package2'.
./storkutil.py addfile package1.tar.gz package2.tar.gz
No tpfile found for foo. Generating new tpfile.
File 'foo.tpfile' successfully signed
File 'foo.tpfile' has been copied to 'foo.~~~.tpfile'
Unembeded trusted packages file 'foo.tpfile' removed.
|
This script will run whenever Stork fails to find a trusted package
file for the default user. It will then continue to the regular package adding script:
Using trustedpackages file '~~~~.tpfile'
Successfully added file 'package1.tar.gz'.
Successfully added file 'package2.tar.gz'.
File '~~~.tpfile' successfully signed
|
Storkutil will automatically sign your trusted packages file using
your key. Also note that your TPFile will be named as your public key followed by
your username, with the .tpfile extension.
Viewing your tpfile:
Afterwards, we can look at the trusted packages file by running:
./storkutil.py view tpfile
Viewing: foo.~~.tpfile
TRUSTED PACKAGES FILE
ALLOW package1.tar.gz
ALLOW package2.tar.gz
|
We can confirm that the packages were added to our trusted packages
file in this way.
Removing a package from the tpfile:
You can remove a file from your TPFile, just use the 'removefile' command.
./storkutil.py removefile package1.tar.gz
Adding and removing trusted users:
Before adding a user to your TPFile, you will need to obtain their
publickey.
To trust users, you can use the adduser command:
storkutil adduser USER USER.publickey (allow|deny|any) AFFECTEDPACKAGES
Where USER is the name of the user you want to trust, followed by the
public key that user has made available. AFFECTEDPACKAGES is a pattern
that defines the packages you want to accept. If the tpfile does not exist,
storkutil will create one for you.
Examples:
storkutil.py adduser USER1 USER1.publickey allow user-4-2.tar.gz
storkutil.py adduser USER2 USER2.publickey deny java\*
storkutil.py adduser PlanetLab PlanetLab.publickey allow \*
storkutil.py adduser USER3
The first example shows how to accept a single package from the user
USER1. The second example will deny all files from USER2 that have the
pattern java*. The third example shows how to accept all packages from the
user PlanetLab (be sure to shell escape the * character instead of using * by itself).
Note that this is not necessary; the PlanetLab node is trusted by default when
a tpfile is created to allow default PlanetLab files to be installed,
and the PlanetLab publickey is included in the enduser tarball.
You can again use the view tpfile
command to see the contents of your TPFILE. You can also remove users using
the removeuser command.
./storkutil.py removeuser USER2
When you are done, you can upload the trusted packages file to your nodes
through the upload
website.
6b. Pacman
Stork's package manager (pacman) can help you manage your packages
across all your nodes. Rather than installing or uninstalling packages on
each of your nodes individually, you can assign your nodes to groups and
instruct stork to install or remove packages on those groups. Pacman
is therefore efficient to use when a package needs to be installed on
a large number of nodes. Pacman also allows packages to be automatically
updated when the package creator updates them in the repository; this
allows Stork to work near autonomously when set up for use.
Pacman uses two files; groups and packages. Using storkutil, you can create these two
files and upload them. Stork will then distribute these files to your nodes
and perform the tasks you assigned.
Pacman Groups
Your groups.pacman file allows you organize your nodes into separate
groups. You are allowed to include a node into a group or exclude a node
from a group. You can also include, exclude, or intersect groups with each
other. To define our groups, we can use:
./storkutil pacgroups (include|exclude|intersect|remove) group NAME [NAME...]
where group is the name of the group to be affected, and
NAME are the names of the nodes or groups to be included/excluded/removed.
We start by adding a node to a group. If a pacman group file for your
username does not exist, then storkutil will create a new one.
./storkutil.py pacgroups include myGroup foo1.node foo2.node
Using groups file: ./foo.~~~.groups.pacman
File Not Found, Creating New File
File './foo.~~~.groups.pacman' successfully signed
./storkutil.py view groups
Viewing: foo.~~~.groups.pacman
PACMAN GROUP FILE
GROUP: myGroup
INCLUDE: foo1.node
INCLUDE: foo2.node
|
Storkutil will automatically sign the file using your default private key
file, which you created when you generated your first key pairs. If
you want to use a username that is not your default, you can use the --username
option to work on a different file.
We can continue adding more data
into our group definitions. For instance, if we want myGroup to include all the
nodes in the group oldGroup, except for the node badNode which is a member of oldGroup,
we can run the following commands:
./storkutil.py pacgroups include myGroup oldGroup
./storkutil.py pacgroups exclude myGroup badNode
This means that installing a package to myGroup will install it to every node
included in oldGroup except for badNode.
We can remove groups or nodes from a group by simply using the remove
command, as such:
./storkutil.py pacgroups remove myGroup
If you want to
view the contents of the group file, you can use the view groups command in
storkutil, but you will first need to add a node to a group to generate a group file.
When you are done, you will need to upload the groups.pacman file to the Stork repository.
Pacman Packages
The packages.pacman file allows you to manage which packages to
install on which nodes. With the pacman packages file, you can tell a node
or group (as defined by your groups.pacman file) to install or
uninstall a package. To create and modify our packages file, we will use:
./storkutil.py pacpackages (node|group|all) NAME (install|remove|upgrade) PACKAGE
|
We can start by assigning a node to install a package. Be sure your
filename follows the pattern .packages.pacman, where the name is the
one your configured in your Stork configuration file. This will aid
your nodes in finding the correct group file to use. If the file you
specified does not exist, then storkutil will create a new one.
./storkutil.py pacpackages node planetlab.foo.edu install gnuchess
This will assign the planetlab.foo.edu node to install gnuchess.
We can also assign groups to install or remove packages:
./storkutil.py pacpackages group myGroup install lynx man gnuchess
This command will tell myGroup to install the lynx, man, and gnuchess packages
in all the nodes in the group.
We can also instruct Pacman to install packages on multiple nodes without grouping them,
using the all command:
./storkutil.py pacpackages all install gnuchess
Note that in this example, the group and node commands are replaced
with all command, and also that there is no group or node name required. This command
will instruct Stork to install gnuchess on every node to which the packages.pacman file is distributed.
Using the upgrade command, Stork will also instruct all nodes in a group
to update a preexisting package:
./storkutil.py pacpackages group myGroup upgrade gnuchess
This will instruct all myGroup nodes to upgrade gnuchess when necessary.
Storkutil will automatically sign the file using your default private key
file, which you created when you generated your first key pairs.
If you want to work using a different set of keys and files, you can use the
--username option.
You can view the contents of your package file using the view packages
command. When you are done,
upload your Pacman config file to the Stork repository.
6a. Manual Install
To manually install a package on a node, simply run the command Stork
followed by the package name, from root. For instance:
./stork.py gnuchess
will install gnuchess on the node on which it is run. To install gnuchess
on three separate nodes, then, the user would need to log into each node
separately and run Stork gnuchess on that node. This is an efficient way
to install packages when the package only needs to be installed on a single or small
number of nodes.
7. Uploading to the Repository
The Stork repository upload site is located at:
https://stork-repository.cs.arizona.edu .
From this site, you can upload packages,
your public keys, your
Stork configuration file,
trusted packages files, and pacman files
(both groups.pacman and packages.pacman files). All of these files will need
to be uploaded to the repository in order for Stork to work correctly with your
nodes.
Home Top
|