stork logo
Stork Tutorial

Stork Tutorial

For a simple walkthrough of using Stork to manage package installation on a PlanetLab system, use the following instructions:

1. Install Stork and Storkutil
2. Create a package
3. Create an identity
4. Modify the Stork configuration
5. Modify your Trusted Packages File
6. Set packages for installation using Pacman, or install manually
7. Upload files to the Stork repository

To access a print-version of this tutorial, hit the following link: www.cs.arizona.edu/stork/cleantut.html.


1. Installing Stork and Storkutil

To use Stork on your slice, you must run the Stork initscript from PlanetLab. Log into PlanetLab, choose a slice, and select Add New Attribute. Select pl_initscript, and then select arizona_stork2.

After finishing installation, download the Stork Enduser tarball and unpack it to obtain the Stork enduser utilities. This tarball also includes a sample stork configuration file, named sample-stork.conf which will need to be modified in order to successfully use the stork utilities.


2. Creating a Package

Stork currently supports two types of packages: RPMs and tarballs. Stork will install RPM packages according to the package scripts. RPMs are generally suggested for complex installations, or where the package needs to be installed outside of /root. However, for simple packages, it can be more efficient to use a simple tarball package.

Creating a tarball package can be a great way to distribute your own applications. With Stork, you can easily distribute your software to every node that needs it. Stork also allows you to execute scripts before and after you install or uninstall, making tarballs even more powerful. Including scripts in your tarball is very simple. Just package '.preinstall', '.postinstall', '.preremove', and/or '.postremove' scripts along with your own files. Stork will automatically execute the script when the user decides to install or uninstall your package. The scripts will run from the same directory the tarball is unpacked (typically '/root/' or what ever is defined to be the home directory). Note that these scripts are completely optional.

For example, consider a simple directory called directory full of dummy files, and also four scripts, .preinstall, .postinstall, .preremove, .postremove. We can create a simple zipped tarball by running the command:
tar -czf mypackage.tar.gz directory ...
Where "mypackage" is the name of the tarball to be created, and the directory is the directory we wish to make into a package. Instead directories, files can also be used.

Upon installing this package with Stork, Stork will automatically run .preinstall, and will run .postinstall after installation. This allows the package creator to include scripts to confirm that installation is going to occur, and confirm that it worked. Likewise, when the package is uninstalled, the preremove and postremove scripts work in the same way.


3. Creating an Identity

To start using Stork's enduser utilities, you will first need to create an identity. You can do this using the 'genpair' function. This will create a private key and a public key, which will identify you and the software you wish to distribute. The privatekey is used to sign the files that Stork uses to install your software, so be sure to keep it safe. The public key is distributed to all your nodes (and all others that wish to use your software). Note that Stork will generate an error if you attempt to use it before creating a pair of keys and setting it as your default identity.

To create your keys, just run the command:
./storkutil.py genpair USERNAME
where USERNAME is any username you desire. The username exsits so that users can recognize each other without having to base interaction solely on public and private keys. After running the genpair command, Stork will ask you if you want this to be your default identity; if this is your first time using Stork, it is best to set this key as your default. If you've already created keys and would like to use those, you can use the 'storkutil.py setdefault USERNAME' command.

./storkutil.py genpair foo
Generated foo.privatekey...
Extracted foo.publickey...
Would you like to make 'foo' your default identity? (y/n) > y
'foo' is now your default identity

./storkutil.py genpair foo
Error: Key files for 'foo' already exist

or, if you have already created foo.privatekey and foo.publickey:

./storkutil.py setdefault foo
Previous Identity: someguy
'foo' is now your default identity

Be sure that the privatekey and publickey are in your current directory while using storkutil. The publickey can be distributed to all your nodes through the upload website. Also, note that your public key is embedded in your TPFile filename, such that the filename should be *publickey*Username.tpfile, without the *.


4. Configuration File

You will need to change the stork configuration file included in the Storkutil tarball to match the identity you created. Simply open up the file which should be named sample-stork.conf and change the 'username = ' line to match the username you chose for your keys. You will also need to modify the 'publickeyfile = ' line, to match the location of your publickey. By default, this should be "publickeyfile = /usr/local/stork/var/keys/.publickey", where is the name you generated your keys for. You will need to distribute this configuration file to all your nodes through the upload website.

Note! The stork configuration file is different from the storkutil configuration file. The setdefault command will appropriately change the storkutil configuration file, but you will still need to change and distribute the Stork configuration file manually.


5. Trusted Packages File

The Trusted Packages File (TPFILE) defines which files you trust to be autmoatically downloaded and install. This helps your nodes stay secure and autonomous, as files will be verified through a set of hash codes. Even if two files share the same name, your nodes will only recieve the one that matches the hash obtained when the Trusted Packages File was created (preventing your nodes from downloading maliciously modified packages in the repository). You are also allowed to trust other users, allowing your nodes to accept packages from those users exclusively. This will let your nodes download and install applications created by other users. Note that adding packages to your TPFILE will not download the packages; it just notifies Stork which packages you trust to be downloaded.

Adding a package to your tpfile:
To add a package to your tpfile, use the command
./storkutil.py addfile FILE [FILE]...
where FILE is the location of the package you would like to trust. If the tpfile does not exist, storkutil will create one for you. Storkutil will look at the file, create a hash, and write it to your tpfile. If you upload the package to the repository and run stork to install the package, your node will select your specific file and install it.

For example, we will add packages 'package1' and 'package2'.

./storkutil.py addfile package1.tar.gz package2.tar.gz
No tpfile found for foo. Generating new tpfile.
File 'foo.tpfile' successfully signed
File 'foo.tpfile' has been copied to 'foo.~~~.tpfile'
Unembeded trusted packages file 'foo.tpfile' removed.

This script will run whenever Stork fails to find a trusted package file for the default user. It will then continue to the regular package adding script:

Using trustedpackages file '~~~~.tpfile'
Successfully added file 'package1.tar.gz'.
Successfully added file 'package2.tar.gz'.
File '~~~.tpfile' successfully signed

Storkutil will automatically sign your trusted packages file using your key. Also note that your TPFile will be named as your public key followed by your username, with the .tpfile extension.

Viewing your tpfile:
Afterwards, we can look at the trusted packages file by running:

./storkutil.py view tpfile
Viewing: foo.~~.tpfile

TRUSTED PACKAGES FILE
ALLOW package1.tar.gz
ALLOW package2.tar.gz

We can confirm that the packages were added to our trusted packages file in this way.

Removing a package from the tpfile:
You can remove a file from your TPFile, just use the 'removefile' command.

./storkutil.py removefile package1.tar.gz

Adding and removing trusted users:
Before adding a user to your TPFile, you will need to obtain their publickey.
To trust users, you can use the adduser command:
storkutil adduser USER USER.publickey (allow|deny|any) AFFECTEDPACKAGES
Where USER is the name of the user you want to trust, followed by the public key that user has made available. AFFECTEDPACKAGES is a pattern that defines the packages you want to accept. If the tpfile does not exist, storkutil will create one for you.

Examples:
storkutil.py adduser USER1 USER1.publickey allow user-4-2.tar.gz
storkutil.py adduser USER2 USER2.publickey deny java\*
storkutil.py adduser PlanetLab PlanetLab.publickey allow \*
storkutil.py adduser USER3

The first example shows how to accept a single package from the user USER1. The second example will deny all files from USER2 that have the pattern java*. The third example shows how to accept all packages from the user PlanetLab (be sure to shell escape the * character instead of using * by itself). Note that this is not necessary; the PlanetLab node is trusted by default when a tpfile is created to allow default PlanetLab files to be installed, and the PlanetLab publickey is included in the enduser tarball. You can again use the view tpfile command to see the contents of your TPFILE. You can also remove users using the removeuser command.

./storkutil.py removeuser USER2

When you are done, you can upload the trusted packages file to your nodes through the upload website.


6b. Pacman

Stork's package manager (pacman) can help you manage your packages across all your nodes. Rather than installing or uninstalling packages on each of your nodes individually, you can assign your nodes to groups and instruct stork to install or remove packages on those groups. Pacman is therefore efficient to use when a package needs to be installed on a large number of nodes. Pacman also allows packages to be automatically updated when the package creator updates them in the repository; this allows Stork to work near autonomously when set up for use. Pacman uses two files; groups and packages. Using storkutil, you can create these two files and upload them. Stork will then distribute these files to your nodes and perform the tasks you assigned.

Pacman Groups
Your groups.pacman file allows you organize your nodes into separate groups. You are allowed to include a node into a group or exclude a node from a group. You can also include, exclude, or intersect groups with each other. To define our groups, we can use:

./storkutil pacgroups (include|exclude|intersect|remove) group NAME [NAME...]
where group is the name of the group to be affected, and NAME are the names of the nodes or groups to be included/excluded/removed.

We start by adding a node to a group. If a pacman group file for your username does not exist, then storkutil will create a new one.

./storkutil.py pacgroups include myGroup foo1.node foo2.node
Using groups file: ./foo.~~~.groups.pacman
File Not Found, Creating New File
File './foo.~~~.groups.pacman' successfully signed

./storkutil.py view groups
Viewing: foo.~~~.groups.pacman

PACMAN GROUP FILE
GROUP: myGroup
INCLUDE: foo1.node
INCLUDE: foo2.node

Storkutil will automatically sign the file using your default private key file, which you created when you generated your first key pairs. If you want to use a username that is not your default, you can use the --username option to work on a different file.
We can continue adding more data into our group definitions. For instance, if we want myGroup to include all the nodes in the group oldGroup, except for the node badNode which is a member of oldGroup, we can run the following commands:

./storkutil.py pacgroups include myGroup oldGroup
./storkutil.py pacgroups exclude myGroup badNode

This means that installing a package to myGroup will install it to every node included in oldGroup except for badNode.

We can remove groups or nodes from a group by simply using the remove command, as such:

./storkutil.py pacgroups remove myGroup

If you want to view the contents of the group file, you can use the view groups command in storkutil, but you will first need to add a node to a group to generate a group file.
When you are done, you will need to upload the groups.pacman file to the Stork repository.

Pacman Packages
The packages.pacman file allows you to manage which packages to install on which nodes. With the pacman packages file, you can tell a node or group (as defined by your groups.pacman file) to install or uninstall a package. To create and modify our packages file, we will use:

./storkutil.py pacpackages (node|group|all) NAME (install|remove|upgrade) PACKAGE

We can start by assigning a node to install a package. Be sure your filename follows the pattern .packages.pacman, where the name is the one your configured in your Stork configuration file. This will aid your nodes in finding the correct group file to use. If the file you specified does not exist, then storkutil will create a new one.

./storkutil.py pacpackages node planetlab.foo.edu install gnuchess
This will assign the planetlab.foo.edu node to install gnuchess.

We can also assign groups to install or remove packages:
./storkutil.py pacpackages group myGroup install lynx man gnuchess
This command will tell myGroup to install the lynx, man, and gnuchess packages in all the nodes in the group.

We can also instruct Pacman to install packages on multiple nodes without grouping them, using the all command:
./storkutil.py pacpackages all install gnuchess
Note that in this example, the group and node commands are replaced with all command, and also that there is no group or node name required. This command will instruct Stork to install gnuchess on every node to which the packages.pacman file is distributed.

Using the upgrade command, Stork will also instruct all nodes in a group to update a preexisting package:
./storkutil.py pacpackages group myGroup upgrade gnuchess
This will instruct all myGroup nodes to upgrade gnuchess when necessary.

Storkutil will automatically sign the file using your default private key file, which you created when you generated your first key pairs. If you want to work using a different set of keys and files, you can use the --username option. You can view the contents of your package file using the view packages command. When you are done, upload your Pacman config file to the Stork repository.


6a. Manual Install

To manually install a package on a node, simply run the command Stork followed by the package name, from root. For instance:
./stork.py gnuchess
will install gnuchess on the node on which it is run. To install gnuchess on three separate nodes, then, the user would need to log into each node separately and run Stork gnuchess on that node. This is an efficient way to install packages when the package only needs to be installed on a single or small number of nodes.


7. Uploading to the Repository

The Stork repository upload site is located at: https://stork-repository.cs.arizona.edu . From this site, you can upload packages, your public keys, your Stork configuration file, trusted packages files, and pacman files (both groups.pacman and packages.pacman files). All of these files will need to be uploaded to the repository in order for Stork to work correctly with your nodes.
Home Top
powered by planetlab University of Arizona, Computer Science logo