Learn Stork Step-By-Step

Part I: Stork for Package Management

This part of the walkthrough will show you how to setup Stork and how to easily manage which packages are installed on your nodes. This is a good place to start if you are new to Stork. Once you're familiar with the steps outlined here in Part I, you can move on to Part II which will show you how to make your own packages available from the repository so that you can use Stork to install packages you've created.

Initial setup checklist:
1. Download and extract Stork enduser tarball.
2. Choose a username and create a keypair for that username.
3. Prepare your stork.conf file.
4. Upload your public key and stork.conf to the repository.
5. Install stork on your nodes.

Package management checklist:
1. Edit your pacman groups file.
2. Edit your pacman packages file.
3. Edit your trusted packages file.
4. Upload these three files to the repository.

Part II: Stork for Package Distribution

This part of the walkthrough will show you how to use Stork to distribute to your nodes packages that you've created. This part assumes you have completed Part I.

Package distribution checklist:
1. Create a package to be distributed.
2. Add the package to the trusted packages file (TPFile).
3. Add the package to the pacman packages file.
4. Upload the package, TPFile, and pacman packages file to repository.

Conventions used in this tutorial:

1. Commands that are to be run through a shell are displayed with light purple background. Parts to be typed are bold; non-bold text is output.

2. Information that you may see in a file, terminal window, or browser but that is not part of a command to be entered in a shell is displayed with an off-white background.

Part I: Stork for Package Management

Initial Setup

1. Download and extract Stork enduser tarball.

Note: Steps 1-4 apply if you are installing Stork on a non-PlanetLab system of your own. They should not be done on a PlanetLab node

Download the latest Stork enduser tarball linked from the Stork file list, extract it, and rename the extracted directory as desired.

The following shows an example of downloading the enduser utilities and putting them in a directory called stork-enduser:

2. Choose a username and create a keypair for that username

Choose a username. We'll use "example2" in this tutorial.

Next, change directory to where you placed your Stork enduser utilities and create a keypair using the username you've chosen:

This command has created 2 new files in the current directory:

As we chose to make this key our new default identity, the following file (in our home directory) was modified to make use of this key:

Note: ~/.storkutil.conf is using the full path on our filesystem to the key pair you just generated. If you decide to move these keys elsewhere on your filesystem, you will need to edit the full path to these files in your ~/storkutil.conf file so that the storkutil tool can find them in their new location.

3. Prepare your stork.conf file.

The stork enduser tarball comes with a sample stork.conf file, appropriately named sample-stork.conf. You will need to make a copy of the sample-stork.conf file and edit the username in two places.

Now edit the stork.conf file and modify the following lines:

you want the username you have chosen, along with its respective key, to be used. So, change the "default" to your username, which for this tutorial is "example2":

4. Upload your public key and stork.conf to the repository.

Note: To complete this next step, you must have a slice on PlanetLab allocated for your use. Contact your PlanetLab Principal Investigator (PI) to request a slice if you have not already done so. For more information, consult the PlanetLab FAQs.

To upload these files, first login to the Stork repository.

Next, click on the "Public Keys" link in the left navigation. Select the slice the public key is for using the drop-down menu that lists all of your slices. Then, click "Browse" and choose the public key file generated during step 2 above. For this tutorial, we would select the example2.publickey file. After selecting your public key, press the "Upload File" button. You will receive a confirmation that your public key was successfully uploaded.

Next, click on the "Configuration File" link in the left navigation. Repeat the process above, this time selecting the stork.conf file you created in step 3 above. You will once again receive a confirmation of successful upload.

5. Install stork on your nodes.

Now that the repository has the basic files needed for new Stork installs on your nodes to operate, you can proceed with installing Stork on them.

=> Install Method 1: PlanetLab initscript [Recommended]

The easiest way to install Stork is using the initscript through PlanetLab.

To run the initscript through PlanetLab, log into PlanetLab, go to the Slices page of your PlanetLab account, click on the name of the slice in which you want Stork installed when you add new nodes, and click on "Add A Slice Attribute". Next, select plc_initscript_id, set the value to 2, and click the "Add Attribute" button. You will now see on your slice details page the Slice Attribute of:

This slice attribute you have set will cause Stork to be installed when a new node is added to your slice.

=> Install Method 2: Manual initscript

To install Stork on existing nodes without reinitializing them, you can manually download and run the Stork initscript on each of your nodes. To do this, login to each node on which you want to install Stork and run the following:

Package Management

Note: To use PlanetLab, you must associate Nodes with your allocated slice. To add Nodes, log into to the PlanetLab website, click on Slices, and click on your Slice Name. In the "Choose Action" drop-down menu, select "Manage Nodes". Then select the institution whose Nodes you want to associate with your slice, and select any of the listed Nodes to associate them with your slice. For this Package Management Tutorial, you will be working with Princeton University PlanetLab Nodes 4 through 7. Select "Princeton" from the drop-down sites menu, and add the following Nodes to your slice: planetlab-4.cs.princeton.edu, planetlab-5.cs.princeton.edu, planetlab-6.cs.princeton.edu, planetlab-7.cs.princeton.edu."

1. Edit your pacman groups file.

Your pacman groups file will allow you to specify sets of nodes that can be referred to collectively for the purposes of installing, upgrading and removing packages. Specifying groups is not required but can make package management simple in some complex situations.

Note: if you do not have a pacman groups file already when you add a group, a new groups file will be created for you.

To create a group called "exampleGroup" made up of three nodes (planetlab-[5-7].cs.princeton.edu), we will go to the directory where we placed the Stork enduser utilities and run the following command:

Note: a backslash (\) is used in the command below to tell the shell that the command spans to the next line. This is not necessary when you are entering longer storkutil commands and is only done for display reasons in this tuturial.

Note: the pacman groups file is named in the format:

We can view the groups defined in our pacman groups file using the following command:

We see that the group we added and the nodes it consists of are as we just specified.

2. Edit your pacman packages file.

We will now specify which packages we want installed, updated and removed on our nodes. We can specify such actions by individual node, by group (according to the groups we created above), or even for all nodes in our slice.

Let's install vim-enhanced on all nodes in our group called "exampleGroup" and gnupg on one individual node that is not part of any group.

To configure our pacman packages file for this, we do the following:

Note: we could install a package to all nodes in our slice by using a command of the format ./storkutil.py pacpackages all install {package-name}

Note: the pacman packages file is named in the format:

We can now check the contents of our pacman packages file:

This is correct. Note the two additional UPDATE commands that we didn't define. These are part of every newly-created pacman packages file. They serve to keep Stork itself updated.

3. Edit your trusted packages file.

We'll now create a trusted packages file (a.k.a. TPFile). For the purposes of this tutorial, we just want to allow packages from user "default". This default user trusts Stork and PlanetLab, as well as a few other packages, meaning you should be able to install the newest Stork and PlanetLab releases without adding them to your TPFile.

Note: When a TPFile does not already exist, the 'default' user is added to your is newly-created TPFile's trusted list automatically. Here we add this user explicitly because we won't be performing any other action on the TPFile.

The command we run to add this default user is as follows (note that default.publickey is included in the Stork enduser tarball):

Note: the asterisk (*) is escaped so that the shell does not treat it as a glob.

Note: the trusted packages file is named in the format:

We can verify the contents of our TPFile with the following:

Note: because we explicitly added the user 'default' and storkutil adds this user automatically when creating a new TPFile, the user is listed twice. This is OK.

4. Upload these three files to the repository.

We log in to the Stork repository.

Next, we go to the link in the left navigation titled "Trusted Package Files" and upload the TPFile we created. When browsing for this file, we will be looking for a file named in the format {username}.{publickey}.tpfile.

We then go to the link in the left navigation titled "Pacman Files". We will first upload our pacman groups file (the file named in the format {username}.{publickey}.groups.pacman). After uploading the groups file, we will return to this same page and upload our pacman packages file (the file named in the format {username}.{publickey}.packages.pacman).

"Package Management" Completed

We have now completed the steps necessary to install the files we desired on the nodes we wanted them installed on. These files will normally begin installing on the appropriate nodes within ten minutes. The total time required for the installation to finish on a given node will depend on which packages are being installed and the load on that node.

In the case of our example, we check a little while later on our various nodes and see the following:

On all nodes in group "exampleGroup" we see:

And only on planetlab-4.cs.princeton.edu:

If we wanted to have these packages upgraded or removed on specific groups or nodes, or even on all nodes in our slice, we would use storkutil to modify our pacman packages file and add entries to "upgrade" and "remove" whichever packages we wanted affected.

If you want to use Stork to distribute your own packages, you can now move on to the tutorial titled "Using Stork to distribute your own packages".

Part II: Stork for Package Distribution

Package Distribution

1. Create a package to be distributed.

Stork currently supports two types of packages: RPMs and tarballs. For more information on using each, please see the Stork tutorial. In this tutorial we will create and distribute a tarball.

Our tarball package will be quite simple for this example. We've made a directory called example-package and inside of that directory is one file:

We also have a separate file called .postinstall that is not in the examplepackage-0.0.1-1 directory that we are going to put in the archive. Stork will run this .postinstall file after the package has been installed. For more information on other special files that can be used in tarball packages, see the Stork tutorial.

In our .postinstall file we have put the following:

So that it can be run, we make sure the .postinstall file is executable:

and then we create our tarball of the example-package directory:

We are now done creating our tarball package.

2. Add the package to the trusted packages file (TPFile).

We now add our tarball package to our TPFile using storkutil. Note that we need to provide the path to the actual tarball, not just it's name, as a hash specific to this tarball will be used to ensure only this specific file is trusted and not any other that may have the same filename.

We can then check the contents of our TPFile, if desired:

We see our package is now trusted.

3. Add the package to the pacman packages file.

Trusting a package as we have done above will not cause the package to be distributed. It will only allow a given file to be trusted if installation of the package has been requested. Therefore, we still need to get this package distributed to the nodes we want it installed on. To do this, we will add this package to our pacman packages file.

Note: we don't need to provide the full path to the file when specifying it to be installed/upgraded/removed in the pacman packages file.

Note: when specifying a tarball package rather than an RPM package, the entire filename should be used. The example below is for our tarball examplepackage.

For an RPM, you would simply enter: ./storkutil.py pacpackages group exampleGroup install examplepackage

We can verify this has been added to our packages file, if desired:

4. Upload the package, TPFile, and pacman packages file to repository.

We are now ready to upload our files to the repository.

First, we login to the repository at Stork repository. Next we click the link titled "Packages" in the left navigation. We click the "Browse" link, select our examplepackage-0.0.1-1.tar.gz file on our local system, then click "Upload File". We will receive a confirmation when the file has been successfully uploaded.

Note: If the package being uploaded is large, this step may take some time depending on your connection speed and the load on the repository.

Next, we upload our new TPFile and pacman packages file by going to the "Trusted Packages Files" and "Pacman Files" links, respectively.

Note: You do not need to re-upload your pacman groups file unless you have changed any of your group definitions.

After uploading these three files---the tarball package, TPFile, and pacman packages file---we are finished. Our package should begin installation on our desired nodes within ten minutes. The total time required for the installation to finish on a given node will depend on which the package being installed and the load on that node. As our example package is pretty small, this shouldn't take too long.

When we check the /root/ directory a little while later, we see the following:

So we can tell the package was installed. If we look inside that directory:

we see there is a file called new-file. That was the file that we had our .postinstall script create, so that means the .postinstall script did run as expected.

If we wanted to upgrade our nodes to a new version of our package, we would increment the version and/or release numbers, add the new file to our TPFile, set our pacman packages file to upgrade this package rather than just install it, and update our new package, TPFile, and pacman packages file to the repository. We would likely also want more realistic .preinstall and .postremove scripts. Remember that RPMs can be used also. For more complex packages, that can be a better choice than tarballs.

Congratulations, you're almost a Stork expert!

Should you still have unanswered questions after looking through our website, or if you are having problems using this tutorial or suspect it has an error, please let us know! An excellent way to contact us is through the Stork forum