Mirage

Description:

Mirage is a network router that virtualizes multiple NFS file servers as a single large NFS file server. Aggregating multiple file servers allows the NFS file system to scale better by dividing workload amongst several servers. Network administration is also eased by allowing new servers to be added without the need to reconfigure existing client computers. Furthermore, Mirage is able to schedule NFS traffic to prevent Denial of Service (DoS) attacks from less trusted clients. Mirage is fully transparent to both the client machines and the file servers, allowing existing technology to be seamlessly used. Mirage provides these additional services and imposes an overhead of only 7% beyond that of a simple IP forwarding router.

Copyright:

Please refer to the file "COPYRIGHT" for copyright information.

Acknowledgements:

Mirage uses List and Hash packages that are Copyright (C) 1985, 1988 Regents of the University of California.

Mirage uses the "RSA Data Security, Inc. MD5 Message-Digest Algorithm".

Implementation Notes:

The version of Mirage that is described within this document is "User Mode Mirage". It is implemented as a user mode process that runs under the linux operation system.

Unpacking the distribution:

Mirage is dirstributed as a gzipped tar file and should unpacked with gzip and tar. When the distribution is unpacked, it will form several subdirectories:

user/	user mode mirage
common/	common modules
config/	files used by the configure script
hash/	This is a hashing package developed by the Sprite project at U.C. Berkeley
list/	This is a doubly-linked list package developed by the Sprite project at U.C. Berkeley.
include/	common include files

Building Mirage:

Mirage includes a configure script and makefile that will automate the majority of the build process for you.

To run the configure script, change to the root directory of the distribution and type:

./configure

Once the configure script has been run, you can build the distribution itself. From the root directory of the distribution, type:

make

The root Makefile will recursively make library components in the common, list, and hash directories. It will then make the programs in the "user" directory. The programs that are built will be placed in the "bin" directory.

Setting Up Machines:

Mirage requires three computers to operate fully:

• File Server: The file server runs NFS to serve files to network users. We'll use "fileserver_hostname" to designate the machine name of the file server.
• Router: The router runs Mirage and sits between the file server and the client. We'll use "router_hostname" to designate the machine name of the router
• Client: The client mounts the the NFS file service through the router. We'll use "client_hostname" to designate the machine name of the client.

It is possible to run any ccombination of file server, router, and client on the same machine. However, it is easiest to describe them as if they are three different machines. Also, mirage supports the use of multiple file servers and multiple clients.

Whenever a file server exports a file system, it has to have a file system name. For linux machines, the file system names are usually stored in the /etc/exports file. You may wish to create a special test file system export for use with mirage. We will use "filesystem_name" to designate the name of the file system.

Running Mirage

Mirage is run on the router machine. You have specify at least one NFS file server. The file server should be set up and running before you run Mirage. In our examples, we will assume that the file server is named "fileserver" -- substitute your file server's name as appropriate.

cd <root of mirage distribution>
cd bin
./mirage --server fileserver_hostname

This will start up mirage with one file server and all debugging output enabled. The debugging output is quite verbose, but will help you get a feel for what is going on.

Mounting Mirage on a Client

To perform the mount, you'll need root access. Type the following commands:

mkdir /tmp/mnt
mount -t nfs router_hostname:/filesystem_name -o port=6685,mountport=6684 /tmp/mnt

Alternatively, you can set up a line in your /etc/fstab file like the following:

router_hostname:/filesystem_name /tmp/mnt defaults noauto,port=6685,mountport=6684 0 0

The port numbers "6684" and "6685" are defaults that are used by Mirage. Mirage purposely avoids using the standard mount and nfs port numbers for two reasons: 1) they would conflict with an existing NFS server if it was running on the machine and 2) the standard port numbers are secure port numbers and would require root access on the router to run user level mirage.

On the router, you should see a bunch of debugging info that looks like the following: (the IP addresses are from our test network; your IP address will likely be different). The debugging output below shows that mirage processed mount, getattr, and statfs RPC calls. The "orig fh" and "rewt fh" lines show the transformation between VFH and PFH (and vice versa).

handling mount request
from 127.0.0.1:976 uid=0 gid=0
rpc xid=150994944 proc=1
handling mount reply
from 198.0.0.116:32769
rpc xid=150994944 stat=0
orig mnt 0100000000080006384108000000000000000000000000000000000000000000
rewt mnt 000000000000270f0000270f00000000442218af00000000393fb3406c3a9f1e
sending reply to 127.0.0.1:976
handling nfs request
from 127.0.0.1:798 uid=0 gid=0 dosid=0
rpc xid=2630507682 proc=1
orig fh 000000000000270f0000270f00000000442218af00000000393fb3406c3a9f1e
rewt fh 0100000000080006384108000000000000000000000000000000000000000000
handling nfs reply
from 198.0.0.116:2049
rpc xid=167772160 stat=0
sending reply to 127.0.0.1:798
handling nfs request
from 127.0.0.1:798 uid=0 gid=0 dosid=0
rpc xid=2647284898 proc=17
orig fh 000000000000270f0000270f00000000442218af00000000393fb3406c3a9f1e
rewt fh 0100000000080006384108000000000000000000000000000000000000000000
handling nfs reply
from 198.0.0.116:2049
rpc xid=184549376 stat=0
sending reply to 127.0.0.1:798

Using Mirage:

Once you've mounted mirage as above, it works like any standard NFS mounted file system. Following our example, you could cd to /tmp/mnt and create files, list directories, etc. All of the traffic is being forwarded through Mirage.

For benchmark purposes, you should turn off the debugging output on the router. This can be done useing the --quiet option. For example,

./mirage --server fileserver_hostname --quiet

Adding Multiple file servers:

Mirage can support multiple file servers. When running Mirage, just use multiple --server command line options. For example:

./mirage --server fileserver1_hostname --server fileserver2_hostname

When you attempt to mount through Mirage, both file servers will be queried by the mount request and once the mount has completed, future NFS requests will be forwarded to the correct server.

Mirage command line options

Operating Mirage Securely

Some NFS servers require connections to be made through secure port numbers (i.e. port numbers less than 1024). This is an attempt to enhance NFS security. Mirage by default uses insecure port numbers and will cause an error when trying to mount one of these servers. For that reason, you can tell mirage to use secure port numbers:

./mirage --server fileserver_name --outmountport 602 --outnfsport 603

The above command uses the secure port numbers 602 and 603 (these were chosen essentially at random) when connecting to servers.

Mirage also lets you specify that incoming request must be on secure port numbers to offer the same security that NFS servers do. To do this, add the "--secure" option. Any client that attempts to mount Mirage via an insecure port will be dropped.

./mirage --server fileserver_name --outmountport 602 --outnfsport 603 --secure

Use of these options will require root access on the router.