Loading C Functions in Icon

Gregg M. Townsend
Department of Computer Science
The University of Arizona

www.cs.arizona.edu/icon/uguide/cfuncs.htm
Adapted from Icon Analyst 36
Last updated October 27, 2009

Dynamic loading allows Icon programs to use functions coded in C without modifying the Icon system itself. The C code is compiled and placed in a library, then loaded from the library when the Icon program runs. Here is a discussion of the use and construction of such functions.

Program library functions

The Icon program library includes an assortment of loadable Unix interfaces and special-purpose functions. Some are there for their general utility, some for illustration, and some to fill specialized needs. Here is a sampling:

bitcount(i) — count the bits set in an integer
chmod(s, i) — change the permissions of a file
fpoll(f, i) — poll a file for input, with timeout
getpid() — return the process identification number
kill(i1, i2) — send a signal to a process
lgconv(i) — convert a large integer to a string
tconnect(s, i) — connect a file to a TCP port
umask(i) — change the process permission mask

The full set of functions can be found in the library's cfuncs directory. Documentation and code are also available on line. These functions are available automatically to an Icon program that includes link cfunc. The bitcount() function is a good example for detailed examination.

Loading a function

The built-in Icon function loadfunc(libname, funcname) loads the C function funcname() from the library file libname and returns a procedure value. If the function cannot be loaded, the program is terminated.

If loadfunc(libname, "myfunc") produces p, then

p(arguments) calls myfunc() with a list of arguments
type(p) returns "procedure"
image(p) returns "function myfunc"
proc("myfunc") returns p
proc("myfunc", 0) fails

The following program loads the function bitcount() and assigns it to a global variable of the same name. Assigning it to a global variable makes it available to other procedures, although that's not needed here. The bitcount() function returns the number of bits that are set in the binary representation of an integer.

$define Library "/icon/bin/libcfunc.so"
global bitcount

procedure main()
   local i
   bitcount := loadfunc(Library, "bitcount")
   every i := 250 to 260 do
      write(i, " ", bitcount(i))
end

When this program is run, it lists the integers from 250 to 260 along with their bit counts:

Loading from a path

Embedding a file name such as /icon/bin/libcfunc.so in the program is undesirable. An alternative is for the program to find the library file using information from the program environment.

The Icon library procedure pathload(libname, funcname) searches the set of directories given by the FPATH environment variable to find libname and load funcname. As is usual in Icon path searching, the current directory is searched first. If the function cannot be loaded, the program is terminated.

The pathload() procedure is included by linking pathfind from the Icon program library. Using pathload(), the example program becomes:

$define Library "libcfunc.so"
link pathfind
global bitcount

procedure main()
   local i
   bitcount := pathload(Library, "bitcount")
   every i := 250 to 260 do
      write(i, " ", bitcount(i))
end

The default FPATH includes the current directory and the installed Icon program library directory. To find a library located elsewhere, FPATH must be set explicitly before the program is run.

Implicit function loading

It is possible to encapsulate the loading process so that the body of an Icon program is unaware that it is calling a C function. Consider this example:

$define Library "libcfunc.so"
link pathfind

procedure main()
   local i
   every i := 250 to 260 do
      write(i, " ", bitcount(i))
end

procedure bitcount(n)
   bitcount := pathload(Library, "bitcount")
   return bitcount(n)
end

First of all, notice that there is no longer a global declaration for bitcount, and that the main procedure no longer calls pathload(). As far as the main procedure is concerned, bitcount() is just another procedure to call, with no special requirements. This is a nice simplification.

The new bitcount() procedure is a bit tricky, though. To understand it, you must know that an Icon procedure declaration creates a global variable with an initial value of that procedure. A global variable is subject to assignment.

When main() calls bitcount() for the first time, the bitcount() procedure loads the bitcount() C function from the library. The result is assigned to the global variable bitcount, replacing the current procedure value. Consequently, all subsequent calls to bitcount() use the loaded function.

The first call to bitcount() remains incomplete after loading the function; the bits of n still must be counted. So, following loading, the procedure calls bitcount(n). Although this looks like a recursive call, it isn't — the call uses the current value of the global variable bitcount, and so it calls the loaded C function. The bits of n are counted and returned, completing the first call.

After the first time, calls to bitcount() go directly to the loaded code. The Icon procedure bitcount() is no longer accessible.

Implicit library loading

The Icon program library provides an implicit loading procedure for each of the C functions in the library. Small procedures like the bitcount() procedure shown above are included by linking cfunc. Using the library interface procedure, our example now can be simplified to this:

link cfunc

procedure main()
   local i
   every i := 250 to 260 do
      write(i, " ", bitcount(i))
end

The link cfunc declaration is the only hint that bitcount() is written in C.

Making connections

The bit counting example doesn't really illustrate the full potential of using C functions in an Icon program. Bit counting, after all, can be done in Icon. Here's something that can't.

The library function tconnect(host, port) establishes a TCP connection to a specified port number on an Internet host. TCP is a communication protocol used by telnet programs, news servers, web servers, and many other network services.

The following program makes a connection to the Icon web server and writes the contents of the Icon home page — in its original HTML markup language, of course.

link cfunc

procedure main()
   local f
   f := tconnect("www.cs.arizona.edu", 80)
   writes(f, "GET /icon/ HTTP/1.0\n\n")
   flush(f)
   seek(f, 1)
   while write(read(f))
end

The tconnect() call establishes the connection and returns a file that is open for both input and output. The internet host www.cs.arizona.edu is the web server for the University of Arizona's Department of Computer Science. (Port 80 is the standard web server port number.) The program then transmits a request for the /icon/ web page. The details of the request string are specified by the Hypertext Transfer Protocol, not discussed here.

The flush() call ensures that all the data is actually sent, and then the seek() call resets the file in preparation for a switch from output to input. In this situation seek() does not actually reposition the file, but it's required when switching modes.

Finally, lines are read and echoed until an end-of-file is received.

Writing loadable C functions

Now consider the construction of library functions. Because the Icon system expects C functions to implement a certain interface, dynamic loading usually requires specially written C functions. In general, it is not possible to use an existing C function without writing an intermediate "glue" function.

C functions must deal with the data types used by the Icon run-time system, notably the "descriptors" that represent all Icon values. While an understanding of the Icon run-time system is helpful, it is possible to create useful functions by modeling them after existing library functions. Integer and string values are most easily handled.

A loadable C function has the prototype

int funcname(int argc, descriptor *argv)

where argc is the number of arguments and argv is an array of argument descriptors. The first element, argv[0], is used to return an Icon value, and is initialized to a descriptor for the null value. This element is not included in the count argc. The actual arguments begin with argv[1].

If the C function returns zero, the call from Icon succeeds. A negative value indicates failure. If a positive value is returned, it is interpreted as an error number and a fatal error with that number is signalled. In this case, if argv[0] is non-null, it is reported as the "offending value". There is no way for a C function to suspend, and no way to indicate a null value as an offending value in the case of an error.

Interface macros

The C file icall.h contains a set of macros for use in writing loadable functions. Documentation is included as comments. This file can be found in the cfuncs directory in the source code of the Icon program library. Alternatively, it can be loaded from the web. Macros are provided for:

inspecting the type of an Icon value
validating the type of an argument
converting an Icon value into a C value
returning a C value in Icon form
failing or signaling an error

Most macros deal with integers or strings. Some support also is provided for handling real and file values. The macros expect the C arguments to be declared with the names of argc and argv.

Counting bits, again

For a concrete example of a C function, consider the source code of the bitcount() function used earlier:

#include "icall.h"

int bitcount(int argc, descriptor *argv)
{
   unsigned long v;
   int n;
   ArgInteger(1);
   v = IntegerVal(argv[1]);
   n = 0;
   while (v != 0) {
      n += v & 1;
      v >>= 1;
      }
   RetInteger(n);
}

Like all loadable functions, bitcount() is an integer function with two parameters, argc and argv.

The ArgInteger macro call verifies that argument 1 is a simple integer. (Large integers are typically rejected by C functions because of the extra work involved.) If argument 1 is missing or has the wrong type, ArgInteger makes the function return error code 101 (integer expected or out of range).

The IntegerVal macro call extracts the value of the first argument.

In each pass through the while loop, the low order bit of v is extracted (v & 1), added to n, and shifted off (v >>= 1). When no more nonzero bits are left, the loop exits. Note that v is declared unsigned to ensure that only zero bits are inserted by the shift operation.

The RetInteger macro call returns the value of n as an Icon integer.

External data values

Loadable functions can create and return data structures that are treated as an opaque external type by Icon. The use of external types is described separately.

Preparing a library

To be used in an Icon program, a C function must be built and installed in a library. Compilation comes first, usually involving a command such as

cc –c bitcount.c

to produce an object file bitcount.o. The –c option causes a relocatable object file to be produced instead of a stand-alone executable program. If icall.h is not in the current directory, an additional option may be needed to specify its location. Other options, such as optimization options, also could be specified.

A C function can be loaded only from a "shared library". Even if there is just one function, it must be placed in a library. Library names conventionally end with a .so suffix.

It seems that every system has a different way to create libraries, usually involving special flags to cc or ld. The shell script mklib.sh embodies our understanding of shared library creation. It takes one argument naming the library to be created and one or more additional arguments listing object file names. For example, the command

mklib.sh mylib.so bitcount.o

creates a file mylib.so containing the functions read from bitcount.o. Like icall.h, mklib.sh is available in the program library source code or from the web.

Summary

Icon programs can load and call functions written in C.
The C functions must be tailored to Icon's requirements.
Each function must be loaded before it can be called.
A simple Icon procedure can be used to hide the loading details.
pathload() searches FPATH to find a function library.
C functions can embed data in opaque external values.
Some useful functions are provided in the Icon program library.