Large Messages

There is no inherent limit on the total size of a message sent using Mach IPC. MachNetIPC has a default configuration that is limited to sending datagrams no larger than 64K. Programs that transfer large amounts of out-of-line data may be inconvenienced by this. We provide an alternate configuration that supports arbitrarily large messages. We also note that very large message transfers may adversely affect the performance of a host node, due to having to buffer the message in the x-kernel task while it is being transferred (this is worse for the receiver than the sender).

The file mach3/api/netipc/graph.comp.large can be used in place of the standard graph.comp to support large messages. It relies on a newly developed protocol (RAT) that implements a reliable datagram protocol over TCP.

If you wish to use RAT for very large messages (more than 1M byte or so), you will need to recompile several x-kernel files in a special manner. We recommend that you recompile these files in your local build directory in order to avoid conflicts with other users. The relevant files are:

xkernel/pi/msg.c
xkernel/pi/msg_internal.h
xkernel/mach3/pxk/malloc.c
xkernel/mach3/pxk/stack.c

The file msg.c contains the message tool implementation, which must be compiled with the MSG_NON_RECflag to work with large messages. The standard Mach malloc doesn't return and reuse large storage blocks very well, so we recommend compiling our distributed version of malloc.c for large messages. You might also wish to compile and use the stack.c that we provide; it limits cthread stacks to 64K and uses less virtual memory.

To compile special versions of these files:

• Copy (or link) these files to your build directory. Edit your Makefile in your build directory to indicate that these files will be locally compiled, by adding to the PRIVSRC and PRIVOBJ variables:

  
  PRIVSRC = \
          msg.c    \
          malloc.c \
          stack.c  
  
  PRIVOBJ = \
          ./$(HOW)/msg.o    \
          ./$(HOW)/malloc.o \
          ./$(HOW)/stack.o

• Add -DMSG_NON_RECto the TMP_CPPFLAGSvariable in your Makefile.

• Run make depend and make.