Getting Lanai Active Messages Running

Requirements:
  • Sparc 10 or 20's.
  • LANai 2.x boards.
  • Relative addressed Myricom switches.
  • Solaris 2.X* (Can be made to run under SunOS, see below).

    • So you've pulled over the compressed tar ball. Now what? You'll need to do four things: install the drivers, make the library, make some routing tables, and then run the tests.

    Install the Drivers

    The drivers have been slightly modified to enable the am_sleep() call. This call blocks the calling thread until message arrival. If you don't want to use am_sleep(), you can use Myricom's drivers, but the default drivers do not allocate enough memory internally for large configurations. The am_enable() call will complain if the driver hasn't allocated enough memory. If you are seeing this message, you will need to modify the COPY_BLOCK_SIZE parameter in myrihome/include/myri_sizes.h and then recompile the lanai_dev.c library.

    If you have installed the Myricom's drivers already, you just need to copy these drivers over the old ones and reboot. if you haven't, use Myricom's install scripts, then copy these drivers over the old ones and reboot :).

    Go into the drivers/dlpi subdirectory and make. Copy the sun4m/myri into /kernel/drv. Do the same for drivers/mmap, and copy sun4m/mlanai into /kernel/drv. If you know what your doing, you can unload and load in the new drivers while the machine is running, otherwise just reboot.

    Make the Library

    This library works under Solaris. If you are running SunOS, get Solaris! You can't run the Sparc port of DOOM under SunOS! :) LAM will work under SunOS, except for the am_sleep() call. But you will need to do some minor tweaking.

    Go into the src sub-directory. Make sure MYRI_HOME is set to the correct place. Also make sure to set the MPL environment variable to LAM. (This to maintain compatibility with the Split-C releases). Use gmake!

    If you're running Solaris, you'll need to do a "gmake lcp" first on a SunOS machine, unless you set up your /bin/as as the old /bin/as. If you don't like this set-up, complain to myricom.

    On a Solaris machine, the gmake should put libLAM.a into the lib subdirectory.

    Install the Routing Files

    Because of current lack of integration with Glunix you have to generate routing files. Each host needs it's own config file which is read during am_enable(). The format of the config file is: 

    	LCP FILE NAME
	Number of nodes in the system
	Route to node 0		Name of node 0
	Route to node 1		Name of node 1
	...
	Route to node P		Name of node P

    For example, a sample routing file would look like: 

    /disks/barad-dur/now/AM/lam/LAM/src/lcp.dat 
8 
51 dawn0
54 dawn1
53 dawn2
56 dawn3
57E dawn4
57D dawn5
57C dawn6
57B dawn7
    A route represents a set of turns for relative addressed switches. The format is pretty simple, if not a little brain dead. 0 .. +7 represent turns 0-7. A-H represent turns 0 .. -7. i.e. B = -1.

    The hostname should be the first part of the name before the first '.'. If you have hosts outside of one DNS domain, you have some Myrinet cluster.

    While using config files works OK for small clusters, and is tolerable for 32 node clusters with the mapper, it is a pain. This will be automated in the next release.

    Running the tests

    I strongly recommend that you start with two nodes first, then work up to four nodes. A few simple tests are included in the tests sub-directory. ping_pong.c is a very simple pinger. if that works, try running test_xfer.c. This is a more stressful test, and is the acid test I use on our switches. ping_sleep.c tests the the am_sleep() call. The other tests are pretty self-explanatory,

    Got this far?

    Wow. If anyone outside of Berkeley got this far, that's pretty impressive! :) If you have a big cluster and hate generating p^2 routes, then it's time to use the mapper .