Operating System Prerequisites
Connecting the SCI cards and cables
Verifying the Scali installation
Operating System Prerequisites
Before the Scali Software Package is installed every node in the cluster needs the proper OS (Linux in this case) installed and configured as explained before the Scali software is installed. The installation is done as root and you must have root rsh access to all other hosts/nodes, along with rlogin as root. You may need to edit /etc/pamd/rlogin and /etc/pam.d/login for Red Hat Linux distributions and remove the pam_securetty check or add hosts to pam authentication. A working 'at' daemon (atd) is also needed on all nodes. Check this by issuing /etc/rc.d/init.d/atd status. Also do the same for the cron daemon. Next identify the master on the cluster, and set it up as a NIS server so that on boot up the other nodes including master can bind to it. This is explained in more detail in Microway Application Note 16.
Connecting the SCI cards and cables
Figure 1 gives the connection diagram of an 8 node dual pentium 3 cluster. The master node is master and the other nodes are node2 through node8. In this case no switch is used and the SCI cards on each node (main and daughter) are connected directly.
Figure 1: SCI card cabling diagram.Note the values for X and Y. During the installation the X and Y values must be entered correctly for each node.
Note that after the installation there is a way to check for correct cabling and correct working of the SCI drivers. This will be discussed in more detail shortly. Since the configuration is 2 x 4 the X value can be either 1 or 2 and the Y value for each X value varies from 1 to 4. The master node has both X and Y value set at 1.
Now from the master run ./install from the CDROM. The installation is divided into 4 broad sections: Introduction, Configuration, Installation and Verification. Chose the expert installation. Also select package "Complete SSP with SCI" (option 1) when prompted for the package selection. Select 'n' for the choice for installing support for front LED panel as there is no switch being used. Then comes the most important part. This is specifying the node names and the X and Y positions. Always start from the master which has X and Y both 1 and proceed downwards till in this case X is 2 and Y is 4. Chose the default (enter) for the rest of the questions. Finally you will be asked to enter the license information. Typically there will be two feature lines that you will need to enter. One for the scali daemons and one for scali mpi. After this you will be prompted to enter the email and address of the owner. You should now see "Install complete!". Remove the installation CD and reboot the nodes. Make sure that the master boots up first so that the other nodes (master included) can bind the the NIS domain.
Verifying the Scali installation
After logging into the master, switchdesk to your favourite desktop (in this case GNOME) and run startx. From the /opt/scali/bin directory run scadesktop. You will see a screen like that shown in Figure 2 with multiple monitoring views. Before this can be used it has to configured. The configurations are stored in file $HOME/.scali/ScaDesk.conf. Ignore the warning message (press OK) and continue. The system defaults are loaded and these need to be changed. Several configuration options exist including remote logging in and administration, user names and passwords, special system and frontend names, and user types.
Figure 2: Scali Desktop Main Window. Note the peguins on each node indicate the node is operational and reachable with LINUX running on it. Also note the menus "File Run Status Management Software Interconnect" fro, which most of the administrative, system and program commands can be issued.
Now check the status of the links. There are two ways to do this. The first way is to use a term window. To do this cd to /opt/scali/sbin and issue scaconftool. At the prompt enter list nodeid status, and if you see all nodes with Status OK, the system is working. Next issue status nodes. The connection with each node showing 000 must appear. Then cd to /opt/scali/examples/bin. Run ./sh_test. The memcpy test will run on all 16 processors in this case. The other way is to use Status in the menu shown in Figure 2 and select link status. As shown in Figure 3 you will see each link controller with the state (all of them should be UP not DOWN) and the Link controllers error counter. This should be 0 for both the main and daughter card for each node. Similarly under Status you could select
Figure 3: Link status for the cluster. Note all the nodes (main and daughter) are up, and the error codes for each are 0.
system load to view the CPU load status. Figure 4 below shows the result of selecting uptime and load from the Status menu, while Figure 5 shows the results of selecting user list.
Figure 4: Uptime and load information in the cluster.
Figure 5: User list indicating cluster use.
The environment also offers the utility that enables parallel shell commands to be run across all the nodes. Thus to check if the mpi daemon is running on all the nodes the command as shown in Figure 6 can be issued.
Figure 6: Example of a parallel shell being issued for all the nodes in the cluster.
Finally run an MPI test as shown in Figure 7. The program is pmandel and the input data is fed from the file cool.points. The dialog to submit MPI jobs can be pulled from the Run menu under MPI Program.
Figure 7: Running an MPI job pmandel on the cluster. Since we have a dual pentium III cluster the processes per node are 2. Note the display option is set to the IP address of the master similar to the export DISPLAY command.
Another thing to check is the status of the daemons on all the nodes in the cluster. Figure 8 shows this. Note that the master runs ypserv with ypbind while the other nodes run ypbind and bind to the NIS server (master). Note also that the mpid daemon must run on all the nodes. The Scali daemons run from /opt/scali/sbin. To check the Process List go the Status Menu and select Process List. It is a good idea to check that if something goes wrong, the master (front end) and the nodes are correctly running the daemons needed for NIS and Scali as shown. The Scali daemons can be restarted, if they have not started correctly. As a rule always restart the daemons on the master after all the daemons on the nodes are shutdown, and then start the daemons on the nodes. The cluster can be set to boot the daemons at startup. In this case the master (front end) should always be booted up first.
Figure 8: Process List on the Scali cluster for master and nodes.
BACK TO MICROWAY APPLICATION NOTE INDEX