Linux High performance Computing Implementation Guide. By Syed Asim Abbas with help MR Kamarn Azeem CBTs.

For comments and suggestions : This e-mail address is being protected from spambots. You need JavaScript enabled to view it

HPCC Installation :

--------------------

I have 4 node for HPCC. 2 nodes are HP ML 360 servers with 2 x dual core processor. One is 1.7 mhz processor and other is AMD Sempron(tm) Processor 2600+

1) Centos 5.2 Installation on all nodes

2) Stop unnecessary services

3) /etc/hosts

------------

vi /etc/hosts (hpcc2)

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

192.168.1.180 hpcc1.isb.ots.com hpcc1

192.168.1.49 hpcc2.isb.ots.com hpcc2

192.168.1.69 test.isb.com test

192.168.1.40 nworkhorse.isb.ots.com nworkhorse

vi /etc/hosts (test)

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

192.168.1.180 hpcc1.isb.ots.com hpcc1

192.168.1.49 hpcc2.isb.ots.com hpcc2

192.168.1.69 test.isb.com test

192.168.1.40 nworkhorse.isb.ots.com nworkhorse

vi /etc/hosts (nworkhorse)

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

192.168.1.180 hpcc1.isb.ots.com hpcc1

192.168.1.49 hpcc2.isb.ots.com hpcc2

192.168.1.69 test.isb.com test

192.168.1.40 nworkhorse.isb.ots.com nworkhorse

root@hpcc1 ~]# ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/root/.ssh/id_dsa):

/root/.ssh/id_dsa already exists.

Overwrite (y/n)?

[root@hpcc1 ~]# ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/root/.ssh/id_dsa):

/root/.ssh/id_dsa already exists.

Overwrite (y/n)? y

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_dsa.

Your public key has been saved in /root/.ssh/id_dsa.pub.

The key fingerprint is: [root@hpcc2 ~]# ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/root/.ssh/id_dsa):

/root/.ssh/id_dsa already exists.

Overwrite (y/n)? y

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_dsa.

Your public key has been saved in /root/.ssh/id_dsa.pub.

The key fingerprint is:

3a:c0:dc:ad:aa:52:07:f8:43:b2:cb:ee:81:c8:6a:8f This e-mail address is being protected from spambots. You need JavaScript enabled to view it

[root@hpcc2 ~]# scp ~/.ssh/id_dsa.pub root@hpcc1:/root/.ssh/authorized_keys

root@hpcc1's password:

id_dsa.pub 100% 612 0.6KB/s 00:00

[root@hpcc2 ~]#

[root@hpcc2 ~]#

[root@hpcc2 ~]#

[root@hpcc2 ~]#

[root@hpcc2 ~]# ssh hpcc1

Last login: Tue May 5 11:02:26 2009 from 192.168.1.19

[root@hpcc1 ~]# logout

7b:a6:db:3e:55:ce:da:6b:ce:48:28:d3:9a:29:84:82 This e-mail address is being protected from spambots. You need JavaScript enabled to view it

[root@hpcc1 ~]# scp ~/.ssh/id_dsa.pub root@hpcc2:/root/.ssh/authorized_keys

root@hpcc2's password:

id_dsa.pub 100% 612 0.6KB/s 00:00

[root@hpcc1 ~]# ssh hpcc2

Last login: Wed May 13 11:03:40 2009 from hpcc1.isb.ots.com

[root@hpcc2 ~]# logout

Connection to hpcc2 closed.

NTP: for all nodes :

-----

Node 1

[root@hpcc2 ~]# vi /etc/ntp.conf

[root@hpcc2 ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [FAILED]

Starting ntpd: [ OK ]

[root@hpcc2 ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 .LOCL. 10 l 5 64 1 0.000 0.000 0.001

[root@hpcc2 ~]#

Node 2

[root@hpcc2 ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@hpcc2 ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 .LOCL. 10 l 1 64 1 0.000 0.000 0.001

[root@hpcc2 ~]# vi /etc/ntp.conf

[root@hpcc2 ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@hpcc2 ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

10.0.0.1 LOCAL(0) 11 u - 64 1 0.155 -691264 0.001

127.127.1.0 .LOCL. 10 l - 64 0 0.000 0.000 0.001

[root@hpcc2 ~]#

Node 3

[root@test ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@test ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 .LOCL. 10 l 1 64 1 0.000 0.000 0.001

[root@test ~]# vi /etc/ntp.conf

[root@test ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@test ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

192.168.1.180 LOCAL(0) 11 u - 64 1 0.155 -691264 0.001

127.127.1.0 .LOCL. 10 l - 64 0 0.000 0.000 0.001

[root@test ~]#

[root@nworkhorse ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@nworkhorse ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

127.127.1.0 .LOCL. 10 l 1 64 1 0.000 0.000 0.001

[root@nworkhorse ~]# vi /etc/ntp.conf

[root@nworkhorse ~]# /etc/init.d/ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@nworkhorse ~]# ntpq -p -n

remote refid st t when poll reach delay offset jitter

==============================================================================

192.168.1.180 LOCAL(0) 11 u - 64 1 0.155 -691264 0.001

127.127.1.0 .LOCL. 10 l - 64 0 0.000 0.000 0.001

[root@nworkhorse ~]#

PDSH:

-----

[root@hpcc1 ~]# wget http://nchc.dl.sourceforge.net/sourceforge/pdsh/pdsh-2.18-1.src.rpm

--11:52:48-- http://nchc.dl.sourceforge.net/sourceforge/pdsh/pdsh-2.18-1.src.rpm

Resolving nchc.dl.sourceforge.net... 211.79.60.17, 2001:e10:ffff:1f02::17

Connecting to nchc.dl.sourceforge.net|211.79.60.17|:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 462342 (452K) [application/x-wais-source]

Saving to: `pdsh-2.18-1.src.rpm'

100%[======================================================================================================>] 462,342 6.43K/s in 16s

11:53:05 (28.9 KB/s) - `pdsh-2.18-1.src.rpm' saved [462342/462342]

[root@hpcc1 ~]# scp pdsh-2.18-1.src.rpm hpcc2:/root

pdsh-2.18-1.src.rpm 100% 452KB 451.5KB/s 00:00

[root@hpcc1 ~]#

[root@hpcc1 ~]# rpmbuild --rebuild pdsh-2.18-1.src.rpm

Installing pdsh-2.18-1.src.rpm

warning: user grondo does not exist - using root

warning: group grondo does not exist - using root

warning: user grondo does not exist - using root

warning: group grondo does not exist - using root

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.4710

+ umask 022

+ cd /usr/src/redhat/BUILD

+ LANG=C

[root@hpcc2 ~]# rpmbuild --rebuild pdsh-2.18-1.src.rpm

Installing pdsh-2.18-1.src.rpm

warning: user grondo does not exist - using root

warning: group grondo does not exist - using root

warning: user grondo does not exist - using root

warning: group grondo does not exist - using root

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.4710

+ umask 022

+ cd /usr/src/redhat/BUILD

+ LANG=C

[root@hpcc1 i386]# rpm -ivh pdsh-*

Preparing... ########################################### [100%]

1:pdsh-rcmd-ssh ########################################### [ 14%]

2:pdsh ########################################### [ 29%]

3:pdsh-debuginfo ########################################### [ 43%]

4:pdsh-mod-dshgroup ########################################### [ 57%]

5:pdsh-mod-machines ########################################### [ 71%]

6:pdsh-mod-netgroup ########################################### [ 86%]

7:pdsh-rcmd-exec ########################################### [100%]

[root@hpcc2 i386]# rpm -ivh pdsh-*

Preparing... ########################################### [100%]

1:pdsh-rcmd-ssh ########################################### [ 14%]

2:pdsh ########################################### [ 29%]

3:pdsh-debuginfo ########################################### [ 43%]

4:pdsh-mod-dshgroup ########################################### [ 57%]

5:pdsh-mod-machines ########################################### [ 71%]

6:pdsh-mod-netgroup ########################################### [ 86%]

7:pdsh-rcmd-exec ########################################### [100%]

[root@test ~]# rpm -ivh /usr/src/redhat/RPMS/i386/pdsh-*

Preparing... ##### ( 14%)

########################################### [100%]

1:pdsh-rcmd-ssh ########################################### [ 14%]

2:pdsh ########################################### [ 29%]

3:pdsh-debuginfo ########################################### [ 43%]

4:pdsh-mod-dshgroup ########################################### [ 57%]

5:pdsh-mod-machines ########################################### [ 71%]

6:pdsh-mod-netgroup ########################################### [ 86%]

7:pdsh-rcmd-exec ########################################### [100%]

[root@ nworkhorse i386]# rpm -ivh pdsh-*

Preparing... ########################################### [100%]

1:pdsh-rcmd-ssh ########################################### [ 14%]

2:pdsh ########################################### [ 29%]

3:pdsh-debuginfo ########################################### [ 43%]

4:pdsh-mod-dshgroup ########################################### [ 57%]

5:pdsh-mod-machines ########################################### [ 71%]

6:pdsh-mod-netgroup ########################################### [ 86%]

7:pdsh-rcmd-exec ########################################### [100%]

NFS Setup

Shutting down NFS mountd: [FAILED]

Shutting down NFS daemon: [FAILED]

Shutting down NFS quotas: [FAILED]

Shutting down NFS services: [ OK ]

Starting NFS services: [ OK ]

Starting NFS quotas: [ OK ]

Starting NFS daemon: [ OK ]

Starting NFS mountd: [ OK ]

[root@hpcc1 .ssh]#

Node 1

[root@hpcc1 /]# groupadd -g 604 power

[root@hpcc1 /]# useradd -u 700 -g 604 -d /cluster/power hpower

Node 2

[root@hpcc2 ~]# groupadd -g 604 power

[root@hpcc2 ~]# useradd -u 700 -g 604 -d /cluster/power hpower

useradd: warning: the home directory already exists.

Not copying any file from skel directory into it.

[root@hpcc2 ~]#

Node 3

[root@test]# mount -t nfs hpcc1:/cluster/ /cluster/

[root@test ~]# mkdir /cluster

[root@test ~]# groupadd -g 604 power

[root@test ~]# useradd -u 700 -g 604 -d /cluster/power hpower

useradd: warning: the home directory already exists.

Not copying any file from skel directory into it.

Node 4

[root@nworkhorse ~]# mkdir /cluster

[root@nworkhorse ~]# mount -t nfs hpcc1:/cluster/ /cluster/

[root@nworkhorse ~]# groupadd -g 604 power

[root@nworkhorse ~]# useradd -u 700 -g 604 -d /cluster/power hpower

useradd: warning: the home directory already exists.

Not copying any file from skel directory into it.

Node 1

[hpower@hpcc1 ~]$ ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/cluster/power/.ssh/id_dsa):

Created directory '/cluster/power/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /cluster/power/.ssh/id_dsa.

Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.

The key fingerprint is:

79:d4:a0:4e:2e:e5:b1:05:5e:9c:93:56:06:0f:d4:1f This e-mail address is being protected from spambots. You need JavaScript enabled to view it

[hpower@hpcc1 ~]$

[hpower@hpcc1 ~]$

[hpower@hpcc1 ~]$ ls

[hpower@hpcc1 ~]$ cd .ssh/

[hpower@hpcc1 .ssh]$ ls

id_dsa id_dsa.pub

[hpower@hpcc1 .ssh]$ vi authorized

[hpower@hpcc1 .ssh]$ vi id_dsa.pub

[hpower@hpcc1 .ssh]$ cat id_dsa.pub > authorized_keys

Node 2

[hpower@hpcc2 ~]$ ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/cluster/power/.ssh/id_dsa):

/cluster/power/.ssh/id_dsa already exists.

Overwrite (y/n)? y

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /cluster/power/.ssh/id_dsa.

Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.

The key fingerprint is:

85:5a:0f:c2:d6:21:6e:55:38:24:5d:60:02:28:16:7b This e-mail address is being protected from spambots. You need JavaScript enabled to view it

[hpower@hpcc2 ~]$

[hpower@hpcc2 ~]$ cd .ssh/

[hpower@hpcc2 .ssh]$ cat id_dsa.pub >> authorized_keys

Node 3

[hpower@test ~]$ ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/cluster/power/.ssh/id_dsa):

/cluster/power/.ssh/id_dsa already exists.

Overwrite (y/n)? y

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /cluster/power/.ssh/id_dsa.

Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.

The key fingerprint is:

82:d9:8e:34:48:a7:f8:32:21:8d:e6:17:3d:2b:dd:21 This e-mail address is being protected from spambots. You need JavaScript enabled to view it

[hpower@test .ssh]$ cat id_dsa.pub >> authorized_keys

Node 4

[hpower@nworkhorse ~]$ ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/cluster/power/.ssh/id_dsa):

/cluster/power/.ssh/id_dsa already exists.

Overwrite (y/n)? y

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /cluster/power/.ssh/id_dsa.

Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.

The key fingerprint is:

c1:a6:01:ec:1e:a6:e9:8b:5e:89:2d:33:06:8b:24:cb This e-mail address is being protected from spambots. You need JavaScript enabled to view it

[hpower@nworkhorse ~]$ cat id_dsa.pub >> authorized_keys

cat: id_dsa.pub: No such file or directory

[hpower@nworkhorse ~]$ cd .ssh/

[hpower@nworkhorse .ssh]$ cat id_dsa.pub >> authorized_keys

[hpower@hpcc1 .ssh]$ chmod 644 authorized_keys

[hpower@hpcc1 .ssh]$ ssh hpcc2

The authenticity of host 'hpcc2 (192.168.1.49)' can't be established.

RSA key fingerprint is 99:6f:8d:b7:8b:89:ae:63:cf:e8:3f:3e:60:d1:bb:c3.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'hpcc2,192.168.1.49' (RSA) to the list of known hosts.

[hpower@hpcc2 ~]$ logout

Connection to hpcc2 closed.

[hpower@hpcc1 .ssh]$ ssh hpcc1

Last login: Sat Jun 6 14:25:48 2009 from hpcc2.isb.ots.com

[hpower@hpcc1 ~]$ logout

[hpower@hpcc2 .ssh]$ ssh hpcc1

The authenticity of host 'hpcc1 (192.168.1.180)' can't be established.

RSA key fingerprint is a2:04:92:48:b8:58:56:b7:24:9d:e5:da:30:75:92:b6.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'hpcc1,192.168.1.180' (RSA) to the list of known hosts.

[hpower@hpcc1 ~]$ logout

Connection to hpcc1 closed.

[hpower@hpcc2 .ssh]$ ssh hpcc2

Last login: Sat Jun 6 14:25:41 2009 from hpcc1.isb.ots.com

[hpower@hpcc2 ~]$ logout

[root@hpcc1 ~]# cp mpich2-1.0.8p1.tar.gz /cluster/

[root@hpcc1 ~]#

[root@hpcc1 /]# chown hpower.power /cluster/ -R

hpower@hpcc1 ~]$ cd ..

[hpower@hpcc1 cluster]$ ls

mpich2-1.0.8p1.tar.gz power

[hpower@hpcc1 cluster]$ tar zxvf mpich2-1.0.8p1.tar.gz

mpich2-1.0.8p1/

mpich2-1.0.8p1/mpich2.def

mpich2-1.0.8p1/test/

mpich2-1.0.8p1/test/.codingcheck

[hpower@hpcc1 mpich2-1.0.8p1]$ ./configure --prefix=/cluster/mpich2

config.status: executing default-3 commands

Configuration completed.

If done with out any error then

[hpower@hpcc1 mpich2-1.0.8p1]$ make

[hpower@hpcc1 mpich2-1.0.8p1]$ make install

[hpower@hpcc1 ~]$ vi .bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

. ~/.bashrc

fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin:/cluster/mpich2/bin

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/cluster/mpich2/lib

#PATH=$PATH:$HOME/bin

export PATH

export LD_LIBRARY_PATH

[mpiuser@node2 ~]$ source .bash_profile

[mpiuser@node1 ~]$ echo $PATH

/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/cluster/mpiuser/bin:/cluster/mpiuser/bin:/cluster/mpich2/bin

Create new file :

[hpower@hpcc1 ~]$ vi mpd.hosts

hpcc1

hpcc2

Create new file :

[hpower@hpcc1 ~]$ vi mpd.hosts

secretword=asim

[mpiuser@node1 ~]$ chmod 600 .mpd.conf

run

[hpower@hpcc1 ~]$ mpd &

[hpower@hpcc1 ~]$ mpd &

[1] 13499

[hpower@hpcc1 ~]$ mpdtrace

hpcc1

[hpower@hpcc1 ~]$ mpdtrace

hpcc1

[hpower@hpcc1 ~]$ mpdallexit

[hpower@hpcc1 ~]$ mpdboot -n 2

[hpower@hpcc1 ~]$ mpdtrace

hpcc1

hpcc2

[hpower@hpcc1 ~]$

Both nodes added Good

For Checkup

[hpower@hpcc1 ~]$ mpdboot -n 2 --chkuponly

checking hpcc2

there are 2 hosts up (counting local)

Now let's start some becnhmarking

[hpower@hpcc1 cluster]$ cd mpich2-1.0.8p1/examples/

cpi file already complied and executeable

-rw-r--r-- 1 hpower power 678 Nov 3 2007 child.c

-rwxr-xr-x 1 hpower power 576450 Jun 6 14:34 cpi

-rw-r--r-- 1 hpower power 1515 Nov 3 2007 cpi.c

-rw-r--r-- 1 hpower power 1964 Jun 6 14:34 cpi.o

-rw-r--r-- 1 hpower power 4469 Nov 3 2007 cpi.vcproj

drwxr-xr-x 2 hpower power 4096 Jun 6 14:31 cxx

drwxr-xr-x 2 hpower power 4096 Mar 27 01:40 developers

-rw-r--r-- 1 hpower power 10446 Nov 3 2007 examples.sln

drwxr-xr-x 2 hpower power 4096 Jun 6 14:31 f77

drwxr-xr-x 2 hpower power 4096 Jun 6 14:31 f90

-rw-r--r-- 1 hpower power 455 Nov 3 2007 hellow.c

-rw-r--r-- 1 hpower power 1892 Nov 3 2007 icpi.c

-rw-r--r-- 1 hpower power 6802 Jun 6 14:31 Makefile

-rw-r--r-- 1 hpower power 6767 Mar 27 01:40 Makefile.in

-rw-r--r-- 1 hpower power 1490 Mar 12 2008 Makefile.sm

drwxr-xr-x 2 hpower power 4096 Mar 27 01:39 mpiexec

-rw-r--r-- 1 hpower power 1049 Nov 3 2007 parent.c

-rw-r--r-- 1 hpower power 46399 Nov 3 2007 pmandel.c

-rw-r--r-- 1 hpower power 47798 Nov 3 2007 pmandel_fence.c

-rw-r--r-- 1 hpower power 4522 Nov 3 2007 pmandel_fence.vcproj

-rw-r--r-- 1 hpower power 45576 Nov 3 2007 pmandel_service.c

-rw-r--r-- 1 hpower power 4532 Nov 3 2007 pmandel_service.vcproj

-rw-r--r-- 1 hpower power 47510 Nov 3 2007 pmandel_spaserv.c

-rw-r--r-- 1 hpower power 4510 Nov 3 2007 pmandel_spaserv.vcproj

-rw-r--r-- 1 hpower power 49858 Nov 3 2007 pmandel_spawn.c

-rw-r--r-- 1 hpower power 4522 Nov 3 2007 pmandel_spawn.vcproj

-rw-r--r-- 1 hpower power 4492 Nov 3 2007 pmandel.vcproj

-rw-r--r-- 1 hpower power 2175 Nov 3 2007 README

-rw-r--r-- 1 hpower power 1116 Nov 3 2007 spawn_merge_child1.c

-rw-r--r-- 1 hpower power 694 Nov 3 2007 spawn_merge_child2.c

-rw-r--r-- 1 hpower power 1795 Nov 3 2007 spawn_merge_parent.c

-rw-r--r-- 1 hpower power 3554 Nov 3 2007 spawntest.vcproj

-rw-r--r-- 1 hpower power 1642 Nov 3 2007 srtest.c

drwxr-xr-x 7 hpower power 4096 Mar 27 01:40 windows

[hpower@hpcc1 ~]$ cd -

/cluster/mpich2-1.0.8p1/examples

[hpower@hpcc1 examples]$ mpiexec -n 1 ./cpi

Lamnodes Failed!

Check if you had booted lam before calling mpiexec else use -machinefile to pass host file to mpiexec

LAM Error but i m not using LAM ...........

After 20 mins fiding finally i found the reason mpiexec was running from /usr/bin/mpiexec

[hpower@hpcc1 ~]$ which mpiexec

/usr/bin/mpiexec

If run this using absolute path it is working. Means there is some problem with our .bash_profile

[hpower@hpcc1 ~]$ /cluster/mpich2/bin/mpiexec -n 1 /cluster/mpich2-1.0.8p1/examples/cpi

Process 0 of 1 is on hpcc1.isb.ots.com

pi is approximately 3.1415926544231341, Error is 0.0000000008333410

wall clock time = 0.000317

I found only one solution to solve

[root@hpcc1 ~]# mv /usr/bin/mpiexec /tmp/ :)

Its Working Now

[hpower@hpcc1 ~]$ which mpiexec

/cluster/mpich2/bin/mpiexec

[hpower@hpcc1 ~]$

[hpower@hpcc1 ~]$ mpiexec -n 1 /cluster/mpich2-1.0.8p1/examples/cpi

Process 0 of 1 is on hpcc1.isb.ots.com

pi is approximately 3.1415926544231341, Error is 0.0000000008333410

wall clock time = 0.000297

[hpower@hpcc1 ~]$ mpiexec -n 2 /cluster/mpich2-1.0.8p1/examples/cpi

Process 0 of 2 is on hpcc1.isb.ots.com

Process 1 of 2 is on hpcc2.isb.ots.com

pi is approximately 3.1415926544231318, Error is 0.0000000008333387

wall clock time = 0.001234

[hpower@hpcc1 ~]$

There is an other program available for testing lets compile it first

/cluster/mpich2-1.0.8p1/examples

[hpower@hpcc1 examples]$ mpicc -o icpi icpi.c

Now execute it and also open top -c to check the processes.

First we will run with single node

[hpower@hpcc1 examples]$ mpiexec -n 1 ./icpi

Enter the number of intervals: (0 quits) 1000000000

pi is approximately 3.1415926535921401, Error is 0.0000000000023470

wall clock time = 22.537397

Enter the number of intervals: (0 quits)

top -c out put :

[root@hpcc1 ~]# top -c

top - 15:53:08 up 1:57, 3 users, load average: 0.21, 0.11, 0.03

Tasks: 188 total, 2 running, 186 sleeping, 0 stopped, 0 zombie

Cpu(s): 12.5%us, 0.0%sy, 0.0%ni, 87.4%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 3368196k total, 936760k used, 2431436k free, 140504k buffers

Swap: 12289716k total, 0k used, 12289716k free, 634932k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

14289 hpower 25 0 2332 812 676 R 100 0.0 0:07.80./icpi

1 root 15 0 2032 640 552 S 0 0.0 0:01.82 init [5]

Lets run with both nodes

[hpower@hpcc1 examples]$ mpiexec -n 2 ./icpi

Enter the number of intervals: (0 quits) 1000000000

pi is approximately 3.1415926535905170, Error is 0.0000000000007239

wall clock time = 11.285166

Enter the number of intervals: (0 quits)

HPCC2 top -c output

top - 15:56:47 up 2:01, 5 users, load average: 0.27, 0.08, 0.06

Tasks: 205 total, 2 running, 203 sleeping, 0 stopped, 0 zombie

Cpu(s): 12.5%us, 0.0%sy, 0.0%ni, 87.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 3368120k total, 1858180k used, 1509940k free, 197464k buffers

Swap: 13309844k total, 0k used, 13309844k free, 1488212k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

4615 hpower 25 0 2356 812 676 R 100 0.0 0:10.26 ./icpi

4593 root 15 0 2320 1080 800 R 0 0.0 0:01.06 top -c

HPCC1 top -c Out put

[root@hpcc1 ~]# top -c

top - 15:56:46 up 2:01, 3 users, load average: 0.14, 0.10, 0.04

Tasks: 188 total, 2 running, 186 sleeping, 0 stopped, 0 zombie

Cpu(s): 12.5%us, 0.0%sy, 0.0%ni, 87.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 3368196k total, 941472k used, 2426724k free, 140656k buffers

Swap: 12289716k total, 0k used, 12289716k free, 634876k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

14312 hpower 25 0 2328 848 712 R 100 0.0 0:08.84 ./icpi

1 root 15 0 2032 640 552 S 0 0.0 0:01.82 init [5]

Now I am going to add nodes 3 on /etc/mpd.hosts

[hpower@hpcc1 ~]$ vi mpd.hosts

hpcc1

hpcc2

test

[hpower@hpcc1 ~]$ mpdallexit

[hpower@hpcc1 ~]$ mpdboot -n 3

[hpower@hpcc1 ~]$ mpdtrace

hpcc1

test

hpcc2

All 3 nodes addes successfully

Now lets run scripts again for benchmarking

[hpower@hpcc1 ~]$ cd /cluster/mpich2-1.0.8p1/examples/

[hpower@hpcc1 examples]$ mpiexec -n 2 ./icpi

Enter the number of intervals: (0 quits) 1000000000

pi is approximately 3.1415926535905170, Error is 0.0000000000007239

wall clock time = 43.697841

[hpower@hpcc1 examples]$ mpiexec -n 3 ./icpi

Enter the number of intervals: (0 quits) 1000000000

pi is approximately 3.1415926535899761, Error is 0.0000000000001830

wall clock time = 29.853977

Hmmmm better then two nodes add another 0 :) It will take some time .

Add node 4

there are not enough hosts on which to start all processes

[hpower@hpcc1 ~]$ vi mpd.hosts

[hpower@hpcc1 ~]$ mpdboot -n 4

[hpower@hpcc1 ~]$ mpdtrace

hpcc1

test

nworkhorse

hpcc2

[hpower@hpcc1 examples]$ mpiexec -n 4 ./icpi

Enter the number of intervals: (0 quits) 1000000000

pi is approximately 3.1415926535896128, Error is 0.0000000000001803

wall clock time = 21.904199

Lets Test GoTo Blast.

Download Location

http://www.tacc.utexas.edu/resources/software/login/gotoblas/GotoBLAS-1.26.tar.gz

[root@hpcc1 ~]# cp GotoBLAS-1.26.tar.gz /cluster/

[root@hpcc1 ~]# cd /cluster/

[root@hpcc1 cluster]# ls

GotoBLAS-1.26.tar.gz mpich2 mpich2-1.0.8p1 mpich2-1.0.8p1.tar.gz power

[root@hpcc1 cluster]# tar zxvf GotoBLAS-1.26.tar.gz

[root@hpcc1 cluster]# su - hpower

[hpower@hpcc1 cluster]$ cd GotoBLAS

[hpower@hpcc1 GotoBLAS]$ ./quickbuild.32bit

[hpower@hpcc1 GotoBLAS]$ make

It takes some time

Now download hpl

Download Link

http://www.netlib.org/benchmark/hpl/hpl-2.0.tar.gz

http://www.netlib.org/benchmark/hpl/hpl.tgz

root@hpcc1 cluster]# cd

[root@hpcc1 ~]# ls

@ Desktop hpl.tgz ipscan-3.0_beta4-1.i386.rpm mpich2-1.0.8p1.tar.gz snapshot1.jpg snapshot4.jpg

abc.sh GotoBLAS-1.26.tar.gz install.log login.php?product=server20 pdsh-2.10-4.src.rpm snapshot2.jpg tm

anaconda-ks.cfg hpl-2.0.tar.gz install.log.syslog mac.txt

[hpower@hpcc1 cluster]$ ls

GotoBLAS GotoBLAS-1.26.tar.gz hpl-2.0.tar.gz hpl.tgz mpich2 mpich2-1.0.8p1 mpich2-1.0.8p1.tar.gz power

[hpower@hpcc1 cluster]$ tar zxf hpl-2.0.tar.gz

[hpower@hpcc1 cluster]$ tar zxf hpl.tgz

[hpower@hpcc1 hpl]$ cp setup/Make.Linux_PII_FBLAS_gm .

First check GCC version/Path

[hpower@hpcc1 hpl]$ gcc -v

Using built-in specs.

Target: i386-redhat-linux

Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-libgcj-multifile --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic --host=i386-redhat-linux

Thread model: posix

/usr/lib/gcc/i386-redhat-linux/4.1.1/

Now edit Make.Linux_PII_FBLAS_gm file

[hpower@hpcc1 hpl]$ vi Make.Linux_PII_FBLAS_gm

Edit the following lines :

ARCH = Linux_PII_FBLAS_gm

TOPdir = $(HOME)/hpl

INCdir = $(TOPdir)/include

BINdir = $(TOPdir)/bin/$(ARCH)

LIBdir = $(TOPdir)/lib/$(ARCH)

#

HPLlib = $(LIBdir)/libhpl.a

LAdir = /cluster/GotoBLAS

LAlib = $(LAdir)/libgoto.a -1m -L/usr/lib/gcc/i386-redhat-linux/4.1.1/

CC = mpicc

CCFLAGS = $(HPL_DEFS) -O3

LINKER = mpicc

Ok Done now compile it .

[hpower@hpcc1 hpl]$ make arch=Linux_PII_FBLAS_gm

[hpower@test hpl]$ cd /cluster/power/hpl/bin/Linux_PII_FBLAS_gm/

[hpower@test Linux_PII_FBLAS_gm]$ ls

HPL.dat xhpl

[hpower@test Linux_PII_FBLAS_gm]$ cp HPL.dat HPL.dat.orig

MY HPL parameter First testing with 2 nodes

Innovative Computing Laboratory, University of Tennessee

HPL.out output file name (if any)

6 device out (6=stdout,7=stderr,file)

2 # of problems sizes (N)

7000 Ns

4 # of NBs

100 NBs

0 PMAP process mapping (0=Row-,1=Column-major)

3 # of process grids (P x Q)

1 Ps

2 Qs

16.0 threshold

3 # of panel fact

0 1 2 PFACTs (0=left, 1=Crout, 2=Right)

2 # of recursive stopping criterium

2 4 NBMINs (>= 1)

1 # of panels in recursion

2 NDIVs

3 # of recursive panel fact.

0 1 2 RFACTs (0=left, 1=Crout, 2=Right)

1 # of broadcast

0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)

1 # of lookahead depth

0 DEPTHs (>=0)

2 SWAP (0=bin-exch,1=long,2=mix)

64 swapping threshold

0 L1 in (0=transposed,1=no-transposed) form

0 U in (0=transposed,1=no-transposed) form

1 Equilibration (0=no,1=yes)

8 memory alignment in double (> 0)

~

~

[hpower@hpcc1 Linux_PII_FBLAS_gm]$ mpiexec -n 2 ./xhpl

The following parameter values will be used:

N : 6000

NB : 100

PMAP : Row-major process mapping

P : 1

Q : 2

PFACT : Left Crout Right

NBMIN : 2 4

NDIV : 2

RFACT : Left Crout Right

BCAST : 1ring

DEPTH : 0

SWAP : Mix (threshold = 64)

L1 : transposed form

U : transposed form

EQUIL : yes

ALIGN : 8 double precision words

top -c on node 1 (hpcc1) Procssor speed 3.0 dual core

Cpu(s): 6.2%us, 1.5%sy, 0.0%ni, 92.1%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 3368196k total, 1229388k used, 2138808k free, 215208k buffers

Swap: 12289716k total, 0k used, 12289716k free, 695944k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

16856 hpower 15 0 172m 141m 1016 S 62 4.3 0:55.41 ./xhpl

top -c on node 3 (test ) 1.7 MHZ PC class machine

Cpu(s): 78.1%us, 1.3%sy, 0.0%ni, 19.6%id, 1.0%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 443392k total, 437712k used, 5680k free, 21312k buffers

Swap: 1052248k total, 0k used, 1052248k free, 189308k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

14055 hpower 25 0 175m 144m 960 R 78.6 33.4 0:59.60 ./xhpl

- The matrix A is randomly generated for each test.

- The following scaled residual checks will be computed:

1) ||Ax-b||_oo / ( eps * ||A||_1 * N )

2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )

3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )

- The relative machine precision (eps) is taken to be 1.110223e-16

- Computational tests pass if scaled residuals are less than 16.0

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2L2 6000 100 1 2 46.28 3.113e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0034911 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0072807 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0014339 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2L4 6000 100 1 2 49.01 2.939e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0038909 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0081145 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0015981 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2C2 6000 100 1 2 45.78 3.146e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0034911 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0072807 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0014339 ...... PASSED

Again RUN now incrase the size of NS.

The following parameter values will be used:

N : 7000

NB : 100

PMAP : Row-major process mapping

P : 1

Q : 2

PFACT : Left Crout Right

NBMIN : 2 4

NDIV : 2

RFACT : Left Crout Right

BCAST : 1ring

DEPTH : 0

SWAP : Mix (threshold = 64)

L1 : transposed form

U : transposed form

EQUIL : yes

ALIGN : 8 double precision words

- Computational tests pass if scaled residuals are less than 16.0

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2L2 7000 100 1 2 72.10 3.173e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.1808081 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091439 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0016514 ...... PASSED

With 4 node

Cpu(s): 2.9%us, 0.7%sy, 0.0%ni, 96.3%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 3368196k total, 1120648k used, 2247548k free, 215220k buffers

Swap: 12289716k total, 0k used, 12289716k free, 695976k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

17007 hpower 15 0 68892 34m 1016 S 28 1.0 0:28.31 ./xhpl

Cpu(s): 62.2%us, 20.4%sy, 0.0%ni, 8.6%id, 8.6%wa, 0.0%hi, 0.3%si, 0.0%st

Mem: 443392k total, 175608k used, 267784k free, 2532k buffers

Swap: 1052248k total, 41448k used, 1010800k free, 86836k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

14144 hpower 15 0 70652 36m 960 R 63.1 8.5 0:10.83 ./xhpl

14066 root 15 0 2212 948 752 R 0.3 0.2 0:01.75 top -c

Cpu(s): 36.0%us, 10.3%sy, 0.0%ni, 50.0%id, 0.0%wa, 1.7%hi, 2.0%si, 0.0%st

Mem: 969896k total, 602732k used, 367164k free, 31900k buffers

Swap: 2096408k total, 0k used, 2096408k free, 374868k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

4803 hpower 16 0 70416 36m 960 R 46.5 3.9 0:30.35 ./xhpl

============================================================================

HPLinpack 1.0a -- High-Performance Linpack benchmark -- January 20, 2004

Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs., UTK

============================================================================

An explanation of the input/output parameters follows:

T/V : Wall time / encoded variant.

N : The order of the coefficient matrix A.

NB : The partitioning blocking factor.

P : The number of process rows.

Q : The number of process columns.

Time : Time in seconds to solve the linear system.

Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N : 4000

NB : 100

PMAP : Row-major process mapping

P : 2

Q : 2

PFACT : Left Crout Right

NBMIN : 2 4

NDIV : 2

RFACT : Left Crout Right

BCAST : 1ring

DEPTH : 0

SWAP : Mix (threshold = 64)

L1 : transposed form

U : transposed form

EQUIL : yes

ALIGN : 8 double precision words

----------------------------------------------------------------------------

- The matrix A is randomly generated for each test.

- The following scaled residual checks will be computed:

1) ||Ax-b||_oo / ( eps * ||A||_1 * N )

2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )

3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )

- The relative machine precision (eps) is taken to be 1.110223e-16

- Computational tests pass if scaled residuals are less than 16.0

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2L2 4000 100 2 2 46.16 9.248e-01

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2L4 4000 100 2 2 18.48 2.310e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2C2 4000 100 2 2 21.46 1.990e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2C4 4000 100 2 2 18.39 2.322e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2R2 4000 100 2 2 18.68 2.285e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00L2R4 4000 100 2 2 17.58 2.428e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098668 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091118 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020710 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00C2L2 4000 100 2 2 19.05 2.241e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00C2L4 4000 100 2 2 17.81 2.398e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00C2C2 4000 100 2 2 17.51 2.438e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00C2C4 4000 100 2 2 18.25 2.340e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00C2R2 4000 100 2 2 18.62 2.293e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00C2R4 4000 100 2 2 17.84 2.393e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098668 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091118 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020710 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00R2L2 4000 100 2 2 18.31 2.331e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00R2L4 4000 100 2 2 17.48 2.442e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00R2C2 4000 100 2 2 18.03 2.368e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00R2C4 4000 100 2 2 18.90 2.259e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00R2R2 4000 100 2 2 18.64 2.290e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED

============================================================================

T/V N NB P Q Time Gflops

----------------------------------------------------------------------------

WR00R2R4 4000 100 2 2 18.59 2.296e+00

----------------------------------------------------------------------------

||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098668 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091118 ...... PASSED

||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020710 ...... PASSED

============================================================================

Finished 18 tests with the following results:

18 tests completed and passed residual checks,

0 tests completed and failed residual checks,

0 tests skipped because of illegal input values.

----------------------------------------------------------------------------

End of Tests.

============================================================================