RQMD on Cray T3E

RQMD (Version 2.3) Source Code
Modifications to RQMD Source
RQMD IO Handling on Cray T3E
RQMD Random Number Seed
Make RQMD executable
Running RQMD on Cray T3E

RQMD (Version 2.3) Source Code

Source code of RQMD version 2.3 is obtained from Dr. Heinz Sorge.

The source code is split into many files, so that each file contains only one subroutine or function for easy identification.

Modifications to RQMD source

Random Number Function ranf

RQMD has a routine ranf which conflicts with an intrinsic fortran routine on the Cray T3E. So the routine name is changed to rrrr. This requires change of ranf -> rrrr in all following routines. (One could also declare the routine as external in all following files without change the routine name.)

absorb.f
angel.f
angin.f
angnp.f
antibb.f
bele.f
bplist.f
bresdc.f
bwdist.f
chains.f
cmass.f
coll.f
coltim.f
delten.f
diffra.f
emuls.f
enlk.f
flavor.f
gamcon.f
hit34.f
imsed.f
imspin.f
initev.f
iqqrnd.f
isocgk.f
jetcon.f
kfindb.f
kmass.f
kqqret.f
kresb.f
kresm.f
lddec.f
lmass.f
lusysj.f
makjet.f
mbafnd.f
mbtage.f
mesmes.f
mesres.f
mlt810.f
mmefnd.f
ndmed.f
nghbor.f
paulin.f
piweg.f
pspsv.f
qdrot.f
reject.f
resdec.f
rlu.f
rrrr.f
scattr.f
string.f
strist.f
timelf.f
torsts.f
twodec.f

Intrinsic Function sign
Intrinsic routine sign seems missing on the Cray T3E. So a routine rsign is added, and changes of sign -> rsign are made in files:
```
angel.f
angin.f
angnp.f
lusysj.f
plu.f
relham.f
rsign.f
ulangl.f
```
Function Statement
The following function statement is commented out in routines
```
cross1.f
diffra.f
```
because T3E Fortran 90 complier seems not to support function statements.
```
       tblfit(plab,a,b,en,c,d)=a+b*plab**en+c*(log(plab))**2+
     a                              d*log(plab)                         
```
Routine tblfit is added to define the function.
Double Precision etc.
All numbers in Cray T3E Fortran are 64 bits. Declarations of
```
	real*4
	integer*4
	double precision
```
will cause program crash or uncertain behavor. All these declarations are changed to
```
	real
	integer
```
There are minor changes of 1d0 -> 1.0 in routines
```
lurobo.f
trobo.f
```
CERN Routines
RQMD uses two CERN routines timex (to manage cpu times) and datimh (to manipulate random numbers). Since there isn't CERN libraries on the Cray T3E, these two routines (timex.f and datimh.f) were written to do the same (but not exactly identical) tasks.
Intrinsic Function secnds
RQMD uses an intrinsic function secnds that is not generic (i.e., it exists on some machines, and doesn't exist on other machines). The routine gives time in seconds from mid-night minus the only argument passed in. This routine (secnds.f) is written.

RQMD IO Handling on Cray T3E

Input: Fortran namelist
RQMD uses namelist input to pass in control parameters. Cray T3E Fortran doesn't support logicals and character strings in namelist which RQMD uses. Standard RQMD input namelist file (file5) looks like:
```
 $input
        n1      = 32
        iz1     = 16
        n2      = 32
        iz2     = 16
	header  = 'rqmd'
        ref     = 'cld'
        elgev   = 100.
        b       = 0.
        iseed   = -100
        nruns   = 10
	switch  = F F F F F T F T F F F F T F F F T F F F F F T F
 $end
```
The way it is handled on the Cray T3E is to replace
- character*4 header with integer jeader
- character*3 ref with integer jef
  - jef=0 for ref=' nn'
  - jef=1 for ref='cld'
  - jef=2 for ref='lab'
- logical switch(24) with integer jwitch(24)
The conversions is down in getinp.f before and after namelist read, so that the rest of RQMD doesn't know these changes in the namelist.

At end of each event, RQMD writes out namelist output which includes ref. On Cray T3E, jef is written out instead.

Output

RQMD writes out 10 output files (file6 - file15). When a RQMD job is submitted onto several processors, all the processors read in the same namelist file and start an event on each processor. With the standard RQMD, all processors write out to the same output files. The way the IO is handled now is to have different processors create output files in different sub-directories (these sub-directories have to be created before running RQMD). The following piece of code is added in getinp.f to achieve this:

      character*4 mype,dpet3e
      external dpet3e
      mype=dpet3e()
c       open (unit=5,file='file5',status='old')                          
c       open (unit=6,file='file6',status='old',form='formatted')         
c       open (unit=7,form='unformatted',file='file7',                    
c     a                  status='unknown')                               
c       open (unit=8,file='file8',status='unknown')                      
c       open (unit=9,file='file9',status='old',form='formatted')         
c       open (unit=10,file='file10',status='unknown')                    
c       open (unit=11,form='unformatted',file='file11',                  
c     a                                    status='unknown')             
c       open (unit=12,form='unformatted',file='file12',                  
c     a                                    status='unknown')             
c       open (unit=13,file='file13',status='unknown')                    
c       open (unit=14,file='file14',status='old')                        
c       open (unit=15,file='file15',status='unknown')                    
       open (unit=5,file=mype//'file5',status='old')
       open (unit=6,file=mype//'file6',status='old',form='formatted')
       open (unit=7,form='unformatted',file=mype//'file7',
     a                  status='unknown')
       open (unit=8,file=mype//'file8',status='unknown')
       open (unit=9,file=mype//'file9',status='old',form='formatted')
       open (unit=10,file=mype//'file10',status='unknown')
       open (unit=11,form='unformatted',file=mype//'file11',
     a                                    status='unknown')
       open (unit=12,form='unformatted',file=mype//'file12',
     a                                    status='unknown')
       open (unit=13,file=mype//'file13',status='unknown')
       open (unit=14,file=mype//'file14',status='old')
       open (unit=15,file=mype//'file15',status='unknown')

What is commented out above is stardard RQMD. Routine dpet3e returns the sub-directory name as "xxx/", where xxx is the processor number. e.g., 000 for processor 0, 031 for processor 31.

dpet3e in turns calls ipet3e, which gives the processor number.

Random Number Seed

Normally RQMD uses computer time to manipulate the random number seed. This has problem with parallel processing, because many processors could end up with same random seed if they start-up at rougly the same time. The way it is handled is to add the processor number into this manipulation of the random number seed in routine seed.f, which RQMD uses to generate random number seed. The routine has to call ipet3e of cause.

Make RQMD executable

Makefile (which includes Makefile.depend ) to link RQMD executable.

Running RQMD on Cray T3E

Use command

qsub script_file

A sample script is

#
# Embedded flags (default options) for qsub:
#
#QSUB -s /bin/csh
#QSUB -r rqmd
#QSUB -eo -o /u/gc5/fqwang/rqmd2.3/wrk/rqmd.out
#QSUB -l mpp_p=16       #Maximum PEs Needed (Required).
#QSUB -mb -me
#
set echo
#
#       setup rqmd files
#
cp /u5/fqwang/rqmd2.3/nml/SS200cld_b0.nml $BIG/file5
chdir $BIG
cp /u/gc5/fqwang/rqmd2.3/bin/rqmd .
touch file6 file9 file14
/u/gc5/fqwang/rqmd2.3/bin/cp_file.ksh 16
rm file5 file6 file9 file14
#
#       execute
#
ja
mpprun -n 16 ./rqmd
ja -s
#
#       merge output files,
#       move the merged files to storage area,
#       and clean up house.
#
ls -sR
/u/gc5/fqwang/rqmd2.3/bin/merge_file.ksh 16
ls -s file*
/u/gc5/fqwang/rqmd2.3/bin/put_file.csh /usr/tmp/fqwang SS200cldbub_b0.nml
rm -rf *
#
exit 0

For explanation of embedded flags, see Cray T3E Web Page and qsub man page.

The script does several tasks:

set-up RQMD files in temperary work directory $BIG (which is recommended by NERSC):
- cp input namelist file into file5
- create output files file6, file9, file14 that RQMD opens as 'old' files
- create sub-directories for multiple processors, and cp the above files into these sub-directories by script cp_file.ksh
- rm the files from the top work directory $BIG
main task: running RQMD
Since different processors write out different sets of output files, we want to merge them to keep number of files small. This step is not absolutely necessary if one doesn't mind large number of files. The merge is done by script merge_file.ksh. The merge is easy for RQMD output files because they are in ASCII format.
cp the files off the temperary directory to some storage area (by put_file.csh), or to the UniTree mass storage (by masput_file.csh).

Contents

RQMD (Version 2.3) Source Code

Modifications to RQMD source

Random Number Function ranf

Intrinsic Function sign

Function Statement

Double Precision etc.

CERN Routines

Intrinsic Function secnds

RQMD IO Handling on Cray T3E

Input: Fortran namelist

Output

Random Number Seed

Make RQMD executable

Running RQMD on Cray T3E

Last updated: 13 May, 1997

fqwang@lbl.gov

Contents

RQMD (Version 2.3) Source Code

Modifications to RQMD source

Random Number Function ranf

Intrinsic Function sign

Function Statement

Double Precision etc.

CERN Routines

Intrinsic Function secnds

RQMD IO Handling on Cray T3E

Input: Fortran namelist

Output

Random Number Seed

Make RQMD executable

Running RQMD on Cray T3E

Last updated: 13 May, 1997 fqwang@lbl.gov

Last updated: 13 May, 1997

fqwang@lbl.gov