Atlas
From AtmosWiki
Contents |
Introduction
As Stetson, the server of our group, gets sometimes very busy, an other option to run your programs is to use Atlas, which belongs to Jordan Kyriakidis' group. This server is located here: atlas.phys.dal.ca. To start a ssh session on it simply type: ssh -X [login]@atlas.phys.dal. If you don't already have an account on Atlas, just ask Balagopal to create one for you. This server has 37 nodes in total, sharing the same characteristics as stetson's nodes. Our jobs on this server are obviously in a low priority state, but although it could be sometimes very busy, most of the time there are plenty of nodes available, so please check it out when you need to launch a lot of jobs. Note that the storage capability on atlas is limited (11T) compared to what we have on stetson. After your jobs are done you will then have to move your output to stetson so that our disk doesn't get quickly filled. Also, any job on Atlas MUST be submitted through the batch-queuing system (Grid Engine) using "qsub".
Running GEOS-Chem
There is a directory on atlas containing all the input files needed for GEOS-Chem to run (the equivalent of /data2/ctm on stetson) which is located here: /stetson-shared/ctm/ctm. The last version of ifort (version 10) is also installed on Atlas, so that any of the last releases of GEOS-Chem can run on it. The first things to do if you want to run GEOS-Chem on Atlas is to properly set your .bashrc. Here is an example:
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
alias ssh_stetson='ssh -p23 nicolas@stetson.phys.dal.ca'
alias 8_proc='export OMP_NUM_THREADS=8'
alias 4_proc='export OMP_NUM_THREADS=4'
# IDL
# ulimit -s 10000
# export KMP_STACKSIZE=100000000
#GEOS-Chem
ulimit -s 2097152
export F_UFMTENDIAN=big
export KMP_STACKSIZE=2097152000
export OMP_NUM_THREADS=4 #Stetson will use 8 Threads by default anyway
#export PATH=/opt/intel-9.1/fce/9.1.040/bin:$PATH
#export LD_LIBRARY_PATH=/opt/intel-9.1/fce/9.1.040/lib:$LD_LIBRARY_PATH
export PATH=/opt/intel-10/fce/10.0.023/bin:$PATH
export LD_LIBRARY_PATH=/opt/intel-10/fce/10.0.023/lib:$LD_LIBRARY_PATH
After having imported your code and run directories on Atlas, don't forget to replace the old input files path (/data2/ctm) by the new one (/stetson-shared/ctm/ctm/) in input.geos. Then everything works just like on stetson, except that you will have to submit your job through the batch-queuing system. Here is an example of script that launches GEOS-Chem:
#!/bin/bash #$ -S /bin/bash . /etc/profile source /data/nicolas/.bashrc #$ -o /stetson-shared/nicolas/GEOS-Chem/GEOS-Chem.v8-02-04.stdrun/runs/v8-02-04/Run_nobb/geos5/ #$ -M N.Bousserez@dal.ca #$ -m bea #$ -cwd cd /stetson-shared/nicolas/GEOS-Chem/GEOS-Chem.v8-02-04.stdrun/runs/v8-02-04/Run_nobb/geos5/ ./geos exit 0
To submit your job, simply type:
qsub [script_filename]
To see the status of jobs and/or queues:
qstat -u <username> or qstat -f
To terminate a job :
qdel <job-id>
Additional information about the Sun Grid Engine batch-queuing system can be found here: http://docs.sun.com/app/docs/doc/817-6117?q=N1GE
IDL & MATLAB
Both IDL and MATLAB are available on Atlas. The license usage is the same, whether your are working on Stetson or Atlas, that is one IDL license used on stetson is one IDL license less available for both Stetson AND Atlas for instance.
Atlas monitoring
The Ganglia web interface is located here: http://atlas.phys.dal.ca/. You will need to ask Balagopal for login/password information in order to access this server.
