Transcript Getting Started on Emerald - Information Technology Services

Getting Started on Emerald
Research Computing Group
Course Outline
 What is Emerald?
 Logging into Emerald
 File manipulation on Emerald
 Submitting jobs on Emerald
 Interactive programs on Emerald
its.unc.edu
2
Help Documentation
 Getting Started on Emerald
• http://help.unc.edu/6020
• General overview of Emerald for range of
users
 Short Course – Getting Started on
Emerald
• http://help.unc.edu/6479
• Detailed notes for beginning Emerald users
its.unc.edu
3
What is Emerald?
 352-processor Linux Cluster
• Maintained by Research Computing Group
 Appropriate for all users regardless of
expertise level
 Other Servers:
• Cedar/Cypress (128-processor SGI/Altix)
• Topsail (4160-processor Dell Linux Cluster)
 Mass Storage
• Account access
its.unc.edu
4
Advantages of Using
Emerald
 High performance
 Large capacity
 Parallel processing
 Many available software packages
 Variety of compiling options
its.unc.edu
5
Emerald Linux Cluster
its.unc.edu
6
Distributed vs. Shared
Memory
CPU
CPU
CPU
CPU
M
M
M
M
CPU
CPU
CPU
CPU
BUS
NETWORK
Distributed memory - each processor
has it’s own local memory. Must do
message passing to exchange data
between processors.
(examples: Emerald, Topsail)
its.unc.edu
MEMORY
Shared memory - single address
space. All processors have access to a
pool of shared memory. (examples:
Yatta, Cedar/Cypress) Methods of
memory access : Bus and Crossbar
7
Logging Into Emerald
 UNIX/Linux/OSX
• ssh [email protected]
 Windows: SSH Secure Shell
• Setting up a Profile for Emerald
• Forwarding X11 packets
its.unc.edu
8
Home and Work
Directories on Emerald
 Home Directory
• /afs/isis/home/m/y/my_onyen/
• 250MB quota
• ~/private/
• Files backed up daily [ ~/OldFiles ]
• Space quota/usage in Home Directory:
 fs lq
 Work Directory
• /netscr/my_onyen/
• No space limit but periodically cleaned
its.unc.edu
9
File Manipulation
Commands
 SSH Secure File Transfer
 Copy files
• cp command
cp /afs/isis/depts/atn/rcg/example_code/Gaussian/water.com /netscr/my_onyen/.
cp /afs/isis/depts/atn/rcg/example_code/sas/test.sas /netscr/my_onyen/.
• cp -r
 cp -r ~/private/TestDirectory .
 Move files
• mv commands
 mv ~/private/testfile.txt .
its.unc.edu
10
File Manipulation
Commands
 Tar archive
• To create a tar file
 tar –cvzf TestDirectory.tgz ./
• To see a tar file’s table of content
 tar –tvzf TestDirectory.tgz
• To untar a tar file
 tar –xvzf TestDirectory.tgz
its.unc.edu
11
Submitting Jobs: LSF and
Packages
 LSF (Load Sharing Facility)
• Fairly distribute compute nodes among users
• 60 processor per user limit
 Packages
• ipm commands
 ipm add (ipm a)
 ipm remove (ipm r)
 ipm query (ipm q)
• Available packages
 http://www.renci.org/unc/computing/resources/appl
ications.php
its.unc.edu
12
Details of LSF
Submission host other
3
LIM
2
6
7
bsub app
LIM
MLIM
MBD
SBD
RES
its.unc.edu
hosts
MLIM
4
5
Batch API
1
hosts
Master host other
Load
information
MBD
9
8
queue
10
– Load Information Manager
– Master LIM
– Master Batch Daemon
– Slave Batch Daemon
– Remote Execution Server
Execution host
LIM
SBD
11
Child SBD
12
RES
13
User job
13
Submitting Jobs: bsub
Command
 bsub command
 All
files must be in /netscr/my_onyen/
 bsub
[- bsub_opts] executable [-exec_opts]
 Queues – bqueues command
• week
• idle
 bsub –o
• bsub –o out.%J
its.unc.edu
14
Submitting Jobs:
Following Job Progress
 bjobs
• bjobs –l JobID
• Shows current status of job
 bhist
• bhist –l JobID
 bkill
• bkill JobID
• Ends job prematurely
 bfree
its.unc.edu
15
Submitting Jobs: Specialty
Scripts
 bsas
• bsub -q week -R blade sas program.sas
• bsas test.sas
 bmatlab
• bsub -q week -R blade matlab -nodisplay nojvm -nosplash program.m -logfile
program.log
• bmatlab test.m
its.unc.edu
16
Compiling on Emerald
• Compilers
–
–
FORTRAN 77/90/95
C/C++
• Parallel Computing
– OpenMP
– MPI (MPICH, LAM/MPI, MPICH-GM)
its.unc.edu
17
Compiling Details on
Emerald
Compiler
Package name
Command
Intel
intel_fortran, intel_CC
ifort, icc
Portland Group
pgi
pgf77, pgf90,pgcc,pgCC
Absoft
profortran
f77, f90
GNU
gcc
g77,gcc, g++
its.unc.edu
18
Compiling Details on
Emerald
 Add a compiler into your working
environment
• ipm add package_name
 Compile a code
• command code.f –o executable
 Run executable on a compute node using
the bsub command
• bsub –q week –R blade executable
its.unc.edu
19
Submitting Jobs: Job
Output
 Output sent to email
• [email protected]
 bsub –o
• Output saved in working directory
 bsub –u email@address
• Output sent to specified email address
its.unc.edu
20
Interactive Jobs: Setup
 X-Windows
• Linux/OSX
 X11 client
• Windows
 X-Win32

Offered on UNC Software Acquisition site
 Port forwarding on SSH Secure Shell
 Setting up a session on X-Win32
its.unc.edu
21
Interactive Jobs:
Submission
 –Ip
• bsub –q int –R blade –Ip sas
• bsub –q int –R blade –Ip gv
 Specialty Scripts
• xsas
• xstata
its.unc.edu
22
Contacting Research
Computing
 For assistance with Emerald, please
contact the Research Computing Group:
• Email: [email protected]
• Phone: 919-962-HELP
• Submit help ticket at http://help.unc.edu
its.unc.edu
23