Transcript High Performance Computing

CCPR Workshop
Introduction to the
Cluster
July 13, 2006
Outline



What is a cluster?
How can CCPR’s cluster help you?
Using the cluster
• Connecting from Windows
• Unix commands
• Submitting jobs
• Managing jobs
What is a cluster?

Think of a cluster as having two parts:
• Hardware: a collection of computers
• Cluster management system: software for
coordinating hardware with a collection of
computational tasks (batch jobs)
CCPR’s Cluster:
Management System

Jobs are submitted to the cluster via the
cluster management system
• An advanced batch system
• Efficiently allocates resources to jobs
• Remembers job requirements
• Remembers node specs and availability
• Schedules jobs according to requested resources
•
and priority
Users can easily view jobs
How does this help your research?







Easy access to software and data
Faster processors
More RAM
Easy to share data, programs, etc. with
colleagues via the cluster
More advanced users: parallel processing,
compilers, etc.
Your pc is available for other work when you
submit a job to the cluster
Submit a job and forget about it
Using the Cluster




Connecting from Windows
Unix Commands
Submitting jobs
Managing jobs
Secure Shell

Secure Shell software (2 pieces)
1. SSH Secure Shell Client (Telnet)
2. SSH Secure File Transfer Client (FTP).
• Download software from
www.ssc.ucla.edu  Computing  Security
Secure Shell: Connecting to
lexis from Windows

Connect to lexis via SSH Secure Shell Client
•
•
•
•


Startup SSH Secure Shell/File Transfer
Click Quick Connect
Enter:
•
•
•
Host Name: lexis.ccpr.ucla.edu
User Name: your lexis account name
Port Number: 22
Click connect and enter password
Can add “profile” for repeated use
Same process for SSH Secure File Transfer Client
Samba: Transferring files from
pc to lexis



Analagous to “mapping a drive”
Allows user to view lexis files via
Windows Explorer interface
Must use vpn when you’re off campus.
Download from
• http://www.bol.ucla.edu/services/vpn/

Step-by-step instructions:
• http://lexis.ccpr.ucla.edu/using/samba/
Samba: Transferring files from
pc to lexis



Open Windows Explorer on the local machine.
Select Tools->Map Network Drive
In the "Folder" text box enter:
\\lexis.ccpr.ucla.edu\your-user-name



Click "Reconnect at login" if you want the drive to be
mounted each time the computer starts up.
Click the "different user name" link to enter your user
name and password for the connection.
Click "OK" then click the "Finish" button on the Map
Network Drive window.
Unix commands
passwd
Change lexis password
smbpasswd
Change samba password
pwd
Print working directory
cd
Change directory
mv filename1 filename2
Rename/move filename1
to filename2
cp filename1 filename2
Copy filename1 to
filename2
More Unix Commands
cat file
Display contents of file to screen
cat file1 >> file2 Concatenate file1 and file2
more file
Display contents of file to screen
ls
ls –lah
List files
List files in current directory (options
l=LONG, a=ALL, h = HUMANREADABLE
More Unix commands
du –h
.forward
gzip
Disk usage (option h=HUMANREADABLE)
forward mail to another account
Compress file
gunzip
Expand compressed file
ln –s actual link Create symbolic link
which name
search for name
Getting help in Unix
man command
list help for command
man –k
command
keyword search for command
whatis command brief description of command
apropos
keyword
list commands with keyword in their
description
Unix Commands: Using Aliases

Alias
• Create an alias for commonly used
•
commands
Example: alias ls=“ls –lah”
• Anytime you type ls, you will see the ls command
with options lah (l=long, a=all, h=human-readable)
• Edit .aliases file to create and remove aliases
Unix commands: Pipes and grep
history
displays a “history” of previously
entered commands
history | more
history, but one page at a time
history | grep cat sends output of history to commnd
“grep cat”, which displays only lines
containing “cat”
history | wc
Counts number of newlines, words,
and bytes contained in output of
history
Editing Files on Lexis



Pico – easiest editor. Open and type.
Commands listed at bottom of editor.
Vi – more advanced. You can move
around quickly, do copy/paste, search,
etc, but keystrokes required for
everything.
Emacs – most advanced
Submitting Stata and SAS jobs



Submit jobs by typing qjob, qstata, qsas,
qmatlab at Unix prompt
You will then be prompted for:
•
•
•
•
•
filename
memory (number in megs, ex. 1050)
if memory > 1 gig -> Over 2 hours (y/n),
job type (for qjob only)
submit (y/n)
Alternatively, submit all options without being
prompted:
•
qstata sample.do 500 y
Files produced by qstata

filename.log - This is the log file from Stata.
The output of your entire Stata job is caputured
in this file.

filename.do.cmd - This is the Command file
that is submitted to the cluster.

filename.do.joblog - This file will capture the
output of your CMD file and any errors that
may occur (Stata output is not captured by this
log.)
Job Management
RAM
Compute
time
requested
Job sent to…
<1 GB
n/a
Any node, no time constraints
>2 GB
n/a
>4-GB node
1-2 GB <2 hours
“2-hour” node, terminates after 2 hours
1-2 GB >2 hours
>4-GB node
Job Management



qstat or qstatus to view overall cluster
usage
qstat –j jobnumber for information
particular to your job
qdel jobnumber to kill your job
Working interactively qinteractive

To work interactively, type “qinteractive”
at prompt
Default RAM allotted is 256

Please use interactive nodes responsibly

• Request x megs by typing “qinteractive x”
• Limited number of spaces
• Testing only
• Logoff promptly
Questions/Comments