Transcript CCPR Workshop Lexis Cluster Introduction

CCPR Workshop
Lexis Cluster Introduction
October 19, 2007
David Ash
Outline

What is a cluster? ….two components
Outline Continued…


How can CCPR’s cluster help you?
Lexis usage




Connecting from Windows
Unix commands
Submitting jobs
Managing jobs
Lexis Management System

Jobs are submitted via cluster
management system





An advanced batch system
Efficiently allocates resources to jobs
Remembers node specs and availability
Schedules jobs according to requested
resources and priority
Users can easily view jobs
How does this help your research?





Remote access to software and data from
anywhere!
Incredible machines – faster processors and
more memory
Flexible system to share data and programs with
colleagues
Opens up your home/office PC for other things,
instead of grinding away on a Stata job
Advanced users: parallelization, fast compilers
Connecting to Lexis

Secure Shell Software
SSH (command line access)
 SFTP (file transferring access)
 Obtain from:
http://computing.sscnet.ucla.edu/public/security/ssh.aspx


Alternative Client Software


PuTTY (SSH)
http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe
winSCP (SFTP)
http://winscp.net/eng/download.php
Connecting to Lexis: Secure Shell

Directions for SSH Secure Shell:


Click Quick Connect
Enter:






Host Name: lexis.ccpr.ucla.edu
User Name: your lexis account name
Port Number: 22
Click connect and enter password
Can “Add Profile” for repeat usage
Same for SSH Secure File Transfer
Mapping Lexis as a Drive using Samba


Creating Lexis as a drive in Windows
Explorer
Must use the VPN to do this action off
campus!



http://www.bol.ucla.edu/services/vpn/
Instructions I’m going to show are at:
Step-by-step instructions:

http://lexis.ccpr.ucla.edu/using/samba/
Mapping Lexis



Open Windows Explorer
Select Tools -->Map Network Drive
At "Folder" text box enter:
\\lexis.ccpr.ucla.edu\your-user-name

Click "Reconnect at login" if you want the drive to
be connected each time the computer starts

Click the "different user name" link to enter your
user name and password for the connection.

Click "OK" then click the "Finish" button on the
Map Network Drive window.
Linux Commands
passwd
Change your Lexis SSH/SFTP password
smbpasswd
Change your Samba password
pwd
Print your current directory
ls
List files (can take arguments)
ls –lah
L = long, A = all, H = human readable
cd
Change directory
AND cd ..
AND
Go up one directory
mkdir foldername
Create a directory
cp file1 file2
Copy file1 to file2
mv file1 file2
Rename/Move file1 to file2
more file OR cat file
Display file to screen
du –h
Disk Usage (-h = human readable)
tar –cvf AND tar -xvf Compress a file/directory / Expand .tar
gzip AND gunzip
Compress a file (.gz extension) / Expand .gz
ln –s actual link
Creates a symbolic link/shortcut
Getting Help in Linux
man command
list help for command
man –k command
keyword search for command
whatis command
brief description of command
which name
search for name
apropos keyword
list commands with keyword in their description
[email protected]
Bryan and myself.
Linux File Editors




Pico – easiest editor. Open and type.
Commands listed at bottom of editor.
Vi – more advanced. You can move
around quickly, do copy/paste, search,
etc, but keystrokes required for
everything.
Emacs – most advanced
OR edit files over Samba from Windows.
Submitting Stata and SAS jobs


Submit jobs to the Cluster using ONE of the
following commands: qjob, qstata, qsas,
qmatlab
Will be prompted for:






Job type (for qjob only!)
Filename
Amount of Memory (number in megs, ex. 1050)
If memory > 1 gig --> Over 2 hours (y/n)
Confirmation to submit (y/n)
Quick Method:

qstata sample.do 500 y
Qstata example output

Two Logs are created
 filename.log - This is the log file from Stata. The output
of your entire Stata job is captured in this file.


filename.do.joblog - This file will capture the output of
your cluster command file and any errors that may occur
(Stata output is NOT captured by this log.)
filename.do.cmd - This is the command file that is
submitted to the cluster.
 Quick TIP: qsub filename.do.cmd
Cluster job monitoring



qstat or qstatus to view overall cluster
usage
qstat –j jobnumber for information
particular to your job
qdel jobnumber to kill your job
Interactive Sessions

Interactive Usage




Non-GUI Interactive Session


Meant for testing before submitting to cluster
Limited spots
Logoff promptly when finished
Type “qinteractive”, can also specify memory
after command. Default 256mb.
GUI Interactive Session


Login with XliveCD
Type “qlive” – Run sas/matlab/xstata
Questions / Comments?