Transcript UNIX - University of Sheffield

Introduction to Linux on Iceberg
•
•
•
•
•
•
•
•
•
What are UNIX and Linux?
Getting Connected
Basic UNIX/Linux Commands
Help!
Working with Directories
Running Programs
Editors
The Shell
Exercise
History of . UNIX/Linux?
• Unix operating system was developed around 1969
in the Bell Labs
• Originally written using C
• Around 1990 Linus Torvalds of Helsinki University
started off a freely available academic version of
Unix
• Linux is the Antidote to a microsoft dominated future
What is UNIX/Linux ?
• Multi-Tasking O/S
• Multi-User O/S
• Available on a range of Computers
Which UNIX?
• SunOS
Sun Microsystems
• IRIX Silicon Graphics
• HP-UX
Hewlett Packard
• Linux
For IBM PC compatibles
There are a number of certification bodies with published standards
and test suites to ensure quality of products. Such as;
• Posix: Portable Operating System Interface ( IEEE standard)
• COE: The Common Operating Environment (COE) Platform
certification.
UNIX Internals (Simplified)
• Kernel
– System Internals
• Shell
– Command Interpreter
– Programming language
• File System
• Process Management
UNIX Shells
•
•
•
•
•
sh Bourne Shell (Original Shell) (Steven Bourne of AT&T)
bash Bourne Again Shell (GNU Improved Bourne Shell)
csh C-Shell (C-like Syntax)(Bill Joy of Univ. of California)
ksh Korn-Shell (Bourne+some C-shell)(David Korn of AT&T)
tcsh Turbo C-Shell (More User Friendly C-Shell).
You can switch from one shell to another by just typing the name of the
shell. exit return you back to previous shell.
Linux and GNU
• Linux is an implementation of Unix
• Linux/Unix operating system is written in ‘C’
• Linux is not part of the GNU project but uses the same licensing
agreements
• Many of the linux utilities and tools are taken from the GNU
project.
• There are many flavours of linux distributions. The mix of the
kernel (linux) with the utilities (GNU and other) and the installation
procedure determine the flavour. Some of these are;
–
–
–
–
–
–
Fedora
SuSE
Redhat
Debian
Mandrake
Knoppix
Terminal Emulation Software
Used in Sheffield Univ.
• ssh : Secure Shell Client (no XWindows)
• Exceed : ( Xwindows support and also secure shell)
Graphical UNIX via Exceed
•
•
•
•
X Concepts
Using the Exceed Software
Starting an xterm session
Starting a ssh session
Basic X Concepts
• X Server runs on local machine
– PC
Exceed, Cygwin, Xming
– UNIX Workstation Included in OS
– Apple Mac
Exodus
• X Client runs on remote machine
– Graphical Application
• xterm
• xcalc
• Modelling and visualisation packages etc.
ssh on Managed/XP workstations
Secure-shell client can be accessed via;
Start Unix_ConnectivityExceed3Dssh.
When the program initialises you will be asked to enter
hostname (default is iceberg) , username and password.
ssh on Managed/XP workstations
When ssh starts you will be presented with the login panel as
shown below; Default host name is iceberg.shef.ac.uk and can
be changed if you want to login somewhere else.
You may next be asked if you wish to save the public host-key
into a local-database. This is not a crucial question and you may
answer YES or NO.
Also if asked to provide host response just press ENTER.
Exceed on the stand-alone PCs
Starting an Exceed xterm session
Start Exceed ->Xstart and fill in the panel as shown below
Fill in host,username,p/w,command
and click on the run icon to start xterm
Exceed on the stand-alone PCs
An Exceed xterm session
Multiple ssh or xterm shells
There are no limits to the number of ssh or xterm windows one can
start simultaneously by methods described in the previous
slides. You may also start extra xterm windows from the host
by simply typing xterm &
On titania and iceberg we also have a local command named
Xterm that starts up an xterm window with nicer to use
parameters. On iceberg we strongly recommend that you use
Qsh rather than xterm or Xterm command so as to make use of
a free worker node. Qsh will act like Xterm but will make use of
a worker node.
Typing exit will terminate an xterm or ssh session neatly.
This will also close the xterm window but not the ssh window. ssh
windows can be closed via the file exit menu.
Starting gnome
Gnome was the default graphical desktop interface that was
installed on iceberg before delivery. We currently do not allow
the use of gnome on the head-node ‘iceberg’ for efficiency
reasons. You could however use the gnome panel while
working on one of the worker nodes ( via qsh ). This provides
the user with access to all the gnome commands available via
the red-hat menus. To do this type
qsh to start an interactive session
and in that session type
/usr/bin/gnome-panel &
This is the nearest, one would get to using the common-desktop
environment as was provided on titania but it is not worth it.
Some basic rules
• Unix is case sensitive.
• Commands are in lower case.
• Backspace and/or Del Keys correct typing errors.
If the terminal parameters are not correctly set;
try Ctrl+H
• Ctrl+C Aborts a program or command.
• You can use the arrow keys to recall previous
commands, optionally edit and execute them.
Key combinations in bash
Ctrl+A
Move cursor to beginning of line
Ctrl+C
<Break> Stop running program
Ctrl+D
<Logout>
Ctrl+E
Move cursor to end of line
Ctrl+H
Delete character to the left
Ctrl+Z
Suspend the program
Arrow left/right
Move pointer left/right to insert chars etc.
Arrow up/down
Recall previous commands so as to edit them andsubmit
them by pressing ENTER
Tab
Command or filename completion. Auto-complete the
command line (to save typing)
If tab does not work tab again to show possibilities
Tab tab
passwords
• In normal linux environment the passwd command can be used
to change the user passwords. However, because we manage
passwords centrally this command will not work on iceberg.
• If you wish to change your iceberg password you will have to do
this via a web interface at the following URL:
http://www.shef.ac.uk/cics/password
Filenames
• Filenames can comprise of:
a-z, A-Z alphabetic characters
0-9
digits
.-_+
special characters
mon+tue_01.06-03-96
• Wildcards when referencing files
* any character or sequence of characters
? any single character
Format of Unix commands
• command [option ...] [filename ...]
eg:
ls
ls -l tutorial
more tutorial
List Directory
• ls
• ls -l
• ls -a
list directory
list directory in long format
list all (inc. hidden) files
-rw------ l course01
Access
Permissions
57
Number of
bytes in file
Oct 18 11:05
hello.c
Date and time
last modified
Displaying contents of a text_file
•
more filename
This command will start listing the contents of filename on screen and pause after a screenfull
of data. While pausing, use the following characters to control the output.
Spacebar
next screenful
n Spacebar : next n lines
Enter
next line
b
back one screen
n b : back n screen’s full
q
quit
? or h
list commands
where n is a whole number
Displaying contents of a text_file… continued
• cat [options] filename [filename … ]
This command will output the contents of filename[s] to standard-output (
normally screen) without pausing.
Following options are useful;
-v
display non-printing characters
-n display with lines numbered on the left
• tail [-n] filename
This command lists the last 10 lines of a text file.
If a number is specified (.eg. -20 ) lists the last n (i.e 20) lines
Copying files
•
Copy files (optionally directories)
cp fromfile tofile
Some of the useful options are:
-R or –r : Recursive copy ‘fromfile’ is a directory so the entire
directory and its contents are copied.
e.g. cp –r mydir newdir
-p : preserve. Preserves all attributes of the file ,such as access rights and
creation date.
•
Copy and concatenate files by using cat
Cat command concatenates contents of list of files and directs the output to
standard output (normally screen). When used with redirection ‘>’ it can be
used to join files together.
e.g. cat file1 file2 file3 > new_big_file
Renaming and deleting files
•
•
mv :This command will move a file or directory to a new location. It can thus be
used to rename files/directories as well as change their locations in the global
directory structure.
Syntax:
mv source destination
Example:
mv myfile mynewfile
mv myfile subdirectory/myfile
mv mysubdir mynewsubdir
rm : This command will delete a file (optionally a directory if used with –r option).
Syntax: rm object_to_delete
Example:
rm myfile
rm –r mydirectory
Finding files and information about them
•
•
•
find :Finds a file in the directory hierarchy
find root_dir –name filename –print
Example :
find . –name “myprog.*” -print
Note that wild-character containing string must be enclosed in quotas
which : Shows in which directory a command is located.
Syntax: which command_name
file: Can be used to see what type of data a particular file contains. For example, script
, program, library, executable binary etc…
Syntax:
find command_name
Searching in files
• grep string file
This command finds and prints out the lines in the file(s) containing the specified
string
string
file
= word or phrase
= file or list of files (wild_card can be used)
Note: We strongly advise that the string is quoted.
Examples: grep ‘Green Man’ england.dat
grep ‘Zodiac’ t*.dat
grep ‘Zone[a-z]’ security.fil
Directory
Structure
/ (root)
home1
usr
me
cs
cs4un1
ph
cs4un2
Home directory of user cs4un1 : /home1/cs/cs4un1
When you log in you are positioned in your home directory.
The environment variable $HOME is also set to contain this
directory name.
Working with Directories
• pwd
• cd
cd
cd ..
cd mydir
cd /var/adm
print working directory
change directory
move to home directory
move up one level
move into a subdirectory
move to an absolute directory
• mkdir directory_name
• rmdir directory_name
create a new directory
delete an empty directory
Managing your filestore
• Each user has an allocation of filestore (Disk Space)
• Monitor filestore usage with the ‘quota’ command
EG:
quota
Disk quotas for username (uid 20002)
Filesystem usage quota limit grace
/home1
/data
/scratch
1200
2000
4000
files quota limit timeleft
153
0
0
411 99600 100000
13
0
0
0 99600 100000
0
0
0
Note: quota –s will display quota in sensible units.
• The quota value can be exceeded for 7 days
quota < usage < limit for 7 days
• The limit value can not be exceeded at all.
usage < limit always
• If usage > quota for 7 days account is expired and files can
not be created
Manual Pages and Info system
It is possible that there are more than one manual pages with the
same name. This is because the manual pages contain
information about the system library routines as well as the usual
unix commands. Manual pages for commands are contained in
section 1 whereas the system library call related items are in
section 3. To access manual pages within a specific section
simply specify the section_number as the first parameter.
Example: man 3 random
• Note that man – k topic will list the relevant manual pages and
their section numbers.
• The –S parameter can also be specified.
Example:
man files –S 1:2:4
Manual Pages and Info system
Man: Manual pages : Give text-based help on usage.
Manual pages are grouped into sections (1,2,3,4 ..).
There is usually one manual page per command which is
located in one of the directories defined by the MANPATH
environment variable.
To access the Manual page for a command just type;
• man command
To get a list of manual pages that contain a ‘word’ type;
• man – k topic
Info : Similar to man but can scroll with cursor keys and link to
other information.
Using whatis and apropos commands
• whatis displays the short description of a system command ‘if
you know the command name’ .
• apropos searches the same ‘whatis-database` and reports
back the names of all the commands that a key-word occurs.
Example usage: Suppose we wish to create a temporary file with
a unique name but do not know if there is a command for doing
so. You could try :
apropos temporary or apropos file or apropos create
This will give ‘whatis’ output for a list of commands which contain
the keyword. You can now use the man command for
potentially useful commands to get the full details of how to use
them.
man mktemp
Running programs on iceberg
• Iceberg is the gateway to the cluster of worker nodes and the
only one where direct logging in is allowed.
• Iceberg’s main purpose is to allow access to the worker
nodes but NOT to run cpu intensive programs.
• All cpu intensive computations must be performed on the
worker nodes. This is achieved by the qsh command for the
interactive jobs and qsub command for the batch jobs.
• Once you log into iceberg, taking advantage of the power of a
worker node for interactive work is done simply by typing qsh
and working in the new shell window that is opened. This
what appears to be a trivial task has would in fact have
queried all the worker nodes for you and started a session on
the least loaded worker in the cluster.
• The next set of slides assume that you are already working
on one of the worker nodes (qsh session).
Running programs
• Two modes of operation foreground and background
– Foreground
Interact with program via
keyboard/screen
– Background
No connection with keyboard/screen
Submit to backbround by Appending ‘&’
EG:
myprog >& myfile &
The symbols ‘>&’ redirect output and any errors to the file myfile
Although the above method of running jobs on the
background is feasible, the prio we recommend that you
submit your background into the batch queue via the qsub
command.
Redirection
Most unix commands are not aware of the source of their
input or the destination of their output. They simply
read/write from/to stdin/stdout. The shell takes care of
these issues.
• Standard Input (default=>keyboard)
• Standard Output (default=>screen)
• Redirection symbols <,>,>> can be used to specify
files as the source/destination of the read/write
operations to override the above defaults.
Redirection continued …
Most unix commands are not aware of the source of their input or the
destination of their output. They simply read/write from/to
stdin/stdout. The shell takes care of these issues.
• To redirect the output to a file use the ‘>‘ symbol.
Example:
ls -l > dirlist
• The ‘>‘ symbol should be used with care as it may over-write an
existing file. ‘>>’ symbol can be used instead if the output should
be appended to the end of an existing file rather than over-writing
it.
Example:
ls -l >> logfile
• If nothing is directed to a file then a zero size file is created, or if
the file already existed then the contents of the file is removed
Example:
> afile
• The file /dev/null is a special symbol to indicate a ‘black-hole’
Example : ls –l > /dev/null
Redirection continued …
If any program expects any of it’s input from the
standard-input-channel , i.e. the keyboard, it can also
read the same information from a file by redirection.
• To read input from a file use the ‘<‘ symbol.
Example:
write cs1xyz < message.fil
Here any text input {write} program expects from the
keyboard will be simply read from a file named
message.fil.
Each end-of-line will be treated as an <ENTER> on the
keyboard.
Piping
• Feeding the output of one command into the input
of another command
• The symbol ‘|’ is called a pipe
command | command
• eg: ls -al | more
ls -la | grep Nov
Examples of re-direction and piping
•
•
•
•
•
•
•
•
•
ls –l | grep ‘Jun’
ls –l | grep `Jun` > june_files
ls –l | grep `Jun` | cut –c 57-80 > june_files
cut –c 1-10 < test_files
aspell –l < message.txt > report.txt
grep fluent < news.dat
Grep fluent < news.dat | cut –c 1-72
(grep fluent < news.dat ) | cut –c 1-72
(grep fluent < news.dat ) | cut –c 1-72 > fluent.news
Foreground Program Control
• Kill a program
• Stop a program
Ctrl C
Ctrl Z
Note a stopped program still exists in the system
and hence can be re-started.
Program control within current shell
• jobs
Lists jobs (programs)
• bg %job_id
Place a job in the
background
• fg %job_id
Return a job to the foreground
• stop %job_id
Stop a job
• kill %job_id
Kill a job
Process_id can be used in place of %job_id for more
definitive way of identification.
jobs
[1] + Running
time.sh > out
stop %1
[1] + Stopped (signal) time.sh > out
bg %1
[1] + time.sh > out &
kill %1
Terminated
Program control using ‘ps’ and ‘kill’
• ps
Report process status
ps -f -u username
UID
PID PPID C
STIME TTY
username 24816 24585 0 16:23:04 pts/50
username 20169 19956 0 16:05:45 pts/50
username 24585 20169 0 16:35:07 pts/50
• kill
TIME
0:00
0:01
0:00
CMD
sleep 2
-csh
/bin/sh time.sh
Terminate process
– Find its process ID (PID) using the ps command
– Kill the process using the kill command
kill 24585
– Sometimes kill on its own does not work so try
kill -KILL 24585
or kill -9 pid
Program Control using top
• top is a separate utility and not part of the OS
top -Uusername
– k kill program (process) k PID, k -KILL PID
– h help
– q quit top
Editors
There are a number of editors for Linux platforms most are not as
easy to use as the Windows based editors. Our recommendation
is;
Use nedit if you are using XWindows ‘e.g. Exceed’, cygwin.
Use vi or vim if you have a text terminal ‘telnet’.
•
•
•
•
pico
Easy ‘graphical’ , rather basic.
nedit Easy ‘graphical’ , good.
vi, vim
Require some knowledge, vim is like vi but
more and includes help
emacs
Have faithful following, good once learned.
Vi Text Editor
•
•
•
•
Very powerful
Available on all versions of UNIX
No in-built help
Man page gives detailed help
• Vi has three modes
– Command
– Text entry
– Last line
Vi Text Editor
•
•
•
•
Very powerful
Available on all versions of UNIX
No in-built help
Man page gives detailed help
• Vi has three modes
– Command
– Text entry
– Last line
T-Shell - Filename Completion
• Complete a partially typed filename
– Operation
• Type enough characters to uniquely identify the name
• Press the ‘Tab’ key (for C – shell use ‘Esc’ key)
• If the response is a ‘bleep’ press Ctrl-d to list possible matches
– Setup
• Add the following to your .cshrc file
• set filec
T-Shell -Repeating Previous Commands
• Operation
– history
List previous commands
– !!
re-run last command
– !n
re-run the nth command
– !str
last command starting with str
eg:
!vi
• Setup
– Add the following to your .cshrc file
– set history=40
T-Shell - Command Aliasing
• Definitions
– alias
– alias
– alias
ll
h
dir
' ls -lF '
' history 24 '
' ls '
• Setup
– Add alias definitions to your .cshrc file
UNIX Utilities
•
•
•
•
tar
compress
sed
awk
Exercise Tutorial
• Turn on PC
• Login to your PC Managed XP account
login digley/mep98xxx
password:
• Login to iceberg
– Start XSession or telnet titania
Login using the provided username and password