Transcript Intermediate Unix

Intermediate Unix
Presented July 29th, 2001 by:
“Robin” R. Battey ([email protected])
Evgeny Roubinchtein ([email protected])
with tips, suggestions, and corrections by:
Hannah Tang ([email protected])
http://www.cs.washington.edu/people/acm/tutorials/
The “File System”
• Under UNIX, (almost) everything is a “file”:
–
–
–
–
–
Normal files
Directories
Hardware
Sockets
Pipes
• Things that are not files:
– Users
– Groups
– Processes
Ownership
• Files have two owners
– Every file has exactly one user owner
– Every file has exactly one group owner
• Everyone is a user
– Users are in at least one group
• Processes have owners, too (known as an “id”)
– Every process has exactly one user id
– Every process has at least on group id
• Users and groups are really just numbers with
names
– Every username is mapped to a single numeric “uid”
– Every groupname is mapped to a single numeric “gid”
Who am I?
• Commands that tell you who you are:
– whoami
– id
displays your username
displays your username and groups
• Commands that tell you who others are:
– finger [<name>] displays info for <name>
– id [<username>] displays info for <username>
• Commands that change who you are:
– su <username>
– login
“switch user” to <username>
login as a different user
File Permissions
• Every file has three access levels:
– user
– group
– other
(the user owner of the file)
(the group owner of the file)
(everyone else)
• At each level, there are three access types:
– read
– write
– execute
(looking at the contents)
(altering the contents)
(executing the contents)
Strange Things
• There are three “strange” permissions:
– setuid
– setgid
– text
(run program as user owner)
(run program as group owner)
(stay in swap after executing)
• Directories act differently
–
–
–
–
–
“write”
“execute”
“setuid”
“setgid”
“text”
(creating/deleting files)
(cd-ing to that directory)
(ignored)
(created files have same group owner)
(deletion restricted to user owned files)
Examining Permissions
• A “long” ls listing shows file permissions:
[zanfur@odin zanfur]$ id
uid=8774(zanfur) gid=100(users) groups=100(users),101(mp3)
[zanfur@odin zanfur]$ ls -l
total 524
-rwxr-xr-x
1 zanfur users
512668 Jul 31 00:18 bash
prw-r--r-1 zanfur users
0 Jul 31 00:24 fifo
-rw-r--r-1 zanfur users
0 Jul 31 00:18 file
drwxr-xr-x
2 zanfur users
4096 Jul 31 00:16 normal
drwxrws--T
2 zanfur mp3
4096 Jul 31 00:14 shared
drwxrwxrwt
2 zanfur users
4096 Jul 31 00:14 tmp
drwxrwxr-x
2 zanfur www
4096 Jul 31 00:15 www
[zanfur@odin zanfur]$ _
Permissions Listing Breakdown
Field
Description
Valid Values
?---------
file type
- (normal file), d (directory), p (fifo), b, c, s
-?--------
user read
- (no read permissions), r (read permissions)
--?-------
user write
- (no write permissions), w (write permissions)
---?------
user execute
----?-----
group read
- (no read permissions), r (read permissions)
-----?----
group write
- (no write permissions), w (write permissions)
------?---
group execute
-------?--
other read
- (no read permissions), r (read permissions)
--------?-
other write
- (no write permissions), w (write permissions)
---------?
other execute
- (no exec), x (exec), S (setuid), s (both)
- (no exec), x (exec), S (setgid), s (both)
- (no exec), x (exec), T (text), t (both)
What You Can Do With Permissions
Permission
File
Directory
r (read)
Read a file
List files in …
w (write)
Write a file
x (execute)
Execute a file
(eg shell script)
Create a file in …
Rename a file in …
Delete a file in …
Read a file in …
Write to a file in …
Execute a file/shell
script in …
Changing Ownership
• Changing user ownership:
– The command is “change owner”:
chown <username> <filename>
– but, you can only do it if you are root
– so just copy it instead
• Changing group ownership:
– The command is “change group”:
chgrp <groupname> <filename>
– but, you can only do it if you are in group
<groupname>
– and you must be the user owner of the file
Changing Permissions
• The “change mode” command:
chmod <level><op><permissions>[,…]
<filename>
<level>
string of: u, g, o, a (user, group, other, all)
<op>
one of +, -, = (gets, loses, equals)
<permissions> string of: r, w, x, s, t, u, g, o
(read, write, execute, set-id, text,
same as user, same as group, same as other),
• Examples:
chmod u+rwx,go-w foobar
chmod g=u,+t temp/
chmod u=rwx,g=rwxs,o= shared/
Process Management
• What can you do with it?
–
–
–
–
–
–
–
Start programs in the background
Run more than one program per terminal
Kill bad and/or crashing programs
Suspend programs mid-execution
List all jobs running in a shell
Move foreground jobs to the background
More …
Three States of a Process
• Foreground
– Attached to keyboard
– Outputs to the screen
– Shell waits until the process ends
• Background, running
– Not attached to keyboard
– Might output to the screen
– Shell immediately gives you another prompt
• Background, suspended
– Paused mid-execution
– Can be resumed in background or foreground
Background Processes
• Listing jobs:
– jobs
– ps
– %<job#>
lists background “jobs” and job #’s
lists processes and their process id (“pid”)
expands to the process id of the job
• Stopping foreground jobs
– Press ^Z (Ctrl-Z) in the terminal window
• Starting a process in the background
– Append a & character to the command line
– Examples:
ls –lR > ls-lR.out &
xemacs my_program.cc &
• Resuming a stopped job
– In the foreground:
– In the background:
fg [<pid>]
bg [<pid>]
Killing Processes
• The “kill” command:
kill [-<signal>] <pid>
Send <signal> to process <pid>
• The “killall” command:
killall [-<signal>] <command>
Send <signal> to all processes that start with
<command>
• Useful signals (kill –l for the complete list):
TERM
KILL
HUP
STOP
the default, “terminate”, kills things nicely
will kill anything, but not nicely
“hangup”, used to reload configurations
stops (suspends) a running process
What are environment variables?
• The environment is an array of strings of the form
NAME=VALUE
• Each process (program) has its own copy of the
environment
• Processes started by the shell get a (“deep”) copy of
the shell’s current environment
• Each process can examine and modify its own (and
only its own) environment
• The “meaning” of environment variables is merely a
matter of established conventions
Viewing your shell’s environment
• To view a single environment variable’s value:
echo $<name>
• For example:
echo $HOME
will display
/homes/iws/evgenyr
• To view all environment variables at once:
– tcsh/csh/bash: printenv
– sh/ksh: set
Setting environment variables
• tcsh/csh:
– setenv <name> <value>
– Example: setenv PRINTER ps329
• bash/ksh:
!
– export <name>=<value>
– There is no space on either side of ‘=’!
– Example: export PRINTER=ps329
• sh:
– <name>=<value>; export <name>
Appending to environment variables
• Appending /uns/bin/ to your PATH
– bash/ksh:
export PATH=$PATH:/uns/bin
– tcsh/csh:
setenv PATH=$PATH:/uns/bin
• Prepending is similar:
– bash/ksh:
export INFOPATH=/uns/info:$INFOPATH
– tcsh/csh:
setenv INFOPATH /uns/info:$INFOPATH
Common environment variables
• The “meaning” of environment variables is purely a
matter of convention.
• However, these environment variables are quite
common:
– PATH, INFOPATH, MANPATH, EDITOR,
VISUAL, PAGER, HOME, MAIL, USER
• The man page for a program will usually list the
environment variables the program pays attention to.
Useful shell features
(a preview of upcoming attractions)
•
•
•
•
Aliases
Redirecting input and output
Command substitution
Scripts
Shell as a work-saver: aliases
• Aliases: textual substitution, similar to
C-preprocessor macros.
• Syntax:
– bash/ksh:
alias <text>=’<replacement>’
– tcsh/csh
alias <text> ’<replacement>’
• When you type <text>, the shell “substitutes”
<replacement>
• Typing alias by itself lists all your current aliases
• unalias <text> removes the alias
• Check the csh/tcsh man page for extra features
Alias examples
• Always print postscript files double-sided
– tcsh/csh:
alias lpr ’lpr –Zduplex’
– bash/ksh:
alias lpr=’lpr –Zduplex’
• Collect a few tree-friendly options to enscript:
Line continuation character,
– tcsh/csh:
just like in C/C++!
alias print \
’enscript –2rhB –SDuplex=DuplexNoTumble’
What is input/output redirection?
• Normally, a program’s standard output is displayed
on user’s terminal, and its standard input comes from
the keyboard.
• Redirecting the output of a program means asking the
shell to put the program’s output (stdout [C++’s
cout]) into a file.
• Redirecting the input of a program means asking the
shell to feed a file as the program’s standard input
(stdin [C++’s cin]).
• Note: redirection works with files.
Why redirect program’s output?
• You may want to save output and examine it at your
leisure:
– if the program generates a lot of output
– if the program takes a long time to run
• You may want to present the output to another person
• Having the program write to standard output may
make the program simpler to write
Standard output vs Standard error
• By convention, “normal” output of a program is sent
to standard output (stdout [C++’s cout]), while
debugging or error output is sent to standard error
(stderr [C++’s cerr]).
How to redirect program’s output?
• To redirect just the standard output:
<program> > <FILE>
• To redirect just the standard error:
sh/ksh/bash:
<program> 2> <FILE>
csh/tcsh:
( <program>
> STDOUT ) >& STDERR
• To redirect both standard output and standard error:
csh/tcsh/bash:
<program> >& <FILE>
sh/ksh/bash:
<program> > <FILE> 2>&1
> vs. >>
• Both > and >> will create the output file, if it
doesn’t already exist
• If the file does exist, then:
– Using > to redirect output will overwrite the output
file:
• ls > newlisting
• printenv > my_environment
– Using >> to redirect output will append to the
output file
• cat ch1 ch2 ch3 > book
• cat ch4 ch5 ch6 >> book
Why redirect program’s input?
• To run the program repeatedly with the same (or
similar input)
• Having the program read from standard input may
make the program simpler to write.
How to redirect program’s input?
• Simple!
<program> < <FILE>
• Example
sort < my_grades.txt
head < really_long_book.txt
Piping
• Piping is connecting programs together by using the
output of one program as the input to the next.
• Syntax:
<program1> | <program2> | … | <programN>
• A simple example (view a sorted file-listing a page at a
time):
ls | sort | less
• Note: piping deals with the input/output of programs
(that is, stdin, stdout, stderr)
Why piping?
• Because “the whole is bigger than the sum of its
parts.
• By combining Unix utilities in a pipeline, you can
build tools “on-the-fly” as you need them.
Piping examples
• How many .c files are in this directory?
ls *.c | wc –l
• What files were modified most recently?
ls –t | head
• What processes am I running?
ps auxw | grep <mylogin>
• Make an alias for the above, for other Linux boxen too:
alias myps=’ ps auxw | grep `id –un`’
The cat utility
• Especially useful: cat
– When used with pipes, “copies” stdin to stdout
• cat can be used to redirect out of a pipe!
• Example: make a file containing currently running processes
ps aux | grep evgenyr |
cat > my_processes
– When used with a file, “copies” the file into stdout
• cat can be used to place a file’s contents into a pipe!
• Example: list all the items in a list in alphabetical order
cat my_grocery_list.txt | sort | uniq
Command substitution
(aka “what’s up with the backquotes?”)
• Command substitution means that the shell
substitutes a command’s output for the
command itself.
• To get the shell to perform command
substitution, enclose the command in
backquotes (`)
Shell as a work-saver: scripts
• Instead of typing the same series of commands over and
over again, put them in a file and have the shell execute
the (commands in the) file!
• The file must have execute permission
• The first line of the file should be:
#! <YOURSHELL> (e.g., /bin/bash)
• There must be no space (or any other character) between
‘#’ and ‘!’.
• Whether to put space after ‘!’ is a matter of style
• Shell also has control flow, tests, etc. The shell tutorial
on the ACM web page goes into much more detail.
Intermezzo: quoting
• Problem: some characters are special to the shell (*,
?), but you would like to pass those characters to the
programs the shell runs as-is.
• So you quote the character(s):
– A single character: by prefixing it with a \ (backslash)
– A string of characters: by enclosing them in
• Single quotes: no characters are special. None
• Double quotes: some characters (notably $ and `) are still special.
You’ll need to prefix those with a backslash to pass to the program
as-is.
Quoting examples
• List all the files in the current directory:
ls *
• List the file or directory called ‘*’ (or complain if it’s
not there):
ls \*
• Print $HOME:
echo ’$HOME’
• Print the path to your home directory:
echo ”$HOME”
• Print `id –un`:
echo ’`id –un`’
• Print your login:
echo ”`id –un`”
Next stop: utilities
• (No, not electricity, water, and sewer)
• diff and patch
• grep
• find and xargs
diff and patch
• You have two versions of a file; how do you see
what’s changed between the two versions?
– diff shows the differences between two files; you’ll
want to use the –u or –c option to diff to produce
output in human-friendly format:
diff –u my_file my_file.orig | less
– patch applies differences to a file
diff –u my_file my_file.orig > my_patch
patch my_file < mypatch
my_file is now the same as my_file.orig
grep – search for patterns
• grep searches for patterns in texts:
grep <string> <FILE>
• grep uses regular expressions to describe the
string(s) to search for
• In a regular expression, most characters are
treated as-is, but a few are magic
– Ordinary characters just represent themselves
– Magic characters do special things
grep’s magic characters
• A few different kinds of magic characters:
– Some characters “anchor” (parts of) a regular
expressions to specific places in the string:
^ - The beginning of a line
$ - The end of a line
– A dot (.) matches “any one character whasoever”
(except for newline)
– [ ] form a character class:
• [aeiou] – any single vowel
Here, ‘^’ • [a-zA-z0-9] – Any single letter or digit
means “not”
• [^0-9] – Any single character that isn’t a digit
Even more of grep’s magic characters
• Quantifiers say how many times the
preceeding “thing” should be repeated:
– * means “zero or more times”
– ? Means “zero or one time”
Frequently used grep options
-i : do case-insensitive search
-n : print line numbers
-v : print lines that do not match
-l : only list the files from which output would
have been printed
grep examples
• Print all non-blank lines:
grep –v ’^$’ my_file.txt
• Print all the lines on which my_function is
called (or declared):
grep –n ’my_function *(’ my_code.c
• Show all the xterms I am running:
ps auxw | grep ”`id –un` .*xterm”
find
find <PATHS> <OPTIONS> <EXPRESSION>
• Traverse the tree(s) rooted at <PATHs> , and for each
“thing” found, evaluate the <EXPRESSION> until the
result of <EXPRESSION> is known.
• The evaluation of <EXPRESSION> “short-circuits”,
just like expressions in C/C++.
– false && some_expression can never be true
• Options apply to the entire find command, not the
individual expression
Components of a find expression
• An expression is zero or more primaries connected by
operators
• Two kinds of primaries:
– Tests: just return true or false
– Actions: do something, in addition to returning true or false
– Operators work just as they do in C/C++.
• ! / -not
• -a / -and
• -o / -or
• , (comma)
– Parentheses are used for grouping, just as in C/C++.
find examples
• Print all the *.c* and *.h* files in and below the current directory
find . –name ’*.[ch]*’ –a –print
• Show line numbers myfunction calls in the above *.c* files
find . –name ’*.c*’ | xargs grep –n ’myfunction.*(’
• Change permissions on files under your home directory so they’re
inaccessible to everyone but you (except for files in the www directory)
cd; find . –path ”./www*” –prune –o –exec
chmod go-rwx {} \;
• How big are my *.c* files?
expr `find –name ’*.c*’ –printf
”%k + ”` 0
• Read the find manual (info find) for more examples
xargs: combine arguments
xargs <OPTION> <COMMAND> <INITIAL_ARGS>
• Feeds standard input as arguments to <COMMAND>
• Often used with find
(find … | xargs grep)
• But it doesn’t have to be used with find:
cat filelist | xargs cat {} > concatenated
Finding more information – at CSE
•
•
•
•
•
Use info and man
Peruse the “see also” section of the man pages
Peruse info’s i (search index) command
The uw-cs.lab-help group
Look at your .login, .cshrc, etc. to see
how support has set things up
Finding more info – on the web
•
•
•
•
http://www.faqs.org (comp.unix questions FAQ)
http://www.google.com
http://www.deja.com
The support web page:
http://www.cs.washington.edu/lab/
about/ugradcomputing.html
• (Shameless plug) the ACM tutorials page:
http://www.cs.washington.edu/people/acm/tutorials/