Transcript Unix - Jazi Eko Istiyanto

Chapter 4: UNIX File Processing
Input and Output
Extracting Information from Files
Section A: Objectives
After studying this lesson, you should be
able to understand and use:
– Redirection
– Pipes
– Wildcards and Regular Expressions
– Commands:
find, sort, cat
grep
Processing Files
• When performing commands, UNIX
processes data, it receives input from the
standard input device (the keyboard) and
then sends output to the standard output
device (the screen).
• System administrators and programmers
refer to standard input as stdin
• They refer to standard output as stdout
• The third standard output is called standard
error, or stderr (also typically the screen),
where error messages are displayed
Using Input and Error Redirection
• File Redirection operators
> - redirect output, send it to a file
>> - redirect output, appends it to a file
< - redirects input, retrieves input from a
file
Redirecting Outputs
• $ ls > foo
• Send the directory list to the file named
“foo”
• $ cat foo
• Show contents of “foo”
• $ date >> foo
• Append the date to the file name “foo”
Redirecting Outputs (cont)
$ ls -l > bar
Send the long directory list to “bar”
$ cat foo bar > foobar
Combine “foo” and “bar” into a new file
name “foobar”
$ cat foobar
Display contents of new file
Redirecting Inputs
$ who > people
Put the list of users currently on the system
into the file “people”
$ sort < people
Sort by sending “people” as input to sort
$ sort < people > speople
Send sorted list to new file “speople”
Using Pipes
• The pipe operator (|) redirects the output of one
command to the input of another command
• The pipe operator is used in the following
manner:
first_command | second_command
• The pipe operator connects the output of the first
command with the input of the second command
• The pipe operator can connect several
commands on the same command line, in the
following manner:
first_command | second_command | third_command ...
Pipes Examples
$ who > people
$ sort < people
Is there a shortcut? Yes, using pipes.
$ who | sort List of users on the system
$ who | wc –l How many users are on the
system?
Wildcards
The ‘*’ and ‘?’ characters are wildcards,
which match against zero or more
characters in a file or directory name.
‘*’ matches any # of characters (0 or more).
‘?’ matches exactly one character
[ ] and - : character range, a way to specify a
sub range of characters to match.
Wildcard Examples
$ ls foo* Lists all files starting with “foo”
$ ls *.c
Lists all files ending with “.c”
$ ls f.?
Lists all files named “f” plus an
extra character, such as “f.1” or “f.c”
$ ls a* d* f* List all files starting with a, d, f
$ ls [adf]* same
$ ls [a-z]* List files starting with any letter
$ ls [a-z] * List all one letter files, then all
files
Wildcard Examples (cont)
$ ls *unix*
list all files/subdirectories which contain
the word “unix”
$ ls *s*
the following example files will match this
pattern.
grades.txt, prog1.s, summary.doc
$ ls *[0-9._]*
files having 0-9 or . or _
Wildcard Examples (cont)
$ rm *
Delete ALL files
$ rm ???
Delete all filenames with 3 characters
$ cp *.txt ..
Copy all files with a “.txt” extension to the
parent directory
Wildcard Examples (cont)
• Example: to list any file that start with a
lowercase letter range from a to z as
specified
ls -l [a-z]*
• Which of the following files will this
pattern match.
ls hw[0-9]*
For example: hw1, hw3.c, hw50.txt, hw.
Finding Files
• The find command searches for files that have a
specified name
• The command has the form:
find pathname - name filename
• The find command prohibits you from searching
where you do not have system-level permissions
• Although Linux does not use it, other UNIX
versions require the -print option after the
filename to display the names of fields the find
command locates
Finding Files Examples
• find /usr -name Chapter1 -type f -print
search through the "/usr" directory for the file
named "Chapter1".
• find /usr -name "Chapter*" -type f -print
search through the "/usr" directory for all files that
begin with the letters "Chapter". The filename can
end with any other combination of characters.
This will match filenames such as "Chapter",
"Chapter1", "Chapter1.bad", "Chapter_in_life".
Using the Sort Command
• Use the sort command to sort a file’s
contents alphabetically or numerically
• Here is an example of its use:
sort file1 > file2
• Here are more complex examples:
ls –s | sort –n list files in sorted order
ls –s | sort –nr reverse sorted order
ls –s | sort –nr | head -5 show 5 largest files
grep
• grep command is used to match a
pattern in one or more files, and
displays the matching output on
standard output.
• The format of grep command is:
grep [options] regexp [files]
• grep means global regular expression
print.
grep
• Example: grep luo /etc/passwd: to search
a user in the system who called 'luo' as
the first name or the last name.
• Quotes (single or double) are needed if
regular expressions are to be used
grep olga*
will stall
grep ‘olga*’ will work fine
• Wildcards in filename can be used with
grep as usual
grep ‘olga*’ *.txt
Regular Expressions
• b* ( , b, bb, bbb, bbbb, ….)
• .
matches any single character
• ^ beginning of the line
grep ‘^taylor’ file1
find pattern ^taylor
• $ end of the line
grep ‘:$’ file1 entries that ends with :
grep ‘100\$’ f1 searches for 100$
grep ‘100$’ f1
searches for 100 at the end
Regular Expressions
• [xy]
any single character in the set
• [^xy]
any single character not in the set
grep ‘^taylor[a-z]’ file1
starts with taylor and
contains a lower case letter afterwards
grep ‘^taylor[^a-z]’ file1
starts with taylor
and no lower case letter afterwards
grep ‘z*’ f1 zero or more occurrences of z
(everything in f1)
grep ‘zz*’ f1 one or more occurrences of z (does
not return lines without z)
Grep flags
• -c: print a count of matched lines only
• -l: list filenames that contain the specified
pattern
• -i: Ignore the case of the letters in the pattern
• -n: Including line numbers
• -b: precede each line with its block number
• -h: print matched lines but not
filenames(inverse of -l)
• -s: Suppress error messages for nonexistent of
unreadable
• -v: Print all lines that don't match regexp
grep
• Example 1: to indicate how many users on
the system has a last name or first name
as "marylin" ignoring the upper or lower
cases.
grep -ci marylin /etc/passwd
• Example2: to indicate how may casesensitive matches are in the file before
any of them are displayed on the screen
grep -c lu /etc/passwd
• Example3: to number the output lines.
grep -n lu /etc/passwd
Chapter Summary
• Output from a command may be redirected
from stdout to a disk file
• Input to a command may be redirected from
stdin to a disk file
• Use the sort command to sort a file’s contents
alphabetically or numerically
• A pipe directs the standard output of one Unix
command into the standard input of another
Unix command.
• A wildcard is a tool that allow you to specify a
group of filenames or directories.
• Useful commands such as find, grep, cat,