Transcript sarwar_32062X_lctr12

Chapter 12
Redirection and
Piping
Objectives
• To describe the notion of standard files-standard input,
standard output and standard error files-and file
descriptors
• To describe input and output redirection for standard files
• To discuss the concept of error redirection and appanding
to a file
• To explain the concept of pipes in UNIX
• To describe how powerful operations can be performed
by combining pipes , file descriptors and redirection
operators
• To discuss error redireaction in C shell
• To explain the concept of FIFOs and their command line
use
• To cover the commands and primitives
&,|,<,>,>>, cat, diff, grep, lp, mkfifo, more, pr, sort, stderr, stdin, stdout, tee, tr, uniq, wc
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Introduction
• All commands perform at least one of the following:
• Input
• Output
• Processing
• Standard files for commands
• Standard Input (stdin)
• Standard Output (stdout)
• Standard Error (stderr)
• In UNIX these three files are opened automatically
by the kernel for every command
• The input ,output and errors of a command can be
redirected to other files by using file redirection
facilities in UNIX
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Standard Files and File
Descriptors
• By default
– stdin is associated with keyboard
– stdout is associated with display screen
– stderr is associated with display screen
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Standard Files
• By default
– stdin is associated with keyboard
– stdout is associated with display screen
– stderr is associated with display screen
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Input Redirection
• Use ‘<‘ for input redirection
command < input-file
• ‘command’ takes input from ‘input-file’
instead of the keyboard
• Example: cat < tempfile
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Input Redirection
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Output Redirection
• Use ‘>‘ for output redirection
command > output-file
• Send output of ‘command’ to the file
‘output-file’ instead of the monitor
screen
• Example: cat > newfile
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Output Redirection
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Output Redirection (contd.)
• In a network environment, use the
following command to sort on the ‘server’
machine the file called datafile residing on
your machine:
$ rsh server sort < datafile
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Combining Input and Output
Redirection
• command < input-file > output-file
• command > output-file < input-file
• Input to ‘command’ comes from the ‘input-file’ instead of the
keyboard, and the output of the command goes to the ‘outputfile’ instead of the display screen
• Example: cat < lab1 > lab2
The cat command takes input from the lab1 file and sends its
output to the lab2 file
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
I/O Redirection with File
Descriptors
• File descriptor: a small integer that the UNIX
kernel attaches with every open file
standard input (sdin) — 0
standard output (stout) — 1
standard error (sderr) — 2
• By making use of file descriptors, standard
output and standard input can be redirected,
using, 0< and 1> respectively
• Example: $ grep “John” 0< tempfile
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting Standard Error
• command 2> error-file
• Error message generated by ‘command’
and sent to stderr are redirected to
error-file
• Example:
– $ ls –l foo 2> error.log
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting Standard Error
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting stdout and stderr in One
Command
• Redirect stdout and stderr to the same file by using the
descriptors with > symbol
– $ cat lab1 lab2 lab3 1> cat.output 2>cat.errors
• Using string 2>&1 to tell the command shell to make descriptor
2 a duplicate of descriptor 1, resulting in error messages going
to the same place the output is goes to
– $cat lab1 lab2 lab3 1> cat.output.errors 2>&1
• Using string 1>&2 to tell the command shell to make descriptor
1 a duplicate of descriptor 2, resulting in output going to the
same place as the error messages
– $ lab1 lab2 lab3 2> cat.output.errors 1>&2
• The evaluation for command line content for file redirection is
from left to right
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting stdout and stderr in One
Command
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting stdout and stderr in One
Command
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting stdin,stdout and stderr in
One Command
• command 0<input-file 1>output-file 2> error-file
• Input to ‘command’ comes from ‘input-file’ instead of
keyboard, output of ‘command’ goes to ‘output-file’
instead of the display screen and error messages
generated by ‘command’ are sent to ‘error-file’
• The file descriptors 0 and 1 are not required as they
are the default values
$ sort 0< students 1> stuents.sorted 2> sort.error
$ sort 2>sort.error 0< students 1> students.sorted
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting
stdin,
stdout and
stderr in
One
Command
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirecting without Overwriting File
Contents (Appending)
• By default, output and error messages overwrite the
contents of the destination file
• Replace ‘>’ with ‘>>’ operator to append
output or error messages generated by a command,
at the end of a file
• The default operator with >> is 1 but file descriptor 2
can be used to append errors to a file
• Examples:
$ ls –l 1>> output.dat 2>> error.log
$ cat
memo
letter >>stuff 2 > error.log
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
UNIX Pipes (‘|’)
• Used to connect the stdout of one command to the stdin of
another command
• Used for communication between processes related to each
other through a common ancestor process
• A pipe is a main memory buffer
• command1 | command2 | … | commandN
Standard output of ‘command1’ is connected to stdin of
command2,…,stdout of ‘command N-1’ is connected to stdin of
‘command N’
• Filters: a special class of UNIX commands that take input from
stdin process it and send it to stdout
• I/O redirection and pipes can be used in a single command
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
UNIX Pipes (‘|’)
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
UNIX Pipes (‘|’)
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirection and Piping combined
• command1| tee file1…fileN|command2
• Standard output of ‘command1’ is connected
to ‘stdin’ of tee, and tee sends its input to files
‘file1’ through ‘fileN’ and as stdin on
‘command2’
• Example:
$ cat names stuents | grep “John Doe” | tee
file1 file2 | wc –l
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Redirection and Piping combined
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Error Redirection in the C Shell
•
•
•
•
•
•
•
The input, output and append operators(<, > ,>>) work in
the C shell as they do in any other shells
The operator for error redirection is >& in the C Shell.
command >& file
Redirects the stdout and stderr of ‘command’ to ‘file’
Examples:
% ls –l foo >& error.log
The C shell does not have an operator for redirecting stderr
alone
To attach stdout and stderr of a command to different files,
parenthesize the command so it is executed in a subshell
% (find ~ -name foo –print >foo.paths) >&
output.error.log
Use >>& operator to redirect and append stdout and stderr
to a file
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Error Redirection in the C Shell (contd.)
• Allows stdout and stderr of a command to be
attached to stdin of another command with |&
operator
• command1 |& command2
– Redirect stdout and stderr of ‘command1’ to ‘command2’
• Example:
% cat file1 file2 |& grep “John Doe”
% cat file1 file2 |& grep “John Doe”|sort|& wc –l
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Error Redirection in the C Shell (contd.)
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Recap of
I/O and
Error
Redirection
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Recap of
I/O and
Error
Redirection
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
FIFOs
• FIFOs (also called named pipes) can be used for
communication between two processes executing on
a system
• Can be used between independently executed
programs on a system (unlike pipes)
• A FIFO is created on disk and has a name like a
filename
• Like files have to be created and opened before they
can be used for communication between processes
•
To create a FIFO use the mknod() system call or
mkfifo() library call
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
FIFOs
• mkfifo[option] file-list
Purpose: Create FIFOs with Pathnames given in the
‘file-list’
Output: FIFOs for pathnames given in the ‘file-list’
Commonly used options/features:
-m mode Set access permissions for newly created
FIFOs to ‘mode’; the access permissions
are specified in ‘mode’ as they are with
the chmod command.666 means read
write permissions for everyone and
execute permissions for nobody
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
FIFOs
• To enable two commands to
communicate via FIFO, we run the first
one in the background so that we can
run the other command
• cmd1 < myffo1 &
cmd2 infile| tee myfifo1 |cmd3
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.