Transcript PIPE

[Unix Programming]
Interprocess Communication - PIPE
Young-Ju, Han
Email: [email protected]
Contents




Pipes
Using pipe a single process
Using pipe a multiple processes
Using pipe in the shell
2007 UNIX Programming
Overview
 different forms of IPC









pipes(half duplex)
FIFOs(named pipes)
stream pipes(full duplex)
named stream pipes
message queues
semaphores
shared memory
sockets
streams
on the same host
on different host
2007 UNIX Programming
Pipes
 only form of IPC on all UNIX implementations
 The oldest form of UNIX IPC
 most commonly used form of IPC
 limitations
 half-duplex
stream pipes
 one-way communication channel
 used only between processes
that have a common ancestor
(related processes)
FIFOs & named stream pipes
2007 UNIX Programming
Pipes

Pipes
 connects the standard output of one program
to the standard input of another program
I/O Redirections and Pipelines
stdin
myprog1
myprog2
stdout
stdin
file1
stderr
stderr
Stdout
file2
$ (myprog1 | myprog2 ) < file1 > file2
2007 UNIX Programming
Pipes

Pipes at command level
(Example) I/O Redirection and Pipelines Method
1. I/O Redirection Method
$ ls /home/kc > tempfile
$ wc -l < tempfile
19
$ rm tempfile
2. Pipeline Method
$ ls /home/kc | wc -l
19
2007 UNIX Programming
Pipes

pipes at command level
Weak Point of I/O Redirection Method
1. Cannot create temporary files if you don’t have permission
2. May have a invalid result because of temporary file
3. Easy to forget to remove temporary file
(Example) cd /bin
1. I/O Redirection Method (Failure)
$ ls > tempfile
tempfile: permission denied
2. Pipeline Method ( Success)
$ ls | wc -l
777
2007 UNIX Programming
pipes
 %who | sort
who
pipe
write pointer of
another process
sort
read pointer of
one process
2007 UNIX Programming
Pipes
 Create pipes
 pipe() system call
 Data transmitting




data is written into pipes using the write() system call
data is read from a pipe using the read() system call
automatic blocking when full or empty
FIFO basis, so lseek() don’t work
 Types of pipes
 (unnamed) pipes
 named pipes
2007 UNIX Programming
Pipes
 Programming with pipes
 create a pipe
#include <unistd.h>
int
pipe (int
fd[2]);
fd[0] : read only open
fd[1] : write only open
0 : ok
-1 : error
more open than per-user process limit(EMFILE)
kernel’s open file table overflow(ENFILE)
2007 UNIX Programming
Pipes
 ex) pipe on a single process
#include <stdio.h>
#include <unistd.h>
#define MSGSIZE 1024
char *msg1= “hello, world #1”;
char *msg2= “hello, world #2”;
char *msg3= “hello, world #3”;
$a.out
hello, world #1
hello, world #2
hello, world #3
int main(){
char inbuf[MSGSIZE];
int fd[2], j, ret;
if (pipe(fd) == -1) {perror(“pipe call”); exit(1);}
write(fd[1], msg1, strlen(msg1));
write(fd[1], msg2, strlen(msg2));
write(fd[1], msg3, strlen(msg3));
for (j = 0; j < 3; j++) {
ret = read(fd[0], inbuf, MSGSIZE);
inbuf[ret]=0;
printf(“%s\n”, inbuf);
}
}
2007 UNIX Programming
Pipes
 ex) pipe on a single process
process
fd[0]
fd[1]
process
or
fd[0]
fd[1]
pipe
kernel
read in the order in which they were written
first in/first out(FIFO) communication channel
=> Since lseek don’t work on a pipe
2007 UNIX Programming
Pipes
 kernel’s pipe buffer size
 constant PIPE_BUF
 rule
 if a write overfill pipe, then blocked until
reading
 if a read when empty, then blocked until writing
 if a write > pipe, then block after write
 if a read > pipe, then return after read
pipe_size = fpathconf( p[0],
_PC_PIPE_BUF );
2007 UNIX Programming
Pipes
 rules (when one end of a pipe is closed)
 If we read from a pipe whose write end has been closed,
 after all the data has been read, read return 0 to
indicate an end of file
 End of file is not generated until there are no
more writers for the pipe
 If we write to a pipe whose read end has been closed,
 the signal SIGPIPE is generated

Default action : terminate
2007 UNIX Programming
Pipes
 ex) pipe from child to parent
$a.out
hello,
hello,
int fd[2], j;
hello,
pid_t pid;
=> 무한
if (pipe(fd) == -1) {perror(“pipe call”); exit(1);}
int main(){
char inbuf[MSGSIZE];
world #1
world #2
world #3
대기
switch(pid = fork()) {
case -1 :
perror(“fork error”); exit(1);
case 0
write(fd[1], msg1, strlen(msg1));
:
write(fd[1], msg2, strlen(msg1));
write(fd[1], msg3, strlen(msg1)); break;
default :
for (j = 0; j < 3; j++) {
read(fd[0], inbuf, MSGSIZE);
inbuf[ret] = 0;
printf(“%s\n”, inbuf);
}
wait(NULL);
}
}
2007 UNIX Programming
무한대기
Pipes
 ex) pipe from child to parent
parent
fd[0]
child
fork
fd[1]
fd[0]
fd[1]
write()
read()
pipe
kernel
2007 UNIX Programming
Pipes
 ex) pipe from child to parent (recommended)
switch(pid = fork()) {
case -1 : perror(“fork error”); exit(1);
case 0
: close(fd[0]);
write(fd[1], msg1, MSGSIZE);
write(fd[1], msg2, MSGSIZE);
write(fd[1], msg3, MSGSIZE); break;
default : close(fd[1]);
for (j = 0; j < 3; j++) {
read(fd[0], inbuf, MSGSIZE);
printf(“%s\n”, inbuf);
}
wait(NULL);
}
2007 UNIX Programming
Pipes
 pipe from child to parent (recommended)
parent
fork
child
fd[0]
fd[1]
write()
read()
pipe
kernel
2007 UNIX Programming
Pipes
 Non-blocking reads and writes
 to use fstat
 If st_size > 0 , and then read() from pipe
 st_size : number of characters currently in the
pipe
 problem : if several processes are reading the
pipe, another process could read from a pipe in
the gap between fstat and read
 to use fcntl ( recommend)
 O_NONBLOCK flag

errno -1 : EAGAIN
#include <fcntl.h>
if( fcntl(fd, F_SETFL, O_NONBLOCK) == -1 )
perror(“fcntl”);
2007 UNIX Programming
Pipes
 Non-blocking read and write
#include <fcntl.h>
#include <errno.h>
#define MSGSIZE 6
int parent (int *);
int child (int *);
char *msg1 = "hello";
char *msg2 = "bye!!";
void fatal(char* msg) { fprintf(stderr,”%s\n”,msg);exit(1);}
int main(){
int pfd[2];
/* 파이프를 개방한다 */
if(pipe (pfd) == -1) fatal ("pipe call");
/* p[0]의 O_NONBLOCK 플래그를 1로 설정한다 */
if (fcntl (pfd[0], F_SETFL, O_NONBLOCK) == -1)
fatal ("fcntl call");
switch(fork()){
case -1: fatal("fork call");
/* 오류 */
case 0: child(pfd);
/* 자식 */
default: parent (pfd);
/* 부모 */
} }
2007 UNIX Programming
Pipes
 Non-blocking read and write
int parent (int p[2]) {
/* 부모의 코드 */
int nread; char buf[MSGSIZE];
close (p[1]);
for(;;) {
switch (nread = read(p[0], buf, MSGSIZE)){
case -1:
/* 파이프에 아무것도 없는지 검사한다. */
if (errno == EAGAIN){
printf ("(pipe empty)\n");
sleep (1);
break;
}
else { perror("read call"); exit(1);}
case 0:
/* 파이프가 닫혔음. */
printf ("End of conversation\n");
exit (0);
default:
printf ("MSG=%s\n", buf);
}
} }
2007 UNIX Programming
Pipes
 Non-blocking read and write
int child(int p[2])
{
int count;
close (p[0]);
for (count= 0; count < 3; count++)
{
write (p[1], msg1, MSGSIZE);
sleep(3);
}
/* 마지막 메시지를 보낸다 */
write (p[1], msg2, MSGSIZE);
exit (0);
$a.out
(pipe empty)
MSG=hello
(pipe empty)
(pipe empty)
(pipe empty)
MSG=hello
(pipe empty)
(pipe empty)
(pipe empty)
MSG=hello
(pipe empty)
(pipe empty)
MSG=bye!!
(pipe empty)
End of conversation
}
2007 UNIX Programming
Pipes
 Using select to handle multiple pipes
 Server & client Applications
parent
fork
fd2[0]
pipe
fd1[0]
fd3[0]
read()
fork
fork
write()
fd1[1]
Child 1
fd2[1]
Child 2
2007 UNIX Programming
fd3[1]
Child 3
Pipes
 Using select to handle multiple pipes
#include <sys/time.h>
#include <sys/time.h>
int select (int nfds, fd_set *readfds, fd_set *writefds,
fd_set* errorfds, struct timeval *timeout);
struct timeval {
long tv_sec; /*second */
long tv_usec; /*microseconds */
}
0 : timeout, -1 : error
>0 : the number of “interesting” FDs

nfds : specifies the range of file descriptors to be tested.





0 ~ nfds-1
readfds : the specified file descriptors is ready for reading
writefds : FDs is ready for writing
errorfds : FDs is ready for error checking
Timeout



Timeout = NULL : block forever or until something of
interest turns up
timeout.tv_sec = 0 && timeout.tv_usec = 0: returns
immediately without blocking
Timeout.tv_sec != 0 || timeout.tv_usec != 0 : return
after that number of tv_sec or tv_usec specified if there
is no FDs activity
2007 UNIX Programming
Pipes
 Macro for manipulating sets of file descriptor (bit mask)
#include <sys/time.h>
/* initialize the mask pointed to by fdset */
void FD_ZERO(fd_set *fdset);
/* set the bit, fd, in the mask pointed to by fdset */
void FD_SET(int fd, fd_set* fdset);
/* is the bit, fd, set in the mask pointed to by fdset */
void FD_ISSET(int fd, fd_set* fdset);
/* turn off the bit, fd, in the mask pointed to by fdset */
void FD_CLR(int fd, fd_set *fdset);
2007 UNIX Programming
Pipes
 Using select to handle multiple regular files
#include <sys/time.h>
#include <sys/types.h>
#include <fcntl.h>
.
.
.
int fd1, fd2;
fd_set readset;
fd1 = open(“file1”, O_RDONLY);
fd2 = open(“file2”, O_RDONLY);
FD_ZERO(&readset);
FD_SET(fd1, &readset);
FD_SET(fd2, &readset);
select(fd2+1,&readset,NULL,NULL,NULL)
Not recommended
switch(select(5,&readset,NULL,NULL,NULL))
{
/* logic */
}
2007 UNIX Programming
pipes
 Examples (Using select to handle multiple pipes)
#include <sys/time.h>
#include <sys/wait.h>
#include <stdio.h>
#include <unistd.h>
#define MSGSIZE
6
char *msg1 = "hello"; char *msg2 = "bye!!";
void parent(int a[][]); int child(int b[]);
void fatal(char* msg){ fprintf(stderr,"%s\n",msg);exit(1);}
int main(){
int pip[3][2]; int i;
for (i = 0; i < 3; i++) {
if (pipe(pip[i]) == -1)
fatal("pipe call");
switch (fork()){
case -1:
/* 오류 */
fatal("fork call");
case 0:
/* 자식 */
child(pip[i]);
}
}
parent(pip); /* 부모 */
exit(0); }
2007 UNIX Programming
pipes
 Examples (parent process)
void parent(int p[3][2]){
char buf[MSGSIZE], ch; fd_set set, master; int i;
for (i = 0; i < 3; i++) close(p[i][1]);
FD_ZERO(&master); FD_SET(0, &master);
for (i = 0; i <3; i++) FD_SET(p[i][0], &master);
while(set=master, select (p[2][0]+1, &set, NULL, NULL, NULL) > 0){
if(FD_ISSET(0, &set)){ /* standard input check */
printf ("From standard input...");
read (0, &ch, 1);
printf("%c\n", ch);
}
for (i = 0; i < 3; i++) {
if(FD_ISSET(p[i][0], &set)) {
if(read(p[i][0], buf, MSGSIZE) > 0) {
printf ("Message from child%d\n", i);
printf ("MSG=%s\n",buf);
}
}
}
if(waitpid (-1, NULL,WNOHANG) == -1) return;
}
}
2007 UNIX Programming
pipes
 Examples (child process)
int child(int p[2]){
int count;
close(p[0]);
for (count =
{
write
sleep
}
write (p[1],
exit (0);
}
0; count < 2; count++)
(p[1], msg1, MSGSIZE);
(getpid()%4);
msg2, MSGSIZE);
$a.out
Message from child0
MSG=hello
Message from child1
MSG=hello
Message from child2
MSG=hello
Message from child2
MSG=hello
Message from child2
MSG=bye!!
d
From standard input...d
From standard input...
Message from child0
MSG=hello
.
.
.
2007 UNIX Programming
Sell pipe implementation
 Pipes and the exec system call
%ls –l | wc
Shell
/bin/sh
fd[1]
child
ls
parent
fd[0]
pipe
child
wc
/* stdin에서 command read */
/* command parsing */
/* 2개의 command이고 둘이 pipe로 연결되어 있다면*/
int fd[2], c1;
pipe(fd);
c1 = fork ();
If(!c1) {
/*first child*/
close( fd[0] );
close(1); dup2( fd[1],1 ); close( fd[1]);
execlp(/*ls –l first command */);
}
c2= fork ();
If(!c2) {
/*second child*/
close( fd[1] );
close(0); dup2( fd[0],0 ); close( fd[0]);
execlp(/*wc second command */);
}
close( fd[0] ); close( fd[1] );
2007 UNIX Programming
Sell pipe implementation
 dup(), dup2()
#include <unistd.h>
int dup(int fd);
int dup2( int fd, int fd2);
 사용중인 파일 디스크립터의 복사본을 만듦
 dup()는 새 파일 디스크립터가 할당
 dup2()는 fd2를 사용
 리턴 값
 성공하면 복사된 새 파일 디스크립터, 실패하면 -1
 dup()는 할당 가능한 가장 작은 번호를 리턴
 dup2()는 fd2를 리턴
 dup2()는 복사본을 만들기 전에 기존의 fd2를
close함
2007 UNIX Programming
Sell pipe implementation
 Another method: The Join program
parent
wait()
write()
(stdout)
P[1]
pipe
P[0]
read()
(stdin)
Child 2
Child 1
(com1)
(com2)
2007 UNIX Programming
Sell pipe implementation
 Pseudo-code form of the Join program
Process forks, parent waits for child
And child continues
Child create a pipe
Child then forks
In child created by second fork (child 2):
standard output is coupled to
write end of pipe using dup2
excess file descriptors are closed
program described by ‘com1’ is exec’ed
In child of first fork (child 1):
standard input is coupled to
read end of pipe using dup2
excess file descriptors are closed
program described by ‘com2’ is exec’ed
2007 UNIX Programming
Sell pipe implementation
 The Join program
int join(char* com1[], char* com2[]){
int p[2], status;
/* 명령을 수행할 자식을 생성한다. */
switch (fork()){
case -1:
/* 오류 */
fatal ("1st fork call in join");
case 0:
/* 자식 */
break;
default:
/* 부모 */
wait(&status);
return (status);
}
/* 루틴의 나머지 부분으로 자식에 의해 수행된다. */
/* 파이프를 만든다. */
if (pipe(p) == -1)
fatal ("pipe call in join");
2007 UNIX Programming
Sell pipe implementation
 The Join program
switch (fork()){/* 다른 프로세스를 생성한다. */
case -1:
/* 오류 */
fatal ("2nd fork call in join");
case 0:
/* 쓰는 프로세스 */
dup2 (p[1],1); /* 표준 출력이 파이프로 가게 한다. */
close (p[0]); /* 화일 기술자를 절약한다. */
close (p[1]);
execvp (com1[0], com1);
/* execvp가 복귀하면, 오류가 발생한 것임. */
fatal("1st execvp call in join");
default:
/* 읽는 프로세스 */
dup2(p[0], 0); /* 표준 입력이 파이프로부터 오게 한다 */
close (p[0]);
close (p[1]);
execvp (com2[0], com2);
fatal ("2nd execvp call in join");
}}
2007 UNIX Programming
Sell pipe implementation
 The Join program
앞서 join routine은 다음 프로그램을 이용하여 호출될 수 있다:
#include <stdio.h>
main()
{
char *one[4] = {"ls", "-l", "/usr/lib", NULL};
char *two[3] = {"grep", "∧d", NULL};
int ret;
ret = join (one, two);
printf ("join returned %d\n", ret);
exit (0);
}
2007 UNIX Programming