Transcript PIPE
[Unix Programming]
Interprocess Communication - PIPE
Young-Ju, Han
Email: [email protected]
Contents
Pipes
Using pipe a single process
Using pipe a multiple processes
Using pipe in the shell
2007 UNIX Programming
Overview
different forms of IPC
pipes(half duplex)
FIFOs(named pipes)
stream pipes(full duplex)
named stream pipes
message queues
semaphores
shared memory
sockets
streams
on the same host
on different host
2007 UNIX Programming
Pipes
only form of IPC on all UNIX implementations
The oldest form of UNIX IPC
most commonly used form of IPC
limitations
half-duplex
stream pipes
one-way communication channel
used only between processes
that have a common ancestor
(related processes)
FIFOs & named stream pipes
2007 UNIX Programming
Pipes
Pipes
connects the standard output of one program
to the standard input of another program
I/O Redirections and Pipelines
stdin
myprog1
myprog2
stdout
stdin
file1
stderr
stderr
Stdout
file2
$ (myprog1 | myprog2 ) < file1 > file2
2007 UNIX Programming
Pipes
Pipes at command level
(Example) I/O Redirection and Pipelines Method
1. I/O Redirection Method
$ ls /home/kc > tempfile
$ wc -l < tempfile
19
$ rm tempfile
2. Pipeline Method
$ ls /home/kc | wc -l
19
2007 UNIX Programming
Pipes
pipes at command level
Weak Point of I/O Redirection Method
1. Cannot create temporary files if you don’t have permission
2. May have a invalid result because of temporary file
3. Easy to forget to remove temporary file
(Example) cd /bin
1. I/O Redirection Method (Failure)
$ ls > tempfile
tempfile: permission denied
2. Pipeline Method ( Success)
$ ls | wc -l
777
2007 UNIX Programming
pipes
%who | sort
who
pipe
write pointer of
another process
sort
read pointer of
one process
2007 UNIX Programming
Pipes
Create pipes
pipe() system call
Data transmitting
data is written into pipes using the write() system call
data is read from a pipe using the read() system call
automatic blocking when full or empty
FIFO basis, so lseek() don’t work
Types of pipes
(unnamed) pipes
named pipes
2007 UNIX Programming
Pipes
Programming with pipes
create a pipe
#include <unistd.h>
int
pipe (int
fd[2]);
fd[0] : read only open
fd[1] : write only open
0 : ok
-1 : error
more open than per-user process limit(EMFILE)
kernel’s open file table overflow(ENFILE)
2007 UNIX Programming
Pipes
ex) pipe on a single process
#include <stdio.h>
#include <unistd.h>
#define MSGSIZE 1024
char *msg1= “hello, world #1”;
char *msg2= “hello, world #2”;
char *msg3= “hello, world #3”;
$a.out
hello, world #1
hello, world #2
hello, world #3
int main(){
char inbuf[MSGSIZE];
int fd[2], j, ret;
if (pipe(fd) == -1) {perror(“pipe call”); exit(1);}
write(fd[1], msg1, strlen(msg1));
write(fd[1], msg2, strlen(msg2));
write(fd[1], msg3, strlen(msg3));
for (j = 0; j < 3; j++) {
ret = read(fd[0], inbuf, MSGSIZE);
inbuf[ret]=0;
printf(“%s\n”, inbuf);
}
}
2007 UNIX Programming
Pipes
ex) pipe on a single process
process
fd[0]
fd[1]
process
or
fd[0]
fd[1]
pipe
kernel
read in the order in which they were written
first in/first out(FIFO) communication channel
=> Since lseek don’t work on a pipe
2007 UNIX Programming
Pipes
kernel’s pipe buffer size
constant PIPE_BUF
rule
if a write overfill pipe, then blocked until
reading
if a read when empty, then blocked until writing
if a write > pipe, then block after write
if a read > pipe, then return after read
pipe_size = fpathconf( p[0],
_PC_PIPE_BUF );
2007 UNIX Programming
Pipes
rules (when one end of a pipe is closed)
If we read from a pipe whose write end has been closed,
after all the data has been read, read return 0 to
indicate an end of file
End of file is not generated until there are no
more writers for the pipe
If we write to a pipe whose read end has been closed,
the signal SIGPIPE is generated
Default action : terminate
2007 UNIX Programming
Pipes
ex) pipe from child to parent
$a.out
hello,
hello,
int fd[2], j;
hello,
pid_t pid;
=> 무한
if (pipe(fd) == -1) {perror(“pipe call”); exit(1);}
int main(){
char inbuf[MSGSIZE];
world #1
world #2
world #3
대기
switch(pid = fork()) {
case -1 :
perror(“fork error”); exit(1);
case 0
write(fd[1], msg1, strlen(msg1));
:
write(fd[1], msg2, strlen(msg1));
write(fd[1], msg3, strlen(msg1)); break;
default :
for (j = 0; j < 3; j++) {
read(fd[0], inbuf, MSGSIZE);
inbuf[ret] = 0;
printf(“%s\n”, inbuf);
}
wait(NULL);
}
}
2007 UNIX Programming
무한대기
Pipes
ex) pipe from child to parent
parent
fd[0]
child
fork
fd[1]
fd[0]
fd[1]
write()
read()
pipe
kernel
2007 UNIX Programming
Pipes
ex) pipe from child to parent (recommended)
switch(pid = fork()) {
case -1 : perror(“fork error”); exit(1);
case 0
: close(fd[0]);
write(fd[1], msg1, MSGSIZE);
write(fd[1], msg2, MSGSIZE);
write(fd[1], msg3, MSGSIZE); break;
default : close(fd[1]);
for (j = 0; j < 3; j++) {
read(fd[0], inbuf, MSGSIZE);
printf(“%s\n”, inbuf);
}
wait(NULL);
}
2007 UNIX Programming
Pipes
pipe from child to parent (recommended)
parent
fork
child
fd[0]
fd[1]
write()
read()
pipe
kernel
2007 UNIX Programming
Pipes
Non-blocking reads and writes
to use fstat
If st_size > 0 , and then read() from pipe
st_size : number of characters currently in the
pipe
problem : if several processes are reading the
pipe, another process could read from a pipe in
the gap between fstat and read
to use fcntl ( recommend)
O_NONBLOCK flag
errno -1 : EAGAIN
#include <fcntl.h>
if( fcntl(fd, F_SETFL, O_NONBLOCK) == -1 )
perror(“fcntl”);
2007 UNIX Programming
Pipes
Non-blocking read and write
#include <fcntl.h>
#include <errno.h>
#define MSGSIZE 6
int parent (int *);
int child (int *);
char *msg1 = "hello";
char *msg2 = "bye!!";
void fatal(char* msg) { fprintf(stderr,”%s\n”,msg);exit(1);}
int main(){
int pfd[2];
/* 파이프를 개방한다 */
if(pipe (pfd) == -1) fatal ("pipe call");
/* p[0]의 O_NONBLOCK 플래그를 1로 설정한다 */
if (fcntl (pfd[0], F_SETFL, O_NONBLOCK) == -1)
fatal ("fcntl call");
switch(fork()){
case -1: fatal("fork call");
/* 오류 */
case 0: child(pfd);
/* 자식 */
default: parent (pfd);
/* 부모 */
} }
2007 UNIX Programming
Pipes
Non-blocking read and write
int parent (int p[2]) {
/* 부모의 코드 */
int nread; char buf[MSGSIZE];
close (p[1]);
for(;;) {
switch (nread = read(p[0], buf, MSGSIZE)){
case -1:
/* 파이프에 아무것도 없는지 검사한다. */
if (errno == EAGAIN){
printf ("(pipe empty)\n");
sleep (1);
break;
}
else { perror("read call"); exit(1);}
case 0:
/* 파이프가 닫혔음. */
printf ("End of conversation\n");
exit (0);
default:
printf ("MSG=%s\n", buf);
}
} }
2007 UNIX Programming
Pipes
Non-blocking read and write
int child(int p[2])
{
int count;
close (p[0]);
for (count= 0; count < 3; count++)
{
write (p[1], msg1, MSGSIZE);
sleep(3);
}
/* 마지막 메시지를 보낸다 */
write (p[1], msg2, MSGSIZE);
exit (0);
$a.out
(pipe empty)
MSG=hello
(pipe empty)
(pipe empty)
(pipe empty)
MSG=hello
(pipe empty)
(pipe empty)
(pipe empty)
MSG=hello
(pipe empty)
(pipe empty)
MSG=bye!!
(pipe empty)
End of conversation
}
2007 UNIX Programming
Pipes
Using select to handle multiple pipes
Server & client Applications
parent
fork
fd2[0]
pipe
fd1[0]
fd3[0]
read()
fork
fork
write()
fd1[1]
Child 1
fd2[1]
Child 2
2007 UNIX Programming
fd3[1]
Child 3
Pipes
Using select to handle multiple pipes
#include <sys/time.h>
#include <sys/time.h>
int select (int nfds, fd_set *readfds, fd_set *writefds,
fd_set* errorfds, struct timeval *timeout);
struct timeval {
long tv_sec; /*second */
long tv_usec; /*microseconds */
}
0 : timeout, -1 : error
>0 : the number of “interesting” FDs
nfds : specifies the range of file descriptors to be tested.
0 ~ nfds-1
readfds : the specified file descriptors is ready for reading
writefds : FDs is ready for writing
errorfds : FDs is ready for error checking
Timeout
Timeout = NULL : block forever or until something of
interest turns up
timeout.tv_sec = 0 && timeout.tv_usec = 0: returns
immediately without blocking
Timeout.tv_sec != 0 || timeout.tv_usec != 0 : return
after that number of tv_sec or tv_usec specified if there
is no FDs activity
2007 UNIX Programming
Pipes
Macro for manipulating sets of file descriptor (bit mask)
#include <sys/time.h>
/* initialize the mask pointed to by fdset */
void FD_ZERO(fd_set *fdset);
/* set the bit, fd, in the mask pointed to by fdset */
void FD_SET(int fd, fd_set* fdset);
/* is the bit, fd, set in the mask pointed to by fdset */
void FD_ISSET(int fd, fd_set* fdset);
/* turn off the bit, fd, in the mask pointed to by fdset */
void FD_CLR(int fd, fd_set *fdset);
2007 UNIX Programming
Pipes
Using select to handle multiple regular files
#include <sys/time.h>
#include <sys/types.h>
#include <fcntl.h>
.
.
.
int fd1, fd2;
fd_set readset;
fd1 = open(“file1”, O_RDONLY);
fd2 = open(“file2”, O_RDONLY);
FD_ZERO(&readset);
FD_SET(fd1, &readset);
FD_SET(fd2, &readset);
select(fd2+1,&readset,NULL,NULL,NULL)
Not recommended
switch(select(5,&readset,NULL,NULL,NULL))
{
/* logic */
}
2007 UNIX Programming
pipes
Examples (Using select to handle multiple pipes)
#include <sys/time.h>
#include <sys/wait.h>
#include <stdio.h>
#include <unistd.h>
#define MSGSIZE
6
char *msg1 = "hello"; char *msg2 = "bye!!";
void parent(int a[][]); int child(int b[]);
void fatal(char* msg){ fprintf(stderr,"%s\n",msg);exit(1);}
int main(){
int pip[3][2]; int i;
for (i = 0; i < 3; i++) {
if (pipe(pip[i]) == -1)
fatal("pipe call");
switch (fork()){
case -1:
/* 오류 */
fatal("fork call");
case 0:
/* 자식 */
child(pip[i]);
}
}
parent(pip); /* 부모 */
exit(0); }
2007 UNIX Programming
pipes
Examples (parent process)
void parent(int p[3][2]){
char buf[MSGSIZE], ch; fd_set set, master; int i;
for (i = 0; i < 3; i++) close(p[i][1]);
FD_ZERO(&master); FD_SET(0, &master);
for (i = 0; i <3; i++) FD_SET(p[i][0], &master);
while(set=master, select (p[2][0]+1, &set, NULL, NULL, NULL) > 0){
if(FD_ISSET(0, &set)){ /* standard input check */
printf ("From standard input...");
read (0, &ch, 1);
printf("%c\n", ch);
}
for (i = 0; i < 3; i++) {
if(FD_ISSET(p[i][0], &set)) {
if(read(p[i][0], buf, MSGSIZE) > 0) {
printf ("Message from child%d\n", i);
printf ("MSG=%s\n",buf);
}
}
}
if(waitpid (-1, NULL,WNOHANG) == -1) return;
}
}
2007 UNIX Programming
pipes
Examples (child process)
int child(int p[2]){
int count;
close(p[0]);
for (count =
{
write
sleep
}
write (p[1],
exit (0);
}
0; count < 2; count++)
(p[1], msg1, MSGSIZE);
(getpid()%4);
msg2, MSGSIZE);
$a.out
Message from child0
MSG=hello
Message from child1
MSG=hello
Message from child2
MSG=hello
Message from child2
MSG=hello
Message from child2
MSG=bye!!
d
From standard input...d
From standard input...
Message from child0
MSG=hello
.
.
.
2007 UNIX Programming
Sell pipe implementation
Pipes and the exec system call
%ls –l | wc
Shell
/bin/sh
fd[1]
child
ls
parent
fd[0]
pipe
child
wc
/* stdin에서 command read */
/* command parsing */
/* 2개의 command이고 둘이 pipe로 연결되어 있다면*/
int fd[2], c1;
pipe(fd);
c1 = fork ();
If(!c1) {
/*first child*/
close( fd[0] );
close(1); dup2( fd[1],1 ); close( fd[1]);
execlp(/*ls –l first command */);
}
c2= fork ();
If(!c2) {
/*second child*/
close( fd[1] );
close(0); dup2( fd[0],0 ); close( fd[0]);
execlp(/*wc second command */);
}
close( fd[0] ); close( fd[1] );
2007 UNIX Programming
Sell pipe implementation
dup(), dup2()
#include <unistd.h>
int dup(int fd);
int dup2( int fd, int fd2);
사용중인 파일 디스크립터의 복사본을 만듦
dup()는 새 파일 디스크립터가 할당
dup2()는 fd2를 사용
리턴 값
성공하면 복사된 새 파일 디스크립터, 실패하면 -1
dup()는 할당 가능한 가장 작은 번호를 리턴
dup2()는 fd2를 리턴
dup2()는 복사본을 만들기 전에 기존의 fd2를
close함
2007 UNIX Programming
Sell pipe implementation
Another method: The Join program
parent
wait()
write()
(stdout)
P[1]
pipe
P[0]
read()
(stdin)
Child 2
Child 1
(com1)
(com2)
2007 UNIX Programming
Sell pipe implementation
Pseudo-code form of the Join program
Process forks, parent waits for child
And child continues
Child create a pipe
Child then forks
In child created by second fork (child 2):
standard output is coupled to
write end of pipe using dup2
excess file descriptors are closed
program described by ‘com1’ is exec’ed
In child of first fork (child 1):
standard input is coupled to
read end of pipe using dup2
excess file descriptors are closed
program described by ‘com2’ is exec’ed
2007 UNIX Programming
Sell pipe implementation
The Join program
int join(char* com1[], char* com2[]){
int p[2], status;
/* 명령을 수행할 자식을 생성한다. */
switch (fork()){
case -1:
/* 오류 */
fatal ("1st fork call in join");
case 0:
/* 자식 */
break;
default:
/* 부모 */
wait(&status);
return (status);
}
/* 루틴의 나머지 부분으로 자식에 의해 수행된다. */
/* 파이프를 만든다. */
if (pipe(p) == -1)
fatal ("pipe call in join");
2007 UNIX Programming
Sell pipe implementation
The Join program
switch (fork()){/* 다른 프로세스를 생성한다. */
case -1:
/* 오류 */
fatal ("2nd fork call in join");
case 0:
/* 쓰는 프로세스 */
dup2 (p[1],1); /* 표준 출력이 파이프로 가게 한다. */
close (p[0]); /* 화일 기술자를 절약한다. */
close (p[1]);
execvp (com1[0], com1);
/* execvp가 복귀하면, 오류가 발생한 것임. */
fatal("1st execvp call in join");
default:
/* 읽는 프로세스 */
dup2(p[0], 0); /* 표준 입력이 파이프로부터 오게 한다 */
close (p[0]);
close (p[1]);
execvp (com2[0], com2);
fatal ("2nd execvp call in join");
}}
2007 UNIX Programming
Sell pipe implementation
The Join program
앞서 join routine은 다음 프로그램을 이용하여 호출될 수 있다:
#include <stdio.h>
main()
{
char *one[4] = {"ls", "-l", "/usr/lib", NULL};
char *two[3] = {"grep", "∧d", NULL};
int ret;
ret = join (one, two);
printf ("join returned %d\n", ret);
exit (0);
}
2007 UNIX Programming