Transcript UNIX Foundations - Computer Science

UNIX Foundations
The Process and the Kernel
1
Course Description
 The
Goals for this course
 Understand
UNIX
 Understand Operating Systems
 Prerequisites:
 COSC
2P13 : Introduction to Operating Systems
 COSC 2P91 : Procedural Programming
Course Description

An intensive study of computer operating
system design
 Multiprogramming
 Time-sharing
 Real-time
processing
 Job and task control
 Synchronization of concurrent processes and
processors
 Resource scheduling
 Protection
 Management of hierarchical storage.
How to study this course
 Read
and remember
 Read
the book, remember the concepts
and commands
 Think
 Think
operating systems as natural
administrative agents
 Practice
 Coding
with UNIX, use and understand of
UNIX commands and get the results
The Textbook Used
 Uresh
Vahalia, UNIX Internals: The
New Frontiers, Prentice Hall, 1996
 Why we use this book?
 UNIX
is one of the most popular
operating systems of the world.
 If you understand UNIX, you can
understand other operating systems.
References
William Stallings, Operating Systems,5th
Ed., Prentice Hall,2005
 A. Tanenbaum, Modern Operating
Systems, 2nd ed. Prentice Hall 2001
 Kay A. Robbins and Steven Robbins, UNIX
Systems Programming, Prentice Hall,2003
 W. R. Stevens, Advanced Programming in
the UNIX Environment, Addison Wesley,
1992

Summary of UNIX History
XPG
SVID
POSIX
Solaris
Digital UNIX
ULTRIX
SVR4
HP-UX
4.3BSD
SCO UNIX
SVR3
XENIX
SunOS
AIX
4.2BSD
SVR2
4BSD
3BSD
7
4.4BSD
UNIX
V1
.
.
.
V6
Summary of
UNIX History
Xenix
BSD
V7
PWB
Xenix2
PWB2
32V
SIII
V8
SYSV
SCO
2BSD
.
.
.
2.9BSD
3BSD
2.10BSD
4BSD
2.11BSD
V10
4.2BSD
4.3BSD
V.2
Plan9
Ultrix
V.3
SUNOS
AIX
V.3.2
SVR4
8
4.4BSD
Mach
Solaris
LINUX
OSF1
Flexibility
 Traditional
Kernel: file, scheduling,
executable file formats
9
Modern UNIX Kernel
10
Assessment of UNIX
 Advantages:
- open software process,
- well designed, small and simple
kernel,
- text files in system databases,
- simple, uniform interface to I/O
devices,
- portability (written in C)
11
Assessment of UNIX
 Disadvantages:
- expansion of the more and more
complex I/O library,
- unfriendly user interface,
- building block approach to tools, but
not to the kernel,
- too many versions and standards,
- monolithic, unmodular and more and
more complex kernel.
12
UNIX Architecture
 Hardware
is surrounded by the
operating-system
 Operating system is called the kernel
 Comes with a number of user services
and interfaces
 shell
C
13
compiler
UNIX Architecture
Application Programs
14
Compiler Components
Compiler
Shell Editors, and
Private User Programs
Kernel
Hardware
User
Interface
The layers of a UNIX system.
15
UNIX Utility Programs
A few of the more common UNIX utility programs required by
16
POSIX
UNIX shell programming
cp src dest
 head –20 file
 ls *.c
 sort <in >out
 sort <in >temp;head –30<temp;rm temp
 sort <in | head –30
 grep ter *.t | sort | head –20 | tail –5 >foo
 sort <x | head &

17
Process
 In
UNIX
 Process
is an instance of a running
program.
 Lifetime: fork/vfork->exec->exit
 Well-defined hierarchy: parent,child,init,
 init process: the top process
 swapper & pagedeamon
 Orphans: the parent process is terminated.
18
19
Process Creation
 Submission
of a batch job
 User logs on
 Created to provide a service such as
printing
 Spawned by an existing process
20
Process Termination
 Batch
job issues Halt instruction
 User logs off
 Process executes a service request to
terminate
 Error and fault conditions
21
Reasons for Process Termination
Normal completion
 Time limit exceeded
 Memory unavailable
 Bounds violation
 Protection error

 example
write to read-only file
Arithmetic error
 Time overrun

 process
waited longer than a specified maximum
for an event
22
Reasons for Process Termination
 I/O
failure
 Invalid instruction
 happens
when try to execute data
Privileged instruction
 Data misuse
 Operating system intervention

 such
 Parent
as when deadlock occurs
terminates so child processes terminate
 Parent request
23
Process States
 The
Running state
 The
process that gets executed (single
CPU)
 The
Ready state
 any
 The
process that is ready to be executed
Blocked state
 when
a process cannot execute until some
event occurs (ex: the completion of an I/O)
24
Process States
 The
New state
 OS
has performed the necessary actions to create
the process
has created a process identifier
 has created tables needed to manage the process

 but
has not yet committed to execute the process
(not yet admitted)

 The
because resources are limited
Exit state
 Termination
moves the process to this state
 It is no longer eligible for execution
 Tables and other info are temporarily preserved for
auxiliary program
25
Five-State Process
Model
Dispatch
New
Admit
Ready
Release
Running
Time-out
Event
Occurs
Blocked
26
Event
Wait
Exit
Single Blocked Queue
Ready Queue
Release
Dispatch
Admit
Processor
Time-out
Event Wait
Event
Occurs
Blocked Queue
27
Multiple Blocked
Queues
Ready Queue
Release
Dispatch
Admit
Processor
Time-out
Event 1 Wait
Event 1
Occurs
Event 1 Queue
Event 2 Wait
Event 2
Occurs
28
Event 2 Queue
Suspended Processes
 Processor
is faster than I/O so all
processes could be waiting for I/O
 Swap these processes to disk to free
up more memory
 Blocked state becomes suspend state
when swapped to disk
 Two new states
 Blocked,
suspend
 Ready, suspend
29
Process State Transition
Diagram with Two Suspend
States
Ready
Admit
Admit
Suspend
Dispatch
Activate
Ready,
suspend
Ready
Suspend
Time out
Event
Occurs
Event
Occurs
Activate
Blocked,
suspend
30
Running
Blocked
Event
Wait
Exit
Processor State
Information
 Contents
of processor registers
 User-visible
registers
 Control and status registers
 Stack pointers
 Program
status word (PSW)
 contains
31
status information
Process Control
Information
 Additional
information needed by the
operating system to control and
coordinate the various active processes
 scheduling
and state information
 data structuring
 interprocess communication
 process privileges
 memory management
 resource ownership and utilization
32
Process Creation
 Assign
a unique process identifier
 Allocate space for the process
 Initialize process control block
 Set up appropriate linkages
 Ex:
add new process to linked list used for
scheduling queue
 Other
 maintain
33
an accounting file
When to Switch a Process
 Interrupts
 Clock

process has executed for the maximum allowable time slice
 I/O
 Memory
fault
 memory
address is in virtual memory so it must be
brought into main memory
 Trap
 error
occurred
 may cause process to be moved to Exit state
 Supervisor
 such
34
call
as file open
Change of Process State
Save context of processor including program
counter and other registers
 Update the process control block with the new
state and any accounting information
 Move process control block to appropriate
queue - ready, blocked
 Select another process for execution
 Update the process control block of the process
selected
 Update memory-management data structures
 Restore context of the selected process

35
UNIX Process State






36
Initial (idle)
Ready to run
Kernel/User running
Zombie
Asleep
for 4BSD: stopped/suspend
Process states and state
transitions
37
Process Context

User address space:
 code,

data, stack, shared memory regions,
Control information:

u area, proc, kernel stack, ATM
Credentials: UID & GID
 Environment variables:

 inherited

Hardware context(in PCB of u area):
 PC,
38
from the parent
SP, PSW, MMR, FPU
User Credentials
Superuser: UID=0, GID=1
 Real IDs: log in, send signals
 Effective IDs: file creation and access
 exec:



suid mode: that of the owner;
sgid mode: that of the calling process
setuid / setgid:
 SV & BSD are different with these

 saved

39
UID, saved GID in SV
setgroup in BSD
The u Area

part of the process space, needed only when running.










40
PCB
proc pointer
UID s
Arguments, results, error status from system calls
Signal handlers and related information
Info from the program header, text, data, stack size, MM info
Open file descriptor
Vnodes & controlling terminal pointers
CPU usage statistics, profiling info, disk quotas, & resource
limits
Per-process kernel stack
 Process
table
The proc
 Id
 Location
of the kernel address map for the u area
 Current process state
 Forward and backward pointers in scheduled queue
 Sleep channel (7.2.3)
 Scheduling priority and related (Chapter 5)
 Signal handling info(Chapter 4).
 MM info
 Pointers to link active, free, zombie processes in lists
 Flags
 Pointers to keep structures on a hash queue by PID
 Hierarchy info
41
Process ID
A typical process hierarchy
in
Parent Process ID
pointer to
4.3BSD
UNIX
parent’s
proc
Pointer to the
younger sibling
42
Pointer to the oldest child
Typical Functions of an
Operating-System Kernel

Process Management
 Process
creation and termination
 Process scheduling and dispatching
 Process switching
 Process synchronization and support for interprocess communication
 Management of process control blocks

Memory Management
 Allocation
of address space to processes
 Swapping
 Page
43
and segment management
Typical Functions of an
Operating-System Kernel
 I/O
Management
 Buffer
management
 Allocation of I/O channels and devices to
processes
 Support
Functions
 Interrupt
handling
 Accounting
 Monitoring
44
Operating System
Control Structures
An
OS maintains the following tables
for managing processes and
resources:
Memory
tables
I/O tables
File tables
Process tables
45
Memory Tables
 Allocation
of main memory to
processes
 Allocation of secondary memory to
processes
 Protection attributes for access to
shared memory regions
 Information needed to manage virtual
memory
46
I/O Tables
 I/O
device is available or assigned
 Status of I/O operation
 Location in main memory being used
as the source or destination of the I/O
transfer
47
File Tables
 Existence
of files
 Location on secondary memory
 Current Status
 Attributes
 Sometimes this information is
maintained by a file-management
system
48
Process Table
 Process
image consists of program,
data, stack, and attributes
 Attributes
 process
49
control block
Process Control Block
Process Identification
 Unique
numeric identifier
 may
be an index into the primary process
table
 User
identifier
 who
50
is responsible for the job
Execution of the
Operating System
 Nonprocess
Kernel
 execute
kernel outside of any process
 operating system code is executed as a
separate entity that operates in privileged
mode
 Execution
 operating
Within User Processes
system software within context
of a user process
 process executes in privileged mode when
executing operating system code
51
Execution of the
Operating System
 Process-Based
 major
Operating System
kernel functions are separate
processes
 a process is invoked by the operating
system
52
The UNIX kernel
A
special program that runs directly on the
hardware.
 Implements the process model and services.
 Resides on disk, in a file /vmunix or /unix.
 Bootstrapping: loads the kernel.
 Initializes the system and sets up the
environment, remains in memory before shut
down
53
UNIX Services
 System
call interface
 Hardware exceptions
 Divide
by 0, overflowing user stack
 Interrupts
 Devices
 Swapper,
54
pagedaemon
The Kernel interacts with
processes and devices
55
BACK
Mode,Space & Context
 By
modes: some critical resources can
be protected.
 Kernel
Mode: More privileged, kernel
functions
 User Mode: Less privileged, user functions
 Virtual
 VM
Memory
space
 Address Translation Maps
 Memory Management Unit
56
Kernel data
 Current
process & Context switch
 One instance of the kernel
 Global data structure
 Pre-process objects
 System call, Mode Switch
 User area: info. about a process
 Kernel stack:
57
Context
 UNIX
kernel is re-entrant: several
processes may be involved in kernel
activities concurrently.
 Execution context
 Process
context:
 System context (Interrupt context):
58
Execution mode and
Context
59
Executing in Kernel Mode
3
types of events:
 Device
interrupts
 Exceptions
 Traps or software interrupts
 Dispatch
table
 System context: interrupts
 Process context: traps, exceptions &
software interrupts
60
The System Call Interface
syscall(): the starting point
 in kernel mode, but in process context.
 Copy arguments , save hardware
context on the kernel stack.
 Use system call number to index
dispatch vector
 Return results in registers, restore
hardware context, to user mode, control
back to the library routine.

61
UNIX Interrupt Handling
 Interrupt
handler(interrupt service routine):
 runs
in kernel mode and system context,
 not permitted to block.
 the
time used to service an interrupt charged to
the interrupted process
 The clock interrupt handler charges the clock tick
to the current process
 ipl(interrupt priority level)- specified for each
interrupt and saved in interrupt register of the
processor status word
 Interrupts are preemptive
62
Setting the interrupt
priority in 4.3BSD and
SVR4
63
Interrupt handling
64
Synchronization
 UNIX
is reentrant
 UNIX is nonpreemptive,
keeping the
kernel states
consistent.
 Relinquish CPU
voluntarily.
65
Blocking Operations
 Blocks
the process (make it sleep).
 Lock: a single bit flag
 wanted(flag):
 sleep():
 wake(): wake all the waiting processes.
 upon waking up: check once again to
make sure.
66
Algorithm for resource
locking
67
Blocking Interrupts

Blocking interrupts while accessing critical
sections.



int x = splbio();/* raise ipl*/
modify disk buffer cache;
splx(x); /*restore ipl*/
 Critical
region:few & brief.
 Blocked interrupts may access the critical
region.
 Different interrupts may have the same ipl
 Blocking an interrupt may block others.
68
UNIX Process Implementation
69
New Processes & Programs
fork:
creates
a new process.
returns 0 to the child, PID to the
parent

exec:

70
begins to execute a new program
Using fork & exec
if ((result = fork()==0){
 /* child code*/
 ……
 if (execve(“new program”),…)<0)

perror(“execve failed!”);
 } else if (result<0) {
 perror(“fork”);/*fork failed*/
}
 /*parent continures here*/

71
Shell creating a process
A highly simplified shell
72
The ls Command
Steps in executing the command ls typed to the shell
73
Process Creation
 Almost an exact clone of the parent.
 Reserve swap space for the child
 Allocate a new PID and proc structure for the child
 Initialize proc structure
 Allocate address translation map (ATM)
 Allocate u area and copy
 Update the u area to refer to the new ATM & Swap space
 Add the child to the set of processes sharing the text region of
the program
 Duplicate the parent’s data and stack regions update ATM to
refer to these new pages.
 Acquire references to shared resources inherited by the child
 Initialize the hardware context
 Make the child runnable and put it on a scheduler queue
 Arrange to return with 0
 Return the PID to the parent
74
fork Optimization
 It
is wasteful to make an actual copy of
the address space of the parent
 Copy-on-write
(SV) : only the pages that
are modified must be copied
 vfork (BSD): The parent loans the
address space and blocks until the child
returns to it.


75
dangerous!
csh: exploits it.
Invoking a New Program
 Process
 Text:
address space
code
 Initialized data:
 Uninitialized data(bss):
 Shared memory(SV):
 Shared libraries:
 Heap: dynamic space
 User stack: space allocated by the kernel
76
exec System Call










77
Parse & access
Verify the permission
Read the header and check if valid executable
If the file has SUID or SGID bits set in its mode, change the
caller’s effective UID or GID to that of the owner
Copy the arguments to exec and the env. variables into kernel.
Allocate swap space for the data and stack region
Set up the new address space.
Copy the arguments and env. variables back onto the new user
stack.
Reset all signal handlers to default actions.
Initialize the hardware context.
Process Termination (exit())











78
Turns off all signals.
Closes all open files.
Releases the text file and other resources such as the current
directory.
Writes to the accounting log.
Saves resource usage statistics and exit status in the proc
structure.
Changes state to SZOMB, and puts the proc on the zombie
process list.
Makes the init process inherit any live children of the exiting
process.
Releases the address space, u area, ATM, and swap space.
Notifies the parent by sending it a SIGCHLD signal.
Wakes up the parent if it is asleep.
Calls swtch() to schedule a new process to run.
Awaiting Process Termination




79
wait(stat_loc);/* SV, BSD & POSIX*/
wait3(statusp, options, rusagep); /*BSD*/
waitpid(pid, stat_loc, options);/*POSIX*/
waitid(idtype, id, infop, options);/*SVR4*/
Zombie Processes
after exit(), process holds only proc structure
and is in zombie state until cleaned
completely.
 wait() (issued by the parent or the init
process)frees the proc structure and
completes process exit,
 If the child dies before the parent which
does not wait for it, the proc is not released
until the system is rebooted.
 Parent can specify that it will not wait for its
children

80
LINUX



81
Modular structure – collection of modules, some of
them loadable and unloadable on demand,
Module – object file whose code be linked and
unlinked from the kernel at runtime, executed in
kernel mode on behalf of the current process
Loadable module characteristic
– Dynamic linking,
-- Stackable modules – may serve as libraries when
they are referenced by client modules higher up in
the hierarchy, and as clients when they reference
modules further down the hierarchy.
82
Linux components



83
At user level – tasks (specific to Linux – combine
features of processes and threads);
At kernel level – interacting collection of
components;
Hardware components – in this case IA-64 Intel
Itanium architecture
84
Linux Task Data
Structure
State
 Scheduling information
 Identifiers
 Interprocess communication
 Links
 Times and timers
 File system
 Address space
 Processor-specific context

85
Linux States of a
Process/Thread
 Running
 Interruptable
 Uninterruptable
 Stopped
 Zombie
86
87