Unit 3: Concurrency

Instructor: Hengming Zou, Ph.D.

In Pursuit of Absolute Simplicity 求于至

简，归于永恒

Outline of Content



3.1. Critical Sections, Semaphores, and Monitors



3.2. Windows Trap Dispatching, Interrupts, Synchronization



3.3. Advanced Windows Synchronization



3.4. Windows APIs for Synchronization and IPC

Critical Sections, Semaphores, and Monitors



The Critical-Section Problem



Software Solutions



Synchronization Hardware



Semaphores



Synchronization in Windows & Linux

The Critical-Section Problem



n threads all competing to use a shared resource



Each thread has a code segment, called critical section, in which the shared data is accessed



Problem: Ensure that:

–

when one thread is executing in its critical section, no other thread is allowed to execute in its critical section

Solution to Critical-Section Problem



Mutual Exclusion

–

Only one thread at a time is allowed into its CS, among all threads that have CS for the same resource or shared data

–

A thread halted in its non-critical section must not interfere with other threads



Progress

–

A thread remains inside CS for a finite time only

–

No assumptions concerning relative speed of the threads

Solution to Critical-Section Problem



Bounded Waiting

–

It must no be possible for a thread requiring access to a critical section to be delayed indefinitely

–

When no thread is in a critical section, any thread that requests entry must be permitted to enter without delay

Initial Attempts to Solve Problem



Only 2 threads, T 0 and T 1



General structure of thread T

i

(other thread T

j

) do {

enter section

critical section

exit section

} while (1); reminder section



Threads may share some common variables to synchronize their actions

First Attempt: Algorithm 1



Shared variables

–

Initialization: int turn = 0;

–

turn == i



T i

can enter its critical section



Thread T

i

do { while (turn != i) ; critical section turn = j; reminder section } while (1);



Satisfies mutual exclusion, but not progress

Second Attempt: Algorithm 2



Shared variables

–

initialization: int flag[2]; flag[0] = flag[1] = 0;

–

flag[i] == 1



T i can enter its critical section



Thread T i do { flag[i] = 1; while (flag[j] == 1) ; critical section flag[i] = 0; remainder section } while(1);



Satisfies mutual exclusion, not progress requirement

Algorithm 3

(Peterson ’s Algorithm - 1981)



Shared variables of algorithms 1 and 2 - initialization: int flag[2]; flag[0] = flag[1] = 0; int turn = 0;



Thread T i do { flag[i] = 1; turn = j; while ((flag[j] == 1) && turn == j) ; critical section flag[i] = 0; } while (1); remainder section



Solves the critical-section problem for two threads

Dekker ’s Algorithm (1965)



This is the first correct solution proposed for the two-thread (two-process) case



Originally developed by Dekker in a different context, it was applied to the critical section problem by Dijkstra.



Dekker adds the idea of a favored thread and allows access to either thread when the request is uncontested.



When there is a conflict, one thread is favored, and the priority reverses after successful execution of the critical section

Dekker ’s Algorithm (contd.)



Shared variables - initialization: int flag[2]; flag[0] = flag[1] = 0; int turn = 0;



Thread T i do { flag[i] = 1; while (flag[j] ) if (turn == j) { flag[i] = 0; while (turn == j); flag[i] = 1; } critical section turn = j; flag[I] = 0;; remainder section } while (1);

Bakery Algorithm

(Lamport 1979)



A Solution to the Critical Section problem for n threads



Before entering its CS, a thread receives a number



Holder of the smallest number enters the CS.



If threads T i and T j if i < j, then T i receive the same number, is served first; else T j is served first.



The numbering scheme generates numbers in monotonically non-decreasing order:

–

i.e., 1,1,1,2,3,3,3,4,4,5...

Bakery Algorithm



Notation “<“ establishes lexicographical order among 2-tuples (ticket #, thread id #) (a,b) < (c,d) if a < c or if a == c and b < d max (a

0

, …, a n-1 ) = { k | k



a

i for i = 0, …, n – 1 }



Shared data int choosing[n]; int number[n]; - the ticket Data structures are initialized to 0

Bakery Algorithm

do { choosing[i] = 1; number[i] = max(number[0],number[1] ...,number[n-1]) + 1; choosing[i] = 0; for (j = 0; j < n; j++) { while (choosing[j] == 1) ; while ((number[j] != 0) && ((number[j],j) ‘’ < ‘’ (number[i],i))); } critical section number[i] = 0; remainder section } while (1);

Mutual Exclusion -

Hardware Support



Interrupt Disabling

–

Concurrent threads cannot overlap on a uniprocessor

–

Thread will run until performing a system call or interrupt happens



Special Atomic Machine Instructions

–

Test and Set Instruction - read & write a memory location

–

Exchange Instruction - swap register and memory location



Problems with Machine-Instruction Approach

–

Busy waiting

–

Starvation is possible

–

Deadlock is possible

Synchronization Hardware



Test and modify the content of a word atomically boolean TestAndSet(boolean &target) { boolean rv = target; target = true; } return rv;

Mutual Exclusion with Test-and-Set



Shared data:

–

boolean lock = false;



Thread T

i

do { while (TestAndSet(lock)) ; critical section lock = false; remainder section }

Synchronization Hardware



Atomically swap two variables void Swap(boolean &a, boolean &b) { boolean temp = a; a = b; b = temp; }

Mutual Exclusion with Swap



Shared data (initialized to

0

): int lock = 0;



Thread T

i

int key; do { key = 1; while (key == 1) Swap(lock,key); critical section lock = 0; remainder section }

Semaphores



Semaphore S – integer variable



can only be accessed via two atomic operations wait (S): while (S <= 0); S--; signal (S):

S++;

Critical Section of n Threads



Shared data: semaphore mutex; //initially mutex = 1



Thread T i : do { wait(mutex); critical section signal(mutex); remainder section } while (1);

Semaphore Implementation



Semaphores may suspend/resume threads

–

Avoid busy waiting



Define a semaphore as a record typedef struct { int value; struct thread *L; } semaphore;



Assume two simple operations:

–

suspend() suspends the thread that invokes it

–

resume(T) resumes the execution of a blocked thread T

Implementation



Semaphore operations now defined as wait(S): S.value--; if (S.value < 0) { add this thread to S.L; suspend(); } signal(S): S.value++; if (S.value <= 0) { remove a thread T from S.L; resume(T); }

Semaphore as a General Synchronization Tool



Execute B in T j only after A executed in T

i



Use semaphore flag initialized to 0



Code:

T i

 

T j A

signal(flag) wait(flag)

B

Two Types of Semaphores



Counting semaphore

–

integer value can range over an unrestricted domain.



Binary semaphore

–

integer value can range only between 0 and 1;

–

can be simpler to implement.



Counting semaphore S can be implemented as a binary semaphore

Deadlock and Starvation



Deadlock –

–

two or more threads are waiting indefinitely for an event that can be caused by only one of the waiting threads



Let S and Q be two semaphores initialized to 1

T 0

wait(S); wait(Q);

 

signal(S); signal(Q)

T 1

wait(Q); wait(S); signal(Q); signal(S);

Deadlock and Starvation



Starvation – indefinite blocking

–

A thread may never be removed from the semaphore queue in which it is suspended.



Solution –

–

all code should acquire/release semaphores in same order

Windows Synchronization



Uses interrupt masks to protect access to global resources on uniprocessor systems.



Uses spinlocks on multiprocessor systems.



Provides dispatcher objects which may act as mutexes and semaphores.



Dispatcher objects may also provide events. An event acts much like a condition variable

Linux Synchronization



Kernel disables interrupts for synchronizing access to global data on uniprocessor systems.



Uses spinlocks for multiprocessor synchronization.



Uses semaphores and readers-writers locks when longer sections of code need access to data.



Implements POSIX synchronization primitives to support multitasking, multithreading (including real-time threads), and multiprocessing

Further Reading



Ben-Ari, M., Principles of Concurrent Programming, Prentice Hall, 1982



Lamport, L., The Mutual Exclusion Problem, Journal of the ACM, April 1986



Abraham Silberschatz, Peter B. Galvin, Operating System Concepts, John Wiley & Sons, 6th Ed., 2003 ;

–

Chapter 7 - Process Synchronization

–

Chapter 8 - Deadlocks

3.2. Trap Dispatching, Interrupts, Synchronization



Trap and Interrupt dispatching



IRQL levels & Interrupt Precedence



Spinlocks and Kernel Synchronization



Executive Synchronization

Kernel Mode Versus User Mode



A processor state



Controls access to memory



Each memory page is tagged to show the required mode for reading and for writing

–

Protects the system from the users

–

Protects the user (process) from themselves

–

System is not protected from system



Code regions are tagged “no write in any mode”



Controls ability to execute privileged instructions



A Windows abstraction

–

Intel: Ring 0, Ring 3

Kernel Mode Versus User Mode



Control flow (a thread) can change from user to kernel mode and back

–

Does not affect scheduling

–

Thread context includes info about execution mode (along with registers, etc)



PerfMon counters:

–

“Privileged Time” and “User Time”

–

4 levels of granularity: thread, process, processor, system

Getting Into Kernel Mode



Code is run in kernel mode for one of three reasons:



1. Requests from user mode

–

Via the system service dispatch mechanism

–

Kernel-mode code runs in the context of the requesting thread



2. Dedicated kernel-mode system threads

–

Some threads in the system stay in kernel mode at all times



mostly in the “System” process

–

Scheduled, preempted, etc., like any other threads

Getting Into Kernel Mode

3. Interrupts from external devices

–

interrupt dispatcher invokes the interrupt service routine

–

ISR runs in the context of the interrupted thread



so-called “arbitrary thread context”

–

ISR often requests the execution of a “DPC routine,” which also runs in kernel mode

–

Time not charged to interrupted thread

Trap dispatching



Trap: processor ‘s mechanism to capture executing thread

–

Switch from user to kernel mode

–

Interrupts – asynchronous

–

Exceptions - synchronous Interrupt Interrupt dispatcher Interrupt service service routines System service call System service dispatcher System services services HW exceptions SW exceptions Exception dispatcher handlers handlers Virtual address exceptions Virtual memory manager‘s pager

Interrupt Dispatching

user or kernel mode code kernel mode interrupt !

Interrupt dispatch routine Disable interrupts Record machine state (trap frame) to allow resume Mask equal- and lower-IRQL interrupts Find and call appropriate ISR Dismiss interrupt Restore machine state (including mode and enabled interrupts) Note, no thread or process context switch!

Interrupt service routine Tell the device to stop interrupting Interrogate device state, start next operation on device, etc. Request a DPC Return to caller

Interrupt Precedence via IRQLs (x86)



IRQL = Interrupt Request Level

–

Precedence of the interrupt with respect to other interrupts

–

Different interrupt sources have different IRQLs

–

not the same as IRQ



IRQL is also a state of the processor

–

Servicing an interrupt raises processor IRQL to that interrupt ’s IRQL

–

this masks subsequent interrupts at equal and lower IRQLs



User mode is limited to IRQL 0



No waits or page faults at IRQL >= DISPATCH_LEVEL

Interrupt Precedence via IRQLs (x86)

31 30 29 28 2 1 0 High Power fail Interprocessor Interrupt Clock Profile & Synch (Srv 2003) .

.

.

Device 1 Dispatch/DPC APC Passive/Low Hardware interrupts Deferrable software interrupts normal thread execution

Interrupt processing



Interrupt dispatch table (IDT)

–

Links to interrupt service routines



x86:

–

Interrupt controller interrupts processor (single line)

–

Processor queries for interrupt vector; uses vector as index to IDT



Alpha:

–

PAL code (Privileged Architecture Library – Alpha BIOS) determines interrupt vector, calls kernel

–

Kernel uses vector to index IDT



After ISR execution, IRQL is lowered to initial level

Interrupt object



Allows device drivers to register ISRs for their devices

–

Contains dispatch code (initial handler)

–

Dispatch code calls ISR with interrupt object as parameter (HW cannot pass parameters to ISR)



Connecting/disconnecting interrupt objects:

–

Dynamic association between ISR and IDT entry

–

Loadable device drivers (kernel modules)

–

Turn on/off ISR



Interrupt objects can synchronize access to ISR data

–

Multiple instances of ISR may be active simultaneously (MP machine)

–

Multiple ISR may be connected with IRQL

Predefined IRQLs

 High –

used when halting the system (via KeBugCheck())

 Power fail –

originated in the NT design document, but has never been used

 Inter-processor interrupt –

used to request action from other processor (dispatching a thread, updating a processors TLB, system shutdown, system crash)

 Clock –

Used to update system ‘s clock, allocation of CPU time to threads

 Profile –

Used for kernel profiling (see Kernel profiler – Kernprof.exe, Res Kit)

Predefined IRQLs (contd.)

 Device –

Used to prioritize device interrupts

 DPC/dispatch

and

APC –

Software interrupts that kernel and device drivers generate

 Passive –

No interrupt level at all, normal thread execution

15 14 13 12 4 3 2 1 0

IRQLs on 64-bit Systems

x64 High/Profile Interprocessor Interrupt/Power Clock Synch (Srv 2003) Device n .

.

Device 1 Dispatch/DPC APC Passive/Low IA64 High/Profile/Power Interprocessor Interrupt Clock Synch (MP only) Device n .

Device 1 Correctable Machine Check Dispatch/DPC & Synch (UP only) APC Passive/Low

Interrupt Prioritization & Delivery



IRQLs are determined as follows:

–

x86 UP systems: IRQL = 27 - IRQ

–

x86 MP systems: bucketized (random)

–

x64 & IA64 systems: IRQL = IDT vector number / 16



On MP systems, which processor is chosen to deliver an interrupt?

–

By default, any processor can receive an interrupt from any device



Can be configured with IntFilter utility in Resource Kit

–

On x86 and x64 systems, the IOAPIC (I/O advanced programmable interrupt controller) is programmed to interrupt the processor running at the lowest IRQL

–

On IA64 systems, the SAPIC (streamlined advanced programmable interrupt controller) is configured to interrupt one processor for each interrupt source



Processors are assigned round robin for each interrupt vector

Software interrupts



Initiating thread dispatching

–

DPC allow for scheduling actions when kernel is deep within many layers of code

–

Delayed scheduling decision, one DPC queue per processor



Handling timer expiration



Asynchronous execution of a procedure in context of a particular thread



Support for asynchronous I/O operations

Flow of Interrupts

Synchronization on SMP Systems



Sync on MP use spinlocks to coordinate among processors



Spinlock acquisition and release routines implement a one owner-at-a-time algorithm

–

Spinlock is either free or is considered to be owned by a CPU

–

Analogous to using Windows API mutexes from user mode



A spinlock is just a data cell in memory

–

Accessed with a test-and-modify operation that is atomic across all processors

–

KSPIN_LOCK is an opaque data type, typedef ’d as a ULONG

–

To implement synchronization, a single bit is sufficient

Kernel Synchronization

Processor A

.

.

.

do acquire_spinlock(DPC) until (SUCCESS) begin remove DPC from queue end release_spinlock(DPC) spinlock DPC DPC

Processor B

.

.

.

do acquire_spinlock(DPC) until (SUCCESS) begin remove DPC from queue end release_spinlock(DPC) Critical section A spinlock is a locking primitive associated with a global data structure, such as the DPC queue

Queued Spinlocks



Problem: Checking status of spinlock via test-and-set operation creates bus contention



Queued spinlocks maintain queue of waiting processors



First processor acquires lock; other processors wait on processor-local flag

–

Thus, busy-wait loop requires no access to the memory bus



When releasing lock, the 1st processor ’s flag is modified

–

Exactly one processor is being signaled

–

Pre-determined wait order

SMP Scalability Improvements



Windows 2000: queued spinlocks

–

!qlocks in Kernel Debugger



Server 2003:

–

More spinlocks eliminated (context swap, system space, commit)

–

Further reduction of use of spinlocks & length they are held

–

Scheduling database now per-CPU



Allows thread state transitions in parallel

SMP Scalability Improvements



XP/2003:

–

Minimized lock contention for hot locks



PFN or Page Frame Database lock

–

Some locks completely eliminated



Charging nonpaged/paged pool quotas,



allocating and mapping system page table entries,



charging commitment of pages,



allocating/mapping physical memory through AWE functions

–

New, more efficient locking mechanism (pushlocks)



Doesn ’t use spinlocks when no contention



Used for object manager and address windowing extensions (AWE) related locks

Waiting



Flexible wait calls

–

Wait for one or multiple objects in one call

–

Wait for multiple can wait for “any” one or “all” at once



“All”: all objects must be in the signalled state concurrently to resolve the wait

–

All wait calls include optional timeout argument

–

Waiting threads consume no CPU time

Waiting



Waitable objects include:

–

Events (may be auto-reset or manual reset; may be set or “pulsed”)

–

Mutexes ( “mutual exclusion”, one-at-a-time)

–

Semaphores (n-at-a-time)

–

Timers

–

Processes and Threads (signalled upon exit or terminate)

–

Directories (change notification)

Waiting



No guaranteed ordering of wait resolution

–

If multiple threads are waiting for an object, and only one thread is released (e.g. it ’s a mutex or auto-reset event), which thread gets released is unpredictable

Executive Synchronization



Waiting on Dispatcher Objects – outside the kernel

Create and initialize thread object Terminated Thread waits on an object handle Initialized Waiting Wait is complete; Set object to signaled state Ready Transition Standby Running Interaction with thread scheduling

Interaction bet Synchronization & Dispatching



User mode thread waits on an event object ‘s handle



Kernel changes thread ‘s scheduling state from ready to waiting and adds thread to wait-list



Another thread sets the event



Kernel wakes up waiting threads; variable priority threads get priority boost

Interaction bet Synchronization & Dispatching



Dispatcher re-schedules new thread – it may preempt running thread it it has lower priority and issues software interrupt to initiate context switch



If no processor can be preempted, the dispatcher places the ready thread in the dispatcher ready queue to be scheduled later

What signals an object?

Dispatcher object System events and resulting state change Effect of signaled state Owning thread releases mutex Mutex (kernel mode) nonsignaled signaled Resumed thread acquires mutex Owning thread or other thread releases mutex Mutex (exported to user mode) nonsignaled signaled Resumed thread acquires mutex on waiting threads Kernel resumes one waiting thread Kernel resumes one waiting thread Semaphore One thread releases the semaphore, freeing a resource nonsignaled signaled A thread acquires the semaphore.

More resources are not available Kernel resumes one or more waiting threads

What signals an object? (contd.)

Dispatcher object Event Event pair Timer System events and resulting state change A thread sets the event nonsignaled Kernel resumes one or more threads Dedicated thread sets one event in the event pair signaled nonsignaled signaled Kernel resumes the other dedicated thread Timer expires Effect of signaled state on waiting threads Kernel resumes one or more waiting threads Kernel resumes waiting dedicated thread nonsignaled signaled Kernel resumes all waiting threads A thread (re) initializes the timer Thread terminates Thread nonsignaled signaled Kernel resumes all waiting threads A thread reinitializes the thread object

Further Reading



Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals, 4th Edition, Microsoft Press, 2004.



Chapter 3 - System Mechanisms

–

Trap Dispatching (pp. 85 ff.)

–

Synchronization (pp. 149 ff.)

–

Kernel Event Tracing (pp. 175 ff.)

3.3. Advanced Windows Synchronization



Deferred and Asynchronous Procedure Calls



IRQLs and CPU Time Accounting



Wait Queues & Dispatcher Objects

Deferred Procedure Calls (DPCs)



Used to defer processing from higher (device) interrupt level to a lower (dispatch) level

–

Also used for quantum end and timer expiration



Driver (usually ISR) queues request

–

One queue per CPU. DPCs are normally queued to the current processor, but can be targeted to other CPUs

–

Executes specified procedure at dispatch IRQL (or “dispatch level ”, also “DPC level”) when all higher-IRQL work (interrupts) completed

–

Maximum times recommended: ISR: 10 usec, DPC: 25 usec



See http://www.microsoft.com/whdc/driver/perform/mmdrv.mspx

Deferred Procedure Calls (DPCs)

queue head DPC object DPC object DPC object

Delivering a DPC

DPC routines can‘t assume what process address space is currently mapped DPC 1. Timer expires, kernel queues DPC that will release all waiting threads Kernel requests SW int.

high Power failure Interrupt dispatch table 2. DPC interrupt occurs when IRQL drops below dispatch/DPC level DPC DPC DPC DPC queue Dispatch/DPC APC Low 3. After DPC interrupt, control transfers to thread dispatcher dispatcher DPC routines can call kernel functions but can‘t call system services, generate page faults, or create or wait on objects 4. Dispatcher executes each DPC routine in DPC queue

Asynchronous Procedure Calls (APCs)



Execute code in context of a particular user thread

–

APC routines can acquire resources (objects), incur page faults, call system services



APC queue is thread-specific



User mode & kernel mode APCs

–

Permission required for user mode APCs

Asynchronous Procedure Calls (APCs)



Executive uses APCs to complete work in thread space

–

Wait for asynchronous I/O operation

–

Emulate delivery of POSIX signals

–

Make threads suspend/terminate itself (env. subsystems)



APCs are delivered when thread is in alertable wait state

–

WaitForMultipleObjectsEx(), SleepEx()

Asynchronous Procedure Calls

(APCs)



Special kernel APCs

–

Run in kernel mode, at IRQL 1

–

Always deliverable unless thread is already at IRQL 1 or above

–

Used for I/O completion reporting from “arbitrary thread context”

–

Kernel-mode interface is linkable, but not documented

Asynchronous Procedure Calls

(APCs)



“Ordinary” kernel APCs

–

Always deliverable if at IRQL 0, unless explicitly disabled (disable with KeEnterCriticalRegion) User mode APCs

–

Used for I/O completion callback routines (see ReadFileEx, WriteFileEx); also, QueueUserApc

–

Only deliverable when thread is in “alertable wait”

Asynchronous Procedure Calls

(APCs) Thread Object K U APC objects

IRQLs and CPU Time Accounting



Interval clock timer ISR keeps track of time



Clock ISR time accounting:

–

If IRQL<2, charge to thread ’s user or kernel time

–

If IRQL=2 and processing a DPC, charge to DPC time

–

If IRQL=2 & not processing a DPC, charge to thread kernel time

–

If IRQL>2, charge to interrupt time

IRQLs and CPU Time Accounting



Since time servicing interrupts are NOT charged to interrupted thread, if system is busy but no process appears to be running, must be due to interrupt-related activity

–

Note: time at IRQL 2 or more is charged to the current thread ’s quantum (to be described)

Interrupt Time Accounting



Task Manager includes interrupt and DPC time with the Idle process time



Interrupt activity is not charged to any thread/process

–

Process Explorer shows these as separate processes



not really processes

–

Context switches for these are really # of interrupts & DPCs

Time Accounting Quirks



Looking at total CPU time for each process may not reveal where system has spent its time



CPU time accounting is driven by programmable interrupt timer

–

Normally 10 msec (15 msec on some MP Pentiums)



Thread execution and context switches between clock intervals NOT accounted

–

E.g., one or more threads run and enter a wait state before clock fires

–

Thus threads may run but never get charged



View context switch activity with Process Explorer

–

Add Context Switch Delta column

Looking at Waiting Threads



For waiting threads, user-mode utilities only display the wait reason



Example: pstat

Wait Internals 1: Dispatcher Objects



Any kernel object you can wait for is a “dispatcher object”

–

some exclusively for synchronization



e.g. events, mutexes ( “mutants”), semaphores, queues, timers

–

others can be waited for as a side effect of their prime function



e.g. processes, threads, file objects

–

non-waitable kernel objects are called “control objects”



All dispatcher objects have a common header



All dispatcher objects are in one of two states Dispatcher Object

–

“signaled” vs. “nonsignaled”

–

when signalled, a wait on the object is satisfied Size State Type Wait listhead

–

different object types differ in terms of what changes their state

–

wait and unwait implementation is common to all types of dispatcher objects Object-type specific data (see \ntddk\inc\ddk\ntddk.h)

WaitBlockList Dispatcher Objects Size State Type Wait listhead Object-type specific data Size State Type Wait listhead Object-type specific data Thread Objects WaitBlockList Wait blocks List entry Thread Object Key Type Next link List entry Thread Object Key Type Next link Wait Internals 2:

Wait Blocks

   

Represent a thread ’s reference to something it’s waiting for (one per handle passed to WaitFor …) All wait blocks from a given wait call are chained to the waiting thread Type indicates wait for “any” or “all” Key denotes argument list position for WaitForMultipleObjects List entry Thread Object Key Type Next link

3.4. Windows APIs for Synchronization and IPC



Windows API constructs for synchronization and interprocess communication



Synchronization

–

Critical sections

–

Mutexes

–

Semaphores

–

Event objects



Synchronization through interprocess communication

–

Anonymous pipes

–

Named pipes

–

Mailslots

Critical Sections

VOID InitializeCriticalSection( LPCRITICAL_SECTION sec ); VOID DeleteCriticalSection( LPCRITICAL_SECTION sec ); VOID EnterCriticalSection( LPCRITICAL_SECTION sec ); VOID LeaveCriticalSection( LPCRITICAL_SECTION sec ); BOOL TryEnterCriticalSection ( LPCRITICAL_SECTION sec );

Only usable from within the same process



Critical sections are initialized and deleted but do not have handles



Only one thread at a time can be in a critical section



A thread can enter a critical section multiple times - however, the number of Enter- and Leave-operations must match



Leaving a critical section before entering it may cause deadlocks



No way to test whether another thread is in a critical section

Critical Section Example

/* counter is global, shared by all threads */ volatile int counter = 0; CRITICAL_SECTION crit; InitializeCriticalSection ( &crit ); /* … main loop in any of the threads */ while (!done) { _try { EnterCriticalSection ( &crit ); counter += local_value; LeaveCriticalSection ( &crit ); } _finally { LeaveCriticalSection ( &crit ); } } DeleteCriticalSection( &crit );

Synchronizing Threads with Kernel Objects

DWORD WaitForSingleObject( HANDLE hObject, DWORD dwTimeout ); DWORD WaitForMultipleObjects( DWORD cObjects, LPHANDLE lpHandles, BOOL bWaitAll, DWORD dwTimeout );

The following kernel objects can be used to synchronize threads:

–

Processes

–

Threads

–

Files

File change notifications Mutexes –

Console input

Events (auto-reset + manual-reset) Waitable timers

Wait Functions - Details



WaitForSingleObject():

–

hObject specifies kernel object

–

dwTimeout specifies wait time in msec



dwTimeout == 0 - no wait, check whether object is signaled



dwTimeout == INFINITE - wait forever



WaitForMultipleObjects():

–

cObjects <= MAXIMUM_WAIT_OBJECTS (64)

–

lpHandles - pointer to array identifying these objects

–

bWaitAll - whether to wait for first signaled object or all objects



Function returns index of first signaled object



Side effects:

–

Mutexes, auto-reset events and waitable timers will be reset to non signaled state after completing wait functions

Mutexes

Mutexes work across processes



First thread has to call CreateMutex()



When sharing a mutex, second thread (process) calls CreateMutex() or OpenMutex()



fInitialOwner == TRUE gives creator immediate ownership



Threads acquire mutex ownership using WaitForSingleObject() or WaitForMultipleObjects()



ReleaseMutex() gives up ownership



CloseHandle() will free mutex object

Mutexes

HANDLE CreateMutex( LPSECURITY_ATTRIBUTE lpsa, BOOL fInitialOwner, LPTSTR lpszMutexName ); HANDLE OpenMutex( LPSECURITY_ATTRIBUTE lpsa, BOOL fInitialOwner, LPTSTR lpszMutexName ); BOOL ReleaseMutex( HANDLE hMutex );

Mutex Example

/* counter is global, shared by all threads */ volatile int done, counter = 0; HANDLE mutex = CreateMutex( NULL, FALSE, NULL ); /* main loop in any of the threads, ret is local */ DWORD ret; while (!done) { ret = WaitForSingleObject( mutex, INFINITE ); if (ret == WAIT_OBJECT_0) counter += local_value; else /* mutex was abandoned */ break; /* exit the loop */ ReleaseMutex( mutex ); } CloseHandle( mutex );

Comparison - POSIX mutexes



POSIX pthreads specification supports mutexes

–

Synchronization among threads in same process



Five basic functions:

–

pthread_mutex_init()

–

pthread_mutex_destroy()

–

pthread_mutex_lock()

–

pthread_mutex_unlock()

–

pthread_mutex_trylock()



Comparison:

–

pthread_mutex_lock() will block - equivalent to WaitForSingleObject( hMutex );

–

pthread_mutex_trylock() is nonblocking (polling) - equivalent to WaitForSingleObject() with timeout == 0

Semaphores



Semaphore objects are used for resource counting

–

A semaphore is signaled when count > 0



Threads/processes use wait functions

–

Each wait function decreases semaphore count by 1

–

ReleaseSemaphore() may increment count by any value

–

ReleaseSemaphore() returns old semaphore count

Semaphores

HANDLE CreateSemaphore( LPSECURITY_ATTRIBUTE lpsa, LONG cSemInit, LONG cSemMax, LPTSTR lpszSemName ); HANDLE OpenSemaphore( LPSECURITY_ATTRIBUTE lpsa, LONG cSemInit, LONG cSemMax, LPTSTR lpszSemName ); HANDLE ReleaseSemaphore( HANDLE hSemaphore, LONG cReleaseCount, LPLONG lpPreviousCount );

Events



Multiple threads can be released when a single event is signaled (barrier synchronization)

–

Manual-reset event can signal several thread simultaneously; must be reset manually

–

PulseEvent() will release all threads waiting on a manual-reset event and automatically reset the event

–

Auto-reset event signals a single thread; event is reset automatically

–

fInitialState == TRUE - create event in signaled state

Events

HANDLE CreateEvent( LPSECURITY_ATTRIBUTE lpsa, BOOL fManualReset, BOOL fInititalState LPTSTR lpszEventName ); BOOL SetEvent( HANDLE hEvent ); BOOL ResetEvent( HANDLE hEvent ); BOOL PulseEvent( HANDLE hEvent );

Comparison - POSIX condition variables



pthread ’s condition variables are comparable to events

–

pthread_cond_init()

–

pthread_cond_destroy()



Wait functions:

–

pthread_cond_wait()

–

pthread_cond_timedwait()



Signaling:

–

pthread_cond_signal() - one thread

–

pthread_cond_broadcast() - all waiting threads



No exact equivalent to manual-reset events

Anonymous pipes

Half-duplex character-based IPC



cbPipe: pipe byte size; zero == default



Read on pipe handle will block if pipe is empty



Write operation to a full pipe will block



Anonymous pipes are oneway

BOOL CreatePipe( PHANDLE phRead, PHANDLE phWrite, LPSECURITY_ATTRIBUTES lpsa, DWORD cbPipe ) main prog1 pipe prog2

I/O Redirection using an Anonymous Pipe

/* Create default size anonymous pipe, handles are inheritable. */ if (!CreatePipe (&hReadPipe, &hWritePipe, &PipeSA, 0)) { fprintf(stderr, “ Anon pipe create failed\n ” ); exit(1); } /* Set output handle to pipe handle, create first processes. */ StartInfoCh1.hStdInput = GetStdHandle (STD_INPUT_HANDLE); StartInfoCh1.hStdError = GetStdHandle (STD_ERROR_HANDLE); StartInfoCh1.hStdOutput = hWritePipe; StartInfoCh1.dwFlags = STARTF_USESTDHANDLES; if (!CreateProcess (NULL, (LPTSTR)Command1, NULL, NULL, TRUE, 0, NULL, NULL, &StartInfoCh1, &ProcInfo1)) { fprintf(stderr, “ CreateProc1 failed\n ” ); exit(2); } CloseHandle (hWritePipe);

Pipe example (contd.)

/* Repeat (symmetrically) for the second process. */ StartInfoCh2.hStdInput = hReadPipe; StartInfoCh2.hStdError = GetStdHandle (STD_ERROR_HANDLE); StartInfoCh2.hStdOutput = GetStdHandle (STD_OUTPUT_HANDLE); StartInfoCh2.dwFlags = STARTF_USESTDHANDLES; if (!CreateProcess (NULL, (LPTSTR)targv, NULL, NULL,TRUE,/* Inherit handles. */ 0, NULL, NULL, &StartInfoCh2, &ProcInfo2)) { fprintf(stderr, “ CreateProc2 failed\n ” ); exit(3); } CloseHandle (hReadPipe); /* Wait for both processes to complete. */ WaitForSingleObject (ProcInfo1.hProcess, INFINITE); WaitForSingleObject (ProcInfo2.hProcess, INFINITE); CloseHandle (ProcInfo1.hThread); CloseHandle (ProcInfo1.hProcess); CloseHandle (ProcInfo2.hThread); CloseHandle (ProcInfo2.hProcess); return 0;

Named Pipes



Message oriented:

–

Reading process can read varying-length messages precisely as sent by the writing process



Bi-directional

–

Two processes can exchange messages over the same pipe



Multiple, independent instances of a named pipe:

–

Several clients can communicate with a single server using the same instance

–

Server can respond to client using the same instance



Pipe can be accessed over the network

–

location transparency



Convenience and connection functions

Using Named Pipes



lpszPipeName: \\.\pipe\[path]pipename

–

Not possible to create a pipe on remote machine (. – local machine)



fdwOpenMode:

–

PIPE_ACCESS_DUPLEX, PIPE_ACCESS_INBOUND, PIPE_ACCESS_OUTBOUND



fdwPipeMode:

–

PIPE_TYPE_BYTE or PIPE_TYPE_MESSAGE

–

PIPE_READMODE_BYTE or PIPE_READMODE_MESSAGE

–

PIPE_WAIT or PIPE_NOWAIT (will ReadFile block?)

all instances of a named pipe HANDLE CreateNamedPipe (LPCTSTR lpszPipeName, DWORD fdwOpenMode, DWORD fdwPipMode DWORD nMaxInstances, DWORD cbOutBuf, DWORD cbInBuf, DWORD dwTimeOut, LPSECURITY_ATTRIBUTES lpsa );

Named Pipes (contd.)



nMaxInstances:

–

Number of instances,

–

PIPE_UNLIMITED_INSTANCES: OS choice based on resources



dwTimeOut

–

Default time-out period (in msec) for WaitNamedPipe()



First CreateNamedPipe creates named pipe

–

Closing handle to last instance deletes named pipe



Polling a pipe:

–

Nondestructive – is there a message waiting for ReadFile

BOOL PeekNamedPipe (HANDLE hPipe, LPVOID lpvBuffer, DWORD cbBuffer, LPDWORD lpcbRead, LPDWORD lpcbAvail, LPDWORD lpcbMessage);

Named Pipe Client Connections



CreateFile with named pipe name:

–

\\.\pipe\[path]pipename

–

\\servername\pipe\[path]pipename

–

First method gives better performance (local server)



Status Functions:

–

GetNamedPipeHandleState

–

SetNamedPipeHandleState

–

GetNamedPipeInfo

Convenience Functions



WriteFile / ReadFile sequence:

BOOL TransactNamedPipe( HANDLE hNamedPipe, LPVOID lpvWriteBuf, DWORD cbWriteBuf, LPVOID lpvReadBuf, DWORD cbReadBuf, LPDOWRD lpcbRead, LPOVERLAPPED lpa);

Convenience Functions

 CreateFile / WriteFile / ReadFile / CloseHandle: – dwTimeOut: NMPWAIT_NOWAIT, NMPWAIT_WIAT_FOREVER, NMPWAIT_USE_DEFAULT_WAIT

:

BOOL CallNamedPipe( LPCTSTR lpszPipeName, LPVOID lpvWriteBuf, DWORD cbWriteBuf, LPVOID lpvReadBuf, DWORD cbReadBuf, LPDWORD lpcbRead, DWORD dwTimeOut);

Server: eliminate the polling loop



lpo == NULL:

–

Call will return as soon as there is a client connection

–

Returns false if client connected between CreateNamed Pipe call and ConnectNamedPipe()



Use DisconnectNamedPipe to free the handle for connection from another client



WaitNamedPipe():

–

Client may wait for server

BOOL ConnectNamedPipe (HANDLE hNamedPipe,

‘s ConnectNamedPipe()



Security rights for named pipes:

–

GENERIC_READ, GENERIC_WRITE, SYNCHRONIZE

Comparison with UNIX



UNIX FIFOs are similar to a named pipe

–

FIFOs are half-duplex

–

FIFOs are limited to a single machine

–

FIFOs are still byte-oriented, so its easiest to use fixed-size records in client/server applications

–

Individual read/writes are atomic



A server using FIFOs must use a separate FIFO for each client ‘s response, although all clients can send requests via a single, well known FIFO



Mkfifo() is the UNIX counterpart to CreateNamedPipe()



Use sockets for networked client/server scenarios

Client Example using Named Pipe

WaitNamedPipe (ServerPipeName, NMPWAIT_WAIT_FOREVER); hNamedPipe = CreateFile (ServerPipeName, GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL); if (hNamedPipe == INVALID_HANDLE_VALUE) { fptinf(stderr, Failure to locate server.\n"); exit(3); } /* Write the request. */ WriteFile (hNamedPipe, &Request, MAX_RQRS_LEN, &nWrite, NULL); /* Read each response and send it to std out. */ while (ReadFile (hNamedPipe, Response.Record, MAX_RQRS_LEN, &nRead, NULL)) printf ("%s", Response.Record); CloseHandle (hNamedPipe); return 0;

Server Example Using a Named Pipe

hNamedPipe = CreateNamedPipe (SERVER_PIPE, PIPE_ACCESS_DUPLEX, PIPE_READMODE_MESSAGE | PIPE_TYPE_MESSAGE | PIPE_WAIT, 1, 0, 0, CS_TIMEOUT, pNPSA); while (!Done) { printf ("Server is awaiting next request.\n"); if (!ConnectNamedPipe (hNamedPipe, NULL) || !ReadFile (hNamedPipe, &Request, RQ_SIZE, &nXfer, NULL)) { fprintf(stderr, “ Connect or Read Named Pipe error\n ” ); exit(4); } printf( “ Request is: %s\n", Request.Record); /* Send the file, one line at a time, to the client. */ fp = fopen (File, "r"); while ((fgets (Response.Record, MAX_RQRS_LEN, fp) != NULL)) WriteFile (hNamedPipe, &Response.Record, (strlen(Response.Record) + 1) * TSIZE, &nXfer, NULL); fclose (fp); DisconnectNamedPipe (hNamedPipe); } /* End of server operation. */

Win32 IPC - Mailslots



Broadcast mechanism:

–

One-directional

Mailslots bear some nasty implementation details; they are almost never used –

Mutliple writers/multiple readers (frequently: one-to-many comm.)

–

Message delivery is unreliable

–

Can be located over a network domain

–

Message lengths are limited (w2k: < 426 byte)



Operations on the mailslot:

–

Each reader (server) creates mailslot with CreateMailslot()

–

Write-only client opens mailslot with CreateFile() and uses WriteFile() – open will fail if there are no waiting readers

–

Client ‘s message can be read by all servers (readers)



Client lookup: \\*\mailslot\mailslotname

–

Client will connect to every server in network domain

Locate a server via mailslot

Mailslot Servers

App client 0 hMS = CreateMailslot( “\\.\mailslot\status“); ReadFile(hMS, &ServStat); /* connect to server */ App client n hMS = CreateMailslot( “\\.\mailslot\status“); ReadFile(hMS, &ServStat); /* connect to server */

Mailslot Client

Message is sent periodically App Server } While (...) { Sleep(...); hMS = CreateFile( “\\.\mailslot\status“); ...

WriteFile(hMS, &StatInfo

Creating a mailslot



lpszName points to a name of the form

–

\\.\mailslot\[path]name

–

Name must be unique; mailslot is created locally



cbMaxMsg is msg size in byte



dwReadTimeout

–

Read operation will wait for so many msec

–

0 – immediate return

–

MAILSLOT_WAIT_FOREVER – infinite wait

HANDLE CreateMailslot(LPCTSTR lpszName, DWORD cbMaxMsg, DWORD dwReadTimeout, LPSECURITY_ATTRIBUTES lpsa);

Opening a mailslot



CreateFile with the following names:

–

\\.\mailslot\[path]name - retrieve handle for local mailslot

–

\\host\mailslot\[path]name - retrieve handle for mailslot on specified host

–

\\domain\mailslot\[path]name - returns handle representing all mailslots on machines in the domain

–

\\*\mailslot\[path]name - returns handle representing mailslots on machines in the system ‘s primary domain: max mesg. len: 400 bytes

–

Client must specifiy FILE_SHARE_READ flag



GetMailslotInfo() and SetMailslotInfo() are similar to their named pipe counterparts

Thoughts Change Life

意念改

变生活