Transcript m-Kernel Construction (2)

L4 Version 4
(Jochen Liedtke)
Uwe Dannowski
Espen Skoglund
Volkmar Uhlig
© 2001 Universität Karlsruhe, System Architecture Group
(X.2)
About presentation


Audience should be familiar with L4
Version 2 and/or Version X.0
Presentation will focus on:



Task/thread creation deletion, and
Communication
In depth discussions afterwards
© 2001 Universität Karlsruhe, System Architecture Group
Major changes from Version 2

No Clans and Chiefs


Use “redirection”
instead
Redesigned IPC API


Architecture
independent API


Two phases
(build and send)


Optimized for IDL
compilers
Support for absolute
timeouts
© 2001 Universität Karlsruhe, System Architecture Group
64-bit/32-bit clean
Use virtual registers
to expand register
set
(E.g., for IPC messages)

Portable
(Recompilation needed)
Thread IDs

Word sized



32 or 64 bit depending
on architecture

Task number not
encapsulated in ID


No restrictions on
number of threads in a
task (address space)
Threads can be migrated
between tasks (address
spaces)
Version numbers not
assigned by kernel

Each thread has a
global and a local ID

Local ID is only valid
inside an address space
(Typically location of
UTCB)

© 2001 Universität Karlsruhe, System Architecture Group
Handled through
privileged
ThreadControl() syscall
Global IDs are unique
across all spaces
Privileged threads and system calls

Privileged threads are able to perform certain
privileged (“unsafe”) syscalls
(e.g., MemoryControl(), ProcessorControl())


Only threads within privileged (tasks) address
spaces are themselves privileged
Only tasks started initially by the kernel are
privileged
(Sigma0 and Root Task)
© 2001 Universität Karlsruhe, System Architecture Group
Version 4:
Task and Thread
Creation/Deletion
Espen Skoglund
© 2001 Universität Karlsruhe, System Architecture Group
Task creation

Three steps required



Create empty space
Configure newly
created space
Start initial thread

Two syscalls
involved



Both syscalls are
privileged

© 2001 Universität Karlsruhe, System Architecture Group
SpaceControl()
ThreadControl()
Only root task can
create/delete tasks
The SpaceControl() system call
ThreadId
Fpage
Fpage
ThreadId
Word


Word
Space Specifier
Kernel Interface Page Area
Utcb Area
Redirector
Control
Address spaces have no IDs
A thread ID specifies the
space in which the thread
resides



Result
The interface page is mapped
by the kernel
Argument specifies the virtual
memory area of interface
page
A nil-page deletes the
specified space
(If space only contains one thread)
© 2001 Universität Karlsruhe, System Architecture Group
The SpaceControl() system call
ThreadId
Fpage
Fpage
ThreadId
Word


Word
Space Specifier
Kernel Interface Page Area
Utcb Area
Redirector
Control
UTCBs are mapped by kernel
Argument specifies virtual
memory area of the UTCBs


© 2001 Universität Karlsruhe, System Architecture Group
Result
Specifies the “redirector” who
controls IPC operations in the
address space
Redirectors currently have
undefined semantics
The SpaceControl() system call
ThreadId
Fpage
Fpage
ThreadId
Word

Controls address space
specific settings



Word
Space Specifier
Kernel Interface Page Area
Utcb Area
Redirector
Control
Small spaces
ASID allocation
Not in Schedule() to allow for
global allocation schemes
© 2001 Universität Karlsruhe, System Architecture Group

Result
Indicates whether operation
was successful or not
The ThreadControl() system call
ThreadId
ThreadId
ThreadId
ThreadId


Word
Destination
Space Specifier
Scheduler
Pager
Identifies addressed thread
May have different version
number than existing thread


(Will change version number)

Non-existent threads will be
created
© 2001 Universität Karlsruhe, System Architecture Group

Result
Specfies space in which to
create thread
A specifier equal to
Destination creates a new
empty address space
A nil-thread specifier deletes
the addressed thread
The ThreadControl() system call
ThreadId
ThreadId
ThreadId
ThreadId



Species scheduler for
addressed thread
Species pager for addressed
thread
Upon thread creation:


Nil ID creates inactive thread
Non-nil ID creates active
thread
© 2001 Universität Karlsruhe, System Architecture Group
Word
Destination
Space Specifier
Scheduler
Pager

Result
Indicates whether operation
was successful or not
Task creation - walkthrough
P
ThreadControl (A, A, nil, nil)
Privileged thread
A
Thread created inactive
Empty address space created
(i.e., without Kernel Interface
Page and UTCB area)
© 2001 Universität Karlsruhe, System Architecture Group
Task creation - walkthrough
P
SpaceControl (A, ~, ~, ~)
A
Thread is still inactive
Address space now contains
Kernel Interface Page
and UTCB area
© 2001 Universität Karlsruhe, System Architecture Group
Task creation - walkthrough
P
ThreadControl (A, A, SA, PA)
A
Thread is now active
How does A know where
to start executing?
© 2001 Universität Karlsruhe, System Architecture Group
Thread startup protocol


Actively created threads start receiving a
startup message from their pager
Pager sends IP and SP
to thread
SP
MR
MR
IP
Kernel sets IP and
MR
~
0
0
SP according to
Startup message received
from pager
startup message
The initial thread starts executing
2

1
0

© 2001 Universität Karlsruhe, System Architecture Group
2
Task creation - walkthrough
P
A
Recv (PA, ~, ~)
Send (A, ip, sp)
PA
© 2001 Universität Karlsruhe, System Architecture Group
Task deletion

Delete all but one
thread from address
space



Address space need not
be completely deleted

ThreadControl (t, nil, ...)
Delete address space
using remaining thread


SpaceControl (t, nil, ...)
Safe since only
privileged threads can
create new threads
inside address space
© 2001 Universität Karlsruhe, System Architecture Group


Leave remaining thread
as inactive
Flush complete address
space
Allows for cheap task
creation
(only one ThreadControl() call
to active initial thread)
The ExchangeRegisters() system call
ThreadId
Word
Word
Word
Word
ThreadId
Word



Destination
Control
SP
IP
FLAGS
Pager
User Defined Handle
Addressed thread
Must be located in same
address space
May use global or local ID
ThreadId
Word
Word
Word
Word
ThreadId
Word

Specifies what to do



© 2001 Universität Karlsruhe, System Architecture Group
Result
Control
SP
IP
FLAGS
Pager
User Defined Handle
Halt/resume thread?
Abort send and/or receive IPC
operations?
Set any of the SP, IP, FLAGS,
Pager, or User Defined Handle
values of addressed thread?
The ExchangeRegisters() system call
ThreadId
Word
Word
Word
Word
ThreadId
Word


Destination
Control
SP
IP
FLAGS
Pager
User Defined Handle
Specifes new SP, IP, FLAGS,
and Pager of addressed
thread
Ignored if the respective flag
in Control is not set
© 2001 Universität Karlsruhe, System Architecture Group
ThreadId
Word
Word
Word
Word
ThreadId
Word

Result
Control
SP
IP
FLAGS
Pager
User Defined Handle
Specifies new User Defined
Handle


Value ignored by kernel
Can be used by, e.g., userlevel thread packages
The ExchangeRegisters() system call
ThreadId
Word
Word
Word
Word
ThreadId
Word

Destination
Control
SP
IP
FLAGS
Pager
User Defined Handle
Addressed thread’s global or
local ID

Depending on whether
Destination was global or local
ThreadId
Word
Word
Word
Word
ThreadId
Word


Old values of SP, IP, etc.
Current state of addressed
thread


© 2001 Universität Karlsruhe, System Architecture Group
Result
Control
SP
IP
FLAGS
Pager
User Defined Handle
Halted or not?
Within IPC send or receive
phase?
Thread creation - two alternatives


Create new threads using ThreadControl()
Alternative A:


Generate a pool of new threads
Each thread is halted upon startup
(using ExchangeRegisters())

Threads can be “created” locally from the pool
(using ExchangeRegisters() to resume them)

Alternative B:


No pool is generated
Each thread creation must happen externally
© 2001 Universität Karlsruhe, System Architecture Group
Version 4:
Communication
Espen Skoglund
© 2001 Universität Karlsruhe, System Architecture Group
Version 4 IPC mechanism
IPC operations happen in two steps:

Message construction


Specify message contents (64 words)
Message sending

Invoke IPC system call
© 2001 Universität Karlsruhe, System Architecture Group
Message construction

Messages are stored in
registers (MR0  MR63)



Message registers are
backed by real registers
or memory
First register (MR0) acts
as message tag
Subsequent registers
contain:


Untyped words (u), and
Typed words (t)
(e.g., map item, string item)
© 2001 Universität Karlsruhe, System Architecture Group
Number of
untyped words
Number of typed
words
Various IPC flags
MR0
label
flags
t
u
Message Tag
Freely available
(e.g., request type)
Message construction

Messages are stored in
registers (MR0  MR63)



Message registers are
backed by real registers
or memory
First register (MR0) acts
as message tag
Subsequent registers
contain:


Untyped words (u), and
Typed words (t)
(e.g., map item, string item)
© 2001 Universität Karlsruhe, System Architecture Group
MR8
MR7
MR6
MR5
MR4
MR3
MR2
MR1
MR0
label
flags
Message
5t
3
u
Message construction


Typed items occupy
one or more words
Three currently
defined items:




Map item (2 words)
Grant item (2 words)
String item (2+ words)
Typed items can have
arbitrary order
© 2001 Universität Karlsruhe, System Architecture Group
MR8
MR7
String Item
MR6
MR5
MR4
Map Item
MR3
MR2
MR1
MR0
label
flags
Message
5t
3
u
Map and Grant items

Two words:

Send base


send base
MRi+1
0
100C MRi
Map Item
Fpage



Similar to Version 2
send fpage
Similar to Version 2
Contains more access bits
Lower bits of send base
indicates map or grant
item
© 2001 Universität Karlsruhe, System Architecture Group
send fpage
send base
MRi+1
0
101C MRi
Grant Item
size 0wrx
location
Fpage
String items



No alignment required
Max size 4MB (per string)
Compound strings
supported


Allows scatter-gather
Incorporates
cacheability hints

Reduce cache pollution
for long copy operations
© 2001 Universität Karlsruhe, System Architecture Group
string pointer
string length
String Item
“hh” indicates
cacheability hints
for the string
MRi+1
0
0
0hhC MRi
String items
New string specifier
may of course contain
substrings
Different size compound
strings require a new
string specifier
All substrings are of
same size
string pointer
MRi+j+k
string pointer
MRi+j+3
string pointer
MRi+j+2
string length
string pointer
MRi+j
string pointer
MRi+2
string pointer
MRi+1
string length
String Item
“hh” indicates
cacheability hints
for the string
© 2001 Universität Karlsruhe, System Architecture Group
0 k 0- 1 0hhC MRi+j+1
0 j -0 1 0hhC MRi
1
Message sending


The Ipc() syscall will transfer message
registers from sender to receiver
Strings will be copied to receiver
(if receiver has specified sufficiently large receive buffers)

Mappings will be mapped/granted
(if receiver has specified an appropriate receive window)

No page-faults will occur if no strings are
copied


No page-faults on accessing message registers
“Short IPCs” are thus up to 64 words
© 2001 Universität Karlsruhe, System Architecture Group
The Ipc() system call
ThreadId
ThreadId
Word


To
From Specifier
Timeouts
Receiver of message
Nil if no send-phase included
ThreadId

Specifies who to receive from:



Different To and From
specifiers allows for, e.g, a
combined send reply and
wait for other thread!
© 2001 Universität Karlsruhe, System Architecture Group
From

Nil indicates no receive-phase
Anythread indicates an open
receive
Anylocalthread indicates an
open receive from a local
thread
Any other thread ID
The Ipc() system call
ThreadId
ThreadId
Word

Specifies Send and Receive
timeouts


To
From Specifier
Timeouts
Timeout before IPC operation
initiates
Transfer timeout specified in a
separate virtual register

Combined timeout of all page
faults during IPC string copy
© 2001 Universität Karlsruhe, System Architecture Group
ThreadId

From
Indicates sender of message
(only for IPCs with receive phase)


Result of IPC is stored in MR0
Received untyped words and
typed items are stored in
remaining MRs
The Lipc() system call (local IPC)


Identical semantics as
normal Ipc()
Lipc() has some
restrictions





Sender and receiver are
in same address space
Sender and receiver runs
on same processor
Includes both send and
receive phase
Receive timeout is 
(infinite)
IPC includes no
map/grant operation
© 2001 Universität Karlsruhe, System Architecture Group

Lipc() can fallback to
normal Ipc()


Enables fast local IPC
implementation, useful
for, e.g.:



If restrictions are not met
Distributing requests
Intra-address-space
synchronization
Lipc() not necessarily
needed for all
architectures
Receiving messages

Receiver buffers are
specified in registers
(BR0 … BR33)


Backed by real registers
or memory
First BR (BR0) contains
“Acceptor”


May specify receive
window (if not nil-fpage)
May indicate presence of
receive strings/buffers
(if s-bit set)
© 2001 Universität Karlsruhe, System Architecture Group
receive window
Acceptor
000s BR0
Receiving messages
If C-bit in string item is cleared,
it indicates that no more
receive buffers are present
A receive buffer can of course
be a compound string
If C-bit in string item is set, it
indicates presence of more
receive buffers
string pointer
BR4+j
string pointer
BR5
string pointer
BR4
string length
string pointer
string length
receive window
The s-bit set indicates presence
of string items acting as receive
buffers
© 2001 Universität Karlsruhe, System Architecture Group
0 j -0 1 0hhC
0 BR3
Acceptor
BR2
0
0
0hhC
1 BR1
0001
000s BR0
The Unmap() system call
Word
Void
Control



Specifies unmap or flush, and
How many fpages to unmap


Fpages are listed in MRs
Each fpage indicate
permissions to remove





© 2001 Universität Karlsruhe, System Architecture Group
Each fpage (MR) will on
return contain page access
bits
Written
Referenced
eXecuted
Page access bits are reset
upon unmap
A no revocation unmap will
just reset and return previous
access bits