Transcript ppt

Week 7
System Calls, Kernel
Threads, Kernel Debugging
Sarah Diesburg
Florida State University
1
First…

Any questions on


Part 1 – 5 system calls
Part 2 – xtime proc module
2
Story of Kernel
Development
Some context…
3
In the old days…


There were no modules or virtual machines
The kernel is a program


Has code, can compile, re-compile, make
executable
When changes needed to be made, developers
make changes in source and re-compile
4
How is the kernel different from a
regular program?

Mostly in how it is executed




Boot loader loads the kernel image/executable
during boot time
Sets kernel mode
Jumps to the entry point in the image/executable
Remember the generic booting sequence?
5
Quick Question

How would you make changes to the kernel
and run those changes?
1.
2.
3.
4.
5.
Make changes to the source
Re-complie the kernel source
Re-install the kernel source
Make sure the bootloader sees the new kernel
image (grub)
Reboot and profit!
6
Getting more modern..


Modules were created as bits of code that
can be loaded and unloaded by the kernel in
kernel mode
Made development easier

Instead of re-compiling, re-installing, and
rebooting into the new kernel, one could just recompile and load a module
7
Quick Question

How would you make changes to a module
and run those changes?
1.
2.
3.
Make changes to module source code
Re-compile the module
Load the new module
8
Present Day


Reboots into new kernels and loading new
modules often freezes machines
Enter virtual machine software


Process that emulates the hardware necessary to
run an OS in user-space
Guest OS is executed inside the virtual machine
process!
9
New System Calls
Fun but tricky!
10
Implementing System Calls
int start_elevator(void);
int issue_request(int #1, int #2,
int #3);
int stop_elevator(void);

Need to implement the functions above. But
how?
Adding a System Call to Kernel
Files to add:




LINUX_DIR/PROJECT_NAME/Makefile
LINUX_DIR/PROJECT_NAME/PROJECT_NAME.c
LINUX_DIR/PROJECT_NAME/NEW_SYSCALLS.c
Files to modify:





LINUX_DIR/arch/x86/kernel/syscall_table_32.S
LINUX_DIR/include/asm-generic/unistd.h
LINUX_DIR/include/linux/syscalls.h
LINUX_DIR/Makefile
Sample System Call

Let’s add a sample module to the kernel that
defines a sample system call

test_newsyscall(int test_int);



Takes an int test_int and issues a printk on it
Returns test_int
Problem – If our module isn’t loaded, what
happens if we call our sample system call?
13
Project 2 System Call Model
Elevator module
Core kernel
User program
14
Project 2 System Call Model
Elevator module
Core kernel
User issues system call,
core kernel looks up
system call in system call
table
User program
15
Project 2 System Call Model
Elevator module
performs system call
action
Elevator module
Core kernel
User program
16
Project 2 System Call Model
Elevator module
returns result of
system call
Elevator module
Core kernel
User program
17
Project 2 System Call Model
Elevator module
Core kernel
Core kernel forwards
result of system call to
user program
User program
18
What happens if elevator module is
not loaded?
Core kernel
User program
19
What happens if elevator module is
not loaded?
Core kernel
User issues system call,
core kernel looks up
system call in system call
table
User program
20
What happens if elevator module is
not loaded?
Elevator module is not
loaded to perform the
action…. OOPS!
Core kernel
User program
21
Module System Calls

We must create a wrapper system call!


Wrapper will call module function if module
loaded, else returns an error
Must be created in a separate, built-in kernel file
in the project folder
22
Function Pointers

We will implement our system call wrapper
with a function pointer


Pointer to a function
Function pointer can point to any function
that you implement that


Takes the same input variable types
Returns the same return type
23
Function Pointers
long (*STUB_test_newsyscall)(int test_int) = NULL;

Function pointer that





Returns a long
Name is STUB_test_newsyscall
Takes parameter int test_int
Function pointer set to NULL
Can set function pointer to a local function
you implement
24
Elevator Project

Create a file in your elevator project that just
contains the system call information

KERNEL_DIR/PROJECT_DIR/newsyscalls.c
25
KERNEL_DIR/PROJECT_DIR/
newsyscalls.c
#include <linux/linkage.h>
#include <linux/kernel.h>
#include <linux/module.h>
/* System call stub. We initialize the stub function to be NULL.
*/
long (*STUB_test_newsyscall)(int test_int) = NULL;
EXPORT_SYMBOL(STUB_test_newsyscall);
/* System call wrapper. If the stub is not NULL, it will be run,
otherwise returns -ENOSYS */
asmlinkage long sys_test_newsyscall(int test_int)
{
if (STUB_test_newsyscall)
return STUB_test_newsyscall(test_int)
else
return -ENOSYS;
26
}
KERNEL_DIR/PROJECT_DIR/
newsyscalls.c
#include <linux/linkage.h>
#include <linux/kernel.h>
#include <linux/module.h>
Function pointer
/* System call stub. We initialize the stub function to be NULL.
*/
long (*STUB_test_newsyscall)(int test_int) = NULL;
EXPORT_SYMBOL(STUB_test_newsyscall);
/* System call wrapper. If the stub is not NULL, it will be run,
otherwise returns -ENOSYS */
asmlinkage long sys_test_newsyscall(int test_int)
{
if (STUB_test_newsyscall)
return STUB_test_newsyscall(test_int)
else
return -ENOSYS;
27
}
KERNEL_DIR/PROJECT_DIR/
newsyscalls.c
#include <linux/linkage.h>
#include <linux/kernel.h>
#include <linux/module.h>
Export the pointer
so we can access
/* System call stub. We initialize the stub function to be NULL.
it later
*/
long (*STUB_test_newsyscall)(int test_int) = NULL;
EXPORT_SYMBOL(STUB_test_newsyscall);
/* System call wrapper. If the stub is not NULL, it will be run,
otherwise returns -ENOSYS */
asmlinkage long sys_test_newsyscall(int test_int)
{
if (STUB_test_newsyscall)
return STUB_test_newsyscall(test_int)
else
return -ENOSYS;
28
}
KERNEL_DIR/PROJECT_DIR/
newsyscalls.c
#include <linux/linkage.h>
#include <linux/kernel.h>
#include <linux/module.h>
/* System call stub. We initialize the stub function to be NULL.
*/
long (*STUB_test_newsyscall)(int test_int) = NULL;
EXPORT_SYMBOL(STUB_test_newsyscall);
/* System call wrapper. If the stub is not NULL, it will
be run,
System
call
otherwise returns -ENOSYS */
wrapper
asmlinkage long sys_test_newsyscall(int test_int)
{
if (STUB_test_newsyscall)
return STUB_test_newsyscall(test_int)
else
return -ENOSYS;
29
}
Elevator Project

Next create a separate file that



Holds your module code
Registers the system call pointer
Actually implements the system call behavior
30
Inside KERNEL_DIR/PROJECT_DIR/
PROJECT_NAME.C
/* Extern system call stub declarations */
extern long (*STUB_test_newsyscall)(int test_int);
long my_test_newsyscall(int test)
{
printk("%s: Your int is %i\n", __FUNCTION__, test);
return test;
}
my_module_init() {
STUB_test_newsyscall=&(my_test_newsyscall);
return 0;
}
my_module_exit() {
STUB_test_newsyscall=NULL;
}
31
Inside KERNEL_DIR/PROJECT_DIR/
PROJECT_NAME.C
/* Extern system call stub declarations */
extern long (*STUB_test_newsyscall)(int test_int);
Gain access to stub
function pointer.
long my_test_newsyscall(int test)
{
printk("%s: Your int is %i\n", __FUNCTION__, test);
return test;
}
my_module_init() {
STUB_test_newsyscall=&(my_test_newsyscall);
return 0;
}
my_module_exit() {
STUB_test_newsyscall=NULL;
}
32
Inside KERNEL_DIR/PROJECT_DIR/
PROJECT_NAME.C
/* Extern system call stub declarations */
extern long (*STUB_test_newsyscall)(int test_int);
Local function that
implements syscall
long my_test_newsyscall(int test)
{
printk("%s: Your int is %i\n", __FUNCTION__, test);
return test;
}
my_module_init() {
STUB_test_newsyscall=&(my_test_newsyscall);
return 0;
}
my_module_exit() {
STUB_test_newsyscall=NULL;
}
33
Inside KERNEL_DIR/PROJECT_DIR/
PROJECT_NAME.C
/* Extern system call stub declarations */
extern long (*STUB_test_newsyscall)(int test_int);
long my_test_newsyscall(int test)
{
printk("%s: Your int is %i\n", __FUNCTION__, test);
return test;
}
Set stub function
pointer
to local function in init
my_module_init() {
STUB_test_newsyscall=&(my_test_newsyscall);
return 0;
}
my_module_exit() {
STUB_test_newsyscall=NULL;
}
34
Inside KERNEL_DIR/PROJECT_DIR/
PROJECT_NAME.C
/* Extern system call stub declarations */
extern long (*STUB_test_newsyscall)(int test_int);
long my_test_newsyscall(int test)
{
printk("%s: Your int is %i\n", __FUNCTION__, test);
return test;
}
my_module_init() {
STUB_test_newsyscall=&(my_test_newsyscall);
return 0;
Reset stub
}
my_module_exit() {
STUB_test_newsyscall=NULL;
}
function
pointer to NULL on
module unload
35
KERNEL_DIR/PROJECT_DIR/
Makefile
obj-m := my_module.o
obj-y := newsyscalls.o
KDIR := /lib/modules/2.6.32/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
36
KERNEL_DIR/PROJECT_DIR/
Makefile
obj-m := my_module.o
obj-y := newsyscalls.o
Compile as a module
KDIR := /lib/modules/2.6.32/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
37
KERNEL_DIR/PROJECT_DIR/
Makefile
obj-m := my_module.o
obj-y := newsyscalls.o
Compile as kernel built-in
KDIR := /lib/modules/2.6.32/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
38
Core Kernel Additions

Add the new system call to the core kernel
system call table


Modify three files
Add the project directory to the main Makefile

Modify one file
39
Modifying syscall_table_32.S
…
.long
.long
.long
.long
sys_preadv
sys_pwritev
sys_rt_tgsigqueueinfo /* 335 */
sys_perf_event_open
Modifying syscall_table_32.S
…
.long
.long
.long
.long
.long
sys_preadv
sys_pwritev
sys_rt_tgsigqueueinfo /* 335 */
sys_perf_event_open
sys_test_newsyscall
/* 337 */
 Add new system call to the end of the file.
 Remember the number – you’ll need it in
userspace!
Modifying unistd.h
/* midfile */
#define __NR_perf_event_open 241
__SYSCALL(__NR_perf_event_open, sys_perf_event_open)
#undef __NR_syscalls
#define __NR_syscalls 242
/* midfile */
 Can be found around line 623…
Modifying unistd.h
/* midfile */
#define __NR_perf_event_open 241
__SYSCALL(__NR_perf_event_open, sys_perf_event_open)
#define __NR_test_newsyscall 242
__SYSCALL(__NR_test_newsyscall, sys_test_new_syscall)
#undef __NR_syscalls
#define __NR_syscalls 242
/* midfile */
Modifying unistd.h
/* midfile */
#define __NR_perf_event_open 241
__SYSCALL(__NR_perf_event_open, sys_perf_event_open)
#define __NR_test_newsyscall 242
__SYSCALL(__NR_test_newsyscall, sys_test_new_syscall)
#undef __NR_syscalls
#define __NR_syscalls 243
/* midfile */
Modifying syscalls.h
asmlinkage long sys_perf_event_open(
struct perf_event_attr __user *attr_uptr,
pid_t pid, int cpu, int group_fd, unsigned
long flags);
#endif
/* EOF */
45
Modifying syscalls.h
asmlinkage long sys_perf_event_open(
struct perf_event_attr __user *attr_uptr,
pid_t pid, int cpu, int group_fd, unsigned
long flags);
asmlinkage long sys_test_newsyscall(int test_int);
#endif
/* EOF */
46
Modifying
KERNEL_DIR/Makefile
# Objects we will link into vmlinux /
subdirs we need to visit
init-y
:= init/
drivers-y
:= drivers/ sound/ firmware/
net-y
:= net/
libs-y
:= lib/
core-y
:= usr/
endif # KBUILD_EXTMOD
47
Modifying
KERNEL_DIR/Makefile
# Objects we will link into vmlinux /
subdirs we need to visit
init-y
:= init/
drivers-y
:= drivers/ sound/ firmware/
net-y
:= net/
libs-y
:= lib/
core-y
:= usr/ my_module/
endif # KBUILD_EXTMOD
 Found around line 475…
 Can replace “my_module” with the name of your
PROJECT_DIR
48
Getting it all to work
1.
2.
3.
4.
5.
Re-compile the kernel
Install modules, install kernel
Make new initramfs image
Reboot
Test with a user-space program…
Sample User-space Program
#include
#include
#include
#include
<stdio.h>
<stdlib.h>
<sys/syscall.h>
<linux/unistd.h>
#define __SYS_TEST_ELEVATOR 337
int main()
{
int test=5;
long ret;
ret=syscall(__SYS_TEST_ELEVATOR, test);
if(ret<0)
perror("system call error");
else
printf("Function successful, returned %i\n", ret);
return 0;
}
50
syscall()
int syscall(int number, ...);

Performs the system call based on the system call’s
number

Number can be found in the syscall_table_32.S file (our example
was 337)
51
User-space Program Output

Output when my_module not loaded
system call error: Function not implemented

Output when my_module loaded
Function successful, returned 5
52
Kthreads
Run the main logic of your module in a
kthread!
53
Refresher: hello.c
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE(“Dual BSD/GPL”);
static int hello_init(void)
{
printk(KERN_ALERT “Hello, world!\n”);
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT “Goodbye, sleepy world.\n”);
}
module_init(hello_init);
module_exit(hello_exit);
54
Kernel Modules


Remember, kernel modules are very eventbased
We need a way to start an independent
thread of execution in response to an event

e.g. start_elevator() for project 2…
55
kthread_run
kthread_run(threadfn, data, namefmt, ...)

Creates a new thread and tells it to run




threadfn – the name of the function the thread should run
data – data pointer for threadfn (can be NULL if the function
does not take any args)
namefmt – name of the thread (displayed during “ps” command)
Returns a task_struct
56
kthread_run example
struct task_struct *t;
t = kthread_run(run, NULL, “my_elevator");
if (IS_ERR(t)){
ret=PTR_ERR(t);
}
57
kthread_stop
int kthread_stop(struct task_struct * k);


Sets kthread_should_stop for k to return true, wakes
the thread, and waits for the thread to exit
Returns the result of the thread function
58
kthread_stop_example
ret=kthread_stop(t);
if(ret != -EINTR)
printk("Main logic tread stopped.\n“);
59
Thread Function Example
static int run(void *arg)
{
/* Lock here */
while(!kthread_should_stop()) {
/* Do stuff */
/* Unlock here */
schedule();
/* Lock here */
}
/* Unlock here */
printk("%s: kernel thread exits.\n", __FUNCTION__);
return 0;
}
60
Thread Function Example
static int run(void *arg)
{
/* Lock here */
while(!kthread_should_stop()) {
/* Do stuff */
/* Unlock here */
schedule();
/* Lock here */
schedule() is very
important here. Why?
}
/* Unlock here */
printk("%s: kernel thread exits.\n", __FUNCTION__);
return 0;
}
61
Inefficient Solution

Thread will continue to run even though it has
nothing to do


Investigate the kthread interface to find ways
to



Eats up resources
Put thread to sleep
Wake up thread
There is more than one way to do this…
62
Debugging
63
Kernel Debugging Configurations









Timing info on printks
__depreciated logic
Detection of hung tasks
SLUB debugging
Kernel memory leak detector
Mutex/lock debugging
Kmemcheck
Check for stack overflow
Linked list debugging
64
Select Kernel Hacking
65
Enable Debugging Options
66
Debugging through procfs


Necessary for elevator project!
General process




Identify data to monitor in your module
Create a proc entry to monitor this data
Run your module
Query /proc/<entry> for that information at any
time
67
Kernel Oops and Other Errors

Kernel errors often only appear on first tty
(terminal interface)


Why?
How can I see my first tty?

On regular system – CTRL+ALT+F1


CTRL+ALT+F7 to go back to X screen
On VMware – CTRL+ALT+SPACE+F1

CTRL+ALT+SPACE+F7 to go back to X screen
68
Oops!
69
Reason
for failure
70
Current
drivers
71
Call Trace
72
Call Trace
73
Failed
command
74
Defensive Programming
•
Infinite loops and deadlocks at the kernel
level hang your machine
–
–
–
–
•
Ctrl-Alt-Del has NO effect
Ctrl-C does not matter
Ctrl-D does not matter
You may only reboot
How do you protect yourself?
–
–
Use schedule() strategically
Use preemptable versions of functions
Debugging Tools not Covered





LTT – Linux Tracing Framework
gdb – Invoking gbd on the kernel image
kgdb – A remote debugger for the kernel
Magic SysRq
printk – Rate limiting, turning on/off
Next Time



Locks
Linked lists
Elevator algorithms
77