Transcript Unit OS C: Programming
Unit OS C: Interoperability
C.2. Programming Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze
Copyright Notice
© 2000-2005 David A. Solomon and Mark Russinovich These materials are part of the
Windows Operating System Internals Curriculum Development Kit,
developed by David A. Solomon and Mark E. Russinovich with Andreas Polze Microsoft has licensed these materials from David Solomon Expert Seminars, Inc. for distribution to academic organizations solely for use in academic environments (and not for commercial use) 2
Roadmap for Section C.2
UNIX Environments on Windows Approaches to UNIX Programming on Windows Interix - the POSIX Subsystem for Windows Common Desktop, Filesystem, IPC Interix in Action: /proc-fs, gcc, truss, ksh, csh Integration scenarios: X11 on Windows Cygwin Compatibility Library and tools UWIN from ATT Research MKS Compatibility Toolkit 3
UNIX Programming on Windows
Interix - POSIX subsystem for Windows Part of Services for UNIX 3.5
Applicable to Windows NT/2000/XP/2003 Compatibility libraries for Windows-API: Applicable to all Windows variants Visual Studio libraries for C/C++ Cygwin library and runtime UWIN library and runtime MKS toolkit (Mortice Kern Systems) Virtual Machine Environments VMware (Intel only), VirtualPC (Intel + PowerPC) 4
The Interix Subsystem as part of Windows Services for UNIX 3.5
A full POSIX subsystem for Windows Interix replaces the original POSIX subsystem on Windows NT, Windows 2000, Windows XP, and Windows Server 2003 Full network support Interoperability between POSIX and Windows subsystems (CreateProcess()) Bundled with many essential UNIX tools (including X11R6 clients) From Windows Server 2003 R2 forward SFU does not apply and will be blocked from installing Microsoft acquired Softway Systems, the makers of Interix on Sept. 17, 1999 Interix allows to run UNIX applications and scripts on Windows. Interix is now shipped as part of Services for Unix 3.5
5
Interix Features
KornShell, C Shell and Bourne Shell with full job control Over 300 utilities, including scripting tools such as awk, sed, perl, Tcl/Tk Berkeley System Distribution (BSD) sockets mapped to Winsock Memory-mapped files System V (SVID) interprocess communication (IPC): semaphores, message queues and shared memory UNIX Look and Feel: Pseudo-terminal support Color curses support Ability to execute Windows applications from Interix Full tty semantics mapped to console windows Pseudo-terminal support File-link support and true case-sensitive file names 6
Interix Features (contd.)
Full integration with Windows security model, administration, file systems, networking and printers Support for POSIX.1 (system services) and POSIX.2 (shell and utilities) X11R6 + X11R5 runtime, including X clients:
xterm, twm, xrdb,
and
xlsclients
Support for daemons (inetd) running as Windows services
telnetd
&
rlogind
services (multiuser login support) Berkeley r-utilities (servers and clients)
cron, syslogd
& other daemon/service support Internet clients:
ftp, telnet, ping, rsh
Integrated tape device support 7
Interix Software Development Kit
Development tools, including
make, RCS, lex, yacc, cc, c89, nm, ar, strip
ANSI C, POSIX.1 and POSIX.2 interfaces Color curses library BSD-style sockets library support pthread implementation X11R6 + X11R5 libraries and header files
gcc
(GNU C) compiler (gcc 3.3)
g++
(GNU C++) compiler
g77
(GNU FORTRAN77) compiler Support for Microsoft Visual C/C++ 8
Interix The POSIX Subsystem for Windows
Environment Subsystems User Mode System & Service Processes User Application Subsystem DLL OS/2 Kernel Mode Executive Device Drivers Hardware Abstraction Layer (HAL) Windows Kernel POSIX Windows Windows User/GDI Device Driver
9
Interix/Windows API Integration
Interix App Interix App Interix Subsystem
Common File Systems
CDFS Common Desktop
Pipes, sockets Shared memory Win32exec COM
Windows Kernel Win App Win App Windows Subsystem NTFS FAT
10
Common Desktop Interix shell appears in a console window with standard controls Cut-and-paste support between Interix, Windows, & X11 applications Launch Interix applications from a desktop icon or menu Standard Windows tools apply
InstallShield, etc.
11
Common File System
Interix supports NTFS, FAT, CDFS.
NTFS provides case sensitive filenames, group ownership, file modes (ACLs) and hard links Windows groups to UNIX uid/gid mapping % id uid=197611(andreas) gid=197121(None) groups=197121(None), 65792(+Everyone), 1976 14(Debugger Users), 131616(+Administrators), 131617(+Users), 66820(+INTERACTIVE) , 66827(+Authenticated Users), 4095(CurrentSession), 66048(+LOCAL) % % who am I andreas ttyn01 Dec 5 20:51 NTFS I/O operation audit features are available to ported UNIX apps on Interix 12
Common Execution Environment
Interix applications can fork/exec Windows applications Can shell script Windows apps ksh and csh are supported Used in the product in lp and cc Interix implements UNIX directory hierarchy /usr, /bin, /sbin, /etc, /dev … Mapping for drive letters: /dev/fs/C/… /proc file system - statistics about Windows + Interix apps Windows apps explicitly for the UNIX environment 13
/proc filesystem in action
Demo: list active processes via ksh-script /proc/$pid/cmdline contains 0-terminated list of arguments for each process on the system $ $ for pid in `ls /proc` > do > echo "$pid\t\c" > cat /proc/$pid/cmdline | \ tr "\000" "\n" | \ tail -n 1 > done The script’s output (shortened) 0 IdleProcess 1168 C:\WINDOWS\system32\spoolsv.exe
1264 C:\WINDOWS\VMADD\VMSRVC.EXE
1276 C:\WINDOWS\system32\nfsclnt.exe
1336 "C:\Program Files\Common Files\Microsoft Shared\VS7Debug\mdm.exe" 1428 C:\SFU\common\rshsvc.exe
1477 sh 1572 C:\WINDOWS\System32\VPCMap.exe
1632 C:\WINDOWS\system32\PSXRUN.EXE
164 C:\WINDOWS\System32\alg.exe
1672 C:\WINDOWS\system32\psxss.exe
1688 C:\SFU\Mapper\mapsvc.exe
1857 /usr/sbin/cron 2032 C:\WINDOWS\system32\wscntfy.exe
2164 C:\WINDOWS\System32\svchost.exe -k HTTPFilter 2876 "C:\WINDOWS\system32\posix.exe" /u /c /bin/csh -l 321 /usr/sbin/init 328 \SystemRoot\System32\smss.exe
… 14
Interix with gcc and truss in action
% % cat > hello.c
# include
Interprocess Communication Applications can share data between subsystems via
Pipes Sockets (Local and Remote) Shared Memory (Windows mapped files to System V shared memory) 16
INTERIX Integration with COM ( Example App)
Existing X11 Graphical Application Existing UNIX Character-based Application
xtide
UNIX X11 C Application
I
NTERIX
tide
UNIX C Application TIDEWRAP.DLL
I NTERIX COM DLL Module
Subsystem
New Windows Graphical Application VBTIDE.EXE
TIDE.XLS
Visual Basic GUI Or Excel Plug-in
Windows Subsystem Windows Kernel
17
Virtual PC + Interix + X11
18
Local X server for SFU 3.5
Starnet Communications offers Free X Server Download X-Win32 LX complements SFU 3.5 by enabling the display of local X Windows applications 19
UNIX Integration in Windows Server 2003 R2 New Subsystem for UNIX Applications (SUA)
An extensively upgraded subsystem derived from the Interix subsystem Biggest change: SUA permits UNIX applications it hosts to also make Win32 API calls OCI and ODBC connectivity Visual Studio Debugger Extension 64-bit Application Support This feature is supported and tested for accessing the Oracle oci.dll library and for accessing the Windows standard ODBC library 20
SUA Architecture
Full Subsystem, not an Emulator
UNIX applications Win32 ® applications
Complete UNIX subsystem on the Windows Kernel Ability to access Win32 from SUA applications 21
UNIX Integration in Windows Server 2003 R2 (contd.)
Active Directory Services NIS schema and Kerberos authentication extensions Identity Management for UNIX Administration Components Password Synchronization Server For NIS Other Network File and Print Services Microsoft Services for NFS Mapping Server NFS AdminUI + NFS client + NFS server Portmap + RpcXdr Server for NFS Authentication 22
Windows Server 2003 R2 UNIX SDKs and Tools The following are available for download:
Utilities and SDK for UNIX-based Applications Base Utilities SVR-5 Utilities Base SDK GNU SDK GNU Utilities UNIX Perl Visual Studio Debugger Add-in 23
Cygwin compatibility library – principles and problems
Cygwin DLL is loaded into an application's text segment UNIX kernel emulation Needs access to all processes running under it First Cygwin DLL to run creates shared memory areas that other processes using separate instances of the DLL can access Keeping track of open file descriptors and assisting fork/exec Per-process structures for process id, user id, signal masks, etc.
Potential security hole - shared memory region writable to all processes 24
Highlights of Cygwin Functionality
Supporting both Windows NT/2000/XP and 9x Cygwin runs as Windows app; not a subsystem Permissions and Security maps Windows file ownership and permissions to the more standard, older UNIX model File Access Windows- and POSIX-style paths Using either forward or back slashes as the directory delimiter Symbolic links are emulated by files containing a magic cookie followed by the path to which the link points Text Mode vs. Binary Mode compatibility at the expense of violating the POSIX standard that states that text and binary mode will be identical 25
Cygwin features (contd.)
Process Creation The fork call in Cygwin is particularly interesting because it does not map well on top of the Windows API Cygwin fork is a non-copy-on-write implementation similar to what was present in early flavors of UNIX Signals When a Cygwin process starts, the library starts a secondary thread for use in signal handling This thread waits for Windows events used to pass signals When a process notices it has a signal, it scans its signal bitmask and handles the signal in the appropriate fashion Most standard UNIX signals are provided Job control works as expected 26
Cygwin features (contd.)
Sockets Socket-related calls in Cygwin are mapped onto Winsock Cygwin has to perform winsock initialization when appropriate Child proc. initialize winsock if any inherited file descriptor is a socket Select The UNIX select function is another call that does not map cleanly on top of the Windows-API Windows select in Winsock only works on socket handles Cygwin select works for different types of file descriptors (sockets, pipes, handles, and a custom /dev/windows Windows messages pseudo-device) 27
Software, that has been ported to Cygwin
Apache 1.3.9 and php-4.0b2, CD Recording Software Data Plotting Library DISLIN 7.3
Differential X Protocol Compressor 3.8.0
DirectX FreeCIV 1.8.0
gcc 2.95.2
Gimp for Win32 groff 1.16.1
IBM's OpenDX Icarus Verilog
Install Cygwin now
Insight debugger, pine, wget, info, other ports Kerberos V4 eBones Distribution v1.0.3
LAPACK Libraries LessTif 0.89.9
• • • • • • • • • • • • • • • • libstdc++-v3 2.90.7
lynx-2.8.2r1
mc-4.1.36, tcltk-8.3.1
Open PTC for Windows perl-5.6.0
Perl, gdbm, cvs, and cygipc PGPlot 5.2
Python headers for mingw RPM 3.04, Lynx 2.8.3
RXVT screen-3.9.8, xchat-1.6.0
Screen Saver Library TCL/Tk 8.1
UCD SNMP X11R6.4 Archive and Zyacc XChat-text 1.5.7
28
UWIN
The UWIN package contains the following three elements: Libraries that provide the UNIX Application Programming Interface (API) Include files and development tools such as cc, yacc, lex, and make. Korn Shell and over 200 utilities such as ls, sed, cp, stty etc. www.research.att.com/sw/tools/uwin/ (free non-commercial license) The library functions are implemented in a DLL (POSIX.DLL) Programs linked with POSIX.DLL run under the Windows subsystem instead of the POSIX subsystem A cc command is provided to compile and link programs for UWIN on Windows CC calls the Visual C/C++ compiler to perform the actual compilation and linking 29
UWIN features
Process control and management: UWIN includes a spawn() family of functions that combines the functionality of fork/exec for efficiency; vfork() support With exec(), an existing process can be overlaid with another process Each process has a unique process id and belongs to a process group File descriptor semantics: Open files, pipes, sockets, fifos, and character and block special devices files have file descriptors associated with them They can be duped and inherited with UNIX semantics UNIX signal semantics: Nearly all of the UNIX signals are provided including job control signals so that ksh can stop and restart jobs A process can catch, block or ignore signals Signals can be sent to processes or to process group -D_BSDCOMPAT compiler flag for UCB UNIX signal semantics 30
UWIN features (contd.)
Support for devices Character and block devices with major and minor numbers Support for direct access to floppy drives (/dev/fd0), and SCSI tape drives (/dev/mt0), as well as ptys and ttys Terminal interface for consoles, sockets, and serial lines: The POSIX termios interface is supported for consoles with vt100 emulation UWIN supports pseudo-ttys Desktop integration for console windows: Select text, copy/paste to the clipboard The /dev/clipboard pseudo-device can be used to redirect command input and output to the clipboard The /dev/windows pseudo-device can be used within select() to block until mouse or keyboard events are received 31
UWIN features (contd.)
UCB sockets based on WINSOCK: Sockets are file descriptors and obey file descriptor semantics Internet domain protocol (AF_INET) and UNIX domain sockets are provided The multicast socket protocol is supported Pathname mapping from UNIX to Windows: UNIX style naming for all files with / as the delimiter by creating mounts. Each drive letter in Windows is mounted in a single letter directory name under / so that /a/file is the UWIN name for a:\file. Implicitly mounted directories: /sys on the system directory, /win on the windows directory, and /msdev on the directory containing the Microsoft tools Character and block special devices are in /dev UWIN also supports UNC names for files that begin with // 32
UWIN features (contd.)
UNIX naming conventions: The PATH variable is a : separated list rather than a ; separated list The cc command generates files with a .o suffix by default, etc Mapping UNIX ids/permissions to Windows permissions: Windows subject identifiers are mapped to UNIX user ids and group ids. UNIX permissions are mapped onto Windows Windows file ACLs. chown() to change the owner and or group of a file. File control locking: UNIX file control advisory locking is supported with deadlock detection. Memory mapping and shared memory and System V IPC: mmap(), System V shared memory, semaphores, message passing API Runtime linking of dynamically linked libraries: The dlopen(), dlsym() interface from System V Release 4 is provided Errors returned by Windows API functions are mapped into UNIX
errnos.
33
UWIN features (contd.)
i
-node numbers: An
i
-node number is returned when reading a file and when stating a file Hard links Hard links are supported on both NTFS and FAT file systems Symbolic links Symbolic links to files and directories can be created in UWIN and are implemented as Windows shortcuts A .lnk suffix will be appended to these file names but will not appear when reading directories with UWIN Shortcuts created by Windows 9x and Windows NT/2k/XP appear as symbolic links in UWIN but do not have the .lnk suffix removed. Fifo's Fifo's (UNIX named pipes) can be created with mkfifo() and opened as an ordinary file 34
UWIN features (contd.)
Setuid and setgid programs Use chmod() to turn on setuid permission for a program. The program must be on an NTFS file system. inet Daemons and commands UWIN 4.0 comes with the UCB inetd daemon, the telnet daemon, the rlogin daemon, and the rsh daemon. It also includes the telnet, ftp, rsh, and rlogin programs. The ssh daemon and program can be built from the source Access of Registry through the File System UWIN treats the Windows registry as a file system with registry keys treated as files, and keys that have sub-keys treated as directories. You can use UWIN tools directly on registry keys. The registry is automatically mounted under /reg during startup. Cron daemon UWIN comes with a cron daemon that is used for running at and cron jobs The cron daemon is started by /etc/rc on Windows 35
MKS Interoperability Products
Mortice Kern Systems - http://www.datafocus.com/ 36
MKS Toolkit for Developers Features
Over 300 UNIX and Windows command-line utilities (full POSIX.2 specification). MKS Korn shell (ksh) and MKS C shell (csh) Command-line build utilities including ar, cc, make, grep, vi. Tools and utilities for manipulating HTML content (htdiff, htsplit, url, mkurl). Web CGI scripting, and active Perl scripting (web, mkscgi, PScript™) File system commands (mount, umount) and symbolic link support. smtpmail and mapimail. Win32 file and user security utilities (chacl, lsacl, su). Utilities for setting up users, groups, and permissions on Windows userinfo, groupinfo, member Tape and archive commands tar, pax, cpio, mt UNIX-compatible backups. 37
NuTCracker Packaging
MKS Toolkit features (contd.): Commands to manipulate device drivers, Windows domains, and file associations (dev, domain, ftype). Remote utilities: rsh, rexec, rcp, rlogin Packaging: N
u
TCRACKER Workstation, N
u
TCRACKER Server, Power User Utilities MKS Toolkit contains the SCO XVision Eclipse X Server. Hummingbird 3D OpenGL X Server an add-on to MKS Toolkit for Enterprise Developers 38
Further Reading
Windows Services for UNIX 3.5
http://www.microsoft.com/windowsserversystem/sfu/ http://www.starnet.com/xwin32LX/get_xwin32LX.htm
UNIX Integration in Windows Server 2003 R2 http://www.microsoft.com/technet/community/events/windows2 003srvR2/add-52.mspx.
Cygwin - Cygnus Solutions http://www.cygwin.com/ UWIN - ATT Research http://www.research.att.com/sw/tools/uwin/ MKS toolkit - Mortice Kern Systems http://www.mkssoftware.com/products/tk/ds_tkdev.asp
39