Transcript Unix_talk_wkt.ppt

Unix: Building a Development
Environment from Scratch
Warren Toomey
TAFE Queensland
Unix: An Influential System
• An important operating system in its own
right: System V, BSD, AIX, SunOS, Solaris etc.
– And a most impressive family tree
• Unix is the most influential operating system
– MS-DOS has pipes & named devices
– POSIX: the only int’l operating system standard
– Darwin (core OS X component) is based on BSD
– Linux is a clean-room re-implementation of Unix
Unix: An Impressive Lifespan
•
•
•
•
•
•
Unix is nearly 50 years old
1969-70: Began life on the PDP-7
1970s: Grew up on the PDP-11
1980s: Teenager: VAXen and “killer micros”
1990s: Maturity: scores of platforms
2000s onwards: semi-retirement
– The family business passed to its descendants the
BSDs and its godson Linux
The Unix Philosophy
From Doug McIlroy:
•Make each program do one thing well.
•Expect the output of every program to become the
input to another, as yet unknown, program.
•Design and build software to be tried early. Don't
hesitate to throw away the clumsy parts.
•Use tools to lighten a programming task, even if
you have to build the tools and throw some of them
out afterwards.
Some More (Less?) Philosophy
Ken Thompson’s advice:
When in doubt, use brute force.
Whence This Philosophy?
• So where did this Unix philosophy come from,
and why was Unix written in the first place?
• Unix was created to fill the void when AT&T
pulled out of the Multics project in 1969
The toy had gone. The computer room was empty. People were
just despondent. Some people were leaving. There was a clear
lack of momentum.
– Sandy
Fraser
Multics
•
•
•
•
•
•
Timesharing system with elegant design
Memory-mapped files (segments)
Multi-user: authentication, access rights
Hierarchical filesystem, command-line shell
Written in a high-level language: PL/I
But … overly complex
three people could absolutely overload this giant room full of
equipment
– Doug
McIlroy
The Multics Void
• The AT&T folk that had worked on Multics
were impressed by its facilities, and by the
interactivity it provided
• April 1969: AT&T withdrew, Multics was gone
• Ken Thompson, Dennis Ritchie and Rudd
Canaday began designing a filesystem
• Using a cast-off PDP-7, Thompson began to
simulate the filesystem
The Summer of ‘69
My wife went on vacation to visit my parents -- we’d just
had a new son in August ’68 and they hadn’t seen the kid
-- and she was gone a month.
I allocated a week each to the operating system, the shell,
the editor, and the assembler, to reproduce itself.
During the month it was totally rewritten in a form that
looked like an operating system with these tools.
If not maintaining itself, then right on the verge of
maintaining itself, to totally sever the GECOS connection.
-- Ken Thompson
The Need for Unix
• From the start, Unix was built to provide a
development environment for programmers
“of the programmers, by the programmers, for the
programmers” – Abraham Lincoln
• Build the tools: file manipulation tools, a
debugger, a document markup system (roff)
• And, of course, programming languages
• But … on the PDP-7 of all platforms!
The PDP-7
• Only one data type: 18-bit word
– You could pack two ASCII characters in a word
• 8Kwords of memory: Unix in half of this
– One running program in the other half
• Only one useful register, the accumulator
• Extremely limited instruction set and
addressing modes, e.g. add but no sub
• No memory management
The Need for a Better Language
• Doug McIlroy: helped write PL/I compiler for Multics
using Bob McClure’s TMG compiler writing tool
• Unix needed a high-level language
– PDP-7 assembly is horrible, I know!
Ken one day said the PDP-7 Unix system needed a Fortran
compiler in order to be a serious system. So he actually
sat down and started to do the Fortran grammar in TMG.
It took him about a day to realize that we wouldn't want
to do a Fortran compiler at all.
– Dennis Ritchie
BCPL led to B
• BCPL from Cambridge University
• One data type: the word
• Easily portable, and a systems language
We were really taken by the language and did a lot of work
with it. It was available on CTSS and we pulled it off of
CTSS and got a version running on GECOS and did system
programming. It was too big a language to run on Unix.
That’s when B was developed.
– Ken Thompson
BCPL led to B
B was almost exactly the same as BCPL, although it looked
completely different; syntactically it was, you know, a redo.
The syntax of it was, if you didn’t look too close, you would
say it was C. Because in fact it was C, without types.
There was a keyword for extern, which means to declare an
external thing. There was a keyword auto, which declared
an auto thing. So, it would be like auto XYZ, instead of int
XYZ and it meant "word".
-- Ken Thompson
Aside: Resurrecting PDP-7 Unix
• Phil Budne and I brought this system back to
life from a set of nth-generation photocopies
– kernel, editor, assembler, some tools
• Phil rewrote the shell from scratch, I wrote a
simulator and the filesystem creation tool
• We both wrote missing tools: ls, mv, roff
• Robert Swierczek rebuilt the B compiler by
“back-porting” the 1972 C compiler
The Move to the PDP-11/20
• A PDP-11/20 was acquired, ostensibly to
process documents for the Patents dep’t
• Unix and the tools were rewritten into PDP-11
assembly code
• B was ported, but it was not the best fit
– The PDP-11 can address characters and words
• Dennis Ritchie began work on extending the B
language
B becomes C via NB
I started trying to add types to the
language B, and fairly soon afterwards write
a compiler for it.
The first phase of C was adding types to B without too much
change in the syntax. For a while it was called NB for New B.
The second phase was adding structures.
And over the summer of ‘72 we made the concerted effort
and actually did redo the whole operating system in C.
-- Dennis Ritchie
Documenting the System
• The first edition of the Unix manuals were written
in November 1971
McIlroy insisted on a high standard in the manual and of
every one of the programs that was documented. We
documented our bugs, too. I think a level of intellectual
honesty that was present in that activity is rare.
Every time another edition of the manual would be made,
there would be a flurry of cleaning activity.
-- Sandy Fraser
Features in 1st Edition Unix
•
•
•
•
•
•
•
Assembly, B, BASIC, FORTRAN, shell scripts
Editor, debugger, roff, utilities: cat, sort etc.
Multiuser, multiprogramming
Hierarchical filesystem
Devices as files: /dev/tty, /dev/mem
File redirection: cat file > /dev/tty0
Background tasks: cat file > /dev/tty0 &
The Legacy of Unix System Calls
• System calls common to 1st Edition Unix and
current Linux:
open(), close(), read(), write(), creat()
fork(), exec(), exit(), wait()
chown(), chmod(), chdir()
link(), unlink(), seek(), stat(), getuid(), brk()
• Most still have the same system call numbers
Pipes and the Unix Philosophy
• Make each program do one thing well.
• Expect the output of every program to become
the input to another, as yet unknown, program.
• Design and build software to be tried early. Don't
hesitate to throw away the clumsy parts.
• Use tools to lighten a programming task, even if
you have to build the tools and throw some of
them out afterwards.
Pipes and the Unix Philosophy
• Doug McIlroy had proposed several ideas in
1964, including this:
We should have some ways of connecting programs
like a garden hose -- screw in another segment when
it becomes necessary to massage data in another
way. This is the way of I/O also.
• For years, he pestered Thompson to
implement the idea
Pipes and the Unix Philosophy
Then one day Ken said, "I’m gonna do it." He was tired
of hearing all this stuff. He put pipes into Unix all in one
night. Most of the programs up until that time couldn’t
take standard input. Ken went in and changed all those
programs in the same night. I don’t know how.
-- Doug McIlroy
Pipes and the Unix Philosophy
I happened to have been visiting the research crew the
day they implemented pipes. It was clear to everyone
practically minutes after the system came up with pipes
working that it was a wonderful thing. Nobody would
ever go back and give that up if they could help it.
-- Dick Haight
Pipes and the Unix Philosophy
• Doug McIlroy believes that pipes crystallised the Unix
philosophy:
“Write programs that do one thing and do it well. Write
programs to work together. Write programs that
handle text streams, because that is a universal
interface.”
All of those ideas, which add up to the tool approach,
might have been there in some unformed way prior
to pipes, but they really came in after pipes.
Unix: Built by Programmers
• “of the programmers, by the programmers, for
the programmers”
• Small core of developers: Thompson, Ritchie,
McIlroy et al., influenced by other systems
• Tool builders at heart
• Simplicity:
– Avoid the second system effect of Multics
– Shoehorn Unix into the PDP-7 and PDP-11
Unix: Do One Thing & Do It Well