Transcript INFO 320 Server Technology I Week 3 Unix/Linux concepts INFO 320 week 3 www.ischool.drexel.edu Overview • This set of notes covers the history that brought us UNIX/Linux,

INFO 320
Server Technology I
Week 3
Unix/Linux concepts
INFO 320 week 3
1
www.ischool.drexel.edu
Overview
• This set of notes covers the history that
brought us UNIX/Linux, describes the
philosophy of Unix to help understand its
viewpoint, and covers basics of Linux use
and installation
– Some background
– The Unix philosophy
– Practical Unix Concepts
– Installing Ubuntu
INFO 320 week 3
2
www.ischool.drexel.edu
Some background
INFO 320 week 3
3
www.ischool.drexel.edu
Pre-Unix OS’s
• Mainframe transistor-based systems were
available in the mid-1950’s, e.g. IBM 701
– Glorified calculators
– Magnetic tape I/O
– Over 18 units sold
– 20 kB memory
– 80 kB drum
– Single “user”
INFO 320 week 3
4
www.ischool.drexel.edu
Pre-Unix OS’s
• In the 60’s we got the 500 kHz IBM 7094
– Ran batch FORTRAN programs via Fortran
Monitoring System (FMS) one at a time
– OS was called IBSYS
– Input was via punch cards to magnetic tape
– Output was magnetic tape to printers
INFO 320 week 3
5
www.ischool.drexel.edu
Pre-Unix OS’s
A punch card (7 3/8” by 3 ¼”) and a
magnetic tape (10.5” dia., ½” wide
9 track; 6250 CPI * 2400’ long = 140 MB)
INFO 320 week 3
6
www.ischool.drexel.edu
Pre-Unix OS’s
• During the 60’s, more advanced OS
concepts came into play, such as OS/360
for IBM System/360 mainframes
– Yay, multiprogramming and spooling!
• Simultaneous Peripheral Operation On Line
– Computers now use integrated circuits
• Greatly increased reliability  commercially viable
– Start of timesharing among many users
• The first was CTSS* at MIT circa 1962
* Compatible Time-Sharing System
INFO 320 week 3
7
www.ischool.drexel.edu
Pre-Unix OS’s
• MULTICS was the best example for large
scale timesharing, using a GE-645 system
– MULTiplexed Information and Computing Service
– About as powerful as a 1991 Intel 386 PC
• DEC (Digital Equipment Corp) led
minicomputers with the PDP series
– PDP-1 had 4k 18-bit words for $120k in 1961
– PDP-11 was most famous
INFO 320 week 3
8
www.ischool.drexel.edu
Pre-Unix OS’s
• DEC VAX mainframes ran VMS
– A VAX 11/780 lasted until the early 90’s
• IBM OS/400 has supported AS/400
minicomputers for decades
• IBM OS/390 and now z/OS superseded
OS/360 for ‘enterprise class servers’
– Is 248 GB RAM better? How about 1.5 TB?
INFO 320 week 3
9
www.ischool.drexel.edu
Memory Types
• An OS needs to access storage, but how
much and how long it takes to obtain vary
– Registers <1kB, 1 nsec
– Cache 4-24 MB, 2 nsec
– RAM 2 – 200+ GB, 10 nsec
– Hard drive ~1 TB, 10 msec
– Magnetic tape many TB, 100 sec
Adapted from (Tanenbaum, 2008)
INFO 320 week 3
10
www.ischool.drexel.edu
PC OS’s
• The PC (microcomputer) started in 1974
with the Intel 4040 8-bit chip and CP/M OS
• The Intel 8080 and 8088 became the basis
of the first IBM Personal Computer in 1981
– And Bill Gates created MS-DOS, after buying
its core (DOS) from Seattle Computer
Products
INFO 320 week 3
11
www.ischool.drexel.edu
PC OS’s
• MS-DOS supported Intel 286, 386, 486
CPUs with little competition (DR-DOS)
• Windows started as a DOS add-on (3.1)
– Then Windows 95/98/ME
– And 32-bit Windows NT/2000/XP/Vista/7?
• Microsoft even tried Unix in the early 80’s
– XENIX
– Oops, time to go back and pick up Unix…
INFO 320 week 3
12
www.ischool.drexel.edu
Unix
• A PDP-7 was used to create a single user
version of MULTICS, which became UNIX
– Ken Thompson from Bell Labs was the papa
of UNIX in 1969
– Originally called UNICS, for UNiplexed
Information and Computer Service, a pun on
MULTICS
– First had to compile Unix code on a GE
computer, then copy it to the PDP-7
Some of this from (Raymond, 2004)
INFO 320 week 3
13
www.ischool.drexel.edu
Lessons Learned
• UNIX learned from CTSS and MULTICS
– Especially the latter became very bloated,
with 1000s of pages of specs before anyone
touched hardware
– UNIX benefitted from being the ‘third project,’
to avoid the mistakes of the first two
– This also influenced its philosophy a great
deal
INFO 320 week 3
14
www.ischool.drexel.edu
Consider 1968-69
• Man on the moon, Apollo 11
– Their navigation computer used 22 kB RAM
• Summer of Love, Woodstock
– Monty Python, Sesame Street
• ARPAnet invented
• UNIX invented
– Computers viewed as the nucleus of
communities, not just number crunchers
•
C programming language invented*
* Dennis Ritchie
INFO 320 week 3
15
www.ischool.drexel.edu
Unix
• Soon (1971) the Unix project moved to a
powerful $65k PDP-11 minicomputer
– 24 kB of core memory (RAM)
• 16 kB for the system
• 8 kB for the user
– Hard disk held 512 kB in 1 kB blocks
– Files were limited to 64 kB at most
INFO 320 week 3
16
www.ischool.drexel.edu
Unix and portability
• UNIX was written in a third generation
programming language (C) in 1973
– Before that it was written in B. Really.
– Before UNIX, most OS’s were written in
assembly language for a specific platform
• Its source code was distributed from the
start, hence it encouraged portability
– But this also helped UNIX fragment quickly
INFO 320 week 3
17
www.ischool.drexel.edu
Unix
• Unix evolution followed two major paths
– (IBM) System V versus BSD (Berkeley
Software Distribution)
– Many commands differ between them
– Most current Unix's are a blend of V and BSD
• 1983 UNIX became a commercial product
– Spread UNIX internationally
– Almost killed it with proprietary variants
INFO 320 week 3
18
www.ischool.drexel.edu
Unix Lineage (roughly)
• System V Unix’s
–
–
–
–
–
–
• BSD Unix’s
System V (1983-89)
XENIX (1980-87)
SCO UNIX (1995)
DEC OSF/1 (1990)
DEC UNIX (1992)
Novell UNIXware
(1993)
– Solaris (1992)
– AIX, HP-UX, IRIX
–
–
–
–
–
BSD (1977-1980)
SunOS (1984)
Linux (1993)
FreeBSD (1992)
Mac OS X (2001)
• From (McKusick,
1996) and (Frisch,
2002)
INFO 320 week 3
19
www.ischool.drexel.edu
Unix Standards
• UNIX or Unix?
– UNIX is a trademark of The Open Group
• “…the definition of UNIX® takes the form of the
worldwide Single UNIX Specification”
– Unix is used as a generic term for any UNIX
variant, including Linux
• POSIX is an ISO standard of core UNIX
functionality (ISO/IEC FDIS 9945:2008)
– POSIX = "Portable Operating System
Interface [for Unix"]
INFO 320 week 3
20
www.ischool.drexel.edu
1985
• The Intel 386 chip was released
– 32-bit clean addressing, up to 4 GB of RAM
– Basis of the first UNIX-based PC
– Major threat to the workstation market
• Richard Stallman issued GNU manifesto,
launched the Free Software Foundation
– Notice that UNIX followed the open source
model even before it had that name
INFO 320 week 3
21
www.ischool.drexel.edu
Linux
• Linux was started in 1991 by Linus
Torvalds, a Finnish student, in response
to the high cost of Sun UNIX
– His goal was to port BSD UNIX to a 386
– He didn’t know the Internet was about to
become very public
– I doubt he intended to create a revolution
• He used GNU open source tools to add
onto the foundation he and others laid
INFO 320 week 3
22
www.ischool.drexel.edu
The Unix philosophy
INFO 320 week 3
23
www.ischool.drexel.edu
The Unix philosophy
• The UNIX philosophy influences how its
apps are designed and used
– It isn’t a formal doctrine, just a collection of
observations (Raymond, 2004)
• They are summarized here to help
understand how Unix works, and why it’s
structured the way it is
– A key motivation is to avoid bloat, and
manage application complexity effectively
INFO 320 week 3
24
www.ischool.drexel.edu
The Unix philosophy
• Rule of Modularity
– Write simple parts connected by clean
interfaces
– This is why Unix is a zillion little specialized
programs
• Rule of Composition
– Design programs to be connected to other
programs
– This will help understand ‘pipes’
INFO 320 week 3
25
www.ischool.drexel.edu
The Unix philosophy
• Rule of Clarity
– Clarity is better than cleverness
• Rule of Representation
– Fold knowledge and complexity into data, so
program logic can be stupid and robust
• Rule of Parsimony
– Write a big complex program only when
nothing else will do
INFO 320 week 3
26
www.ischool.drexel.edu
The Unix philosophy
• Rule of Separation
– Separate policy from mechanism; separate
interfaces from engines
• Policy (front end) changes often, so keep a
sound mechanism (back end) to implement
those changes
– This supports tiered application architectures,
like client/server or n-tier client/server
INFO 320 week 3
27
www.ischool.drexel.edu
The Unix philosophy
• Rule of Transparency
– Design for visibility and discoverability to
make inspection and debugging easier
• Rule of Simplicity
– Design for simplicity; add complexity only
when needed
• Rule of Robustness
– Robustness is the child of transparency and
simplicity
INFO 320 week 3
28
www.ischool.drexel.edu
The Unix philosophy
• Rule of Least Surprise*
– In interface design, always do the least
surprising thing; so know your audience
• Rule of Silence
– When a program has nothing surprising to
say, it should say nothing
• Rule of Repair
– When you must fail, do so as noisily and as
soon as possible
* a.k.a. the Principle of Least Astonishment
INFO 320 week 3
29
www.ischool.drexel.edu
The Unix philosophy
• Rule of Economy
– Programmer time is expensive; conserve
it in preference to machine time
• Rule of Generation
– Avoid hand hacking; write programs to
write programs when you can
• Rule of Optimization
– Prototype, then polish. Make it run, then make
it run right, then make it fast. (Kent Beck)
INFO 320 week 3
30
www.ischool.drexel.edu
The Unix philosophy
• Rule of Diversity
– Distrust all claims for ‘one true way’
– No one programming language, design, tool,
or approach is always the answer
• Rule of Extensibility
– Design for the future, because it will be here
sooner than you think
– Leave room for growth, version numbers, new
formats, new options, etc.
INFO 320 week 3
31
www.ischool.drexel.edu
Other implications
• Unix tradition encourages programs that
read and write simple text streams in
device-independent format
– Many programs are text filters
• Take one text stream as input, and produce
another text stream as output
• Text streams are like messages in networking
– This makes it easier to connect programs
INFO 320 week 3
32
www.ischool.drexel.edu
Other implications
• Keep programs independent of each other
– Each program should be a black box with a
clearly defined purpose
• When a GUI is needed, separate
interactive parts from computational parts
– Again emphasizing modularity and tiered
architecture
INFO 320 week 3
33
www.ischool.drexel.edu
In a nutshell
• The best app is made from small, simple,
beautiful cooperating pieces
– Silence is golden
– Be liberal in what you accept, and
conservative in what you send (Jon Postel)
• The antithesis of this is bloated, buggy,
useless-feature-rich code that doesn’t
work because the pieces are too complex
and heavily interconnected
INFO 320 week 3
34
www.ischool.drexel.edu
The psychology of Unix
• Be loyal to excellence
– When you see the right thing, do it
• Value your time enough not to waste it
– Lean on tools and other existing code
when possible
• Software design and implementation
should be a joyous art.
– You need to care. You need to play.
You need to be willing to explore.
INFO 320 week 3
35
www.ischool.drexel.edu
Practical Unix Concepts
INFO 320 week 3
36
www.ischool.drexel.edu
Enough theory!
• Focus on Ubuntu distribution of Linux
• Why?
– It’s committed to fully open source code
– It’s free
– It works well
– It fosters a sense of community, not just
random developers and users
• Ubuntu is based on the Debian distribution
INFO 320 week 3
37
www.ischool.drexel.edu
Ubuntu Desktop vs Server
• The Server version has optional GUI, the
Desktop version has one by default
• The Server’s kernel is better optimized
• Desktop versions are supported for three
years, Server LTS versions for five years
– LTS = Long Term Support
INFO 320 week 3
38
www.ischool.drexel.edu
Time in Unix
• In Windows, the difference between two
dates is the number of days between them
– Time zero is January 0, 1900 12:00:00 AM
• In Unix, the difference between two dates
is the number of seconds between them
– Time zero is GMT 00:00:00, January 1, 1970
– On a 32-bit system, the clock will roll over on
January 19, 2038, causing the 2038 Problem
– 64-bit clocks won’t run out for billions of years
INFO 320 week 3
39
www.ischool.drexel.edu
The Unix approach
• Everything in Unix is a file in a directory
– Files & directories don’t need extensions
– The highest level directory is the root
directory, /
– Even peripherals (CD-ROM, flash drives, etc.)
are treated as directories
• Based on a text command interface
– Fits with the text stream concept
– GUI just does the text commands for you
INFO 320 week 3
40
www.ischool.drexel.edu
The Unix approach
• UNIX developers apparently couldn’t type
to save themselves – secretaries did that
– Hence use abbreviations even for four letter
commands (ls, mv, cp)
– Commands are very terse as a result
• Unix is case and space sensitive
– Spaces and punctuation always matter!!
– Even directory listings are case sensitive
INFO 320 week 3
41
www.ischool.drexel.edu
Command syntax
• General command syntax
– Command -qualifiers target
– The –qualifiers and target are optional
• Every letter in a qualifier is a separate, case
sensitive option or modifier for that command
– ls –al 320*
• The * is a wildcard for any number of
characters, can be used anywhere in the
string e.g. 320*.txt or *.java
INFO 320 week 3
42
www.ischool.drexel.edu
Command prompt
• The command prompt may be $ or #
– The # is used in lab instructions, don’t type it!
– You can change the command prompt
• The default shell for Ubuntu is a
BASH shell (bash)
– Other shells exist, such as a C shell (csh),
Bourne shell (sh), Korn shell (ksh)
– Look in /bin/ to see which are available
INFO 320 week 3
43
www.ischool.drexel.edu
Directories
• To refer to a file, no directory implies
the current directory
– pwd tells what your current directory is
(means ‘print working directory’)
• A directory relative to your current
directory can be cited by starting the
location with a period
– ./folder/filename
INFO 320 week 3
44
www.ischool.drexel.edu
Directories
• The relative directory reference .. (two
periods) goes up one folder
– ../otherfile
• You can nest these to go up multiple
folders
– ../../work/thirdfile
• Or just cite the full path to the file, an
absolute reference, starting with /
– /etc/hosts
INFO 320 week 3
45
www.ischool.drexel.edu
Unix users
• The administrative user is often called
‘root’, a.k.a. the superuser
– Don’t log in as root, it’s too powerful!
• Users are also assigned to groups
– Groups are created by root
– Permissions can be controlled by user,
group, or anyone else (‘other’)
INFO 320 week 3
46
www.ischool.drexel.edu
Root is king
• Most critical files can only be modified by
the administrator (root)
– Never log in as root
• To run or edit key configuration files, use
‘sudo’ command to briefly act as the
administrator
– sudo command qualifiers target
– You’ll be prompted for your account’s
password (not the root password)
INFO 320 week 3
47
www.ischool.drexel.edu
Help!
• Help is via ‘manual’ pages, hence the
‘man’ command
– man ls
• On many systems, ‘info’ does the same
as ‘man’
– info ls
INFO 320 week 3
48
www.ischool.drexel.edu
STOP!
• To stop what you’re doing, control-c (or
rarely, control-z) will do it
– If you’re in the middle of a man page,
type q to quit
• Otherwise, you need to identify the rogue
process with ps –ef and then kill that
process id number (PID)
– ps –ef
– kill PID
INFO 320 week 3
49
www.ischool.drexel.edu
Unix configuration files
• Much of Unix is controlled by scripts or
configuration files
– Text configuration files have commands that
control most parameters
– For BASH shells, .bashrc and
.bash_profile are configuration files
• The presence of a file in a certain directory
can activate that feature
INFO 320 week 3
50
www.ischool.drexel.edu
GUI for Unix
• There are three major graphic interfaces
for Unix
– X11 is the daddy of them all (circa 1987), also
called X-Windows
– KDE is the K Desktop Environment, and is the
default GUI for Kubuntu
– Unity is the default GUI for Ubuntu (GNOME )
• They all do roughly the same thing
INFO 320 week 3
51
www.ischool.drexel.edu
Important commands
• Show files in a directory with ls
– Common qualifiers are –a (show all files), and
–l (long format), as in ls –a or ls –al
• Show contents of a file with cat or more
– cat filename
• Create a file or edit an existing file with vi
or nano (text editors)
– vi is horrible to learn, nano is more user
cuddly
INFO 320 week 3
52
www.ischool.drexel.edu
Directing output
• The > can be used to redirect the output
text stream from a command
• Could use it to create a file
– ls –al > mynewfile
– ls –l *.c > sourcecodelisting
• This isn’t the same as the pipe | , which
routes output from one command into
another
INFO 320 week 3
53
www.ischool.drexel.edu
Important commands
• To copy a file, use cp (cp = copy)
– cp myfile newfile
• To rename a file, use mv (mv = move)
– If it stays in the same directory, it can still
change name
• To move a file, use mv
– mv myfile ../folder/myfile
• To delete a file, use rm (rm = remove)
INFO 320 week 3
54
www.ischool.drexel.edu
Installing Ubuntu
INFO 320 week 3
55
www.ischool.drexel.edu
Installation
• Most of the Ubuntu installation process is
obvious (selecting keyboard, time zone,
etc.)
• Partitioning and formatting hard drives is a
bit odd though
• If you want multiple OS’s on your system,
look for the GRUB boot loader
– Windows dual boot information
INFO 320 week 3
56
www.ischool.drexel.edu
Partitions
• In Windows, each drive is often a partition
– The C: drive often represents one hard drive
• In Linux, there are a minimum of two
partitions needed
– ‘/’ is the root partition, and can include all files
and directories
– A swap space is also needed, equal in size to
your RAM
INFO 320 week 3
57
www.ischool.drexel.edu
Partitions
• It’s possible to set aside other key directories as
separate partitions (possibly on other drives)
–
–
–
–
–
–
/boot holds the Linux kernel
/home has personal files for each user
/opt holds much third party software
/srv holds server app data
/tmp is for temporary application files
/usr doesn’t change much after installation, can be
set side for security
– /var has OS files that vary in size, print spoolers
and such
INFO 320 week 3
58
www.ischool.drexel.edu
Partitions
• You can use LVM (Logical Volume
Management) to group partitions, disks,
or arrays into a single logical volume
– The volume can be expanded later, for
example, without reinstalling Ubuntu
– The /boot partition can’t use LVM
• An SATA hard drive is limited to four
partitions
INFO 320 week 3
59
www.ischool.drexel.edu
File systems
• Once a partition is created, a file system
can be installed on it
• FAT-32 was derived from DOS’ FAT-12
– Only uses 28 bits for addressing
– Partitions can be up to 2 TB
– Blocks sizes range from 4-32 kB
– Still used for digital camera memory, flash
drives, etc.
This section mostly from (Tanenbaum, 2008)
INFO 320 week 3
60
www.ischool.drexel.edu
File systems
• NTFS (New Technology File System)
– Developed for Windows NT, still widely used
– Uses 64-bit addresses
– Case sensitive file names in Unicode, up to
255 characters
– Block sizes typically from 4-64 kB
– Maximum volume of just under 256 TB
INFO 320 week 3
61
www.ischool.drexel.edu
File systems
• ext3 (third extended filesystem)
– Block size is 4 kB
– Supports files up to 4 TB
– Supports volumes up to 8 TB (some say
16 TB)
– Supports 32- and 64-bit architectures
• The ext3 or ext4 file system is required for
the /boot partition
INFO 320 week 3
62
www.ischool.drexel.edu
File systems
• ext4
– The ext4 file system was added to Linux in
January 2009
– It uses 64-bit addressing, and is noticeably
faster than ext3
– Single files up to 16 TB
– File system (volume) up to 1 EB (1 million TB)
• Given a choice, use this or LVM
INFO 320 week 3
63
www.ischool.drexel.edu
File systems
• NFS is a network file system under Linux
– NFS allows a system to share directories and
files with others over a network
– By using NFS, users and programs can
access files on remote systems as if they
were local files
• Install NFS with
– sudo apt-get install nfs-kernel-server
INFO 320 week 3
64
www.ischool.drexel.edu
Packages
• Software is installed and uninstalled as
‘packages’
– There are over 24,000 packages available
• dpkg is a package manager for Debian
based systems
– It can install, remove, and build packages, but
unlike other package management system's it
can not automatically download and install
packages and their dependencies
INFO 320 week 3
65
www.ischool.drexel.edu
Packages
• Ubuntu's Advanced Packaging Tool (APT)
can also manage packages
– The command is apt-get
• For example, to install nmap, the
command is
– sudo apt-get install nmap
– /etc/apt/sources.list contains APT
configuration data, and can be edited directly
INFO 320 week 3
66
www.ischool.drexel.edu
Packages
• Aptitude is a menu-driven, text-based
front-end to the APT system
– Many of the common package management
functions, such as installation, removal, and
upgrade, are performed in Aptitude with
single-key commands, which are typically
lowercase letters
INFO 320 week 3
67
www.ischool.drexel.edu
Packages
• The unattended-upgrades package can
be used to automatically install updated
packages, and can be configured to
update all packages or just install security
updates
• Install it with
– sudo apt-get install unattended-upgrades
INFO 320 week 3
68
www.ischool.drexel.edu
References
• Assorted peripherals,
http://www.cs.nott.ac.uk/~ef/ComputerXHi
story/Peripherals/index.htm
• The UNIX time-sharing system, Dennis M.
Ritchie, Ken Thompson July 1974
Communications of the ACM , Volume
17 Issue 7 (historic interest)
• Ubuntu Server Guide
INFO 320 week 3
69
www.ischool.drexel.edu