Transcript 1.01 - Fordham University

Virtual Memory,
File-System Interface
Background

Virtual memory – separation of user logical memory
from physical memory.
 Only part of the program needs to be in memory for
execution
 Logical address space can therefore be much larger than
physical address space
 Allows address spaces to be shared by several processes
 Allows for more efficient process creation

Virtual memory can be implemented via Demand paging
2
Virtual Memory Larger Than Physical Memory

3
Demand Paging

Bring a page into memory only when it is needed





Page is needed  reference to it



invalid reference  abort
not-in-memory  bring to memory
Lazy swapper – never swaps a page into memory unless
page will be needed

4
Less I/O needed
Less memory needed
Faster response
More users
Swapper that deals with pages is a pager
Page Table When Some Pages Are Not in Main Memory
5
Handling a Page Fault
6
Process Creation
Virtual memory allows other benefits
Copy-on-Write (COW): more efficient process
creation




7
allows both parent and child processes to initially share same
pages in memory
If either process modifies a shared page, only then is the
page copied
What happens if there is no free frame?


Page replacement – find
some page in memory, but
not really in use, swap it
out
 Goal –minimize number
of page faults
Only modified pages are
written to disk to reduce
overhead of page
transfers
8
Basic Page Replacement
1.
Find the location of the desired page on disk
2.
Find a free frame:
- If there is a free frame, use it
- If there is no free frame, use a page replacement
algorithm to select a victim frame
3.
Bring desired page into the (newly) free frame; update
the page and frame tables
4.
Resume the process
9
Page Replacement Algorithms



Want lowest page-fault rate
Evaluate algorithm by running it on a particular
string of memory references (reference string)
and computing the number of page faults on that
string
In all our examples, the reference string is
1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5
10
First-In-First-Out (FIFO) Algorithm



Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5
3 frames (3 pages can be in memory at a time per process)
4 frames
1
1
4
5
2
2
1
3
3
3
2
4
1
1
5
4
2
2
1
5
3
3
2
4
4
3
9 page faults
10 page faults
Belady’s Anomaly: more frames  more page faults
11
Optimal Page Replacement


Replace page that will not be used for longest period of
time
Used for measuring how well your algorithm performs
12
Least Recently Used (LRU) Page
Replacement


13
Every page entry has a counter; every time
page is referenced through this entry, copy
the clock into the counter
When a page needs to be changed, look at
the counters to determine which to change
LRU Algorithm (Cont.)

14
Stack implementation – keep a stack of page numbers
in a double link form:
 Page referenced:
 move it to the top
 requires 6 pointers to be changed
Use Stack to Record The Most Recent Page References

keep a stack of page numbers in a double link form:

15
Page referenced move it to the top, requires 6 pointers to
be changed
Counting Algorithms

Keep a counter of the number of references that
have been made to each page

Least Frequently Used (LFU) Algorithm:
replaces page with smallest count

Most Frequently Used (MFU) Algorithm:
based on the argument that the page with the
smallest count was probably just brought in and
has yet to be used
16
Frame Allocation


Equal allocation – For example, if there are 100 frames
and 5 processes, give each process 20 frames.
Proportional allocation – Allocate according to the size
of process
si  size of process pi
S   si
m  total number of frames
s
ai  allocation for pi  i  m
S
17
m  64
si  10
s2  127
10
 64  5
137
127
a2 
 64  59
137
a1 
Thrashing

If a process does not have “enough” pages, the pagefault rate is very high.



Thrashing  a process is busy swapping pages in and out
Demand paging works because of locality model
 Process migrates from one locality to another
 Localities may overlap
Why does thrashing occur?
 size of locality > total memory size
18
Working-Set Model


  working-set window  a fixed number of page
references
Example: 10,000 instruction
WSSi (working set of Process Pi) =
total number of pages referenced in the most recent 
(varies in time)






19
if  too small will not encompass entire locality
if  too large will encompass several localities
if  =   will encompass entire program
D =  WSSi  total demand frames
if D > m  Thrashing
Policy if D > m, then suspend one of the processes
Move on to File System



To explain the function of file systems
To describe the interfaces to file systems
To explore file-system protection
20
File Concept




A named collection of related information that is
stored on secondary storage
The smallest allotment of secondary storage
A sequence of bits, bytes, lines or records…
Types:
 Data
 numeric
 character
 binary
 Program
21
File Structure


None - sequence of words, bytes
Simple record structure




Complex Structures




Lines
Fixed length
Variable length
Formatted document
Relocatable load file: executable files, library files
Indexed file: for fast access to data
Can simulate last two with first method by inserting
appropriate control characters
22
Example of Index and Relative Files
23
File Attributes








Name – only information kept in human-readable form
Identifier – unique tag (number) identifies file within file
system
Type – needed for systems that support different types
Location – pointer to file location on device
Size – current file size
Protection – controls who can do reading, writing, executing
Time, date, and user identification – for creation/last
modification/access, used for protection, security, and usage
monitoring
Information about files are kept in directory structure, which is
maintained on the disk
24
File Operations

File is an abstract data type with operations such as:








25
Create
Write
Read
Reposition within file
Delete
Truncate
Open(Fi) – search the directory structure on disk for entry Fi,
and move the content of entry to memory
Close (Fi) – move the content of entry Fi in memory to
directory structure on disk
Open Files

Several pieces of data are needed to manage open files:
 File pointer: pointer to last read/write location, per
process that has the file open
 File-open count: counter of number of times a file is open
– to allow removal of data from open-file table when last
processes closes it
 Disk location of the file: cache of data access information
 Access rights: per-process access mode information
26
Open File Locking



Provided by some operating systems and file systems
Mediates access to a file
Mandatory or advisory:
 Mandatory – access is denied depending on locks
held and requested
 Advisory – processes can find status of locks and
decide what to do
27
File Types – Name, Extension
28
Access Methods

29
Sequential Access
read next
write next
reset
no read after last write
(rewrite)
Simulation of Sequential Access on Direct-access File

30
Direct Access, n = relative block number
read n
write n
position to n
read next
write next
rewrite n
Directory Structure

Directory: a collection of nodes containing information about
all files
Directory
Files
F1
F2
F3
F4
Fn
Both the directory structure and the files reside on disk
31
Disk Structure

Disk can be subdivided into partitions




Disks or partitions can be protected against failure using:
RAID (Redundant Array of Independent Disks)
Disk or partition can be used raw – without a file system, or
formatted with a file system
Entity containing file system known as a volume


also known as minidisks, slices
Each volume containing file system also tracks that file system’s info
in device directory or volume table of contents
general-purpose file systems vs special-purpose file systems
32
A Typical File-system Organization
33
Operations Performed on Directory






Search for a file
Create a file
Delete a file
List a directory
Rename a file
Traverse the file system
34
Organize the Directory (Logically) to Obtain



Efficiency – locating a file quickly
Naming – convenient to users
 Two users can have same name for different files
 The same file can have several different names
Grouping – logical grouping of files by properties,
(e.g., all Java programs, all games, …)
35
Single-Level Directory

A single directory for all users
Unique naming problem
Grouping problem
36
Two-Level Directory

Separate directory for each user
 Path name
 Can have the same file name for different user
 Efficient searching
 No grouping capability
37
Tree-Structured Directories
38
Tree-Structured Directories (Cont)

Efficient searching

Grouping Capability

Current directory (working directory)

39
cd /spell/mail/prog
Tree-Structured Directories (Cont)




Absolute or relative path name
Creating a new file is done in current directory
Delete a file
rm <file-name>
Creating a new subdirectory is done in current
directory
mkdir <dir-name>
Example: if in current directory /mail
mkdir count
mail
prog
copy prt exp count
Deleting “mail”  deleting the entire subtree rooted by “mail”
40
Acyclic-Graph Directories

41
Have shared subdirectories and files
Acyclic-Graph Directories (Cont.)
Issues:
 A file can have more than one path (aliasing problem)
 If dict deletes list  dangling pointer
Solutions:



Backpointers, so we can delete all pointers
Count number of references to a file
Implement shared files / directories:

New directory entry type:


42
Link – another name (pointer) to an existing file
Resolve the link – follow pointer to locate the file
General Graph Directory
43
General Graph Directory (Cont.)

How do we guarantee no cycles? (avoid infinite loops)
 Allow only links to files, not subdirectories
 Garbage collection: delete items that have no reference
to it
 Traverse file system and mark everything that can be
accessed
 Collected everything that is not marked onto a list of free
space
 Every time a new link is added, use a cycle detection
algorithm to determine whether it is OK
44
File Sharing in Multiple User System

Sharing of files on multi-user systems is desirable

Sharing may be done through a protection scheme

Identify users
45

User IDs identify users, allowing permissions and protections to be
per-user

Group IDs allow users to be in groups, permitting group access rights
Protection

File owner/creator should be able to control:
what can be done
by whom



Change owner user or group
chgrp: change group associated with file
chown: change owner of file



Types of access






Read
Write
Execute
Append
Delete
List
46
Access Lists and Groups
chmod: change access modes



chmod 761 prog1.out
Mode of access: read, write, execute, setuid, setgid
Three classes of users
RWX
a) owner access 7
 111
RWX
b) group access
6
 110
RWX
c) public access
1
 001
47
Digit
Permissions
Binary
Meaning
0
---
000
All types of access are denied
1
--x
001
Execute access is allowed only
2
-w-
010
Write access is allowed only
3
-wx
011
Write and execute access are allowed
4
r--
100
Read access is allowed only
5
r-x
101
Read and execute access are allowed
6
rw-
110
Read and write access are allowed
7
rwx
111
Everything is allowed
setuid, setgid access right

Mode of access: read, write, execute, setuid, setgid

setuid, setgid: Unix access rights flags that allow users to run an
executable with permissions of the executable's owner or group.
Used to allow users to run programs with temporarily elevated privileges
in order to perform a specific task.
When an executable file has been given setuid attribute, normal users
who have permission to execute this file gain the privileges of the user
who owns the file (commonly root) within the created process. When
root privileges have been gained within the process, the application can
then perform tasks on the system that regular users normally would be
restricted from doing.
E.g. passwd, chsh commands for changing password or login shell





Need to modify system file /etc/passwd
Another example: program you used for submitting programs
49
File Sharing – Remote File Systems

Network allow file system access between systems




Client-server model allows clients to mount remote file
systems from servers






Manually via FTP
Automatically, seamlessly using distributed file systems
Semi automatically via world wide web
Server can serve multiple clients
Client and user-on-client identification is insecure or
complicated
NFS is standard UNIX client-server file sharing protocol
CIFS is standard Windows protocol
Standard operating system file calls are translated into remote
calls
Distributed Information Systems (distributed naming
services) such as LDAP, DNS, NIS, Active Directory
implement unified access to information needed for
remote computing
50
File Sharing – Consistency Semantics

Consistency semantics specify how multiple users are
to access a shared file simultaneously

Similar to Ch 7 process synchronization algorithms



Andrew File System (AFS) implemented complex remote file
sharing semantics
Unix file system (UFS) implements:



Writes to an open file visible immediately to other users of the same
open file
Sharing file pointer to allow multiple users to read and write
concurrently
AFS has session semantics

51
Tend to be less complex due to disk I/O and network latency (for
remote file systems
Writes only visible to sessions starting after the file is closed
File System Mounting


52
A file system must be mounted before it can be
accessed
A unmounted file system is mounted at a mount
point
Mount Point
53