Transcript Introduction - unisannio.it

DISTRIBUTED SYSTEMS
Principles and Paradigms
Second Edition
ANDREW S. TANENBAUM
MAARTEN VAN STEEN
Chapter 11
DISTRIBUTED FILE
SYSTEMS
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File system distribuiti
•
Nascondono il fs remoto;
•
Offrono una vista standard dei fs remoti;
•
Offrono un’interfaccia per l’accesso ai file remoti
consistente con le interfacce per l’accesso ai file locali:
l’accesso è indistinguibile dal punto di vista
dell’interfaccia.
•
Es.: NFS
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Client-Server Architectures (1)
Figure 11-1. (a) The remote access model (es. nfs).
(b) The upload/download model (ex. using ftp).
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Linux VFS
• VFS: Virtual File System
• Introduced in BSD
• Goal: manage different file systems and perform operations common to
all file systems
open()
VFS
ext3
NFS
procfs
disk
network
kernel
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Client-Server Architectures (2)
Figure 11-2. The basic NFS architecture for UNIX systems.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File system distribuiti: NFS
•
Il VFS remoto deve essere consistente con il modello
NFS;
–
•
•
•
•
Es. MSDOS con il limite di 8+3 caratteri per i nomi dei file non lo è.
I file sono visti come una sequenza di byte, non
strutturata;
Sono organizzati gerarchicamente in file e directory;
Supporta hard link e link simbolici;
I file hanno degli attributi.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
NFS
•
Versione 3:
–
•
senza stato;
Versione 4
–
–
–
Tratta in modo più coerente la creazione dei file speciali (directory,
link, …);
Semplifica alcune operazioni (lookup);
mantiene alcune informazioni di stato.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File System Model (1)
Figure 11-3. An incomplete list of file system
operations supported by NFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File System Model (2)
Figure 11-3. An incomplete list of file system
operations supported by NFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
NFS - server
Definire mount point in /etc/exports
– /public (ro)
– /public2 host1 (ro), host2 (rw)
– /mnt/cdrom *unisannio.it (rw,async)
Riavviare NFS:
– /etc/init.d/nfs restart
Verifica NFS:
– rpcinfo -p
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
NFS - client
Verfica support per NFS
– cat /proc/filesystems | grep nfs
Se manca, eseguire
– modprobe nfs
Verifica:
– Ricontrollare /proc/filesystems
– Eseguire lsmod
Montare:
– # mount –t nfs 10.0.0.1:/mnt/cdrom /mnt/tmp
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Cluster-based file systems
•
Tecniche di file-streaping: un singolo file è distribuito
su più server.
•
Per sistemi che usano decine di migliaia di server i
problemi sono diversi:
–
–
In ogni istante c’è qualche server che non funziona: si considera
normale che qualche server non funzioni;
Es. Google ha implementato il GFS, IBM ha implentato GPFS per
AIX (e linux).
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Cluster-Based Distributed File Systems (1)
Figure 11-4. The difference between (a) distributing whole
files across several servers and
(b) striping files for parallel access.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Google File System
•
I file generalmente trattati sono di grandi dimensioni
(fino all’ordine dei GB) e le operazioni di scrittura fatte
sono in maggioranza append;
•
Si devono trattare frequenti failure;
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Cluster-Based Distributed File Systems (2)
Figure 11-5. The organization of a Google cluster of servers.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Google File System
•
I chunk son di grandi dimensioni (64MB);
•
Il master mantiene i metadati dei file e la loro
associazione ai chunk:
–
L’elenco dei chunk può non essere consistente: viene aggiornato
contattando i chunk server (polling);
•
I Chunk sono replicati (primary-backup);
•
I client contattano il master solo per avere informazioni
su quali chunk server contattare, poi si rivolgono
direttamente ai chunk server;
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
GPFS
GPFS e’ un file system commerciale sviluppato da IBM per AIX e portato su linux:
–
–
–
–
standard POSIX + quota + ACL
ridimensionamento dinamico dei volumi
prestazioni elevate: parallelizzazione degli accessi e bilanciamento del carico
alta affidabilità per
• conservazione dei dati tramite repliche di dati e metadati
• accesso ai dati grazie a server secondari di volumi che possono subentrare in
failover
– esportabilita’ dei volumi via NFS (V3/V4) e samba
Nodo GPFS: singolo sistema operativo su cui gira il software GPFS
– moduli del kernel
– utility di management del file system
– utility di management del cluster
Cluster GPFS: insieme di nodi che condividono le configurazioni e l’accesso ai file
system, che possono avere funzioni di
– manager/client
– quorum/non quorum
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Network Shared Disk
Tutti i nodi del cluster devono poter eseguire comandi privilegiati su ciascun
altro nodo, senza introdurre password
– e’ possibile utilizzare rsh o ssh
– tecnologia che limita la sicurezza e complica il management del cluster
Ogni partizione che dovra’ contenere dati viene configurata come NSD (ha un
nome ed un device file)
L’accesso al singolo NSD avviene:
– direttamente se il nodo vede il device fisico associato all’NDS
– indirettamente tramite export GPFS di un NSD server
Ridondanza per l’accesso all’NSD
– il singolo NSD puo’ essere esportato da un server secondario, che in caso di necessita’
subentra al primario senza perdita di continuita’ di servizio
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File System
Il file system GPFS e’ costituito da uno o piu’ NSD
– POSIX: l’applicativo accede tramite standard file I/O, con supporto di quota ed ACL
(GPFS o compatibili con NFS V4) a livello di file
– rapid recovery: GPFS e’ un logging file system che registra le operazioni sui metadati
– parallelizzazione: le operazioni di I/O vengono realizzate in striping sugli NSD
disponibili, mettendo a disposizione la banda aggregata e bilanciando il carico tra gli
NSD server
– dinamicità: NSD possono essere aggiunti e rimossi dal file system on line
– affidabiltà: possibilita’ di replicare dati e metadati su diverse NSD senza punti critici
in comune
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Configuration Manager, File System
Manager e locking distribuito
Il configuration manager viene eletto tra i quorum node attivi e si occupa di
– controllare il quorum e la consistenza delle configurazioni
– recovery dei dischi in seguito a node failure
– definizione del file system manager (per ogni fs)
Il file system manager ha le funzioni di:
–
–
–
–
aggiunta/rimozione di dischi
recovery del file system
disk space allocation e quota
definire i token manager
Entrambe le funzionalita’ possono migrare senza perdita di operativita’ in
caso di failure di un manager
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Locking distribuito
Consistenza di dati e metadati tramite locking distribuito, mantenuto
tramite meccanismi di token
– l’accesso al file viene permesso tramite la concessione di un token, il cui stato
risiede sul nodo e sul token manager
– un nodo chiede al token manager (attraverso il file system manager) un token per
accedere ad un file
– il token manager concede il token o comunica al nodo la lista dei nodi che hanno
un token in conflitto
– il nodo contatta i nodi che impediscono l’accesso per chiedere il rilascio del token
Come per gli altri, anche le funzioni di token manager possono migrare su
altre macchine senza perdita di funzionalita’
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Processi
•
Sistemi stateless (NFS3):
–
–
•
Non c’è necessità di recovery dopo i crash;
C’è necessità di componenti aggiuntivi per operazioni più complesse
come i lock;
Sistemi con stato (NFS4):
–
–
–
Si tende a minimizzare le informazioni di stato;
Supporto dei lease per l’uso esclusivo dei file;
Supporto di callback;
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Remote Procedure Calls in NFS
•
Open Network Computing RPC (ONC RPC)
•
Nfs3: ogni operazione corrisponde a una chiamata
RPC;
–
•
Le operazioni devono essere molto semplici (es. lookup, read, …);
Nfs4: cerca di minimizzare le chiamate RPC
raggruppandole;
–
Non supporta semantica transazionale;
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Remote Procedure Calls in NFS
Figure 11-7. (a) Reading data from a file in NFS version 3. (b)
Reading data using a compound procedure in version 4.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
RPC2
•
RPC2 implementa RPC affidabili su protocollo UDP
(inaffidabile)
•
Ad ogni invocazione
–
–
•
Il client avvia un thread e resta in attesa della risposta del server;
Il server invia messaggi di stato periodicamente;
Side effect
–
•
Comunicazioni Applicatio-specific ottenute con API delle RPC;
Multicasting
–
•
Per le copie locali
Parallel RPC (MultiRPC)
–
Per le chiamate parallele
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
What is a side effect?
It is a collection of hooks
Primary purpose is to do a “pre” and “post” RPC
processing, e.g.:
Fetch(0x21.0x3.0x6) initiates and completes a side
effect in RPC2 stub code
Most hooks are functions: at setup, binding, before and
after RPC’s
Also hook in the connection area.
SideEffect are not part of RPC2 protocol but are active
parameters (bind and RPC)
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
The RPC2 Subsystem (1)
Figure 11-8. Side effects in Coda’s RPC2 system.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
The RPC2 Subsystem (2)
Figure 11-9. (a) Sending an invalidation message one at a time.
(b) Sending invalidation messages in parallel.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Naming in NFS
•
Obiettivi:
–
•
Mantenere la trasparenza per l’accesso ai file;
L’accesso avviene non all’intero file system remoto, ma
solo a una directory (export):
–
–
Gli utenti non condividono un namespace;
Un server NFS può montare ma non esportare directory remote (risolto
da lookup in nfs4);
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Naming in NFS (1)
Figure 11-11. Mounting (part of) a remote file system in NFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Naming in NFS (2)
Figure 11-12. Mounting nested
directories from
multiple servers in NFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File Handles
•
Riferimento ad un file indipendente dal nome:
–
–
–
•
•
E’ creato dal server remoto;
E’ unico rispetto a tutti i fs esportati dal server
E’ opaco per il client che lo usa.
Utilizzare i FH è efficiente in quanto evita le operazioni di
lookup
Un server non può riutilizzare un FH dopo aver
cancellato un file
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
UNIX v-node
UNIX v-node
– a data structure used in UNIX to represent an open file, directory, or other
entities that can appear in the file system name-space
– keep track of all file activity
– a unique vnode allocated for each active file/directory
v-node Interface
– a vnode does not expose what type of physical file system it implements
– vnode interface allows higher level operating system modules to perform
operations on vnodes uniformly
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Virtual File System (VFS)
Virtual File System
– Contains the common file system code of the v-node interface
v-node
– a reference to a file handle if the file is remote
– or an i-node if the file is local
File system identifier
– Unique number generated for each file system (in UNIX stored in super
block)
i-node generation number
– number of links to the i-node
File handle
File System identifier
v-node
i-node
i-node
i-node generation
number
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Automounting (1)
Figure 11-13. A simple automounter for NFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Automounting (2)
Figure 11-14. Using symbolic links with automounting.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Constructing a Global Name Space
Figure 11-15. Junctions in GNS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Semantics of File Sharing (1)
Figure 11-16. (a) On a single
processor, when a read
follows a write, the
value returned by the
read is the value just
written (Unix semantic).
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Semantics of
File Sharing (2)
Figure 11-16. (b) In a
distributed system with
caching, obsolete values
may be returned.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Semantics of File Sharing (3)
Figure 11-17. Four ways of dealing with the
shared files in a distributed system.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File Locking
•
Si usa un Lock Manager
•
Le operazioni di lock si usano per bloccare un range di
bytes (solo parte di un file).
–
•
Se il blocco è già esistente nfs restituisce un messaggio di errore.
Share reservation
–
–
Sistema indipendente dal locking
Quando un client apre un file specifica il tipo di operazioni e le
operazioni negate agli accessi concorrenti.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File Locking (1)
Figure 11-18. NFSv4 operations related to file locking.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File Locking (2)
Figure 11-19. The result of an open operation with share
reservations in NFS. (a) When the client requests shared
access given the current denial state.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
File Locking (3)
Figure 11-19. The result of an open operation with share
reservations in NFS. (b) When the client requests a denial
state given the current file access state.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Client-Side Caching (1)
Figure 11-21. Client-side caching in NFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Client-Side Caching (1)
–
–
–
•
Operazioni di read e write preparano e modificano la cache
Operazioni di close oggiornano il server
Dopo il close i dati possono essere conservati nella cache
Open delegation
–
–
Si delega al client la gestione del file
E’ necessario un sistema di lock più avanzato
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Client-Side Caching (2)
Figure 11-22. Using the NFSv4 callback mechanism
to recall file delegation.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Handling Byzantine Failures
Figure 11-26. The different phases in Byzantine fault tolerance.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Security in NFS
Figure 11-28. The NFS security architecture.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Secure RPCs
Figure 11-29. Secure RPC in NFSv4.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Access Control
Figure 11-30. The various kinds of users and processes
distinguished by NFS with respect to access control.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Decentralized Authentication (1)
Figure 11-31. The organization of SFS (Secure File System).
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5
Decentralized Authentication (2)
Figure 11-32. A self-certifying pathname in SFS.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5