pNFS BOF SC08 2008-11-19
Download
Report
Transcript pNFS BOF SC08 2008-11-19
pNFS Update 2010-09-17
Sorin
Faibish, EMC
Outline
What is pNFS?
pNFS Timeline
Standards Status
Industry Support
pNFS EMC Status and Roadmap
pNFS Update 2010-09-17
2
pNFS
pNFS protocol
standardized: NFSv4.1
Storage-access protocol
files (NFSv4.1)
blocks (FC, iSCSI, FCoE, IB)
objects (OSD2)
Control protocol
Outside of the pNFS standard
Distributes data across storage
pNFS
cluster
protocol
Eliminates or reduces load and
capacity balancing
And yes: can accelerate I/O
Client
Storage-access
protocol
Metadata
Server
Control
protocol
pNFS Update 2010-09-17
Data Servers
3
pNFS Multiple Layouts
Client gets a layout from the NFSv4.1 server
The layout maps the file onto storage devices and addresses
The client uses the layout to perform direct I/O to storage
At any time the server can recall the layout
Client commits changes and returns the layout when it’s done
pNFS is optional, the client can always use regular NFSv4.1 I/O`
layout
Storage
Clients
NFSv4.1 Server
pNFS Update 2010-09-17
4
Linux pNFS Clients
Transparent to applications
Common client for different storage back ends
Fewer support issues for storage vendors
Normalizes access to clustered file systems
Client Apps
pNFS Client
Layout
Driver
1.
2.
3.
4.
files (NFSv4.1)
objects (OSD2)
blocks (SCSI)
Future back ends …
pNFS
Server
Cluster
Control Protocol
File System
pNFS Update 2010-09-17
5
NFSv4.1 – Open Source Status
Two OpenSource Implementations
OpenSolaris and Linux (file, osd and block)
OpenSolaris Client and Server
Support only file-based layout
Support for multi-device striping already present (NFSv4.1 +
pNFS)
“Simple Policy Engine” for policy-driven layouts also in the gate
Linux Client and Server
Support files (NFSv4.1)
Support currently also blocks (SCSI), objects (OSD T10)
Client consists of generic pNFS client and “plug ins” for “layout
drivers
Windows NFSv4.1 Client from CITI - NEW
pNFS Update 2010-09-17
6
6
Timeline
2004 – CMU, NetApp and Panasas draft pNFS problem
and requirement statements
2005 – CITI, EMC, NetApp and Panasas draft pNFS
extensions to NFS
2005 – NetApp and Sun demonstrate pNFS at
Connectathon
2005 – pNFS added to NFSv4.1 draft
2006 - 2008 – specification baked
Bake-a-thons, Connectathons
26 iterations of NFSv4.1/pNFS spec
2009 – RFC submitted (680 pages)
2010 – RFC 5661 published
2010 – Fedora 13 includes pNFS server/client rpms (did
you triy it yet?)
pNFS Update 2010-09-17
7
pNFS Standards Status
NFSv4.1/pNFS were standardized at IETF
All done including RFC 5661,3,4:
NFSv4 working group (WG)
WG last call (DONE)
Area Director review (DONE)
IETF last call (DONE)
IESG approval for publication (DONE)
IANA review (DONE)
RFC publication (2010)
Consists of several documents:
RFC 5661 - NFSv4.1/pNFS/file layout
RFC 5662 - NFSv4.1 protocol description for IDL (rpcgen) compiler
RFC 5663 - blocks layout
RFC 5664 - objects layout
RFC 5665 - netid specification for transport protocol independence (IPv4,
IPv6, RDMA)
pNFS Update 2010-09-17
8
Industry Contributors to pNFS Standard
BlueArc
NetApp
CITI
Ohio
CMU
Panasas
EMC
Seagate
IBM
Oracle/SUN
SuperComputer Center
LSI
pNFS Update 2010-09-17
9
Traditional pNFS Use Cases
Seismic Data Processing /
Geosciences' Applications
Broadcast & Video
Production
High Performance
Streaming Video
Finite Element Analysis for
Modeling & Simulation
HPC for Simulation &
Modeling
Data Intensive Searching
for Computational
Infrastructures
pNFS Update 2010-09-17
pNFS Ethernet Solution
Application Server Racks
FCoE
NFS
pNFS
iSCSI /NFS
FC Network
iSCSI / FC
pNFS
pNFS
Server
Storage
Heads
10
Questions – Client Technology
When will a valid pNFS client be available?
How does the parallel NFS block layout work?
What’s an object layout, how do objects compare to
blocks/files?
How do current cluster file systems work with pNFS
client and server?
How is client access to data servers coordinated and
controlled?
Attendee Questions Encouraged…
pNFS Update 2010-09-17
11
Questions – Files, Blocks and Objects
How many layout types can there be?
How does pNFS make managing a lot of systems
easier?
Can my application control how its data is striped?
When can we expect to see real pNFS performance,
not vendor claims based on older technology?
How can I use pNFS in an enterprise environment;
would it support non-disruptive (NDU) upgrades?
Attendee Questions Encouraged…
pNFS Update 2010-09-17
12
Questions – Data Management
Can I retain data management practices I use today
based on Snapshots and Volume replication?
Can File, Object and Block layouts co-exist in the
same storage network?
Can a client use volumes accessed via each layout
concurrently?
I’m deploying a Unified Ethernet Fabric; how do I
secure data access – files, blocks & objects?
Attendee questions encouraged…
pNFS Update 2010-09-17
13
Questions – What else
NFSv4.1 and pNFS what are the expectations
compared to 4.0?
What additional function do you want to see added to
4.1:
FedFS
Copy server side
What would you like to see in NFSv4.2 additional to
what is proposed:
Sparse files
Access Permissions
pNFS Update 2010-09-17
14
Industry Support - Implementations
Clients
Linux
Oracle (Open Solaris)
ESX (prototype in 5.0)
Servers
Desy (file)
EMC (block)
IBM (file)
Linux (all layouts)
LSI (block)
NetApp (file)
Panasas (object)
Oracle (file/OpenSolaris)
Several other implementations have been
tested at Bake-a-thons and Connectathons
pNFS Update 2010-09-17
15
Linux Status
Client
Server
Consists of generic pNFS client and “plug ins” for “layout
drivers”
Supports files, blocks, objects
Contributors: CITI, EMC, NetApp, Panasas
Supports files, blocks, objects
Contributors: CITI, EMC, IBM, NetApp, Panasas
Finalizing patches for kernel.org – NFSv4.1 sessions
Predicted timeline:
Basic NFSv4.1 features 2H2010
NFSv4.1 pNFS and layout drivers by 1H2011
Linux distributions shipping supported pNFS in 2H2011.
pNFS Update 2010-09-17
16
OpenSolaris Status
Developing both pNFS client and server in open
• [email protected]
File-based implementation exploiting ZFS capabilities
Management via Simple Policy Engine (SPE)
pNFS data path over RDMA (on Infiniband)
pNFS Update 2010-09-17
17
EMC and pNFS
History and Roadmap
18
Parallel NFS - pNFS
Data Network - LAN
pNFS
Data
Storage
Network
NFS Server
pNFS
Control
NFS file naming, management, and administration
Parallel high bandwidth file access (via Storage Network)
Block Layout leverages existent SAN infrastructures
19
pNFS Block Layout - high I/O speeds
pNFS addresses storage access issues
– Remove servers layer between CE and shared
storage
– Separates MD traffic from Data Traffic
– Asymmetric storage architectures increase
scalability
– Leverages SSD to increase I/O speed
Automatic tiering
Storage must be Networked
HPC Architecture
HPC Jobs
MIDDLEWARE
– Improves utilization to any SAN infrastructure:
FCoE, Infiniband, FC, iSCSI
– Enable access to PB’s of storage at GB’s/sec
speeds
Demonstrated by existent MPFS deployments
– Combine multiple MD servers in a unified storage
system
– MD server is any Celerra NAS server supporting:
Compute Engines
pNFS NFS S E R V E R S
CONNECTIVITY
CONNECTIVITY
NFSv3, CIFS, MPFS and pNFS
Tiered services for increased scalability
SAN STORAGE
20
pNFS Block Layout – The beginning
The ancestors of pNFS Block Layout are NAS
accelerators - 1998:
– EMC-MPFS, Quantum-StoreNext and Mercury-Sanergy
EMC donate the FMP (MPFS) protocol and IP
– Open source version of FMP client (iRoad) - 2003
– IETF pNFS Block Layout = modified open storage FMP protocol - 2004
EMC support pNFS Block Layout in Linux kernel by join
work with CITI: Peter Honeyman, Fred Isaman, Bruce
Fields
– Current pNFS block layout open source client and NFSv4.1
demonstrated at bake-a-thons
– Ongoing funding the project, in 5th year = strong EMC commitment
– Customers can experience the value of pNFS using the EMC FMP
open source driver, or by installing current shipping MPFS product.
21
pNFS Block Layout – Now
pNFS will supports any SAN storage (LSI, EMC, other SAN)
– Working with other SAN vendors to promote pNFS Block Layout
EMC supports NFSv4.1 and pNFS server using pNFS clients in Linux
kernel
– Prototype demonstrated at latest Bake-a-thon
– Demo on Laptop with VM and real clients
EMC is working with all the pNFS developers to accelerate adoption by
HPC
– The goal is to combine all flavors of pNFS servers accessed by each Linux
client in one single infrastructure
– Working with Linux Distributions and Linux kernel developers
What value brings pNFS block layout
– Leverage existent SAN storage and connectivity
– Allow access to SAN storage by NFSv4 network clients
– Virtualizes multi-vendor storage arrays into a single unified view
22
pNFS Block Layout – Roadmap
EMC supports pNFS block in current Celerra release
– Q3 2010 – First vendor to announce pNFS server
– Fedora 13 has support for pNFS block funded by EMC
EMC plans to support NFSv4.1 and pNFS file in future
– Prototype will be presented at Bake-A-Thon at EMC
EMC hosts the Fall Bake-a-Thon in at EMC
– October 4-8, 2010
EMC contributes new protocol enhancement for NFSv4.2
– Draft: http://www.ietf.org/id/draft-faibish-nfsv4-pnfs-access-permissions-check-03.txt
EMC is working with all the pNFS vendors to accelerate
adoption
– Sponsor SNIA ESF
– Support pNFS for MS Windows (we are already testing)
– Working with Linux Distributions RHEL 6.1 is the plan (Q2 2011)
23