Transcript CineGrid @ SURFnet 2008 “Building a New User Community for

CineGrid @ ASIS&T 2010
Building a Collaborative Testbed for Very High-Quality Media
Applications Using Very High-Speed Networks
April 9, 2010
Laurin Herr
President, Pacific Interface Inc.
[email protected]
What is CineGrid?

Formed 2004 – non-profit international membership
organization

Mission – to build an interdisciplinary community
Focus – the research, development & demonstration of
networked collaborative tools
To enable – the production, use, preservation & exchange of
very high-quality digital media over high-speed photonic
networks

Members – media arts schools, research universities, scientific
labs, post-production facilities & hardware and software
developers around the world
Connected via 1 G Ethernet & 10 G Ethernet networks
For – research & education

Convergence Motivates CineGrid

Media technology historically driven by 3 sectors
 Entertainment, media, art and culture
 Science, medicine, education and research
 Military, intelligence, security and police

Adoption of digital media means all 3 face converging needs
 Fast networking for distributed applications
 Access to shared devices
 Specialized computers and massive storage
 Collaboration tools for distributed teams
 Robust security for intellectual property
 Higher quality sound and picture
 A next generation of trained professionals
CineGrid: A Scalable Approach
More
1 - 24 Gbps
Tiled Displays
Camera Arrays
8K x 60
UHDTV (far future)
4K2 x 24/30/60
Stereo 4K (future)
SHD (Quad HD)
250 Mbps - 7.6 Gbps
SHD x 24/30/60
4K x 24
2K2 x 24
2K x 24
200 Mbps - 3 Gbps
HD2 x 24/25/30
20 Mbps - 1.5 Gbps
HDTV x 24/25/30/60
500 Mbps - 15.2 Gbps
250 Mbs - 6 Gbps
5 - 25 Mbps
HDV x 24/25/30/60
Digital Cinema
Stereo HD
HDTV
Consumer HD
Why is more resolution is better?
HDTV (2K)
1080
1. More Resolution Allows Closer Viewing of Larger Image
2. Closer Viewing of Larger Image Increases Viewing Angle
Visual acuity=1.0=20/20
3. Increased Viewing Angle Produces Stronger Emotional Response
1920
Standard viewing distance
30º
UHDTV(8K)
7680
4320
3.0 × Picture Height
2160
3840
UHDTV(4K)
0.75 × Picture Height
Yutaka TANAKA
SHARP CORPORATION
Advanced Image Research Laboratories
100º
60º
1.5 × Picture Height
Saint Issac’s cathedral in St.
Petersburg, Russia
Original art is 3 meters tall by 1
meter wide
Original photograph shot from
about 3 meters distance.
Camera resolution of
12 Mpixels (4K x 3K)
Original shot from about 1 meters distance
Original shot from about 0.3 meters distance
1981
Francis Ford Coppola with Dr. Takashi Fujio
“First Look” at Electronic Cinema
2001
NTT Network Innovations Laboratory
“First Look” at 4K Digital Cinema
2004 OptIPuter Vision for the Next Decade
Gigapixels @ Terabits/sec
4K
Streaming Video
Gigapixel
Wall Paper
Augmented Reality
No Glasses
1 GigaPixel x 3 bytes/pixel x 8 bits/byte x 30 frames/sec ~ 1 Terabit/sec!
Source: Jason Leigh, EVL
CineGrid projects run over the Global Lambda
Integrated Facility (GLIF) backbone
In California, CineGrid access primarily via CENIC
2008 GLIF Visualization by Bob Patterson, NCSA/UIUC
“Learning by Doing”
Early CineGrid Projects
CineGrid @ iGrid 2005
CineGrid @ Holland Festival 2007
CineGrid @ AES 2006
CineGrid @ GLIF 2007
CineGrid Exchange

TERABYTES PILING UP. To store &
distribute its own collection of digital media
assets. Members access materials for
experiments and demonstrations.

THE DIGITAL DILEMMA. Report
published by AMPAS – lessons learned by
NDIPP and NARA + pioneering research
at Stanford (LOCKSS) & UCSD (SRB
and iRODS)

GLOBAL SCALE TESTBED = distributed
storage + fast networks + high quality
digital media assets. Explore strategic
issues in digital media storage, access,
distribution and preservation.
Trends in Cinema
• Movie-making is going digital and going global. Local talent is key!
• Post-production is almost all digital already. Increasingly
distributed!
• Many movies still shot using 35mm film cameras, then scanned
as the first step in the “digital intermediate” process, but digital
cameras are getting better and better!
• Digital cinema projection technology is enabling better picture
quality: From 2K digital to 4K digital, plus a new generation of 3D
stereo movies… Avatar !
Trends in Cinema
• Cinema sound quality is improving. From stereo to compressed
5.1, to uncompressed 5.1, to uncompressed 7.1 surround sound!
• Of the roughly 100,000 screens worldwide, only 15% are digital
today. But recent financing announcements in the USA lead to
forecasts that 60-70% of American screens will be digital by 2013!
• Fast networks are becoming critical infrastructure for media
companies involved in production, post-production and distribution!
• Education of the next generation of media professional is critical!
Cinema Archives vs. Libraries
• Archive
• Goal: preservation without errors, access without end
• Access is “WORSE” case: Write Once Read Seldom if Ever
• Long-term “insurance” for expensive intellectual property
• Mixed motivations
• Potential for future revenue
• Corporate asset protection
• Cultural record
• Library
• Goal: customer service, active management, temporary storage
• Access is “BEST” case: Best Effort Sells Titles
• Mixed inventory: film, video, digital in many formats
• Increasing going online/near-line
• quicker search of inventory
• faster time-to-market
• lower cost of replication & distribution
The Digital Dilemma of Cinema
• 35mm film is the quality standard for motion pictures
• Properly stored lasts 50-100 years
• Survives “benign neglect” gracefully
• Directly readable without reliance on “software stack”
• Very cost effective storage media
• 4K digital cinema needed to match 35mm analog film quality
• More than 8 TB per master version of 2 hour theatrical movie
• More than 5 million objects, ~ 2 PB total all elements per production
• Hollywood produce approx. 400 motion pictures per year
• Hollywood archives are privately funded for future profit
• Digital cinema preservation relying on endless migration is less
reliable and much more expensive (TCO) than film preservation
CineGrid Exchange 2008
Geographically Distributed Repositories + Fast
Networks
High-quality digital media
assets: 4K, 2K, HD, mono &
stereo, still & motion pictures
+ audio
 San Diego @ UCSD/Calit2
40 TB with 10 Gbps connectivity
 Amsterdam @ UvA
30 TB with 10 Gbps connectivity
 Tokyo @ Keio/DMC
6 TB with 10 Gbps connectivity
Total storage = 76 TB
CineGrid Exchange 2009
Building on Basic Concept
• Receive “seed” funding from AMPAS to start building multi-layer open
source asset management & user access framework for distributed
digital media repository
• Establish Working Group
• Bi-annual workshops
• Weekly conference calls
• Basecamp for online coordination
• Write CineGrid Exchange functional requirements document
• Expand CineGrid Exchange storage capacity with more nodes
• Diversify to reduce risk!
CineGrid Exchange 2009
Phase 1 Development Goals
• Define basic workflows for ingest, preservation, distribution, & deletion
• Implement basic workflows to automate management of collection
• Expand distributed repository: more nodes, more assets in collection
• Deploy consistently at all CineGrid Exchange nodes
• Enable easier large data transfers between nodes
• Prepare for integration of cataloging application + distributed repository
• Define and validate CineGrid Exchange metadata schema
CineGrid Exchange Development
Multi-layer Open Software Stack
User Interface & Access
Open Source Application for
Collections Management & On-Line Access
CineGrid Exchange Access Portal
For CineGrid by CineGrid
Collective Access
Extended by Whirl-I-Gig for AMPAS
Open Source Digital Repository Interface
Open Source Middleware for Rule-based
Management of Distributed Digital
Repository Resources & Asset
Testbed Infrastructure of
Distributed Storage and Network Links
iRODS by DICE
For CineGrid By Calit2
Resource Description Framework
For GLIF and CineGrid by UvA
CineGrid Exchange (CX) MiddleWare
iRODS (Integrated Rules Oriented Data Services)
• Programmable micro-services to implement management policies as
non-ambiguous “rules” for automated operations across distributed
storage repository
• Transparent management of CX files in a distributed repository
• Dedicated CX storage
• Non-dedicated CX storage
• Centralized catalog (iCAT)
• File transfer using parallel TCP & UDP
• Suitable for experimenting with digital media preservation strategies
CineGrid Exchange 2010
 Build multi-layer
open-source
framework for
distributed digital
media repository
 Refine iRODS
middleware “rules”
for robustness
 Integrate iRODS +
Collective Access
CX Node Site
Storage Type
UCSD/Calit2, San Diego, USA
Sun Thumper (x4540)
CX
Allocation
66 TB
UvA, Amsterdam, Netherlands
Sun Thumper (x4540)
30 TB
UIC/EVL, Chicago, USA
RAID Array
10 TB
Keio U./DMC, Tokyo, Japan
RAID Array
8 TB
CESNET, Prague, Czech
Republic
Ryerson U, Toronto, Canada
Sun Thumper (x4540)
48 TB
Sun Thumper (x4540)
57 TB
AMPAS, Los Angeles, USA
Sun Thumper (x4540)
24 TB
Total CineGrid Exchange Capacity
243 TB
Acknowledgements
• Academy of Motion Picture Arts and Sciences, STC
• Naval Postgraduate School, MOVES Institute
• University of California San Diego, Calit2
• University of Illinois Chicago, EVL
• University of Washington, Research Channel
• Ryerson University, Rogers Communications Ctr.
• CESNET
• NTT Network Innovation Laboratory
• Keio University, DMC
• University of Amsterdam
• Pacific Interface
• DICE at UNC & SDSC
CineGrid International Workshop
December 13-15, 2010 @ UCSD
www.cinegrid.org