Transcript API’s for Digital Media

Bringing 3D Gaming
to Cell Phones
Being developed by the
© Copyright Khronos Group, 2003 - Page 1
Topics
• Khronos and the OpenGL ES initiative
- Neil Trevett, Secretary of the Khronos Group
• OpenGL ES API specification overview
- David Blythe, OpenGL ES Specification Editor
• Demonstrations and industry support
- Nokia, Hi, Motorola, PowerVR
• Call to action
- Q&A
Neil Trevett
Khronos Secretary
[email protected]
© Copyright Khronos Group, 2003 - Page 2
Large Opportunity for Graphics Software
3D capability is about to be everywhere
Beach head application is
3D games on cell phones which will drive
3D acceleration hardware into handsets in 2004
100Ms /
annum
10Ms /
annum
100Ks /
annum
80’s
3D Hardware
Acceleration in
workstations
90’s
3D Hardware
Acceleration in
everyday PCs
00’s
3D Hardware
Acceleration in
Handhelds
© Copyright Khronos Group, 2003 - Page 3
Embedded Market has API Problem
No suitable low-level graphics APIs exist - yet
• These APIs are the foundation for all advanced graphics applications
• It’s like having Desktop Windows without Direct3D or OpenGL
Games
Productivity Apps
OpenGL is widely available
across platforms – but is way
too big
Applications
Games Engines
JSR-184
High Level Graphics Libraries
No suitable APIs
exist at this level
Low-level Graphics APIs
?
?
Embedded Hardware
Direct3D - not cross platform – so
every platform other than Windows
CE has a real problem
© Copyright Khronos Group, 2003 - Page 4
Inverse Screen Size Law
The need for advanced graphics
• Smaller screens need more advanced graphics processing per pixel
- To make best use of restricted screen real-estate
• Advanced graphics techniques provided by state-of-the-art APIs
- Anti-aliasing, multi-level compositing, rich media, 3D
Interactive
3D Gaming Content
Video Processing and
synchronization
Translucent Menus don’t
obscure lower layers
Perspective windows
maximize screen real-estate
Anti-aliased text to make
small text extremely legible
© Copyright Khronos Group, 2003 - Page 5
Cross Platform Solution Essential
Many platforms and initiatives need these APIs
Wireless Applications Platforms
Need advanced graphics capability in small footprint for BREW and chipsets
Symbian OS for mobile consumer devices
Needs minimal API size and increasing functionality
Embedded Linux – need for low-footprint APIs
No standardized 2D API other than X Windows – not low footprint
Increasing use of OpenGL for 3D – but not low footprint
Java Community – Java2D, Java Media Framework, Java Advanced Imaging
Java3D is a high-level API – need low-footprint 3D API for embedded applications
ARINC – standards for advanced cockpit displays
Need standardized 2D & 3D APIs
Severe certification demands drive need for minimum API size
WindML® – expanding multimedia capabilities of VxWorks®
Looking to further expand 3D and rich media delivery capabilities
Web3D Consortium
Extensible standards for 3D delivery over the internet, driving 3D in MPEG-4
BUT needs target small footprint 2D/3D API for higher-level standards
© Copyright Khronos Group, 2003 - Page 6
OpenGL ES Industry Momentum
Safety
Critical
Line-powered
Appliances
Consoles
Avionics
Strong support from
ARINC and aerospace
companies …
STBs
Internet
Terminals
Automotive
New Visual Processing
APIs
Phones
PDAs
Web pads
Cameras
Hand-held
Appliances
Working to define fullfeatured 3D API.
Discussions with
BroadcastCL groupto avoid divergence
At a meeting in April 2002,
Symbian and the cell phone
industry selected OpenGL ES
as the graphics API for cell
phones with accelerated 3D in
2004
© Copyright Khronos Group, 2003 - Page 7
Khronos – an Open Standards Group
Our Members are creating OpenGL-related APIs
• Working closely with the OpenGL ARB
Feedback and Ratification
ARB
Promoting Members
Contributing Members
© Copyright Khronos Group, 2003 - Page 8
OpenGL ES
A proposed embedded graphics API
• Create well-defined subset profiles of OpenGL
- A series of increasingly compact profiles
- Selecting functionality needed for key target markets
Feedback and Ratification
Embedded
Focus
ARB
Workstation
Focus
Eliminate
Redundancy
Eliminate Pure
Workstation
Functionality
© Copyright Khronos Group, 2003 - Page 9
OpenGL and OpenGL ES
A close synergy
• OpenGL is already the most widely available cross-platform graphics API
- Workstations to embedded appliances, PCs to Macs, Windows to Linux
• Strong future roadmap with OpenGL 2.0
- Enabling access to the new generation of programmable hardware
• Through Khronos – OpenGL ES be royalty free to the embedded industry
- Agreed by SGI – MAJOR OPPORTUNITY
• Just two profiles - minimizing fragmentation
- Common Profile
- Minimum footprint full function 3D with texture-mapping
- Good gaming platform
- Implementable on cell phones
The focus for the rest of
today’s presentation
- Safety Critical Profile
- Absolute minimum 3D to ease safety certifications
- Used in avionics and automotive displays
Common
Safety
Critical
© Copyright Khronos Group, 2003 - Page 10
API Overview
David Blythe
OpenGL ES Specification Editor
[email protected]
© Copyright Khronos Group, 2003 - Page 11
Overview
•
•
•
•
•
Where are we in OpenGL ES development?
Handheld problem space
Design goals
What’s In – What’s Out
Summary
© Copyright Khronos Group, 2003 - Page 12
Where Are We?
•
•
•
•
Handouts are a snapshot from January
Presentation reflects specification as of March 4th
Specification undergoing public review
Specification still needs to be ratified
4Q02
First draft
specification
1Q03
“OpenGL ES 1.0”
ratifiable by Khronos
September
Official Working Group
Meetings
August
Ad hoc Working Group
Meetings
July
Khronos Participation
agreement available
at Siggraph
© Copyright Khronos Group, 2003 - Page 13
Profile Design Goals
• Preserve OpenGL’s Goodness
- Don’t try to fix OpenGL, it isn’t broken
• Proper subsets of OpenGL 1.3
- Match market space requirements
• Eliminate un-needed functionality
- Redundancy
- Expensive
- Unused
• Leverage Workstation/PC experience
- Extension mechanism, feature set
• Allow extensions to enable innovation
- But should be the exception rather than the rule
• Defining conformance tests as well as the API
- Minimum quality defined by conformance tests
© Copyright Khronos Group, 2003 - Page 14
Handheld Problem Space
• Low power
- Battery powered devices
• Low memory
- 50KB code budget
• Low resolution
- 120x120 display
• Rapidly evolving platforms
- Cell phone, PDA, Set top, Wrist watch, ????
© Copyright Khronos Group, 2003 - Page 15
Low Power Devices
• Low power consumption
• Modest processing power
• 25-400MHz integer processing
- 400Mhz PDA, 50MHz Cell phone, 10MHz Wrist Watch, 100MHz Set top Box
• Assume 32-bit integer
• Floating-point not available
• Battery power
- Maximize battery life  minimize instruction/data traffic
• Where is the sweet spot?
© Copyright Khronos Group, 2003 - Page 16
Memory Footprint
• Platforms with limited storage
- RAM 1-8-64MB, NAND FLASH 1-8-64MB
• Minimize library footprint
- Reduce command footprint
- Reduce number of ‘code paths’
• Minimize data storage
- Vertex data and attributes
- Texture storage
© Copyright Khronos Group, 2003 - Page 17
Floating-Point Arithmetic
• No HW floating-point in most of the target space
- Today!
• Be both:
- Integer friendly
- Floating-point ready
• Key Issue
• Lots of Options
- Multiple profiles
- Fixed-point arithmetic
- Floating-point emulation
• Lots of Tradeoffs
-
Robustness/Stability
Performance
Memory Footprint
API Complexity
Ease of Use
© Copyright Khronos Group, 2003 - Page 18
Evolving Device Space
• Regular review of profiles
- Target is yearly revision
• Profiles don’t become obsolete
- Use with emerging devices
• Use OpenGL extension model
- Enable HW vendors to ship extra features
- Add to profile when demand is there
• Avoid creating fragmentation!
© Copyright Khronos Group, 2003 - Page 19
Floating- and Fixed-point
• Critical Issue
• Want:
- Efficient vertex data representation
- Dynamic range for transformations
- Accommodate platforms with and without floating-point
• Don’t want:
- Redundant or seldom used code paths
- API bloat
- Confused application writers
© Copyright Khronos Group, 2003 - Page 20
What’s In/What’s Out
•
•
•
•
•
•
•
•
•
•
A Trip Down the OpenGL Pipeline
Vertex & Geometry Processing
Rasterization
Texture Mapping
Fragment Processing
Buffer Operations
Miscellaneous Operations
Profile Structure
Platform Embedding
Utility Libraries
© Copyright Khronos Group, 2003 - Page 21
Vertex and Geometry Processing
In
Vertex Arrays
Out
Begin/End, Edge Flags
All Primitives
TexGen
Matrix Stack
User Clip Planes
Viewport, DepthRange
Vertex Lighting
Back Materials
Local Viewer, Secondary Color
ShadeModel
© Copyright Khronos Group, 2003 - Page 22
Rasterization
In
Out
Multisampling (optional)
Points and AA Points
Lines & AA Lines
Line Stipple
Polygons
Polygon Stipple, Polygon Smooth
© Copyright Khronos Group, 2003 - Page 23
Rasterization
In
Out
Face Culling
Polygon Mode
PolygonOffset – fill mode
PolygonOffset – line, point mode
Bitmap
DrawPixels
© Copyright Khronos Group, 2003 - Page 24
Texture Mapping
In
Out
2D Textures
Wrap repeat, edge_clamp
Compressed Texture
1D, 3D, Cube Maps, Proxy
clamp, border_clamp
Texture Borders
GetCompressedTexImage
TexSubImage, CopyTexImage
Multitexture
Combine Environment
BGRA / packed formats, L, LA
All Filters
All other image formats
© Copyright Khronos Group, 2003 - Page 25
Fragment Processing
In
Fog
Out
Scissor Test
Alpha Test
Stencil Test (optional)
Depth Test (required)
Blending
Logic Op
Dither
© Copyright Khronos Group, 2003 - Page 26
Framebuffer Ops
In
Out
Clear
Accum
ReadPixels
CopyPixels
Draw/ReadBuffer
© Copyright Khronos Group, 2003 - Page 27
Miscellaneous
In
Out
Flush/Finish
Evaluators
Hint
Feedback
Selection
Display Lists
Get – static state (constants)
Get – dynamic state
Push/PopAttrib
© Copyright Khronos Group, 2003 - Page 28
Profile Assumptions
• Lots of profiles doesn’t help
- Creates support problems for developers
- Tuned profiles not fully exploited
• Optional extensions are similar to multiple profiles
• Floating-point a medium-term trend
- Eventually want both floating-point transforms and vertex attributes
- Some platforms support it well now
- Need to understand migration path now
• Want to cover today’s platforms well
- Important for adoption
• Fixed-point is practical
- Not ideal, but …
- Successfully used in the past
© Copyright Khronos Group, 2003 - Page 29
Profile Structure
• Two profiles for the entertainment and related spaces:
-
Mobile
PDA
Set top box
Consoles
Specialty devices, kiosks, terminals
• Start with profile that covers as much of this space as possible
- Common profile
• Address less-capable platforms with subset
- Common-light profile
• Vernacular
- Integer = GLbyte, GLubyte, GLshort
- Fixed-point = 16.16 representation (GLfixed)
- Floating-point = GLfloat (single-precision), GLdouble (double-precision)
© Copyright Khronos Group, 2003 - Page 30
Common Profile
• Includes entire feature set just described
- Integer + single-precision floating-point vertex attributes
- Integer + single-precision commands as described in the spec
- Double-precision floating-point commands where no pre-existing single-precision
• Adds fixed-point data type extension
- Vertex attributes, command parameters
- Pipeline implements floating-point dynamic range, even with fixed-point input
• Adds signed byte vertex data type extension
- Vertex and texture attributes (normal, color already present)
• Adds extension to determine ‘implementation-specific’ read format
- Minimize number of formats
- Allow efficient read
© Copyright Khronos Group, 2003 - Page 31
Common-Lite Profile
•
•
•
•
•
•
•
Proper subset of common profile
Currently, ‘one’ difference from common profile
No floating-point (single- or double-precision) anywhere
Command parameters, vertex data are fixed-point and integer
Support contemporary and trailing-envelope platforms
Support only one form of commands
Nominally 16.16 dynamic range in all computations (including transforms)
- Subject to discussion
• No overflow checking or reporting
- Subject to discussion
• Compile & run-time (dynamic libraries) with common profile
© Copyright Khronos Group, 2003 - Page 32
Common Vs Common-Lite
Common
Common-Lite
Modest
Handset
Game
Device
Set Top
Smart-phone
PDA
Kiosk
Footprint
Performance
Power Consumption
© Copyright Khronos Group, 2003 - Page 33
Extensions
•
•
•
-
-
Definition
Commands/features that change/extend the behavior of the OpenGL state machine
In the ‘gl’ name space
Leverage current OpenGL extension mechanism
Extension string, function pointer query
ARB extensions
OpenGL ES extensions (OES)
Other OpenGL extensions
Vendor extensions
Rules
1.
2.
3.
4.
-
Can’t re-add commands a profile removed (no begin/end)
Can’t create negative extensions (contractions) to remove features
Can add existing OpenGL extensions (e.G., Texture_cube_map)
Can add OpenGL ES working group (ESWG) exceptions to 1
Candidates: texgen
5. Can add vendor-specific exceptions
-
Use to violate rule 1 
© Copyright Khronos Group, 2003 - Page 34
Platform Embedding
• Interface OpenGL renderer to platform window system
- Similar to GLX/AGL/WGL
• Issues:
• Platform specific or not
- Is there significant value in a cross-platform API
• Feature set
- Minimum platform capabilities
- Video, p-buffers, render-to-texture
© Copyright Khronos Group, 2003 - Page 35
Packaging
• Retain ‘GL’ namespace
• Single C header file
-
#defines for versions, profile variants
GL_ES_PROFILE_XX
GL_ES_PROFILE_XX_1_0
XX = CM, CL, SC
• Runtime queries
- GetString(GL_VERSION) OpenGL ES -XX 1.0
- XX = CM, CL, SC, …
- GetString(GL_EXTENSIONS) “foo_OES bar_OES …”
• Proprosal for link/runtime library packaging
- libGLESXX.{a,so,lib}
- Versioning uses platform native versioning mechanism
© Copyright Khronos Group, 2003 - Page 36
Utility Libraries
• No GLU subset
- Project, Unproject, perspective useful, not hard to implement
- Quadrics – questionable value (demo programs)
- Tessellator, NURBS too large
• GUI libraries
- Run away!
© Copyright Khronos Group, 2003 - Page 37
Summary
• Embedded market space is very diverse
- Focusing on handheld and safety-critical markets
• Creating OpenGL profile that enable handheld devices
- Small footprint
- Software renderers
- Integer/fixed-point
• Evolving standard
- Yearly revisions
• Draft spec undergoing public review
- Now is the time to comment
© Copyright Khronos Group, 2003 - Page 38
Demonstrations
© Copyright Khronos Group, 2003 - Page 39
Demonstrators
Proto OpenGL ES Implementations
•
•
•
•
•
Kari Pulli – Nokia
Mark Callow – Hi Corporation
Mike Olivarez – Motorola
Mitsubishi
MBX and PowerVR Technology overview – Khronos Booth
© Copyright Khronos Group, 2003 - Page 40
OpenGL ES Hardware Support
• The technology is here now!
- New hardware generation for mobile platforms.
• MBX and PowerVR Technology overview
- Tile Based Rendering and Deferred Rendering
• More information at:
www.imgtec.com www.powervr.com www.pvrdev.com
Advanced Graphic
Hardware
OpenGL ES
Embedded Software
Developers
Advanced
Consumer
Content
© Copyright Khronos Group, 2003 - Page 41
OpenGL Standards Universe
How things COULD develop
Aiming for Ratification by Siggraph 2003
OpenGL 2.0 includes full 1.4 functionality –
applications run unmodified
Applications can
incrementally can use
shading language extensions
OpenGL 2.0
“Pure” OpenGL 2.0 subset
OpenGL 1.4
GL2 extensions
“Pure OpenGL 2.0”
subset uses
programmability to
form a simple,
compact, efficient and
flexible API
OpenGL ES 2.0
OpenGL 1.3
OpenGL ES 1.0
Full shader
programmability
for embedded
devices
© Copyright Khronos Group, 2003 - Page 42
Call to Action
Your chance to influence OpenGL ES!
• Join Khronos – as a Promoter or Contributor
- Have a voice in OpenGL’s development
- Reasonable annual fee plus participation agreement
- Talk to Elizabeth Riegel, the Khronos Group Manager here at the session,
on our booth, or via email at [email protected]
• Review the draft specification – we are looking for ISV volunteers
- Simple review agreement required to participate
- Email [email protected]
• Information at www.khronos.org
- Or email the Secretary at [email protected]
© Copyright Khronos Group, 2003 - Page 43
Any Questions?
www.khronos.org
OpenGL is a registered trademarks of Silicon Graphics, Inc., and OpenML is a trademark of Silicon
Graphics, Inc., used with permission by the Khronos Group. All other product names, trademarks,
and/or company names are used solely for identification and belong to their respective owners.
© Copyright Khronos Group, 2003 - Page 44