Transcript DirectFB Issues for CE

Our Experiences on DirectFB in
Embedded Application Development
IGEL Co., Ltd / Renesas Solution Corp.
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
1
Today’s Topics
 DirectFB porting experiences on embedded platforms
– SH7751 + SM501
– SH7770
 Missing pieces in DirectFB
– Functionality missing to write specific applications
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
2
Porting DirectFB
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
3
DirectFB Architecture
–
–
–
Core API Module
Generic GFX Driver
GFX Drivers for Specific Hardware
–
Generic GFX Driver checks whether
the hardware acceleration by a GFX
driver is available
•
•
2005/1/26
If yes, it handovers to the GFX driver
If not it uses software rendering
engine
DirectFB
DirectFB Core API Module
Generic
GFX Driver
Device Drivers Hardware
 To bring out the best performance
on a specific graphics hardware
GFX Drivers for the hardware
should be written.
DirectFB
Application
User Level
 DirectFB works on a frame buffer
device(/dev/fb) and provides the
mechanism to use the hardware
acceleration effectively.
 DirectFB consists of the followings:
GFX drivers(※)
Frame Buffer
Driver(※)
Display Unit
2D Graphics
Hardware
※Modules that needs to be developed
IGEL Co.,Ltd. / Renesas Solution Corp.
4
Why do we need GFX drivers?
 Embedded CPU and bus are slow compare to Desktop’s CPU
– 200-400MHz CPU
– 120MHz 32bit Bus
 Therefore, handover the rendering tasks to specialized
hardware is crucial!
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
5
Effects of Hardware Accelerations
CPU
Bus
RAM
2D
Kernel
H/W Accel
A
SH7751 240MHz
SH-Bus
64MB
SMI SM501
2.4.19
Off
B
SH7751 240MHz
SH-Bus
64MB
SMI SM501
2.4.20
On
C
SH7751 240MHz
PCI
64MB
Matrox Millennium
2.4.20
On
D
Celeron 450MHz
PCI
128MB
Matrox Millennium
2.4.20
On
A
Fill Rectangles [MPixel/sec]
Fill RectanglesThe
(blend)
[MPixel/sec]
hardware
Fill Triangles
[MPixel/sec]
acceleration
shows
Fill Triangles
(blend) [MPixel/sec]
remarkable
results.
Draw Rectangles [KRects/sec]
Draw Rectangles (blend) [KRects/sec]
Draw Lines [KLines/sec]
Draw Lines (blend) [KLines/sec]
Blit [MPixel/sec]
Blit with format conversion [MPixel/sec]
2005/1/26
B
C
D
14.07
217.66
63.63
53.25
1.64
1.66
1.2
3.26
12.25
93.69
62.26
50.51
1.63
1.63
1.17
3.17
1.81
15.45
10.67
8.57
0.43
0.84
61.33
48.84
1.94
3.7
38.68
32.56
3.59
17.79
The performance
0.52
0.56
depends on7.1hardware67.09
acceleration
engine 2.43
2.33
rather than
CPU. 102.47
8.12
4.04
4.12
IGEL Co.,Ltd. / Renesas Solution Corp.
6
How to write GFX Drivers?
 Callback routines needs to be written
– GFX Graphics Driver Functions
– GFX Graphics Device Functions
 Good starting point is gfxdrivers/i810/*.[ch]
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
7
GFX Graphics Driver Functions
From core/gfxcard.h:
typedef struct {
int
(*Probe)
void
(*GetDriverInfo)
(GraphicsDevice
(GraphicsDevice
GraphicsDriverInfo
*device);
*device,
*driver_info);
DFBResult (*InitDriver)
(GraphicsDevice
GraphicsDeviceFuncs
void
void
*device,
*funcs,
*driver_data,
*device_data);
DFBResult (*InitDevice)
(GraphicsDevice
GraphicsDeviceInfo
void
void
*device,
*device_info,
*driver_data,
*device_data);
void
(*CloseDevice)
void
(*CloseDriver)
(GraphicsDevice
void
void
(GraphicsDevice
void
*device,
*driver_data,
*device_data);
*device,
*driver_data);
} GraphicsDriverFuncs;
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
8
GFX Graphics Device Functions
From core/gfxcard.h:
typedef struct _GraphicsDeviceFuncs {
/*
* function that is called after variable screeninfo is changed
* (used for buggy fbdev drivers, that reinitialize something when
* calling FBIO_PUT_VSCREENINFO)
*/
void (*AfterSetVar)( void *driver_data, void *device_data );
/*
* Called after driver->InitDevice() and during dfb_gfxcard_unlock( true ).
* The driver should do the one time initialization of the engine,
* e.g. writing some registers that are supposed to have a fixed value.
*
* This happens after mode switching or after returning from
* OpenGL state (e.g. DRI driver).
*/
void (*EngineReset)( void *driver_data, void *device_data );
/*
* Makes sure that graphics hardware has finished all operations.
*
* This method is called before the CPU accesses a surface' buffer
* that had been written to by the hardware after this method has been
* called the last time.
*
* It's also called before entering the OpenGL state (e.g. DRI driver).
*/
void (*EngineSync)( void *driver_data, void *device_data );
2005/1/26
/*
* after the video memory has been written to by the CPU (e.g. modification
* of a texture) make sure the accelerator won't use cached texture data
*/
void (*FlushTextureCache)( void *driver_data, void *device_data );
/*
* Check if the function 'accel' can be accelerated with the 'state'.
* If that's true, the function sets the 'accel' bit in 'state->accel'.
* Otherwise the function just returns, no need to clear the bit.
*/
void (*CheckState)( void *driver_data, void *device_data,
CardState *state, DFBAccelerationMask accel );
/*
* Program card for execution of the function 'accel' with the 'state'.
* 'state->modified' contains information about changed entries.
* This function has to set at least 'accel' in 'state->set'.
* The driver should remember 'state->modified' and clear it.
* The driver may modify 'funcs' depending on 'state' settings.
*/
void (*SetState) ( void *driver_data, void *device_data,
struct _GraphicsDeviceFuncs *funcs,
CardState *state, DFBAccelerationMask accel );
IGEL Co.,Ltd. / Renesas Solution Corp.
9
GFX Graphics Device Interface
(contd.)
/*
* drawing functions
*/
bool (*FillRectangle) ( void *driver_data, void *device_data,
DFBRectangle *rect );
bool (*DrawRectangle) ( void *driver_data, void *device_data,
DFBRectangle *rect );
bool (*DrawLine)
( void *driver_data, void *device_data,
DFBRegion *line );
bool (*FillTriangle) ( void *driver_data, void *device_data,
DFBTriangle *tri );
/*
* blitting functions
*/
bool (*Blit)
( void *driver_data, void *device_data,
DFBRectangle *rect, int dx, int dy );
bool (*StretchBlit) ( void *driver_data, void *device_data,
DFBRectangle *srect, DFBRectangle *drect );
/*
* emit any buffered commands, i.e. trigger processing
*/
void (*EmitCommands) ( void *driver_data, void *device_data );
} GraphicsDeviceFuncs;
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
10
Porting DirectFB on
SH7751 + SM501
 Our first development
 GFX Driver for SM501 just
set registers to issue
rendering commands
– Issuing command is done
on the fly
– Callback functions
immediately set registers to
render
– Rendering comes on screen
instantly
2005/1/26
GFX Driver for SM501
SM501 Registers
IGEL Co.,Ltd. / Renesas Solution Corp.
SM501
11
Porting DirectFB on SH7770
 Our second development
 GFX Driver for SH7770 creates list
of rendering commands, so called
Display List
–
The list is double buffered
GFX Driver for SH7770
 The driver fills the list until they’re
full, and then pass them to the 2D
engine
–
–
–
While the driver is filling one list, the
2D engine reads commands from
another list
Once the 2D engine is done with the
list, it sends an interrupt, and get the
next list
Rendering doesn’t come on screen
instantly
 Sync mechanism is required to sync
with software rendering done by
generic GFX driver
2005/1/26
Display
List 1
Display
List 2
Registers
SH7770 2D Engine
IGEL Co.,Ltd. / Renesas Solution Corp.
12
Missing Pieces in DirectFB
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
13
What’s Missing?
 Access multiple layered frame buffer from a process
– Recent graphics hardware has multiple frame buffers.
 Using ‘scroll’ function on hardware
– New feature not covered by DirectFB API
 Synchronous rendering and display with VSYNC
(QoS, delay handling)
– Real-time motion graphics (e.g. game), car navigation, etc.
 Synchronize 2D Engine and 3D Engine
– Render with 2D and 3D Engine in a single layer
– Synchronous display even 2D and 3D are on different layers
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
14
Access Multiple Layered Frame
Buffer from a Process
 Recent graphics hardware has multiple layered frame
buffer.
 To coordinate layers efficiently, an application
process wants to issue rendering commands and
switch on / off display of each layer.
Layer #1
(/dev/fb0)
Application
process
Layer #2
(/dev/fb1)
Display
Unit
Layer #3
(/dev/fb2)
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
15
Using ‘Scroll’ Function on
Hardware
 Use Case: car navigation system, web browser
 The ‘Scroll’ function reduces re-rendering cost.
Display
start
position
A
C
B’
B
A’
Simple scroll
2005/1/26
C’
Wraparound scroll
IGEL Co.,Ltd. / Renesas Solution Corp.
16
Synchronous Rendering and
Display with VSYNC (QoS)
 In real-time motion graphics applications, the screen
must be updated in sync with the VSYNC signal.
 Under the standard (fairly) task scheduling in Linux,
rendering might miss display timing (VSYNC)
because of signal interrupts or other heavy tasks.
y=16.6ms(for NTSC)
Other tasks
VSYNC
Ideal
Fact
rendering
rendering
rendering
rendering
Rendering task keeps needed resources in every
VSYNC
time slots.miss
miss
miss
miss
Interrupt signal handling
2005/1/26
rendering
A heavy task was joined.
IGEL Co.,Ltd. / Renesas Solution Corp.
17
Synchronous Rendering and
Display with VSYNC (Delay
Handling)
 Real-time motion graphics applications could be
optimized for screen rendering, especially for delay
handling.
 Application needs VSYNC signal timing to notice the
delay.
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
18
Delay Handling by Application
 Example 1: Skipping the next frame rendering
 When an application noticed that display has been
delayed, it could skip the next frame and start
rendering the frame after the next.
y=16.6ms(for NTSC)
Skip the
frame #3
VSYNC
Rendering
Display
Frame #1
Frame #2
Frame #1
Catch up
the delay
Frame #4
Frame #1
Frame #2
Frame #4
Frame #2
Frame #3
Frame #4
miss
Ideal
2005/1/26
Frame #1
IGEL Co.,Ltd. / Renesas Solution Corp.
19
Delay Handling by Application
(contd.)
 Example 2: Updating screen even if the rendering is
not finished
 Additionally, application could give priority to
rendering operations in case of incomplete frame
displaying.
y=16.6ms(for NTSC)
Terminate
rendering #2
VSYNC
Rendering
Frame #1
Display
Ideal
2005/1/26
Enforce
display #2
Frame #2
Frame #3
Frame #1
Uncompleted
frame #2
Frame #1
Frame #2
Frame #4
Frame #3
Frame #3
IGEL Co.,Ltd. / Renesas Solution Corp.
Frame #4
Frame #4
20
Synchronize 2D Engine and 3D
Engine
 Many 3D graphics applications combine 3D graphics
and 2D graphics.
 These 2D graphics must be synchronized with 3D
graphics.
 Some 3D acceleration hardware (nVIDIA, ATI, for
example) are separated from 2D hardware.
 Synchronization mechanism between 2D and 3D
graphics (hardware) is needed.
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
21
Synchronization Problem in 2D
Engine and 3D Engine
 Situation 1: Rendering 2D graphics and 3D graphics
into a single layer simultaneously
 2D engine and 3D engine tries to draw into a frame
without any synchronization. They should be
serialized.
 Issued rendering commands might be performed
asynchronously.
Application
2D Engine
Layer
3D Engine
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
Display
22
Synchronization Problem in 2D
Engine and 3D Engine
 Situation 2: Displaying a 2D graphics layer and a 3D
graphics layer synchronously
 2D/3D engines can draw into each independent layer
asynchronously if the multiple layered frame buffer is
available.
 Synchronous display function is still needed.
2D Engine
2D Layer
Synchronous
display
Application
3D Engine
2005/1/26
3D Layer
IGEL Co.,Ltd. / Renesas Solution Corp.
23
Conclusion
 We need to consider wherther DirectFB API should /
could cover all application requirements or not.
 We would like to submit proposals and distribute
implementations against the issues.
2005/1/26
IGEL Co.,Ltd. / Renesas Solution Corp.
24