Transcript Interactive 2-D Projection Cross Coverage Viewer for

Interactive 2-D Projection
Cross Coverage Viewer for
Coverage Hole Analysis
Keisuke Shimizu
ClueLogic
[email protected]
Tomohisa Iida
Tomohisa Iida
Verifore,
Verifore,
Inc. Inc.
[email protected]
[email protected]
Outline
Introduction and motivation
 Related work
 Cross coverage and views
 Our 2-D viewer
 “Room for improvement” factor
 Results
 Limitations and future improvement
 Summary

Introduction
Constrained-random verification (CRV)
methodology such as VMM and OVM becomes
widely accepted
 The random nature of the methodology requires
functional coverage metrics to monitor what has
been tested



Reaching 100% coverage is one of the goals
Identifying the root cause of the coverage holes
is crucial

Especially when dealing with multi-dimensional
cross coverage that tends to have a huge
coverage space
Motivation

Synopsys Unified Report Generator (URG)
presents cross coverage comprehensively, but


Not easy to grasp the big picture of the coverage
Not easy to recognize hidden patterns of the
coverage holes
URG focuses on displaying the entire coverage
results
 We wanted a tool to discover coverage holes and
their patterns

URG
Related Work
Hole analysis for functional coverage data
[Lachish, DAC ‘02]
 Defining coverage views to improve functional
coverage analysis [Asaf, DAC ‘04]





Selection
Projection
Grouping
We focus on interactivity so that the users can
see the coverage results from different view
points
Cross Coverage Example

Cross coverage with 3 coverage points


Each coverage point has 4 bins
This makes 4x4x4 cross bins
Bin
C
ov
Po e r
in ag
tC e
Coverage Point A
Coverage Point B
Coverage Hole Analysis
Determine the areas that have not been tested
 Is there a pattern here?

Coverage
hole
Covered
cross bin
Coverage View: List View
Exhaustive, but loses patterns
 This is what URG basically provides

List View
List View: URG

URG puts coverage holes together
URG
Coverage View: 2-D Projection

Add bin counts of coverage points you do not
focus on
Coverage Point A
Coverage Point B
Coverage View: 2-D Projection
Coverage Point B
o
Po ve r
in ag
tC e
Projection
Collapse multiple dimensions into two
C

Coverage View: 2-D Projection
Projection
o
Po ve r
in ag
tC e
Coverage Point A
We get C(n, 2) projections,
where n is # of
coverpoints in
a cross
C

Coverage View: Bin Selection

Another way to see the coverage point relation
Selection
Coverage View: Bin Selection

Select a bin of a coverage point
Coverage View: Bin Selection

Can be combined with the 2-D projection
Selection
How the 2-D Viewer Works
VCS
Coverage
Database
(vdb)
URG
Preprocess
XML
Java
Script
HTML
Browser
2-D
Viewer
Views
Interactive
2-D projections/
bin selections
User
Intermediate XML
testbench
cg
cg
cg
cumulative
inst
inst
inst
cp
cp
cp
bn
bn
bn
cp
cp
cp
cc
cc
cc
cc
cc
cc
bn
bn
bn
namedbins
crosspoints
axis
axis
axis
bn
bn
bn
illegal_auto_crosses
domain
domain
domain
excluded_crosses
domain
domain
domain
data
<text>
axis
axis
axis
axis
axis
axis
Coverage Example

Design under Test

SoC interconnect
DMA
Controller
AXI Bridge
AHB
Bridge
Proprietary Bus
Processor
Bridge
Memory
Controller
Memory
Arbiter
AHB
Bridge
Peripheral
Bridge
Proprietary Bus
Verification Environment

VMM-based CRV
Functional Coverage
DMA
Monitor
DMA
Controller
AXI
Transactor
/Monitor
AHB
Transactor
/Monitor
Processor
Transactor
/Monitor
Memory
Transactor
/Monitor
AHB
Transactor
/Monitor
Peripheral
Transactor
/Monitor
AXI Bridge
AHB
Bridge
Processor
Bridge
Memory
Controller
AHB
Bridge
Peripheral
Bridge
Proprietary Bus
Memory
Arbiter
Proprietary Bus
Coverage Data
256 coverage group instances


Cross coverage example:
Target
n

Initiator vs. Target vs. Direction (Read/Write)
36x26x2 bins (including illegal bins)
ct
io

ire

196 cross coverage instances
4,789 coverage bins in total (excluding illegal bins)
Initiator
D

Projection: Initiator vs. Target
D
ire
ct
io
n
Target
Initiator
Bin Selection
D
ire
ct
io
n
Target
Initiator
Projection: Initiator vs. Direction
Initiator
D
ire
ct
io
n
Target
Projection: Target vs. Direction
D
Initiator
ire
ct
io
n
Target
Which coverage group to tackle first?

Motivation



Now that we can analyze coverage holes more
efficiently
But which coverage group should we look at first?
Traditional “coverage score” measures how many
bins are covered within a coverage group

This metric does not show relative importance on
the overall coverage
Coverage Group Comparison

Which coverage group is more important?

Coverage report displays the coverage group A to
the top because it is least covered, but…
Coverage Group
Number of total bins
Number of bins hit
Coverage score
A
B
C
10
20
170
2
8
136
20%
40%
80%
“Room for Improvement” Factor
Coverage Group
A
B
C
Number of total bins
10
20
170
Number of bins hit
2
8
136
Coverage score
20%
40%
80%
“Room for Improvement” factor
4%
6%
17%
Coverage score shows coverage group A is least
covered, but …
 Coverage group A would increase overall
coverage merely by 4%, whereas coverage
group C would increase by 17%


Assuming Wg =number of bins in coverage group g
(see next page)
The Factor Definition
Fg 
(100 Cg ) Wg
W
g
g

Shows the impact on the overall coverage
 “How many more percentages the overall
coverage would increase if this coverage group
were covered ”



Cg is the score of coverage group g in percent
g  set of coverage groups
Wg is the weight associated with coverage group g
Actual Data
The coverage
group having the
lowest coverage
score does not
necessarily mean
the one that has
the most impact
on the overall
coverage
 We need both
metrics

Coverage groups sorted
by coverage score
Coverage groups sorted by
“room for improvement” facor
Results

Using the 2-D Viewer, we found several
verification/coverage bugs including:

Illegal access to targets
 Boundary
crossing
 Access to a reserved area


Missing burst length values
Illegal byte enable patterns
Byte Enable Coverage
Limitations

There exists non-obvious coverage holes


New approach is required to uncover the root
causes of the hard to spot holes
Currently only VCS coverage database is
supported
Future Improvement

Rewrite the 2-D Viewer preprocessor using
Unified Coverage Database API


UCAPI – to support VCS without worrying about
future database format changes
UCDB API – to support simulators other than VCS
Add “grouping feature” of coverage bins
 3-D Viewer?

Summary

The interactive 2-D Viewer gives variable views
for multi-dimensional cross coverage hole
analysis


The interactive features sometimes reveals
unexpected coverage hole patterns
The “room for improvement” factor enables us to
identify the coverage groups which make the
most impact on the overall coverage score
References







S. Asaf, E. Marcus, and A. Ziv. Defining coverage views to
improve functional coverage analysis. In Proceedings of the
41st Conference on Design Automation, pages 41–44, Jun
2004.
J. Bergeron, E. Cerny, A. Hunter, and A. Nightingale.
Verification Methodology Manual for SystemVerilog. Springer,
2005.
Cadence Design Systems, Inc. and Mentor Graphics, Inc. Open
Verification Methodology User Guide Product Version 2.0.1,
Oct 2008.
Cadence Design Systems, Inc. and Mentor Graphics, Inc.
UCDB API Reference Version 1.1, Feb 2009.
O. Lachish, E. Marcus, S. Ur, and A. Ziv. Hole Analysis for
Functional Coverage Data. In Proceedings of the 39th
Conference on Design Automation, pages 807-812, Jun 2002.
Synopsys, Inc. Unified Coverage Database API Reference
Manual Version B-2008.12, Dec 2008.
Synopsys, Inc. Unified Coverage Reporting User Guide Version
B-2008.12, Dec 2008.
Live Demo