Transcript A Novel Hierarchical Matching Algorithm for Intrusion

Towards a Benchmark for
Evaluating Design Pattern Miner
Tools
Date:102/1/17
Publisher:IEEE Software Maintenance and Reengineering, 2008.
CSMR 2008. 12th European Conference on
Author:Lajos Jen˝o F¨ul¨op¤, Rudolf Ferenc and Tibor Gyim´othy
組員:余世淇,游家瑋
Introduction
• Recovering design pattern usage in source code is a very
difficult task. Several tools are described in the literature for
this purpose, but there is little work invested in evaluating
them. The main reason for this is the lack of an approved
benchmark for these tools.
Architecture
• We use the well-known issue and bug tracking system called
Trac [34] (version 0.9.6) as the basis of the benchmark. Trac is
written in Python and it is an easily extendible and
customizable plug-in oriented system.
Architecture
Architecture
Architecture
A. Initialization
B. Pattern Matching processing on GPU
C. Data Output
• True Positives (TP): true instances found by the tool (correctly).
• False Positives (FP): false instances found by the tool
(incorrectly).
• False Negatives (FN): true instances not found by the tool
(incorrectly).
• The precision value is defined as TP/TP+FP , which means the
ratio of correctly identified instances with respect to all found
instances. The recall value is defined as TP/TP+FN , which
means the ratio of correctly identified instances with respect
to all existing real instances.
Adding a New Tool
• Upload. Uploading the results of a tool requires the name of
the tool, name of the mined software, the programming
language of the software, source location, and information
about the found design pattern instances in comma separated
value (CSV) file format
Benchmark Contents
Conclusion and FutureWork
• This paper presents work in progress, the benchmark is
operational, but will certainly need further development
based on user feedback. A possible extension of the
benchmark would be, for instance, a flexible solution for
selecting siblings among the pattern instances. Currently,
users can add comments to an instance to argue about the
judgements. In the future, we want to extend this with
categorizing the votes of a user to control the overrating of his
own tool.
• http://www.inf.u-szeged.hu/designpatterns/