Transcript cs527-05 - University of Illinois at Urbana–Champaign

CS527: Advanced Topics in
Software Engineering
(Software Testing and Analysis)
Darko Marinov
September 9, 2008
Schedule
•
•
•
•
•
•
•
Aug 26 & 28: Intro, reading/writing papers
Sep 2: Random testing
Sep 4: Refactoring (Jeff Overbey)
Sep 9: Testing refactoring engines
Sep 11: Static analysis (more votes)
Sep 16: Model checking (fewer votes)
Sep 18 (and on): Testing (most votes)
Communication Info
• Everyone seems to be on the mailing list
– 17 paper reports received by 1pm
• If you send early, your question may be discussed
– You can use newsgroup, but I won’t be able to
monitor
• People in the course on Wiki
– 18 students listed
• Assignment 0: Add yourself to the list
– Let me know if you still can’t access it
Paper Reports
•
•
Actual Assignment 1 from today
Four items:
1.
2.
3.
4.
•
One good point
One bad point
One potential project
One question for discussion
Looking for depth
– Don’t make reports too long
Report Submission
• Due by lecture time, 2pm CST
• Please follow these rules
– Email presented, Vilas (vbangal2) & me
– Put cs527 in the subject line
– ASCII text (no PDF/DOC/HTML files)
– No attachments
Report Policies
• You can have two reports “late”
– Submit them by the next class
• You can omit two reports altogether
• Please inform us if you plan to be late or
omit a report
• You can discuss papers with others
• Reports must be written individually
Paper Today
• Automated Testing of Refactoring Engines
by Daniel, Dig, Garcia, Marinov (FSE 2007)
• Slides courtesy of Brett Daniel
– My generalization for “test abstractions”
Your Questions (1)
• Can language grammar be used for similar result?
[YYL]
• Can a declarative generator be as effective? [YYL]
• Would it be possible that writing imperative
generators are too complex that tester can
introduce errors when writing one? [YYL]
• How could similar approaches be applied to other
problems but refactoring engines? [DY]
• Do refactoring bugs really cause very big
problems? [WX]
Your Questions (2)
• On oracles: who checks the checker?
• Do better than AST Comparator? [DH]
• Can we automatically define oracles
based on the input patterns? [KB]
• What more oracles can we use? [PP]
• How to verify certain semantic aspects?
• Can we add another refactoring engine?
Your Questions (3)
• Were the bugs found actually bugs that
should have been found during a unit test
phase/white box testing? Which phase of the
IDE developement seemed most flawed?
• What were the root causes of the bugs in
these refactoring engines? What is the best
data model to represent a program such that
refactoring can be performed correctly and
efficiently?
• Which IDE is "best" for refactoring?
Your Questions (4)
• Could these automatic generated tests be
clustered in some way? Do they cover all the
situation very well?
• Do the test cases need to be regenerated every
time there is a change in the target refactoring
engine?
• Performance for Imperative generation of Larger
and more complex ASTs ?
• How does quality of generators differ for different
developers?
• How are Generator and Dependent Compositions
handled in detail?