Transcript Peli de Halleux, Nikolai Tillmann Research in Software Engineering Microsoft Research, Redmond, USA.

Peli de Halleux, Nikolai Tillmann
Research in Software Engineering
Microsoft Research, Redmond, USA

1999… people packing groceries for the Y2k
bug
DateTime.Now == new DateTime(2000,1,1))
if (DateTime.Now
throw Y2KBugException();

How do you replace DateTime.Now?
DateTime.Now
MDateTime.NowGet
= () => new DateTime(2000,1,1);
Moles

Delegates naming convention
delegate void Action<T>(T t); // void f(int i);
delegate R Func<T, R>(T t); // int f(string i);

Lambda Expressions and Statements
// C# 2.0
Func<string, int> f = delegate(string s) {return 0; }
// C# 3.0
Func<string, int> f = (s) => 0
Func<string, int> f = (s) => { return 0; }
Func<string, int> f = _ => 0
DEMO
Motivation


A unit test is a small program with assertions
Tests a single (small) unit of code in isolation
void ReadWrite() {
var list = new List();
list.Add(3);
Assert.AreEqual(1, list.Count);
}

Reality check:
Real unit tests are not that simple!

Components depend on other components
bool IsFileEmpty(string file) {
var content = File.ReadAllText(file);
File.ReadAllText(file);
return content.Length == 0;
}

Hidden Integration Tests
void FileExistsTest() {
File.Write(“foo.txt”,
File.Write(“foo.txt”, “”);
“”);
var result = IsFileEmpty(“foo.txt”)
Assert.IsTrue(result);
}



Slow, complicated setup, non-deterministic
tests
Solution: Replace by Simpler Environment
(“mocking”)
Testable Design: Abstraction layer +
Dependency Injection + Mocks for testing
 Simply uses virtual methods

Hard-coded Design: No abstraction layer,
static methods, sealed types.
 Runtime rewriting needed

Replace Any .NET method with A Delegate

Method can be overridden? Use Stubs
 Interfaces, Abstract classes,
 Virtual methods in non-sealed types

Method cannot be overridden? Use Moles
 Static methods,
 Sealed types,
 Inline Constructor calls


Introduce abstraction for external components
Replace them with something simpler, i.e. a Mock
IFileSystem fs,
fs string file) {
bool IsFileEmpty(IFileSystem
var content = fs.ReadAllText(file);
Mock, Stub, Double, Fake,
return content.Length…== 0;
}
void FileExistsTest() {
IFileSystem fs = ???;
fs.Write(“foo.txt”, “”);
var result = IsFileEmpty(fs,“foo.txt”)
Assert.IsTrue(result);
}

Replace Any .NET method with A Delegate
interface IFileSystem {
stringReadAllText(string
ReadAllText(stringfile);
file);
string
}
class SIFileSystem : IFileSystem {
Func<string,string> ReadAllTextString;
string IFileSystem.ReadAllText(string file) {
return this.ReadAllTextString(file);
}} // </auto-generated>
var fs = new SIFileSystem() {
file =>
=> “”;
“”;
ReadAllTextString = file
};
DEMO

Existing external components
cannot be re-factored
 SharePoint, Asp.NET, VSTO

Need mechanism to stub
non-virtual methods
 Static methods, methods in sealed types,
constructors

MSIL code rewriting required
 Other Tools provide this functionality

Method redirected to user delegate, i.e. moled
MFile.ReadAllTextString = file => “”;
bool result = IsFileEmpty(“foo.txt”);
Assert.IsTrue(result);


Requires Code Instrumentation,
e.g. via Profiler!
Pex provides [HostType(“Pex”)]
 NUnit, xUnit, etc… also supported
mscorlib.dll
File.ReadAllText(string name) {
}
.NET Runtime
Just In Time Compiler
File.ReadAllText(string name) {
var d = GetDetour();
if (d != null) return d();
}
ExtendedReflection
push ecx
push edx
push eax
DEMO

Lightweight Framework
 Type Safe
 Refactorable

Testable and “Hard-coded” Code
 Overridable methods -> Stubs
 Any other -> Moles

Delegate Based – use the language!

A Unit Test has Three essential ingredients:
 Data
 Method Sequence
 Assertions
// for
void
Add()
all item,
{
capacity...
void
intAdd(int
item = 3;
item,
intint
capacity
capacity)
= 4;{
var list = new List(capacity);
list.Add(item);
void List.Add(T item) {
var countif= (this.count
list.Count; >= this.Capacity)
this.ResizeArray();
Capacity = 0
Assert.AreEqual(1, count);
this.items[this.count++] = item;
Test Case!
}
}

Automated White box Analysis does not
‘understand’ the environment
???
if (DateTime.Now
DateTime.Now == new DateTime(2000,1,1))
throw new Y2KException();

Isolate Code using Stubs and Moles
Void Y2k(DateTime dt) {
MDateTime.NowGet = () => dt
...
DateTime.Now == dt
}
DEMO
Z3
Constraint Solver
Future
standalone download
Pex
Test Generation
Automated White box Analysis
Stubs
ExtendedReflection
Runtime Code Instrumentation
Source Code Generation
Moles

Types
Bar.IFoo -> Bar.Stubs.SIFoo

Methods
void Foo(string v) -> FooString

Properties
String Value {get;} -> ValueGet

Types
Bar.Foo -> Bar.Stubs.MFoo

Methods
void Foo(string v) -> FooString

Properties
string Value {get;} -> ValueGet
class Foo {
static int StaticMethod() {…}
int InstanceMethod() {…}
}
class MFoo : MoleBase<Foo> {
static Func<int> StaticMethod { set; }
Func<int> InstanceMethod { set; }
implicit operator Foo (MFoo foo);
}

Compiler generates closures for us
void Test(string content) {
var fs = new SIFileSystem();
bool called = false;
fs.ReadAllText = file => {
called
called == true;
true;
return content;
};
...
Assert.IsTrue(called);
called
}

For free with Object Initializers
interface IBar { IFoo Foo {get;} }
interface IFoo { string Value {get;} }
IBar bar = new
…
SIBar()
if(bar.Foo.Value
.Foo .Value == “hello”) ...
var bar = new SIBar {
FooGet = () => new SIFoo {
ValueGet = () => “hello”
}
};

For free with Object Initializers
class Bar { public Foo Foo {get;} }
class Foo { public string Value {get;} }
if(new
new Bar()
Bar().Foo.Value
.Foo .Value == “hello”) ...
MBar.Constructor = me => {
new Mbar(me) => {
FooGet = () => new MFoo {
ValueGet = () => “hello”
}}}

It just works!
class Bar { public Foo Foo {get;} }
interface IFoo {string Value {get;} }
if(new
new Bar()
Bar().Foo.Value
.Foo .Value == “hello”) ...
MBar.Constructor = (me) => {
new Mbar(me) => {
FooGet = () => new SIFoo {
ValueGet = () => “hello”
}}}

Bind all methods of an interface at once
class Foo : IEnumerable<int> {...}
int[] values = {1,2,3};
var foo = new MFoo()
.Bind(values); // bind all methods of
// Ienumerable<int>

Set a trap to flag any call to a type
MFoo.FallbackToNotImplemented();

Iteratively build the mole sequence

Dispatching moles per instance
class Foo {
public int Value {get;}
}
var foo = new MFoo { ValueGet = () => 1 };
var bar = new MFoo { ValueGet = () => 2 };
Assert.IsTrue(foo.Value != bar.Value);

Attach Mole when Contructor is run
class Foo {
public Foo() {}
public int Bar() {…}
}
MFoo.Constructor = me => {
var foo = new MFoo(me) { Bar = () => 10 };
MFoo.Constructor = null; // only 1 instance
};
MFoo.Constructor = _ =>
new MFoo(_) { Bar = () => 10 };

Stubs inherited from class may call base
implementation
abstract class FooBase {
public virtual string Value {get;}
}
var foo = new SFooBase() { CallBase
CallBase == true;
true; }
// call base class if no stub provided
var value = foo.Value;

Defines behavior when stub not provided
 Default throws exception
interface IFoo {
string Value {get;}
}
StubFallbackBehavior.Current =
StubFallbackBehavior.Default;
var foo = new SIFoo();
var value = foo.Value; // returns null

Defines behavior when mole not provided
 Default throws exception
class Foo {
string Value {get;}
}
var foo = new MFoo() {
InstanceFallbackBehavior =
MoleFallbackBehavior.Default }.Instance;
var value = foo.Value; //returns null


Pex automatically detects and uses Stubs
Pex provides return values
interface IFoo {
string Value {get;}
}
[PexMethod]
void Test(IFoo foo) { // pex uses SIFoo
if (foo.Value == “foo”)
throw ...; // pex chooses value