Transcript Iterators Revisited: Proof Rules and Implementation

Iterators Revisited:
Proof Rules and
Implementation
Bart Jacobs, Erik Meijer,
Frank Piessens, Wolfram Schulte
Outline
• Iterators in C#
• How to specify and verify iterators and
foreach loops?
• How to prevent interference between
iterators and foreach loops?
• What are nested iterators?
• How to implement nested iterators
efficiently?
The Iterator pattern in
C# 2.0
public interface IEnumerator<T> {
T Current { get; }
bool MoveNext();
}
public interface IEnumerable<T> {
IEnumerator<T> GetEnumerator();
}
Foreach Loops
foreach (T x in C) S
is implemented as
IEnumerable<T> c = C;
IEnumerator<T> e = c.GetEnumerator();
while (e.MoveNext())
{ T x = e.Current; S }
C# 2.0 Iterator Methods
IEnumerable<int> FromTo(int a, int b) {
for (int x = a; x < b; x++)
yield return x;
}
is implemented as
IEnumerable<int> FromTo(int a, int b)
{ return new FromTo_Enumerable(a, b); }
Compiler-generated class
C# 2.0 Iterator Methods
class FromTo_Enumerator : IEnumerator<int> {
int a; int b; int pc; int x; int current;
public FromTo_Enumerator(int _a, int _b) { a = _a; b = _b; }
public int Current { get { return current; } }
public bool MoveNext() {
switch (pc) {
case 0: x = a; goto case 1;
case 1: if (!(x < b)) goto case 4;
case 2: current = x; pc = 3; return true;
case 3: x++; goto case 1;
case 4: pc = 4; return false;
}}}
How to specify and
verify iterators?
static IEnumerable<int> FromTo(int a, int b)
requires a <= b;
invariant forall{int i in (0; b – a); values[i] == a + i};
invariant values.Count <= b – a;
ensures values.Count == b – a;
{
Enumeration invariant must be
for (int x = a; x < b; x++) proved at start of iterator method…
invariant values.Count == x – a;
{ yield return x; }
… and after each yield return
statement.
}
Ensures clause must be proved at end of method
(and at yield break statements)
How to specify and
verify foreach loops?
int sum = 0;
Seq<int> values = new Seq<int>();
foreach
while
(*)(int x in FromTo(1, 3))
invariant sum == Math.Sum(values);
free invariant forall{int i in (0:values.Count); values[i]==1+i};
free invariant values.Count <= 3 – 1;
{
int x; havoc x; values.Add(x);
assume forall{int i in (0:values.Count); values[i]==1+i};
assume values.Count <= 3 – 1;
sum += x;
}
assume values.Count == 3 – 1;
assert sum == 6;
Interference
List<int> xs = new List<int>();
class List<T> : IEnumerable<T> {
ArgumentOutOfRangeException
!
xs.Add(1); xs.Add(2);
…
xs.Add(3);
IEnumerator<T>
int sum = 0;
GetEnumerator() {
foreach (int x in xs)
int n = Count;
{ sum += x; xs.Remove(0); }
for (int i = 0; i < n; i++)
//assert sum == 6;
yield return this[i];
}}
Parties execute in an interleaved fashion
But we wish to verify them as if they executed in isolation
Proposed solution:
Prevent either party from seeing the other party’s effects
Error: unsatisfied
requires this.readCount == 0;
Enforced using an extension
of the Boogie methodology
Proposed Solution
List<int> xs = new List<int>();
xs.Add(1); xs.Add(2);
xs.Add(3);
int sum = 0;
foreach (int x in xs)
{ sum += x; xs.Remove(0); }
//assert sum == 6;
reads clause declares the set of
pre-existing objects the iterator
method wishes to read
The iterator method
may not read or write
any other pre-existing
objects
class List<T> : IEnumerable<T> {
…
IEnumerator<T>
GetEnumerator()
reads this; {
int n = Count;
for (int i = 0; i < n; i++)
yield return this[i];
}}
And the foreach
loop body may not
write the objects
in the reads clause
The Boogie methodology
• Enforces object invariants
• Uses a dynamic ownership system
• Each object gets two extra fields:
– bool inv;
– bool writable;
• o.f := x; requires o.writable && !o.inv
• unpack o; requires o.writable && o.inv
– Sets o.inv := false;
– Makes owned objects writable
• pack o; reverses the effect of unpack o;
Adding read-only objects
to the Boogie methodology
• Each object gets three special fields:
– bool inv;
– bool writable;
– int readCount; // never negative
• o.f = x; requires
o.writable && o.readCount == 0 && !o.inv
• x = o.f; requires
o.writable || 0 < o.readCount
read (o) S
means
Read-onlyassert
Methods
o.writable || 0 < o.readCount;
assert o.inv;
partial class List<T> :
o.readCount++;
foreachIEnumerable<T>
([Owned] field f of{o)
o.f.readCount++;
S
IEnumerator<T>
GetEnumerator()
foreach ([Owned]
field f of o)
o.f.readCount--;
reads this; {
o.readCount--;
int n = Count;
partial class List<T> {
[Owned] T[] elems;
T this[int index] {
get
requires
inv &&
(writable ||
0 < readCount);
{ read (this)
{ return elems[index]; }
}
}}
for (int i = 0; i < n; i++)
yield return this[i];
}}
How is this call verified?
General Approach to
“Dependent Objects”
• (Preliminary thoughts)
• Each object has a set of dependee objects
• Object may declare “dependent invariants” that
dereference dependee objects
• Dependent invariants become requires clauses,
unless the dependent object is in Reader mode
• Reader mode is statically nested within read
blocks on dependee objects
• Generic user of Iterator interface will require
that Iterator object is in Reader mode
What are
Nested Iterators?
yield foreach E;
means
foreach (T x in E) yield return x;
but is implemented with better time
complexity if E evaluates to a nested iterator
and with less garbage generation if E is a
recursive call of the same iterator
Nested Enumerations
class Tree {
Number of
IEnumerable<Tree> and
int value; List<Tree>! children;
IEnumerator<Tree>
IEnumerable<Tree> Nodes { get { objects created is O(n)
yield return this;
for (int i = 0; i < children.Count; i++)
foreach (Tree t in children[i].Nodes)
yield return t;
Number of recursive
MoveNext calls is
}}
O(n*log(n))
}
Assume a balanced
tree of n nodes
Nested Iterators
class Tree {
int value; List<Tree>! children;
IEnumerable<Tree> Nodes { get {
yield return this;
for (int i = 0; i < children.Count; i++)
yield foreach children[i].Nodes;
}}
}
Space usage:
O(log(n))
Nb. of reallocations:
O(log(log(n))
Nested Iterators
struct TreeStackFrame { Tree self; int pc; int i; }
class TreeEnumerator : IEnumerator<Tree> {
Tree current; TreeStackFrame[]! stack = new TreeStackFrame[8]; int top;
public TreeEnumerator(Tree self) { Push(self, 0, 0); }
public bool MoveNext() {
Total nb. of loop iterations across all
while (0 <= top) {
MoveNext calls: O(n)
switch (stack[top].pc) {
case 0: current = this; stack[top].pc = 1; return true;
case 1: stack[top].i = 0; goto case 2;
case 2: if (!(stack[top].i < stack[top].self.children.Count)) goto case 4;
stack[top].pc = 3; Push(stack[top].self.children[stack[top].i], 0, 0);
break;
case 3: stack[top].i++; goto case 2;
Nb. of TreeStackFrame copy
case 4: Pop(); break;
}}
operations: O(log(n))
return false; }
public Tree Current { get { return current; } }
void Push(Tree self, int pc, int i) { … } void Pop() { top--; }
}
Nested Iterators
Balanced tree of n nodes
Linked list of length n
Time
Allocations
Time
Allocations
Plain
iterators
O(n*log(n))
O(n)
O(n^2)
O(n)
Nested
iterators
O(n)
O(n)
O(n)
O(n)
Recursive
nested
iterators
O(n)
O(log(log(n)))
O(n)
O(log(n))
But if we can statically detect tail recursion,
the nb. of allocations becomes O(1)