Transcript Program Synthesis for Automating Education Sumit Gulwani ([email protected]) Microsoft Research, Redmond Program Synthesis: Why Today? Program Synthesis = Synthesis of executable code from user intent expressed.

Program Synthesis for
Automating Education
Sumit Gulwani
([email protected])
Microsoft Research, Redmond
Program Synthesis: Why Today?
Program Synthesis = Synthesis of executable code from
user intent expressed using some constraints.
• Enabling Technology is now available
– Better search/logical reasoning techniques
• SAT/SMT solvers
– Faster machines (Good application for multi-cores)
• A natural goal to enable people to easily leverage
computational power.
1
Program Synthesis: Recent Success
Our techniques can synthesize a wide variety of
algorithms/programs from logic and examples.
• Undergraduate book algorithms (e.g., sorting, dynamic prog)
– [Srivastava/Gulwani/Foster, POPL 2010]
• Program Inverses (e.g, deserializers from serializers)
– [Srivastava/Gulwani/Chaudhuri/Foster, MSR-TR-2010-34]
• Graph Algorithms (e.g., bi-partiteness check)
– [Itzhaky/Gulwani/Immerman/Sagiv, OOPSLA 2010]
• String Manipulating Programs(e.g. ”Viktor Kuncak”-> “Kuncak, V.”)
– [Gulwani, POPL 2011]
• Bit-vector algorithms (e.g., turn-off rightmost one bit)
– [Jha/Gulwani/Seshia/Tiwari, ICSE 2010]
2
Bitvector Algorithms
• Straight-line programs that use
– Arithmetic Operators: +,-,*,/
– Logical Operators: Bitwise and/or/not, Shift left/right
• Challenge: Combination of arithmetic + logical operators
leads to unintuitive algorithms
• Application: Provides most-efficient way to accomplish
a given task on a given architecture
3
Examples of Bitvector Algorithms
Turn-off rightmost 1-bit
10101100
&
Z
10101000
Z & (Z-1)
10101100
Z
10101011
Z-1
10101000
Z & (Z-1)
4
Examples of Bitvector Algorithms
P24: Round up to next
highest power of 2
o1 := sub(x,1);
o2 := shr(o1,1);
o3 := or(o1,o2);
o4 := shr(o3,2);
o5 := or(o3,o4);
o6 := shr(o5,4);
o7 := or(o5,o6);
o8 := shr(o7,8);
o9 := or(o7,o8);
o10 := shr(o9,16);
o11 := or(o9,o10);
res := add(o10,1);
P25: Higher order half
of product of x and y
o1 := and(x,0xFFFF);
o2 := shr(x,16);
o3 := and(y,0xFFFF);
o4 := shr(y,16);
o5 := mul(o1,o3);
o6 := mul(o2,o3);
o7 := mul(o1,o4);
o8 := mul(o2,o4);
o9 := shr(o5,16);
o10 := add(o6,o9);
o11 := and(o10,0xFFFF);
o12 := shr(o10,16);
o13 := add(o7,o11);
o14 := shr(o13,16);
o15 := add(o14,o12);
res := add(o15,o8);
[ICSE 2010] Joint work with Susmit Jha, Sanjit Seshia (UC-Berkeley),
Ashish Tiwari (SRI) and Venkie (MSR Redmond)
5
Pyramid of Technology Users
Algorithm
Designers
Software Developers
End-Users
Most
Transformational
Target
Students and Teachers
Consumers of Program Synthesis Technology
6
Automating Education
Make education interactive and fun
• Automated problem solving (for students)
– Provide hints
– Point out mistakes and suggest fixes
• Creation of teaching material (for teachers)
– Authoring tools
– Problem construction
• Group interaction (for teachers/students)
– Ask questions
– Share annotations
Domains: Geometry, Algebra, Probability, Mechanics,
Electrical Circuits, etc.
7
Geometry Constructions Domain
What is the role of PL + Logic + Synthesis?
• Programming Language for Geometry
– Objects: Point, Line, Circle
– Constructors
•
•
•
•
•
Ruler(Point, Point) -> Line
Compass(Point, Point) -> Circle
Intersect(Circle, Circle) -> Pair of Points
Intersect(Line, Circle) -> Pair of Points
Intersect(Line, Line) -> Point
• Logic for Geometry
– Inequality predicates over arithmetic expressions
• Distance(Point, Point), Angle(Line, Line), …
• Automated Problem Solving
– Given pre/postcondition, synthesize a straight-line program
8
Geometry Domain: Automated Problem Solving
Automated Problem Solving
• Given pre/postcondition, synthesize a straight-line program
Example: Draw a line L’ perpendicular to a given line L.
Precondition: true
Postcondition: Angle(L’,L) = 90
Program
Step 1: P1, P2 = ChoosePoint(L);
Step 2: C1 = Circle(P1,P2);
Step 3: C2 = Circle(P2,P1);
Step 4: <P3, P4> = Intersect(C1,C2);
Step 5: L’ = Line(P3,P4);
9
Constructing line L’ perpendicular to given line L
Step 1: P1, P2 = ChoosePoint(L);
Step 2: C1 = Circle(P1,P2);
Step 3: C2 = Circle(P2,P1);
Step 4: <P3, P4> = Intersect(C1,C2);
Step 5: L’ = Line(P3,P4);
L’
P3
C2
C1
P1
L
P2
P4
10
Examples of Geometry Constructions
•
•
•
•
•
•
•
Bisect a given line.
Bisect an angle.
Copy an angle.
Draw a line parallel to a given line.
Draw an equilateral triangle given two points.
Draw a regular hexagon given a side.
Given 4 points, draw a square with each of the
sides passing through a different point.
Other Applications:
• New approximate geometric constructions
• 2D/3D planning problems
11
Synthesis Algorithm for Geometry Constructions
• Synthesis, in general, is harder than verification.
– Synthesis Problem: Given pre/postcondition, synthesize a
straight-line program
– Verification Problem: Given pre/postcondition, and a straightline program, determine whether the Hoare triple holds.
Precondition: True
Postcondition: Angle(L,L’) = 90
Step
Step
Step
Step
Step
1: P1, P2 = ChoosePoint(L);
2: C1 = Circle(P1,P2);
3: C2 = Circle(P2,P1);
4: <P3, P4> = Intersect(C1,C2);
5: L’ = Line(P3,P4);
• Decision procedures for verification of geometry
constructions are known, but are complex.
– Because of symbolic reasoning.
12
A simpler strategy for verification of Constructions
• Symbolic reasoning based decision procedures are complex.
• How about property testing?
Theorem: A construction that works (i.e., satisfies the
postcondition) for a randomly chosen model of precondition
also works for all models (w.h.p.).
Proof:
• Objects constructed using ruler/compass can be described
using polynomial ops (+,-,*), square-root & division operator.
• The randomized polynomial identity testing algorithm lifts
to square-root and division operators as well !
13
Randomized Polynomial Identity Testing
• Problem: Given two polynomials P1 and P2, determine
whether they are equivalent.
• The naïve deterministic algorithm of expanding
polynomials to compare them term-wise is exponential.
• A simple randomized test is probabilistically sufficient:
– Choose random values r for polynomial variables x
– If P1(r) ≠ P2(r), then P1 is not equivalent to P2.
– Otherwise P1 is equivalent to P2 with high probability,
14
Synthesis Algorithm for Geometry Constructions
Problem: Symbolic reasoning is hard.
Idea #1: Leverage Property Testing to reduce symbolic
reasoning to concrete reasoning.
• Construct a random input-output example (I,O) for the problem
and find a construction that can generate O from I.
• Example: Construct incenter of a triangle.
– If I chose my input triangle to be an equilateral one, then the
circumcenter construction also appears to work!
• Since incenter = circumcenter for an equilateral traingle.
– But what are the chances of choosing an random triangle to be
an equilateral one?
15
Synthesis Algorithm for Geometry Constructions
Exhuastive Search Strategy: Given input objects I and
desired objects O, keep constructing new objects from I
using ruler and compass until objects O are obtained.
Problem: Search blows up, i.e., too many (useless) objects get
constructed.
– Example: n points lead to O(n^2) lines, which leads to O(n^4)
points, and so on…
16
Synthesis Algorithm for Geometry Constructions
Problem: Search space is huge.
• Idea #2: Perform goal-directed reasoning.
– Example: If an operation leads to construction of a line L that
passes through a desired output point, it is worthwhile
constructing line L.
– Mimics human intelligence.
– For this to be effective, we need solutions with small depth.
• Idea #3: Work with a richer library of primitives.
– Common constructions picked up from chapters of text-books.
– A search space of (small width, large depth) is converted into
one of (large width, small depth).
– Mimics human experience/knowledge.
17
Search space Exploration: With/without goal-directness
18
Problem Solving Engine with Natural Interfaces
Problem Description in English
Natural Language
Processing
Problem Description as Logical Relation
Synthesis
Engine
Solution as Functional Program
Paraphrasing
Solution in English
Joint work with: Kalika Bali, Monojit Chaudhuri (MSR Bangalore)
Vijay Korthikanti (UIUC), Ashish Tiwari (SRI)
19
Useful modules powered by problem solving engine
The next step is to architect several useful modules on
top of the problem-solving architecture such as:
• Interactive feedback to students
–
–
Provide hints
Point out mistakes and suggest fixes
• Creation of teaching material (for teachers)
– Problem construction
– Authoring tools
20
Other Domains
What domains should we prioritize for automation?
• Mathematics
– Algebra
– Probability
• Physics
– Mechanics
– Electrical Circuits
– Optics
• Chemistry
– Quantitative Chemistry
– Organic Chemistry
21
Electrical Circuits: Concept-specific solutions
• Consider the problem of computing effective resistance
between two nodes in a graph of resistances.
• MATLAB implements Kirchoff’s law based decision procedure
– Algebraic sum of the currents at any circuit junction = 0
– Sum of changes in potential in any complete loop = 0
Joint work with: Swarat Chaudhuri (Penn State University)
22
Electrical Circuits: Concept-specific solutions
• Consider the problem of computing effective resistance
between two nodes in a graph of resistances.
• Kirchoff’s law based decision procedure is not useful for
students who are expected to know only simpler concepts.
• Solutions need be parameterized by specific concepts such as
– Series/Parallel composition of resistances
– Symmetry Reduction
– Wheatstone Bridge
Joint work with: Swarat Chaudhuri (Penn State University)
23
Resistance Reduction Concepts
Series
Combination
Parallel
Combination
Wheat-stone
Bridge
If R3/R1 = R4/R2, then VD = VB
24
Pyramid of Technology Users
Algorithm
Designers
Software Developers
End-Users
Most
Transformational
Target
Students and Teachers
Consumers of Program Synthesis Technology
25
Automating Education: Long-term Goals
• Ultra-intelligent computer
• Model of human mind
• Inter-stellar travel 
26
References
• Tutorial at FMCAD 2010 (Oct 20, Wednesday, 9 am)
– “Dimensions in Program Synthesis”
– (More) applications and methodologies/techniques for
synthesis
– A cool demo 
• Technical report
– “Synthesizing Geometry Constructions” by Sumit
Gulwani, Vijay Korthikanti, Ashish Tiwari
– Send email to sumitg@microsoft for copy of the draft.
27