OSHL: A Propositional Prover with Semantics for First-Order Logic David A. Plaisted UNC Chapel Hill.
Download ReportTranscript OSHL: A Propositional Prover with Semantics for First-Order Logic David A. Plaisted UNC Chapel Hill.
OSHL: A Propositional Prover with Semantics for First-Order Logic
David A. Plaisted UNC Chapel Hill
Current theorem provers
Largely syntactic Resolution or ME (tableau) based First-order provers are often poor on non Horn clauses Rarely can solve hard problems Human interaction needed for hard problems
4/28/2020
Unit Resolution and General Resolution
Resolution is efficient for Horn and renameable Horn problems.
Resolution is efficient if the proof can be found by UR resolution.
Hard problems tend not to be Horn, renameable Horn, or UR resolvable.
Of 1697 TPTP problems provable by Otter in 30 seconds, 1042 can be proved by UR resolution.
Unit Resolution and General Resolution
Of the 1697 problems provable by Otter, only 297 were both non Horn and had rating greater than zero.
Of these 297, at most 215 are not UR resolvable.
Otter can do hundreds of thousands of resolutions in 30 seconds on this machine.
Resolution is inefficient on hard, non UR resolvable problems.
Need for new approaches.
How do humans prove theorems?
Semantics Case analysis Sequential search through space of possible structures Focus on the theorem
“Systematic methods can now routinely solve verification problems with thousands or tens of thousands of variables, while local search methods can solve hard random 3SAT problems with millions of variables.” (from a conference announcement)
DPLL Example
{p,r},{
p,
q,r},{p,
r} p=T {T,r},{
T,
q,r},{T,
r} SIMPLIFY {
q,r} p=F {F,r},{
F,
q,r},{F,
r} SIMPLIFY {r},{
r} SIMPLIFY {}
Hyper Linking
Problem Input Clauses Ph5 45 Ph9 Latinsq 297 16 Salt Zebra 44 128 OTTER (sec) Hyper Linking 38606.76 1.8 >24 hrs >24 hrs 2266.6 56.4 1523.82 28.0 >24 hrs 866.2
Eliminating Duplication with the Hyper Linking Strategy, Shie-Jue Lee and David A. Plaisted, Journal of Automated Reasoning 9 (1992) 25-42.
Definition Detection
Problem OSHL Time P1 0.3 P2 2.3 P3 P4 P5 11.25 1.35 2.0 Otter Time 0.03 1000+ 1000+ 1000+ 1000+ Otter Clauses 51 41867 27656 105244 54660
Replacement Rules with Definition Detection, David A. Plaisted and Yunshan Zhu, in Caferra and Salzer, eds., Automated Deduction in Classical and Non-Classical Logics, LNAI 1761 (1998) 80-94.
2 3 4 5 6 7 8 n
S
1
S
2
More Definitions …
S
n
=S
n
S
n-1
Left Associative
…
S
1 OSHL time Gen 0.175 41 Kept 36 Otter time Gen 600+ Kept 100303 24712 Vampire time Gen 0.00
103 Kept 90 E Setheo DCTP time time 0.0
0.01
0.678 85 2.107 141 5.317 207 12.02 283 38.97 7 77.94 7 80 136 202 278 3 3 600+ 66753 600+ 47219 600+ 46054 600+ 60247 600+ 56299 600+ 56352 31496 22119 20941 22923 19660 18932 70.1
3606742 50382 0.3
300+ 25898955 68385 0.3
300+ 25298293 67864 2.6
300+ 25612105 68457 300+ 300+ 25641650 67977 300+ 300+ 25863117 68542 300+ 300+ 300+ 300+ 300+ 300+ 300+
More Definitions
Similar results for other definitions: S 1
S 2
…
S n =S n
S n-1
…
S 1 , left side left associated, right side right associated S 1
S 2 …
…
S n = S 1
S 2
…
S n
S 1
S 2 S n , both sides associated to the left S 1
S 2 …
… S n
S n = S 1
right associated S 2
…
S n
S 1
S 2 , left side left associated, right side Similar results for ∩
Later propositional strategies
Billon’s disconnection calculus, derived from hyper-linking Disconnection calculus theorem prover (DCTP), derived from Billon’s work FDPLL
Performance of DCTP on TPTP, 2003
DCTP 1.3 first in EPS and EPR (largely propositional) DCTP 10.2p third in FNE (first-order, no equality) solving same number as best provers DCTP 10.2p fourth in FOF and FEQ (all first order formulae, and formulae with equality) DCTP 1.3 is a single strategy prover.
Strategy Selection in E
Strategy Selection
Schulz, Stephan, E-A Brainiac Theorem Prover, Journal of AI Communications 15(2/3):111-126, 2002.
Strategy Selection
The Vampire kernel provides a fairly large number of features for strategy selection. The most important ones are: Choice of the main saturation procedure : (i) OTTER loop, with or without the Limited Resource Strategy, (ii) DISCOUNT loop. A variety of optional simplifications. Parameterised reduction orderings. A number of built-in literal selection functions and different modes of comparing literals. Age-weight ratio that specifies how strongly lighter clauses are preferred for inference selection. Set-of-support strategy.
Strategy Selection
The automatic mode of Vampire 7.0 is derived from extensive experimental data obtained on problems from TPTP v2.6.0. Input problems are classified taking into account simple syntactic properties, such as being Horn or non-Horn, presence of equality, etc. Additionally, we take into account the presence of some important kinds of axioms, such as set theory axioms, associativity and commutativity. Every class of problems is assigned a fixed schedule consisting of a number of kernel strategies called one by one with different time limits.
Various Provers
PTTP solved 999 of 2200 tested problems.
Otter proved 1595.
leanCoP proved 745.
Source: Jens Otten and Wolfgang Bibel.
leanCoP: Lean Connection-Based Theorem Proving. Journal of Symbolic Computation, Volume 36, pages 139-161. Elsevier Science, 2003.
Vampire 6.0: 3286 refutations of 7267 problems, more solved
DCTP Strategy Selection
DCTP 1.31 has been implemented as a monolithic system in the Bigloo dialect of the Scheme language.
DCTP 1.31 is a single strategy prover. Individual strategies are started by DCTP 10.21p using the schedule based resource allocation scheme known from the E-SETHEO system. Of course, different schedules have been precomputed for the syntactic problem classes. The problem classes are more or less identical with the sub-classes of the competition organisers.
In CASC-J2 DCTP 10.21p performed substantially better.
Semantics
Gelernter 1959 Geometry Theorem Prover Adapt semantics to clause form: An interpretation (semantics) assignment of truth values to literals so that I I is an assigns opposite truth values to L and
L for atoms L .
The literals L and
L
complementary
.
are said to be
Semantics
We write I C ( I
satisfies
C ) to indicate that semantics I makes the clause C true.
If C is a ground clause then I satisfies satisfies at least one of its literals.
C if I Otherwise I satisfies C instances D interpretations.) if I satisfies all ground of C . (Herbrand If I does not satisfy C then we say I
falsifies
C .
Example Semantics
Specify I by interpreting symbols Interpret predicate p(x,y) as x = y Interpret function f(x,y) as x + y Interpret a as 1, b as 2, c as 3 Then p(f(a,b),c) interprets to TRUE but p(a,b) interprets to FALSE Thus I satisfies p(f(a,b),c) but I falsifies p(a,b)
Obtaining Semantics
Humans using mathematical knowledge Automatic methods (finite models) Trivial semantics
Goal of OSHL
First-order logic Clause form Propositional efficiency Semantics Requires ground decidability
Structure of OSHL
Goal sensitivity if semantics chosen properly Choose initial semantics to satisfy axioms Use of natural semantics For group theory problems, can specify a group Sequential search through possible interpretations Thus similar to Davis and Putnam’s method Propositional Efficiency Constructs a semantic tree
Ordered Semantic Hyperlinking (Oshl) Reduce first-order logic problem to propositional problem Imports propositional efficiency into first-order logic The algorithm Imposes an ordering on clauses Progresses by generating instances and refining interpretations
I0 I1 I2 I3 … D0 D1 D2 unsatisfiable T
OSHL
I 0 D i is specified by the user is chosen
minimal
so that I i falsifies D i D i I i is an instance of a clause in S is chosen all j < i
minimal
so that I i satisfies D j for Let T i be {D 0 ,D 1 , …, D i-1 }.
I i falsifies D i but satisfies T i When T i is unsatisfiable OSHL stops and reports that S is unsatisfiable.
Clause Ordering
||L|| lin ||P(f(x),g(x,c))|| lin ||L|| dag = 6 ||P(f(x),f(x))|| dag = 4 Extend to clauses additively, ignoring negations OSHL chooses D i ordering minimal in such an
Alternate version of OSHL
Want to keep the size of T small Do this by throwing away clauses of T subject to the condition: The minimal model of T i+1 minimal model of T i is larger than the for all i.
This guarantees completeness.
Leads to a formulation using sequences of clauses and resolutions between clauses.
Rules of OSHL Start with empty sequence
(C 1 ,C 2 , …, C n ), D minimal contradict I, I minimal model (C 1 ,C 2 , …, C n ,D) (C 1 ,C 2 , …, C n , D), C n not needed (C 1 ,C 2 , …, C n-1 ,D) (C 1 ,C 2 , …, C n ,D), max resolution possible (C 1 ,C 2 , …, C n-1 ,res(C n ,D,L))
Proof if empty clause derived
Propositional Example ( p I 0 () p) ({-p1, -p2, -p3 }) I 0 [-p3] ({-p1, -p2, -p3 }, {-p4, -p5, -p6 }) I 0 [-p3,-p6] ({…}, {…}, { -p7 }) I 0 [-p3,-p6,-p7] ({…}, {…}, { -p7 }, {p3, p7 }) ({…}, {-p4, -p5, -p6 }, { p3 }) ({-p1, -p2, -p3 },{ p3 }) ({-p1, -p2 }) I 0 [-p2]
Semantics
Trivial semantics: Positive: Choose I 0 to falsify all atoms, first D is all positive. Forward chaining.
Negative: Choose I 0 to satisfy all atoms, first D is all negative. Backward chaining.
Natural semantics: I 0 chosen by user
Semantics Ordering
< t a well founded ordering on atoms, extended to literals Extend < t to interpretations as follows: I and J agree on L if they interpret L the same Suppose I 0 is given I < t J if I and J are not identical, A is the minimal atom on which they disagree, and I agrees with I 0 on A
Semantics Ordering
< t is not a well founded ordering on interpretations. But < t minimal models of T always exist.
I i is always chosen as the < T.
t minimal model of Theorem: Such I L m ] where L i i always has the form I 0 [L 1 are literals of clauses of T.
I 0 [L 1 … L m ] L iff at(L)
{at(L 1 I 0 L, or for some i L = L i .
… L n … )} and
Instantiation Example
Suppose I 0 interprets arithmetic in the standard way.
Suppose S contains axioms of arithmetic and the clause X+3
5.
Then the first instance chosen could be 2+3
5, (1+1)+3
5, (3-1)+3
5 et cetera but it could not be 3+3
5, nor could it be an instance of an axiom.
Instantiation Example
Suppose the first instance chosen is 2+3
5.
Then I 1 is I 0 [2+3
5], which interprets all atoms as in standard arithmetic except that the statement 2+3
5 is true.
The next instance chosen might be 2+3-1 = 5 1
2+3 = 5. This contradicts I 1 . It is an instance of the clause X-1 = Y-1
and corresponds to generating the subgoal 2+3-1 = 5-1.
X = Y
U Rules
Choose clauses instances to match existing literals. Look for a contradiction.
Basic clauses and U clauses Basic clauses are used in three rules given Sequence can also have U clauses on the end U clauses have a selected literal In basic clauses the max. lit. is selected In U clauses other literals can be selected.
Significant performance enhancement.
U Rules
UR resolution: Find C in S having a ground UR resolvent with selected literals. Let C' be the corresponding instance of C. Add C' to the end of the sequence of clauses and select the UR resolvent from it.
Filtering: Find C in S such that NIL is derivable by unit resolution from selected literals and C. Let C' be the corresponding instance of C. Add C' to the end of the sequence of clauses. Select a literal from it.
U Rules
Case Analysis: Find C in S and L in C such that L has all the variables of C. Find instance L' of L that is complementary to a selected literal of some clause in the sequence. Let C' be the corresponding instance of C. Add C' to the end of the sequence and select a literal from it.
This rule expands definitions.
Examples of U Rules
UR resolution: Given the sequence ({s(a), p(b) p(b) }, {t(a), }, {t(a), q(b) q(b) }) and the clause {not p(X), not q(X), r(X)} create the sequence ({s(a), }, {not p(b), not q(b), r(b) } ) Filtering: Given the sequence ({s(a), p(b) }, {t(a), q(b) }) and the clause {not p(X), not q(X)} create the sequence ({s(a), p(b) }, {t(a), q(b) }, {not p(b), not q(b) } )
Examples of U Rules
Case analysis: Given the sequence ({s(a), p(b) }, {t(a), q(b) }) and the clause {not q(X), r(X), s(X)} create the sequence ({s(a), p(b) }, {t(a), q(b) }, {not q(b), r(b), s(b) } )
Example Proof Using U Rules
All positive semantics Clauses: A1.
X
Y,
Y
X, X=Y A2.
Z
X,
X
Y, Z
Y A3. g(X,Y)
X, X
Y A4.
g(X,Y)
Y, X
Y A5.
Z
X, Z
X A6. A7.
Z Z
T.
A
Y, Z X
X B = B
A Y Y Y, Z
X, Z
Y
Example Proof Using U Rules
1. {
A
2. {
A
B
B
A ,
B
A} (Case Analysis, A1) 3. {
g(A B = B
B, B
resolution, A4) A} (T) A)
B
A
A
A , A
B, A B
B
B = B
A} (UR 4. {g(A
B, B
A)
resolution, A5) 5. {g(A
B, B
A)
resolution, A6) 6. {g(…)
B, g(…)
resolution, A7) 7. {A
B
B
A, B B A,
g(…)
A,
g(…)
A, g(…) A
g(…)
B A
B A B } (UR } (UR } (UR } (Filtering, A3)
Example Proof Using U Rules
1. {
A
2. {
A
B
B
A ,
B
A} (Case Analysis) 3. {
g(A B = B
B, B resolution)
A} A)
B
A
A
A , A
B, A B
B
B = B
A} (UR 4. {g(A
B, B
resolution) 5. {g(A
B, B
resolution) A) A)
B B
A, A, 8. {g(…)
B, g(…)
A , A
(Resolution of 6. and 7.)
B g(…) g(…)
B
B A } (UR } (UR A,}
Example Proof Using U Rules
1. {
A
2. {
A
B
B
A ,
B
A} (Case Analysis) 3. {
g(A B = B
B, B resolution)
A} A)
B
A
A
A , A
B, A B
B
B = B
A} (UR 4. {g(A
B, B
resolution) A)
B
A,
g(…) 9. {g(A
B, B
A)
B
A, g(…)
A} (Resolution of 8. and 5.)
B B } (UR , A
B
B
Example Proof Using U Rules
1. {
A
2. {
A
B
B
A ,
B
A} (Case Analysis) 3. {
g(A B = B
B, B resolution)
A} A)
B
A
A
A , A
10. { g(A
4.) B, B
A)
B
A B, A B
B
B = B
A} (UR } (Resolution of 9. and
Example Proof Using U Rules
1. {
A
11. { A
B = B
B
B
A} 2. {
A
B
B
A ,
B
A} (Case Analysis) A A
A
B, A
B = B } (Resolution of 10. and 3.)
Example Proof Using U Rules
1. {
A
12. {
B
B = B
A 11 and 2)
A
A} B , A
B = B
A} (Resolution of
Now the other half of the proof will be done. Note that there is only one ascending sequence of clauses constructed by OSHL and we are only indicating part of it.
Implementation Results
Slower implementation speed of OSHL Uniform strategy versus strategy selection The choice of Otter Influence of U rules on an earlier version: None: 233 proofs in 30 seconds on TPTP problems Using them: 900 proofs in 30 seconds All results for trivial semantics
Implementation Results
OSHL has no special data structures.
Implemented in OCaML No special equality methods Semantics was implemented but frequently only trivial semantics was used.
Thus significant performance improvements are possible.
All # P R O B S
Implementation Results
# Otter Proofs # OSHL Proofs All H O R N Non-Horn All All R = 0 R > 0 H O R N Non-Horn All R = 0 R > 0 4417 1697 764 933 297 636 1027 311 716 265 451 FLD 143 28 0 28 11 17 68 0 68 47 21 SET 604 168 2 166 40 126 211 2 209 93 116 Total Number of Proofs, 30 seconds
Implementation Results
Shows that a prover working entirely at the ground level can come into the range of performance of a respectable resolution theorem prover.
DCTP and FDPLL probably perform better than OSHL.
DCTP and FDPLL do not work entirely at the ground level and do not use natural semantics.
Implementation Results
All Horn Non Horn R=0 Clauses, Otter Clauses, OSHL Ratio 3483094 215290 3267804 915737 17212 202 8110 26.6
9102 359 14888 61.5
R>0 2567357 Non Horn, R>0 2460992 2324 1105 2216 1111 For problems for which both provers found proofs in 30 seconds.
.
Implementation Results
In a given number of inferences OSHL finds more proofs than Otter for non Horn problems
Summary of theoretical results about semantics
Several results show that OSHL with an appropriate semantics is implicitly performing unifications. Thus the choice of semantics has a profound effect on the operation of OSHL.
OSHL has some features of propositional methods and some features of unification based methods.
Semantics might significantly improve OSHL.
Number of Clauses Generated
Problem GRP005-1 GRP006-1 GRO007-1 GRP018-1 GRP019-1 GRP020-1 GRP021-1 GRP023-1 GRP032-3 GRP034-3 GRP034-4 GRP042-2 GRP043-2 GRP136-1 GRP137-1 #clauses, Otter Oshl+semantics 57 3 62 7 85 266 22 16 267 265 264 79 83 141 222 21 80 0 0 15 18 19 22 14 30 6 15 81 8 8