GÖDEL'S GOURD Fuzzing for logic and state issues

Introductions  Michael Eddington  CTO and Principal Consultant @ Déjà vu Security  12+ years in security consulting  Senior developer/architect in prior life  Author of Peach, an open source fuzzer  Device, Kernel, User, Web, Network

DARPA Cyber Fast track Thanks Mudge!

Defining the Problem  Fuzzers are good at findings implementation issues  …that crash the target  …that are generically detectable (sqli, xss)  Not good at finding design, logic, and state issues  …that do not crash the target  …that are not generically detectable

Examples

Easy

 Buffer Overflows  Memory Corruption  Resource Usage  SQL Injection

Hard

 Missing authentication  State corruption  Incorrect logic

Authentication Examples  Out of 100 admin pages, 5 are missing authentication  Microsoft SSPI skip a step auth bypass  OpenBSD IPSEC incorrect if/then logic

Authentication – Detect  Web – Missing Auth  Trigger  Request page w/o logging in  Status Code (200/403)  What pages require auth  MS SSPI/OBSD IPSEC  Trigger  Skip a step  Result (Pass)  Did we complete all steps

Logic Example  Windows 95 SMB Flaw  Logic error in password checking code  Length of loop determined by client input  Modified SMB client, ~32 attemps always wins  We never throw an exception or crash  Typical generic fuzzer will never find this

Logic – Win95 SMB { bool CheckPw( int userdata_len, char* userdata, int sysdata_len, char* sysdata ) for(int i=0; i<userdata_len; i++) if(userdata[i] != sysdata[i]) return false; } return true;

Logic – Detect  Win 95 SMB  Trigger  Try all chars  Remove NULL  Result  Does password match

State Example  Device (phone/tablet/laptop) with theft system  Agent “heartbeats” to server  Server can trigger “stolen” mode in laptop  Laptop will trigger if unable to “heartbeat”  Timer/counter runs down

State – Detect  System Server  Trigger  Cause exception  Flow locked  Unable to heartbeat  Can we perform state flow?

 Check result of each step

How to detect?

 Goal – Modify existing fuzzer to detect these issues  We already produce triggers  How do we add detection?

How to detect?

 What do we need to detect these issues?

 Provide system constraints  If not authenticated result is 402  If steps 1, 2, and 3 not performed step 4 is error  Result is never 500  Verify we are still working  Perform state flow w/o mutations

Proposed Solution  Gödel's Gourd  Re-use Peach fuzzing engine  Mutation engine  Fault detection/reporting  Constraint language  Control iterations (non mutation iterations)  Mutate state model (skip, order, etc.)

Control Iterations  Goal: Verify target is working correctly  No mutations  Constraints pass  State model is followed  Matches recorded control iteration

How it works        R – Record iteration 1 – Fuzzing iteration C – Control iteration 2 – Fuzzing Iteration C – Control iteration 3 – Fuzzing iteration …  Remember all states/actions from record iteration  Verify on control iterations  Control iterations every N fuzzing iterations

Outcome  If control does not match record – throw fault  Identify conditions that stop normal operation

Constraints  Verify logic via simple constraint expressions  Apply constraints to state model  State  Action  Does not modify fuzzer state

Language Options  Existing Traditional Languages  JavaScript  Python  Ruby  etc.

 Pro  Well known  Available via .NET scripting interface  Cons  Allows modification of fuzzer state.

Other Options  Domain Specific Language (DSL)  Use existing  Create our own  Pros  Meet all requirements  Cons  Must implement  Not well known

DSL Selection  Object Constraint Language (OCL)  Specification language, no side effects  Developed as part of new UML standards  Familiar syntax  Relatively easy to implement

Object Constraint Language (OCL)  Expression types    Invariant (inv)  Always true Pre (pre)  Evaluated before [ something ] Post (post)  Evaluated after [ something ]  Can access state from Pre. (@pre)

OCL Examples “Car owner must be at least 18 years old” context Car inv: self.owner.age >= 18 “If passwords match result is true” context Login post: result = true implies pass1 = pass2

OCL Context  Groups sets of constraints  Constraints for a context are run together  Association based on context

Normal Fuzzing Iteration  Enter State Model  State 1  Action 1.1

 Send Data  Action 1.2

 Receive Data  State N  …

Fuzzing Iteration With Constraints  Enter State Model  State 1  Action 1.1

 Send Data  Action 1.2

 Receive Data  Inv(pre)  Pre  EVENT  Inv(post)  Post  State N  …

Applying (Authentication)  Web Authentication # Verify authentication occurred post: (reply = 200 && url.indexOf(‘/admin’) > -1) implies auth.reply = 200

Applying (Authentication)  Windows SSPI # Verify all steps completed post: reply = true implies ( auth.step1.reply = true && auth.step2.reply = true && auth.step3.reply = true)

Applying (Logic)  Windows 95 Bug post: reply = true implies userpw = ‘password’

Applying (State)  Antitheft System  Perform control iteration

Implementation

Technologies Used  Microsoft .NET Framework – C#  Peach Fuzzer 3  Cross platform using Mono  OS X  Linux

Implementation Diagram

OCL Implementation  Irony .NET Language Toolkit  Many differences from traditional  Grammar is code  Easy AST hookups  LINQ Expressions  From IronPython work  Last mile is already done

LINQ Expressions  Exposes language constructs for use in AST classes.  Does all the heavy lifting.

return Expression.Condition( (Expression)ifNode.Evaluate(thread), (Expression)thenNode.Evaluate(thread), (Expression)elseNode.Evaluate(thread));

Gödel Usage All the things that do the stuff

Peach Pit vs. Gödel Gourd  Data Model  State Model  Agents  Test  Data Model  OCL Definitions  State Model  OCL Associations  Agents  Test

Gödel: Define Constraints

Gödel: Associate Constraints Constraints will now run with this Action.

Gödel: Control Iterations controlIteration=“1”> Define how often control iterations occur.

Usage Feasibility Time and Cost

Adding Gödel  Process:  Existing Peach PIT  Add OCL Constraints  Test and Verify Definition  Not recreating full application logic  Just our “view of the world”

Time per Protocol  Based on current experience of limited protocol set  Decent in 1 – 2 days  Complete in 1 week or less

Performance  What performance impact does Gödel incur?

 Constraint evaluation  Control iterations  No performance optimizations…yet

Performance of Constraints 30 20 10 0 60 50 40 1 5 10 15

Constraint Count

20 25

Performance Control Iterations  Depends on how often, worst case half speed  Never longer than mutation iterations

Performance Conclusions  Performance impact dependent on speed of fuzzing  Ability to scale fuzzing lowers impact  For fast fuzzers, acceptable impact  For slower fuzzers, adjust control iterations to occur less often

Conclusions  Pentesting/Quick fuzzing  Reasonable for “basics” (verify state’s work, critical logic flows)  General definition building  Reasonable to implement decent coverage  1-2 days “good enough”

Wrapping it up…

Lessons Learned  Constraints applied only to control iterations  Writing good constraints that apply to all mutation cases is challenging  A few constraints can go along ways  Performance overhead needs to be lowered when many constraints used.

 Optimize access to most used variables/objects

Looking towards next rev…  Can we “learn” basic constraints?

 Performance optimizations  Shorted “name” of common objects  context.test.publishers[self.publisher].Result

 self.Result

Thanks for all the fish!

 Michael Eddington  [email protected]

 http://dejavusecurity.com

 http://peachfuzzer.com