Transcript Security Aspects in Java Bytecode Engineering (Blackhat Briefings 2002, Las Vegas, Aug 01,02) Marc Schönefeld Software-Architect [email protected].

Security Aspects
in
Java Bytecode Engineering
(Blackhat Briefings 2002, Las Vegas, Aug 01,02)
Marc Schönefeld
Software-Architect
[email protected]
Agenda
• Java Security Architecture: an Overview
• The JVM: Structures and Concepts
• Bytecode Basics
• Bytecode (Reverse) Engineering
• Applications of Bytecode (Reverse) Engineering
2
Java Security
Architecture:
an overview
3
How the court defines Java
Findings of Judge Thomas Penfield Jackson, Nov. 5, 1999
• 73. The term "Java" refers to four
interlocking elements.
• First, there is a Java programming language with
which developers can write applications.
• Second, there is a set of programs written in Java that
expose APIs on which developers writing in Java can
rely. These programs are called the "Java class
libraries."
• The third element is the Java compiler, which
translates the code written by the developer into Java
"bytecode."
• Finally, there are programs called "Java virtual
machines," or "JVMs" which translate Java bytecode
into instructions comprehensible to the underlying
operating system.
5
Java Architectural Stack
EJBeans
Servlets, JSP
J2EE
App-Server
Applet
Application
Browser JRE
Standalone JRE,
J2SE Web Start
Java Virtual Machine
Operating System
Hardware
6
The typical JVM
development environment
Development
Javac/
jikes
Dog.
class
Class
loader
Cat.py
jython
Cat.
class
Class
loader
Bird.j
J
A
S
M
Bird.
class
Class
loader
Dog.java
V
E
R
I
F
I
E
R
J
V
M
Runtime
String
system
class
System
Class
loader
Object
system
class
<other>
System
class
7
The basic security
architecture
• Java security (APIs)
–
–
–
–
(access): The Security manager
(origin): Signed Codebases
(behalf): Principle-based access control (JAAS)
cryptography
• JVM security
– Class loaders
– Class file verification process
– JVM intrinsic security features
8
Java security
Security Manager and its API
• Central instance for access control as far as code
is concerned
• Policies define access to outer-domain ressources
• SecurityManager objects instances enforce
policies, throwing SecurityExceptions
• By Default java programs do not have a security
manager, therefor it is a good precaution to
instantiate one
9
Java security
Security Manager and its API
• Fine-grained control to Limit access on:
–
–
–
–
–
SocketConnections (create, accept, multicast)
Thread Groups
Dynamic Library Loading (JNI)
Files (read, write, delete)
Access to External shared ressources (printjob,
clipboard)
– Program control (exit, toplevelwindow)
– Runtime components (member, package, classloader)
10
Java security
Code Base Authentication
• Java-Archives (JARs) store codebases
• Proof of Origin can be be achieved by signing the
jars
JAR
JAR
Cat.class
Dog.class
Bird.class
Private
key
hash
Cat.class
Dog.class
Bird.class
sign
Signed hash
11
Java security
JAAS: Security based on principals
• Enables login functionality
– Username, password
– Fingerprint
– ...
• Execution permitted/denied depending on the
identity who runs the code
– Policy based access to functionality
– Fine-grained permission handling possible
12
JVM security
intrinsic features
• Non-continuous memory model, distinct data
areas
– Java stack frames (execution state)
– Method area (bytecode storage)
– Garbage-collected heap (object storage)
• Type-safe casting
• No self-modifying code
• Automated garbage-collecting disallows explicit
free operation
• Automatic Array bounds-checking prevents
off-by-one and buffer overflow scenarios
13
JVM security
Class loaders
• Classloaders load a classfile as byte array into the
JVM
• Can load from
–
–
–
–
file,
network or
dynamically generated byte array
Can even compile on the fly (so Java behaves like Perl)
• Security features
– Establishing name spaces
– Enforcing separation of trusted system library code
from user-supplied code via parent-delegation
14
JVM security
Verifier
• Task: check loaded classfile for integrity
• 4-step process
– 1st step: structural correctness
– 2nd step: data type correctness
– 3rd step: bytecode checks
– 4th step: symbolical references management
(runtime)
15
JVM security
Classfile verification
public class Cat {
void bite (int times)
{
...
}
}
JAVAC
CA FE BA BE 00 03 00 2D
00CA13FE07BA00BE170012033000112D
..00..13..07 00 17 12 30 11
.. .. ..
bytecode assembler
Class
loader
P
A
S
S
1
Verifier
P
P
A
A
S
S
S
S
2
3
P
A
S
S
J
V
M
4
.class public Dog
.method bite I
.invokestatic seekVictim
...
.end method
.end class
16
The Verification Process
Pass 1: Basic Structural checks
• the classloader delivers byte array
• Magic number = 0xCAFEBABE ?
• Version id: 1.1=45.3, 1.2=46.0, 1.3=47.0,
1.4=48.0
• All recognized attributes need to be in proper
length
• The class file must not be truncated or have extra
bytes at the end
• The constant pool must not contain any
„superficially unrecognizable information“
17
The Verification Process
Pass 2: Check Context-Pool (CP) information
• final classes are not subclassed, and final methods
are not overridden.
• All classes (except java.lang.Object) must have a
superclass.
• Check constraints for CP-entries: For example,
class references in the CP can be resolved via a
field to a string reference in the CP.
• Checking that all field references and method
references in the CP must have legal names,
classes, and type signature.
18
The Verification Process
Pass 3 : Bytecode verification
• Core part of verification
• Static constraints
– Checking maximal local variable count throughout
control flow
– Checking control-flow correctness (branch always to
start of instruction, not beyond end of code)
– all exception-handlers are valid (no partial overlap)
– ...
• Structural constraints
– Reachability : subroutines (scope), exception handlers
– data-flow : Instances initialization and new objects,
stack size
19
The Verification Process
Pass 4: delayed checks during runtime
• Verifies that currently executing class is allowed to
reference the given class.
• The first time an instruction calls a method, or
accesses or modifies a field, the verifier checks
the following:
– method or field class
– Method or field signature
– that the currently executing method has access
to the given method or field
• insert „quick“ optimized instructions
20
Problems with JDK verifier
Not enabled by default
public class IllegalAccess2{
public String str = "The trade secret";
public void test(){
System.out.println("Test2: " + str);
}
}
public class IllegalAccess{
public static void main(String args[]){
IllegalAccess2 t2 = new IllegalAccess2();
System.out.println("Test1: " + t2.str);
t2.test();
t2.str = "an open hint";
System.out.println("Test1: " + t2.str);
System.exit(0);
}
}
D:\entw\java\blackhat>java -classpath .
Test1: The trade secret
Test2: The trade secret
Test1: an open hint
IllegalAccess
21
Problems with JDK verifier
Not enabled by default
public class IllegalAccess2{
private String str = "The trade secret";
public void test(){
System.out.println("Test2: " + str);
}
}
public class IllegalAccess{
public static void main(String[] args){
IllegalAccess2 t2 = new IllegalAccess2();
System.out.println("Test1: " + t2.str);
t2.test();
t2.str = "an open hint";
System.out.println("Test1: " + t2.str);
System.exit(0);
}
}
Variable str now restricted, JVM should now
complain access, but at least on JDK1.3.1 and
1.4.0_01 2000 the following happens...
22
Problems with JDK verifier
Not enabled by default
D:\entw\java\blackhat>java IllegalAccess
Test1: The trade secret
Test2: The trade secret
Test1: an open hint
D:\entw\java\blackhat>java -verify IllegalAccess
Exception in thread "main" java.lang.IllegalAccessError: try to
access field IllegalAccess2.str from class IllegalAccess
at IllegalAccess.main(IllegalAccess.java:7)
Only the explicit -verify flag restricts access to
restricted variable „str“ !
23
Problems with Inner classes
• Inner classes can access private fields of
outer classes
final class Outer {
private String secret = "you will never be able to read this" ;
public void alter_secret(String x) {
secret = x;
}
private String reverseSecret() {
StringBuffer b = new StringBuffer(secret);
return b.reverse().toString();
}
class Inner {
private String innersecret = secret;
}
}
private String reverseinner = reverseSecret();
new
public static void main(String[] args) {
Outer outer = new Outer();
}
24
Problems with Inner classes
• You can‘t access these private fields from
other classes via java code, but you can
with a handcrafted bytecode class
new
new Outer
dup
invokespecial Outer/<init> ()V
// new Outer
dup // dup here to avoid local vars, take it directly from stack
dup
invokestatic Outer/access$000 (LOuter;)Ljava/lang/String;
getstatic java/lang/System/out Ljava/io/PrintStream;
swap // correct the positions
invokevirtual java/io/PrintStream/println (Ljava/lang/String;)V
invokestatic Outer/access$100 (LOuter;)Ljava/lang/String;
getstatic java/lang/System/out Ljava/io/PrintStream;
swap // correct the positions
invokevirtual java/io/PrintStream/println (Ljava/lang/String;)V
return
25
Other Problems with JDK
verifier
• SDK and JRE 1.3.1_01 or earlier
– „A vulnerability in the Java(TM) Runtime
Environment Bytecode Verifier may be exploited
by an untrusted applet to escalate privileges. „
• Also some Netscape browser versions were
affected
26
Problems with JDK verifier
• You can check your classes with a
standalone verifier
• Open source solution „Justice“ supplied in
Apache Jakarta project BCEL
new
27
Problems with Java security
What is also still missing
• Checks in terms of hard and soft limits on
– memory allocation
– Thread activation
• Excessive memory usage and threading utilization
often leads to
Denial of Service problems
28
The JVM:
Structures
and
Concepts
29
JVM Internals
• The architecture
– JVM is an abstract concept
– Sun just specified the interface
– implementation details depend on specific product (SUN
JDK, IBM JDK, Blackdown)
• Java bytecode, the internal language
– independent from CPU-type (bytecode)
– Stackoriented, object-oriented, type-safe
30
Runtime view on a JVM
Runtime Data storage
Class
loader
Method
Area (Classes)
Heap (Objects)
PC registers
JVM
runtime
Stack Frames
Native method
stacks
Native
methods
31
Runtime data
• Frame:
• Saves runtime state of execution threads, therefore
holds information for method execution (program
counter)
• All frames of a thread are managed in a stack frame
32
Runtime data
• Method area
– Runtime information of the class file
– Type information
– Constant Pool
– Method information
– Field information
– Class static fields
– Reference to the classloader of the class
– Reference to reflection anchor (Class)
33
The Constant Pool
• The "constant pool" is a heterogenous array of
data. Each entry in the constant pool can be one
of the following:
– string , class or interface name , reference to a field or
method , numeric value , constant String value
• No other part of the class file makes specific
references to strings, classes, fields, or methods.
All references constants and also for names of
methods and fields are via lookup into the
constant pool.
34
The Class File Structure
H
E
A
D
E
R
CONSTANTPOOL
FIELDS
METHODS
I
N
T
E
R
F
A
C
E
S
ACCESS
FLAGS
(Final,
Native,
Private,
Protected,
...)
A
T
T
R
I
B
U
T
E
S
• You can use a classdumper like javap -c or DumpClass
to analyze these inner details
35
The Class File Format
• Java class files are brought into the JVM via the
classloader
• The class file is basically just a plain byte array,
following the rules of the byte code verifier.
• All 16-bit and 32-bit quantities are formed by
reading in two or four 8-bit bytes, respectively,
and joining them together in big-endian format.
36
Methods and Fields
• The type of a field or method is indicated by a
string called its signature.
• Fields may have an additional attribute giving the
field's initial value.
• Methods have an additional CODE attribute
giving the java bytecode for executing that
method.
37
The CODE Attribute
• maximum stack space
• maximum number of local variables
• The actual bytecode for executing the
method.
• A table of exception handlers,
– start and end offset into the bytecodes,
– an exception type, and
– the offset of a handler for the exception
38
Bytecode
Basics
39
The JVM types
• JVM-Types and their prefixes
–
–
–
–
–
–
–
–
Byte
Short
Integer
Long
Character
Single float
double float
References
b
s
i (java booleans are mapped to jvm ints!)
l
c
f
d
a to Classes, Interfaces, Arrays
• These Prefixes used in opcodes (iadd, astore,...)
40
The JVM Instruction
Mnemonics
•
•
•
•
•
•
Shuffling (pop, swap, dup, ...)
Calculating (iadd, isub, imul, idiv, ineg,...)
Conversion (d2i, i2b, d2f, i2z,...)
Local storage operation (iload, istore,...)
Array Operation (arraylength, newarray,...)
Object management (get/putfield, invokevirtual,
new)
• Push operation (aconst_null, iconst_m1,....)
• Control flow (nop, goto, jsr, ret, tableswitch,...)
• Threading (monitorenter, monitorexit,...)
41
Bytecode
• Java Bytecode (JBC) are followed by zero or more
bytes of additional operand information.
• Table lookup instructions (tableswitch,
lookupswitch) have a flexible length
• The wide operation extension allows the base
operations to use „large“ operands
• No self-modifying code
• No branching to arbitrary locations, only to
beginning of instructions limited to scope of
current method (enforced by verifier!)
42
Bytecode
(Reverse)
Engineering
43
Bytecode Engineering tools
•
Obfuscators
–
•
Native compilers
–
•
Remove/Manipulate all information that can be used
for reverse engineering
„Real“ compile of java bytecodes to native instructions
(x86/sparc)
Build your own bytecode
–
–
Programmatic Generation
Manipulate classfiles with an API
44
Obfuscators
Techniques used
•
Identifier Name Mangling
•
•
•
The JVM does not need useful names for Methods and
Fields
They can be renamed to single letter identifiers
Constant Pool Name Mangling
•
•
Decrypts constant pool entries on runtime
Control flow obfuscation
•
•
Insertion of phantom variables, stack scrambling
And by relying on their default values inserting ghost
branch instructions, which never execute
45
Obfuscators
Problems with Obfuscation
•
•
•
Constant value Mangling implies overhead
processing in extra method call of an
„deobfuscatename“ method in each retrieval
from constant pool
Dynamic class loading may become broken as
classes get new names and reflection calls like
class.forName(„Account“) will fail because class
„Account“ now known as by it‘s obfuscated
name „b16“!
And: Obfuscation breaks patterns that can
be recognized by JIT-engines for
optimization
46
Protecting the Source Code:
Native Compilers
• Convert Java bytecode to C
• Generate executable via normal c-build
fast execution
Additional decompilation effort needed
Long turnaround times
Even for small java programs you get monster
size executable files (67mb source for Viva.java)
from some commercial products
Transformed program may than be vulnerable
to buffer overflows and off-by-ones
48
Bytecode Reverse Engineering
• Decompilation
– Get Source code from class files
• Graphical Analysis
– Rebuild the logical control flow
• Disassembly
– Get symbolic bytecode from class files
49
Decompilers
But only one really works
• Several available,
– MOCHA , by Hanpeter van Vliet was the first one
– JASMINE , patch of Mocha - crashed less often
– JAD, the fast Java Decompiler (written in C++) by
Pavel Kouznetsov is quite usable, but can also be fooled
• DJ JAVA , GUI for JAD
• NMI, GUI for JAD
50
Decompilers
• General method of decompilation
– Rebuilding control flow from class file code
segment
– Matching flow patterns to java language contructs
– Associating constants and external references from
constant pool entries
– Do not redo field/method and constant pool entry
mangling
– Condensed constant pool entries therefore sometimes
result in non-valid java identifiers in the generated
source
51
Graphical Analysis
Overview
• All Java Classfiles (those which are obfuscated,
too) have to be compliant to the JVM specification
• Although Control flow is interleaved in the
obfuscated class file, a graphical flow reveals most
of the original control flow
• The inner structures and dependencies of the
methods can be discovered by graphical analysis
52
53
Graphical Analysis
Tools
• BCEL: Byte-Code Engineering Library
– Written by to Markus Dahm
– Maintained by the Apache Jakarta Project
– Exposes internal structure of *.class files as
java objects
• aiSee: graphical visualisation tool
– Cross-platform
– Developed by the university of saarland
– Uses graphical description language
54
Graphical Analysis
Procedure
• Walk thru a specific code attribute of
methods in a classfile
• Retrieve target instructions
• Calculate control blocks
• Calculate successor relationships
• Rebuild control flow
• Export control flow graph (CFG) as GDL
• Display the GDL file for the control flow in
aiSee
55
Disassembling
• Javap
– The disassembler from the JDK
• Gnoloo
– disassembler by Joshua Engel from
„Programming for the JVM“
(ISBN: 0201309726)
– Can be used in pair with Oolong, the
corresponding assembler
– Comes with a good DumpClass tool
56
Applications of
Bytecode
(Reverse)
Engineering
57
Build your own bytecode
Tools
• Javac & Javap
– Included in the JDK
– Shows you how to translate high language
constructs to bytecode
• Hex-Editor for binary patching
– Ultraedit or Xemacs
• Oolong/Gnoloo
– Assembler and disassembler
• BCEL
59
Managing the Bytecode
with BCEL
• BCEL comes with several tools
– org.apache.bcel.util.Class2HTML
• Generates nice HTML pages for a class
– org.apache.bcel.util.JavaWrapper
• Replaces standard java interpreter to use own class
loader
• Lets you modify a loaded class via modifyClass
method, that can be overloaded
– org.apache.bcel.verifier.GraphicalVerifier
• Allows you to verify your class files with verbose
output
60
Binary Patching
with your favorite editor
• Read the output of „javap –c“ or DumpClass
• You got to know the opcodes values to find the
injection point
• Possibilities:
– so you can blank out some unwanted code with 00 00..
(nop)..
– Adjust some binary checks
– For example x9a0016 to x570000 („ifne +22“ to „pop
nop nop“)
• Remember to keep the right balance on the stack
61
Sample bytecode
The java source
public class Viva {
public static void main(String[] args) {
String[] TOTD = {"Viva","Las","Vegas"};
for (int i = 0; i < TOTD.length; i++) {
new Viva(TOTD[i]);
}
}
public Viva(String a) {
System.out.println(a.toUpperCase());
}
}
62
Sample bytecode
The disassembled Viva bytecode (Gnoloo) –I.method public static main
([Ljava/lang/String;)V
.limit stack 4
.limit locals 3
.line 3
iconst_3
anewarray java/lang/String
dup
iconst_0
ldc "Viva"
aastore
dup
iconst_1
ldc "Las"
aastore
dup
iconst_2
ldc "Vegas"
aastore
astore_1
.line 4
iconst_0
istore_2
goto l39
.line 5
l25:
new Viva
dup
aload_1
iload_2
aaload
invokespecial Viva/<init>
(Ljava/lang/String;)V
pop
.line 4
iinc 2 1
l39:
iload_2
aload_1
arraylength
if_icmplt l25
.line 7
return
63
Sample bytecode
The disassembled Viva bytecode (Gnoloo) –II.method public <init> (Ljava/lang/String;)V
.limit stack 2
.limit locals 2
.line 8
aload_0
invokespecial java/lang/Object/<init> ()V
.line 9
getstatic java/lang/System/out Ljava/io/PrintStream;
aload_1
invokevirtual java/lang/String/toUpperCase
()Ljava/lang/String;
invokevirtual java/io/PrintStream/println
(Ljava/lang/String;)V
.line 10
return
.end method
64
Analysis of some obfuscator‘s
work
public class a extends java.lang.Object {
public static boolean a;
public static boolean b;
public static void main(java.lang.String[]);
public a(java.lang.String);
}
public class Viva extends java.lang.Object {
Method void main(java.lang.String[])
0 getstatic #42 <Field boolean b>
3 istore_3
4 iconst_3
5 anewarray class #1 <Class java.lang.String>
8 dup
9 iconst_0
10 ldc #2 <String "cLxt">
12 invokestatic #61 <Method java.lang.String deobfuscateName(java.lang.String)>
15 aastore
16 dup
17 iconst_1
18 ldc #3 <String "yD ">
20 invokestatic #61 <Method java.lang.String deobfuscateName(java.lang.String)>
23 aastore
24 dup
25 iconst_2
26 ldc #4 <String "c@ t ">
28 invokestatic #61 <Method java.lang.String deobfuscateName(java.lang.String)>
31 aastore
Method void main(java.lang.String[])
public static void main(java.lang.String[]);
public Viva(java.lang.String);
}
0
1
4
5
6
iconst_3
anewarray class #1 <Class java.lang.String>
dup
iconst_0
ldc #2 <String "Viva">
DEMO
8 aastore
9 dup
10 iconst_1
11 ldc #3 <String "Las">
13
14
15
16
aastore
dup
iconst_2
ldc #4 <String "Vegas">
18 aastore
65
Things you can only do in JBC
methods that only differ in return type!
.class
.super
.field
.field
public overload
java/lang/Object
static myint I
static mydouble D
.method static jvmoverload
()I
.limit stack 3
.limit locals 1
iconst_1
ireturn
.end method
.method static jvmoverload
()D
.limit stack 3
.limit locals 1
dconst_1
dreturn
.end method
.method public static main
([Ljava/lang/String;)V
.limit stack 3
.limit locals 1
invokestatic
overload/jvmoverload ()I
putstatic overload/myint I
invokestatic
overload/jvmoverload ()D
putstatic
overload/mydouble D
return
.end method
.end class
66
Things you can only do in JBC
methods that only differ in return type!
If the decompiled source is compiled again:
>javac -classpath . overload.java
overload.java:15: jvmoverload() is already defined in
overload
static double jvmoverload()
^
1 error
• The signatures are considered duplicate in java
• But valid in java bytecode, and therefore can be used in
obfuscation techniques
67
DEMO
68
Usage Patterns
• Scenarios
– Tools for database access
– Adding dynamically Logging, profiling facilities
without changing sourcecode
– Verifying class files on domain boundaries
(add-in to content filtering systems)
– Find Frontier bugs, before they may be
exploited for DoS-attacks by calling code
which crashes the JVM
– Port your own language to Java bytecode
69
Java Library Holes
• In JNI-based applications checking for null
pointers is often given too little regard
• Also the routines in the core Java Runtime classes
use native code (included in jvm.dll)
• But not every entry into these trusted libs is
shielded appropriatly
• In some cases you can force a JVM crash when
pumping „null“ values to that routines
70
Frontier Bugs
Sample
package crashtest;
import sun.dc.pr.PathDasher;
public class CrashTest
{
public CrashTest()
{
PathDasher d = new PathDasher(null);
}
public static void main(String args[])
{
CrashTest crashTest1 = new CrashTest();
}
}
Unexpected Signal : EXCEPTION_ACCESS_VIOLATION occurred at PC=0x6d443081
Function name=JVM_DisableCompiler
Library=H:\programme\JB6\jdk1.3.1\jre\bin\hotspot\jvm.dll
Current Java thread:
at sun.dc.pr.PathDasher.cInitialize(Native Method)
at sun.dc.pr.PathDasher.<init>(PathDasher.java:48)
at crashtest.CrashTest.<init>(CrashTest.java:91)
at crashtest.CrashTest.main(CrashTest.java:98)
Dynamic libraries:
0x00400000 - 0x00405000
0x77F40000 - 0x77FF0000
....
H:\programme\JB6\jdk1.3.1\bin\javaw.exe
C:\WINDOWS\System32\ntdll.dll
71
Frontier Bugs
Localization
• This possible vulnerability may also be harmful in
other tools as if they rely on JNI (native interface)
classes calling to some native core (written in
some appropriate language, C or C++)
• Example JDK: The frontier between
– java library code (rt.jar) and the
– jvm (jvm.dll, libjvm.so)
has official and inofficial entrypoints
– The official ones are called by Native
Java-routines recommended by Sun (java.*)
– The inofficial are called by sun.* routines
72
Frontier Bugs
Semiautomatic Detection
• Traverse java runtime libraries
• Procedure
– Detect public classes
– In the public classes detect public methods
– From that filter native methods
– Now you have a list containing the public entry points to the JVM
library or another JNI-library
• Tool: NativeFinder, which is based on BCEL
73
Frontier Bugs
Semiautomatic Detection
• On this list:
– Start with the static class methods, they are easier to
test
– With the object methods, you need to know how to
build such on object
• In the easy case you have an default constructor
• In the hard case you will have to delve deep into the class
hierarchy to get an object
– Then call the method with some nice „null“ values and
wait 
74
DEMO
75
Soon to become a problem:
Hot Swap
• Until JDK 1.3 class definitions were immutable
during the lifetime of a JVM
• JDK 1.4 now offers a „hot swap“ of class
definitions via JVMDI (debugger API)
• Now you (or someone else?) can „substitute your
class definitions by modified code in a running
application through the debugger“
76
Q&A
Mail to [email protected] for the tools
Or visit our website
http://www.illegalaccess.org/
for some more examples and links (sept 02)
77
Thank
you!
To You as a great audience,
To The Blackhat Staff (specially Charel for the tickets),
To Halvar, Johnny and G0dzilla for the helpful discussions,
And to my Girlfriend
78