Transcript Escape from the Spaghetti Code Jungle

Reflective Programming in
Smalltalk
eXterme Metaprogramming
Brian Foote
University of Illinois at Urbana-Champaign
7 May 1998
Smalltalk Solutions ‘98
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 1
Reflective Programming in
Smalltalk
• Definitions
• Building a Language Out of Objects
• Examples
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 2
Frequently Asked Questions
Q: What is reflection?
A: It’s about building your language
out of first-class, dynamic objects that
you can look at and change at runtime.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 3
Frequently Asked Questions
Q: Haven’t people really been doing
this for a long time?
A: They certainly have. Smalltalk
programmers have been doing this
since the ‘70s.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 4
Reflection from the
Eisenhower Era
Hence, [the machine] can, in particular,
change the orders (since these are in
memory!)--the very orders that control
its actions
--John Von Neumann 1958
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 5
Frequently Asked Questions
Q: What about efficiency?
A:
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 6
Frequently Asked Questions
Q: What about efficiency?
A: There are a variety of techniques
that can be used to make reflective
systems faster.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 7
Frequently Asked Questions
Q: Isn’t reflection dangerous?
A: Yes! You bet it is!
A: Yes, if you are not careful.
A: Yes, but you can make it safer.
A: Yes, but so is crossing the street.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 8
Frequently Asked Questions
Q: Isn’t reflection just a fancy name
for a few clever hacks?
A: Well, that and so much more...
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 9
Frequently Asked Questions
Q: Why is so much of the reflection
literature so hard to read?
A: This is due, in part, to the area’s AI
heritage...
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 10
Reflective Terminology
• introspection
• reflection
• reification
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 11
Reflective Terminology
• infinite regress
• causal connection
• reflective tower
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 12
Introspection
When a program can look at at the
objects from which it is built
Smalltalk has a rich, comprehensive,
indeed, unrivaled collection of
introspective facilities
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 13
Reflection
When a program can change the
objects from which it is built
Smalltalk represents as much as it can
as as first-class dynamic objects, and
you can change them at runtime.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 14
Consider these
Meta considered harmful
“Meta” considered harmful
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 15
Neat Hack Hall of Fame
 Parentless Objects
 Does Not Understand
 Metaobjects
 Lightweight Classes
 Method Wrappers
 Byte Code Manipulation
 Compiled Method Copying
 Context Manipulation
 Source Generation
 Association Hacks
 Becomes
 Class Change
 Instance Variable At
 Methods On-Demand
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 16
Objects themselves
Object
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 17
Object Introspection
Object size == basicSize hash identityHash printOn:
storeOn:
dependents ...
allOwners firstOwner nextInstance ownerAfter:
instVarAt:
isKindOf: class isMemberOf: respondsTo:
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 18
Example:
AccessibleObjects
Demonstrates:
doesNotUnderstand:
instVarAt: instVarAt:put:
at: at:put:
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 19
Accessible Objects
AccessibleObject class methods for: examples
example
"AccessibleObject example"
| temp |
temp := AccessibleObject new.
temp dog: 'Fido'.
temp cat: 'Tabby'.
Transcript print: temp dog; cr.
Transcript print: temp items; cr.
temp keysDo: [:key | Transcript print: key; cr].
Transcript print: (temp variableAt: #items); cr.
Transcript endEntry
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 20
Accessible Objects
AccessibleObject methods for: accessing
at: key
"Return the object associated with the given key..."
^self valueAt: key
at: key put: value
"Store the indicated value at the designated place in our item
dictionary... "
^self valueAt: key put: value
size
"Let's say our size is the size of our item dictionary
plus our number of instance variables..."
^self items size + self instVarNames size
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 21
Accessible Objects
AccessibleObject methods for: accessing
valueAt: key
"Return the object associated with the given key..."
^self valueAt: key ifAbsent: [self errorKeyNotFound]
valueAt: key put: value
"Store the indicated value at the designated place in our item
dictionary, unless there is an instance var by that name..."
items isNil ifTrue: [items := IdentityDictionary new: 16].
(self hasVariableNamed: key)
ifTrue: [^self variableAt: key put: value]
ifFalse: [^items at: key put: value]
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 22
Accessible Objects
AccessibleObject methods for: instance variable access
allInstVarNames
"Define a shorthand for this class method..."
^self class allInstVarNames
hasVariableNamed: name
"Say whether we have a variable by the given name..."
^(self variableIndex: name) ~= 0
instVarNames
"Define a shorthand for this class method..."
^self class instVarNames
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 23
Accessible Objects
AccessibleObject methods for: instance variable access
variableAt: name
"Return the named value..."
| index |
index := self variableIndex: name.
index = 0 ifTrue: [self error: 'Bad instance variable name...'].
^self instVarAt: index
variableAt: name put: value
"Set the named instance variable to the indicated value..."
| index |
index := self variableIndex: name.
index = 0 ifTrue: [self error: 'Bad instance variable name...'].
^self instVarAt: index put: value
variableIndex: name
"Return the instance variable index for this name, or zero..."
^self class allInstVarNames indexOf: name asString.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 24
Accessible Objects
AccessibleObject methods for: error interception
doesNotUnderstand: aMessage
"Refer messages we don't understand to our item dictionary..."
1)]].
| selector name args |
selector := aMessage selector.
name := (selector copyWithout: $:) asSymbol.
args := aMessage arguments.
(self hasVariableNamed: name)
ifTrue: [args size = 0
ifTrue: [^self variableAt: name]
ifFalse: [^self variableAt: name put: (args at:
(items respondsTo: selector)
ifTrue: [^items perform: selector withArguments: args].
args size = 1
ifTrue: [^self valueAt: name put: (args at: 1)]
ifFalse: [^self valueAt: name ifAbsent:
[^super doesNotUnderstand: aMessage]]
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 25
Classes and Behavior
Behavior
ClassDescription
Class
Metaclass
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 26
Organizations
ClassBuilder
SystemOrganizer
ClassOrganizer
ClassCategoryReader
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 27
Code Representation
MethodDictionary
CompiledMethod
ByteArray
BlockClosure
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 28
Runtime Enviroment
Context
MethodContext/BlockContext
Message
MessageSend
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 29
Exceptions and Events
Event
Exception
Signal
SignalHandler SignalCollection
HandlerList
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 30
Process Scheduling
Process
ProcessScheduler
Semaphore
SharedQueue
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 31
Viewing the Program
Debugger
Decompiler
Inspector
ChangeList
<Browsers>
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 32
Making a Promise
Future methodsFor: ‘demonstration’
demo
|f|
f := Future promising: [2+2].
f printString '4.0'
Future class methodsFor: 'instance creation'
promising: aBlock
| aFuture |
aFuture := self new.
^aFuture promising: aBlock
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 33
Creating an Orphan
nil subclass: #Future
instanceVariableNames: ‘semaphore '
classVariableNames: ‘ '
poolDictionaries: ' '
category: ‘Reflection-Examples’
In VisualWorks, ClassBuilder does the rest. Default
implementations of doesNotUnderstand: and class are
provided.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 34
We’ll do it eventually...
Future methodsFor: 'initialization/dispatching'
promising: aBlock
"Create a semaphore, and fork a block that
will signal it. The result of this block is stored
in result..."
semaphore := Semaphore new.
[result := aBlock value.
semaphore signal] fork.
^self
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 35
Keeping a Promise
doesNotUndertand: aMessage
"If this is our init message, let it by..."
aMessage selector == #promising:
ifTrue: [^super perform: aMessage selector
withArguments: aMessage arguments].
"Wait until our result is available..."
semaphore wait.
"If our result is a SmallInteger, it has no oop.."
(result isKindOf: SmallInteger)
ifTrue: [result := result asFloat].
"Become the result and do the
deferred message..."
result become: self.
^self perform: aMessage selector
withArguments: aMessage arguments
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 36
Metaobjects and
Lightweight Classes
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 37
Compiler Classes
Compiler
Decompiler
CodeRegenerator
Scanner
SmalltalkCompiler
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 38
Compiler Support
CodeStream
DefineOpcodePool
MethodNodeHolder
ProgramNodeEnumerator
ScannerTable
<more>
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 39
Variables and Scopes
ArgumentVariable InstanceVariable
LocalScope LocalVariable
NameScope NullScope
PseudoVariable ReceiverVariable
RemoteVariable StaticScope
StaticVariable TemporaryVariable
UndeclaredVariable
VariableDefinition
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 40
Parse Tree Nodes
ProgramNode
MethodNode ParameterNode
StatementNode ReturnNode
ValueNode ArithmeticLoopNode
AssignmentNode CascadeNode
ConditionalNode LeafNode BlockNode
LiteralNode VariableNode LoopNode
SequenceNode SimpleMessageNode
MessageNode
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 41
Smalltalk
SystemDictionary
Association/VariableBinding
<class and pool variables>
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 42
Storage and Garbage
ObjectMemory
MemoryPolicy
WeakArray
WeakDictionary
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 43
Levels of Representation
Source
Compiler
Parse Node
ProgramNode ProgramNode
Byte Code
VM
Decompiler
Native Code
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 44
Ways to Wrap
• Source Code Modifications
• Byte Code Modifications
• New Selectors
• Dispatching Wrappers
• Class Wrappers
• Instance Wrappers
• Method Wrappers
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 45
Compiled Methods
OrderedCollection class
methodDict
…
CompiledMethod
mclass
bytes
sourceCode
1
MethodDictionary
#removeFirst
…
#[17 68 …]
removeFirst
self emptyCheck.
…
#emptyCheck
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 46
Method Wrappers
OrderedCollection class
methodDict
…
MethodDictionary
#removeFirst
…
118567696
MethodWrapper
mclass
bytes
clientMethod
sourceCode
1
2
…
#receiver:
7/17/2015 (C) Brian Foote 1998
#[17 68 …]
CompiledMethod
mclass
bytes
sourceCode
1
removeFirst
self emptyCheck.
…
#emptyCheck
Reflective Programming in Smalltalk
Slide -- 47
Method Wrappers
valueWithReceiver: anObject arguments: args
self beforeMethod.
^[clientMethod
valueWithReceiver: anObject
arguments: args]
valueNowOrOnUnwindDo:
[self afterMethod]
originalMethodName: argument
^#() receiver: self value: argument
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 48
Multimethods
OptimizingVisitor>>visitWithNode: aNode <ParseNode>
^self value optimized
OptimizingVisitor>>
visitWithNode: aNode <VariableNode>
^aNode lookupIn: self symbolTable
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 49
Parse Tree
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 50
Issues
Power
Safety
Efficiency
Comprehensibility
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 51
Reflection in Smalltalk
Astonishing Introspective
Highly Reflective
Intelligently Factored
Uniquely Malleable
Unappoligetically Dynamic
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 52
Frequently Asked Questions
Q: What can’t I get my hands on?
A: In ‘89 we said: sending a
message, receiving a message,
returns, and instance variable
access. Oh, and the
ObjectMemory...
Progress has been made.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 53
Metalevel Architecture
Building languages out of objects
allows one to achieve the goals of
reflection
Loops, Lazy Reification, and Induction
Where does the language end?
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 54
Go Ye Forth and Hack
The best way to learn how do reflective
programming in Smalltalk is to read
code and experiment.
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 55
Contact Information
Brian Foote
Dept. of Computer Science
3253 DCL
1304 W. Springfield
Urbana, IL 61801
(217) 328-3523
http://www.laputan.org
[email protected]
7/17/2015 (C) Brian Foote 1998
Reflective Programming in Smalltalk
Slide -- 56