Transcript WildHeart.HTTPListener

WildHeart.HTTPListener
an open source Web Server
for Dyalog APL v11.0
Stefano
“WildHeart”
Lanzavecchia
Agenda
 APL should be a popular language for
developing web pages, because:
– APL has all you need to generating static and
dynamic web content
 It is not used as much as it deserves,
because:
– Even the best tools need good frameworks to
level the learning curve a bit
– No frameworks are readily available
2006-10-17
Dyalog User Group Conference
2
Goal
 Explore the problem which needs to be
solved and demonstrate a proposed
solution... (obviously )
 So, what’s the problem?
 The easy part is generating HTML
 Hopefully nobody will disagree that APL is
an EXCELLENT language for generating
character vectors
2006-10-17
Dyalog User Group Conference
3
Some very simple HTML
A ”web”
2006-10-17
Dyalog User Group Conference
4
The hard part
If we have a form like this and the user hits the submit button…
2006-10-17
Dyalog User Group Conference
5
Before we look at the response…
<tr>
Some of you will want to know
how the HTML looks for the form?
<form action="RUN" method="post" name="FrontPage_Form1">
<div>
<table border="0" cellpadding="3">
<tr>
<td>Purchase Amount</td>
<td><input type="text" size="9" name="LoanAmt"/></td>
</tr>
<tr>
<td>Percent Down</td>
<td><input type="text" size="6" maxlength="6"
name="PercentDown"/></td>
</tr>
<tr>
<td>Maximum Years in Loan</td>
<td><input type="text" size="2" maxlength="2"
name="LenMax"/></td>
</tr>
<tr>
<td>Minimum Years in Loan</td>
<td><input type="text" size="2" maxlength="2"
name="LenMin"/></td>
</tr>
… continued on the right …
2006-10-17
<td>Maximum Interest Rate (%)</td>
<td><select name="IntrMax" size="1">
<option>15.0</option>
<option>14.5</option>
..etc down to…
<option>1.0</option>
</select></td>
</tr>
<tr>
<td>Minimum Interest Rate (%)</td>
<td><select name="IntrMin" size="1">
<option>15.0</option>
<option>14.5</option>
<option>14.0</option>
<option>13.5</option>
<option selected="selected">13.0</option>
<option>12.5</option>
<option>12.0</option>
…etc…
<option>1.5</option>
<option>1.0</option>
</select></td>
</tr>
</table>
<p><input type="submit" value="Calculate
Repayments"/></p>
</div>
</form>
Dyalog User Group Conference
6
When the user hits the button…
POST /loan/RUN HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, application/vnd.ms-excel,
application/vnd.ms-powerpoint, application/msword, */*
Referer: http://localhost:8080/loan/loan.htm
Accept-Language: it
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;
.NET CLR 1.1.4322; .NET CLR 2.0.50727; WinFX RunTime 3.0.50727)
Host: localhost:8080
Content-Length: 74
Connection: Keep-Alive
Cache-Control: no-cache
LoanAmt=10000&PercentDown=50&LenMax=10&LenMin=15&
IntrMax=15.0&IntrMin=13.0
2006-10-17
Dyalog User Group Conference
7
The hard part…
 APL is not a bad tool for taking character
vectors apart, either 
 But: Some knowledge about the HTTP
protocol is required to decipher it
 A rule for calling the code to deal with the
request (in this case #.loan.RUN) is
needed
 Some way to decode the body of the
request
 This is what Web Server frameworks like
Microsoft IIS coupled with ASP.NET do…
2006-10-17
Dyalog User Group Conference
8
What is a web application?
“A collection of functions that take an
HTTP Request as input and produce
an HTTP response as output”
2006-10-17
Dyalog User Group Conference
9
Solutions
 Dyalog APL integrates quite well with Microsoft
IIS+ASP.NET, and can be called from other web
servers, but:
– These solutions require a lot of heavy machinery from
other vendors
– The learning curve is still quite high
 Dyalog provide a sample SERVER workspace,
but:
– It leaves almost all the work of deciphering HTTP to you
– It’s based on old standards
2006-10-17
Dyalog User Group Conference
10
Solutions
The author has:
 A keen (geek) interest in OO, web
and other software development
methodologies
 Written numerous tools on top of the
Dyalog TCPSocket object
2006-10-17
Dyalog User Group Conference
11
Enter WildHeart.HTTPListener
 The advent of Object Orientation in Dyalog 11.0
makes it attractive to start again
 My goal is to create a framework which:
– Requires a little knowledge of HTML and hardly any
–
–
–
–
knowledge of HTTP
Allows ”ordinary” APL developers to easily write Web
applications, yet:
Is built from reusable, replaceable components which
can be extended by APL developers who learn a little
more...
Is portable across all platforms which support TCP
Is run as an open source project, to be shared and
extended by anyone who wishes to contribute
2006-10-17
Dyalog User Group Conference
12
To write or Not to write
 Why NOT TO WRITE a web server in APL:
– This is the third web server I develop:
• I was paid to develop the first (and it was never used)
• I was almost paid to the develop/steal the second
(and it’s heavily used)
• I was NOT paid to develop the third and I had to find
the time to do it at night and during weekends
 A wise man once said:
“only a fool would do that”
2006-10-17
Dyalog User Group Conference
13
Not to write, seriously
 HTTP is easy but tricky
 Reliability
 Security
 Speed
 So many web servers out there
2006-10-17
Dyalog User Group Conference
14
To write or not to write
 Why write a web server in APL?
(I wrote an editorial about it in Vector a few years ago)
– Easily embeddable in an APL
application
– Easily understood by APLers
– Can stay near the core of the
application
2006-10-17
Dyalog User Group Conference
15
To write or not to write
"One of the reasons to use Lisp in
writing Web-based application is that
you can use Lisp. When you're
writing software that is only going to
run on your own servers, you can
use whatever language you want.“
Paul Graham
2006-10-17
Dyalog User Group Conference
16
To write!
“Morten needs a demo. I need an
HTTP/1.1 web server, written in APL.
APL Italiana, the company I work for,
also needs one, even if they haven’t
realised this yet.”
Stefano Lanzavecchia
2006-10-17
Dyalog User Group Conference
17
Components of… WildServer’06
 HTTPListener
 HTTPRequest
 Static file server
 Namespace as
– Headers
– Cookies
– Request Variables
 HTTPResponse
– Cookies
 HTML Utils
 Event pipeline
filesystem…
 File Handlers:
– aplx files
– apls files
– …
– User Validation
– Sessions
2006-10-17
Dyalog User Group Conference
18
Dyalog APL: an OO language
 Web server: a good test bed for the
OO extensions
– Abstractions: divide et impera
– Inheritance: this looks a bit like that
– Keyed properties: syntax tricks
 What is missing in the language?
 Is the IDE good enough?
2006-10-17
Dyalog User Group Conference
19
HTTPListener
fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')
listener←⎕NEW web.HTTPListener(8080 fs)
listener.Start
Then point your web browser at http://localhost:8080/
“It’s a kind of magic”
2006-10-17
Dyalog User Group Conference
20
The trip of an HTTP request
 TCPSocket gets TCPRecv events
 HTTPRequest objects collects them,
and parses the request according to
the HTTP specs
 When the request is complete, it gets
shipped to a dispatcher…
2006-10-17
Dyalog User Group Conference
21
The dispatcher: FileSystem
 To me a FileSystem looks like a
namespace:
– Its folders are sub-namespaces
– Idea: we can inject virtual folders:
• A virtual folder can be a folder on the HD
coming from a different sub-folder
• A virtual folder can be… an APL
namespace! Its functions would be files…
2006-10-17
Dyalog User Group Conference
22
APLNamespace
fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')
fs['loan']←⎕NEW
web.APLNamespace(#.loan'c:\aplserve\loan')
listener←⎕NEW web.HTTPListener(8080 fs)
listener.Start
http://localhost:8080/loan/RUN
executes #.loan.RUN
but
http://localhost:8080/loan/bullet.gif
will transmit the file c:\aplserve\loan\bullet.gif
2006-10-17
Dyalog User Group Conference
23
ZipFileSystem
fs←⎕NEW
web.ZipFileSystem('c:\aplserve.zip?aplserve')
fs['loan']←⎕NEW
web.APLNamespace(#.loan'c:\aplserve\loan')
listener←⎕NEW web.HTTPListener(8080 fs)
listener.Start
 Now the folders and files come from a zip file.
 Neat, uh?
 In fact this presentation…
2006-10-17
Dyalog User Group Conference
24
FileSystem
 Basically one method: ProcessRequest
∇ ProcessRequest ctx;html;ns;raw;aplo
:Access Public Instance Override
ctx←#.WildHeart.HTTPContext.Current
html←_NS⍎MapFileName ctx.URI
r←ctx._Response
r._IsBodyRaw←raw←82≠⎕DR html
r._Body←html
:If ''≡⊃r._Headers[⊂'Content-type']
r._Headers[⊂'Content-type']←u.MimeMappings['htm'
'bin'[1+raw]]
:EndIf
ctx.SendResponse
∇
2006-10-17
Dyalog User Group Conference
25
Tricks: thread local storage
[…]
ctx←#.WildHeart.HTTPContext.Current
[…]
 Thread context: wouldn’t it be nice to
have an object local to a thread?
2006-10-17
Dyalog User Group Conference
26
HTTPHandler
 A FileSystem delegates the handling of
the requested resource to the appropriate
handler:
– StaticFileHandler by default
– AplxHandler
– AplsHandler
– … more to come
fs._FileHandlers[⊂'aplx']←#.WildHeart.AplxFileHandler
2006-10-17
Dyalog User Group Conference
27
HTTPHandler
 One method: ProcessRequest
∇ ProcessRequest ctx;fname;tn;ext;etag
:Access Public Instance Overridable
fname←ctx._FileSystem.MapFileName ctx.URI
tn←fname ⎕NTIE 0
ctx._Response._IsBodyRaw←1
ctx._Response._Body←⎕NREAD tn,83,⎕NSIZE tn
⎕NUNTIE tn
ctx.SendResponse
∇
2006-10-17
Dyalog User Group Conference
28
AplxHandler
“This is only a test”
<?@ master="../newindex.maplx" execute="" ?>
<apl:content id="title">
<title>Rain Graphics: Climate</title>
</apl:content>
<apl:content id="cssstyle">
<link rel="stylesheet" type="text/css" href="../styles/spiffy.css"/>
</apl:content>
<apl:content id="main" execute="#.rain.Climate">
</apl:content>
2006-10-17
Dyalog User Group Conference
29
A master page
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML
1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml
1-transitional.dtd">
<html xmlns=“http://www.w3.org/1999/xhtml”
xmlns:apl="http://wildheart.com/2006/10/
"
>
<head>
<apl:placeholder id="cssstyle">
<link rel="stylesheet"
type="text/css"
href="styles/spiffy.css"/>
</apl:placeholder>
<apl:placeholder id="title">
<title/>
</apl:placeholder>
<body>
<div id="header"><h1
class="banner">WildHeart.HTTPListener</h
1></div>
<div id="subheader"> <apl:placeholder
id="bcrumb"
execute="#.WildHeart.WebDemo
s.BreadCrumb" />
</div>
<div id="toc">
<apl:placeholder id="apltoc">
<div class="toc"><h2>APL Web Server</h2>
<ul>
<li><a
href="/intro.aplx">Introduction</a></li>
<li><a href="/cv/Index.aplx">View
Code</a></li>
…
</head>
2006-10-17
Dyalog User Group Conference
30
The event pipeline
 As a request goes through the
FileSystem hierarchy, events are
raised:
– OnEnter
– OnAuthenticate
– OnAuthorize
–…
2006-10-17
Dyalog User Group Conference
31
Tricks: the observer pattern
∇ Add(obj fnname)
:Class Event
:field _Observers
:Access Public Instance
_Observers,←⊂obj fnname
∇ make name
:Access Public Instance
:Implements Constructor
_Observers←⍬
⎕DF'Event: ',name
∇
∇ Raise(sender arg);obj;fnname
:Access Public Instance
:For obj fnname :In _Observers
(obj⍎fnname)(sender arg)
:EndFor
∇
2006-10-17
∇
∇ Del(obj fnname)
:Access Public Instance
_Observers←
_Observers~⊂obj fnname
∇
:EndClass
Dyalog User Group Conference
32
The observed shows what it’s got
 onAuthenticate←⎕NEW
#.WildHeart.Event('Authenticate')
auth←⎕NEW web.WebDemos.FormsAuth
fs.onAuthenticate.Add
auth'OnAuthenticate'
 onAuthenticate.Raise ⎕THIS lurl
2006-10-17
Dyalog User Group Conference
33
Two simple authentication modules
 SimpleAuth: based on HTTP
challenge response:
– Quick and dirty
 FormsAuth: based on cookies
– More flexible
– More work
2006-10-17
Dyalog User Group Conference
34
Sessions
 HTTP is a stateless protocol
 People want to have conversations
with peers who don’t forget instantly
what they just said
 Have a cookie!
2006-10-17
Dyalog User Group Conference
35
Sessions
 As easy as:
fs.EnableSession ⎕NULL
fs.onSessionStart.Add #’DoSomething'
fs.onSessionEnd.Add #’UndoSomething'
 In the application:
:If ⎕NULL≡ns←ctx._Session[⊂'footer']
ns←⎕NS''
ns.count←0
ctx._Session[⊂'footer']←ns
:EndIf
ns.count+←1
2006-10-17
Dyalog User Group Conference
36
Extensibility
 Some of the features (ZipFileSystem,
session module, authentication, aplx
handler) were hacked together to prove
that the server could support them
 Even before its completion, the WildServer
has:
– Had a file handler written (by Nic & Morten)
– Been used to run a course in writing Web
Applications using Dyalog APL
2006-10-17
Dyalog User Group Conference
37
Open Source
 Open Source means: “free as in free
beer” (except: is there such a thing as free beer?)
 No strings attached:
– You get the source code and whatever
documentations exists with it at the time
– You get the right to do whatever you want with
it (even sell it! But only a bad boy would do that…)
– You get from me all the support I can give you when I
can if I can (alas, I am very busy and maybe one day I’ll
get a life)
2006-10-17
Dyalog User Group Conference
38
Why Open Source?
 Have you looked at Ruby on Rails?
 Tons of contributors do a lot of work
 Tons of clever contributors do a lot
of good work!
 OK, maybe that’s a good product.
2006-10-17
Dyalog User Group Conference
39
TODO list
 It’s long, it’s scary, but…
 APL Prototypes seem to acquire a
life of their own
 The “ducks” have started
contributing
2006-10-17
Dyalog User Group Conference
40
Thanks
 Peter Michael Haager: I have adapted
his XML parser to produce trees of
objects instead of nested arrays
2006-10-17
Dyalog User Group Conference
41
Conclusion
 Initial feedback from users (Nic, Morten
and Mondays course participants) is
promising
 The WildServer is still very much ”work in
progress”, and the author wants to keep
control of it until about Christmas
 However, you are all welcome to try it now
and suggest or code enhancements
2006-10-17
Dyalog User Group Conference
42
Questions?
 If not… now I need some sleep.
輝く空と君の声を
感じたい
2006-10-17
Dyalog User Group Conference
43
What is missing in the language
 No public operators! C# has them!
 No protected members! C# has them!
 No object serialization! C# has it!
 C# is a statically typed language. OK,
substitute in the slide “C#” with
“Ruby”
2006-10-17
Dyalog User Group Conference
44