1. WildHeart.HTTPListener an open source Web Server for Dyalog APL v11.0 Stefano “WildHeart” Lanzavecchia

2. 2006-10-17Dyalog User Group Conference2 Agenda APL should be a popular language for developing web pages, because: – APL has all you need to generating static and dynamic web content It is not used as much as it deserves, because: – Even the best tools need good frameworks to level the learning curve a bit – No frameworks are readily available

3. 2006-10-17Dyalog User Group Conference3 Goal Explore the problem which needs to be solved and demonstrate a proposed solution... (obviously ) So, what’s the problem? The easy part is generating HTML Hopefully nobody will disagree that APL is an EXCELLENT language for generating character vectors

4. 2006-10-17Dyalog User Group Conference4 A ” web ” Some very simple HTML

5. 2006-10-17Dyalog User Group Conference5 The hard part If we have a form like this and the user hits the submit button…

6. 2006-10-17Dyalog User Group Conference6 Before we look at the response… Purchase Amount Percent Down <input type="text" size="6" maxlength="6" name="PercentDown"/> Maximum Years in Loan <input type="text" size="2" maxlength="2" name="LenMax"/> Minimum Years in Loan <input type="text" size="2" maxlength="2" name="LenMin"/> … continued on the right … Maximum Interest Rate (%) 15.0 14.5..etc down to… 1.0 Minimum Interest Rate (%) 15.0 14.5 14.0 13.5 13.0 12.5 12.0 …etc… 1.5 1.0 Some of you will want to know how the HTML looks for the form?

7. 2006-10-17Dyalog User Group Conference7 When the user hits the button… POST /loan/RUN HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */* Referer: http://localhost:8080/loan/loan.htm Accept-Language: it Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;.NET CLR 1.1.4322;.NET CLR 2.0.50727; WinFX RunTime 3.0.50727) Host: localhost:8080 Content-Length: 74 Connection: Keep-Alive Cache-Control: no-cache LoanAmt=10000&PercentDown=50&LenMax=10&LenMin=15& IntrMax=15.0&IntrMin=13.0

8. 2006-10-17Dyalog User Group Conference8 The hard part… APL is not a bad tool for taking character vectors apart, either But: Some knowledge about the HTTP protocol is required to decipher it A rule for calling the code to deal with the request (in this case #.loan.RUN) is needed Some way to decode the body of the request This is what Web Server frameworks like Microsoft IIS coupled with ASP.NET do…

9. 2006-10-17Dyalog User Group Conference9 What is a web application? “A collection of functions that take an HTTP Request as input and produce an HTTP response as output”

10. 2006-10-17Dyalog User Group Conference10 Solutions Dyalog APL integrates quite well with Microsoft IIS+ASP.NET, and can be called from other web servers, but: – These solutions require a lot of heavy machinery from other vendors – The learning curve is still quite high Dyalog provide a sample SERVER workspace, but: – It leaves almost all the work of deciphering HTTP to you – It’s based on old standards

11. 2006-10-17Dyalog User Group Conference11 Solutions The author has: A keen (geek) interest in OO, web and other software development methodologies Written numerous tools on top of the Dyalog TCPSocket object

12. 2006-10-17Dyalog User Group Conference12 Enter WildHeart.HTTPListener The advent of Object Orientation in Dyalog 11.0 makes it attractive to start again My goal is to create a framework which: – Requires a little knowledge of HTML and hardly any knowledge of HTTP – Allows ”ordinary” APL developers to easily write Web applications, yet: – Is built from reusable, replaceable components which can be extended by APL developers who learn a little more... – Is portable across all platforms which support TCP – Is run as an open source project, to be shared and extended by anyone who wishes to contribute

13. 2006-10-17Dyalog User Group Conference13 To write or Not to write Why NOT TO WRITE a web server in APL: – This is the third web server I develop: I was paid to develop the first (and it was never used) I was almost paid to the develop/steal the second (and it’s heavily used) I was NOT paid to develop the third and I had to find the time to do it at night and during weekends A wise man once said: “only a fool would do that”

14. 2006-10-17Dyalog User Group Conference14 Not to write, seriously HTTP is easy but tricky Reliability Security Speed So many web servers out there

15. 2006-10-17Dyalog User Group Conference15 To write or not to write Why write a web server in APL? (I wrote an editorial about it in Vector a few years ago) – Easily embeddable in an APL application – Easily understood by APLers – Can stay near the core of the application

16. 2006-10-17Dyalog User Group Conference16 To write or not to write "One of the reasons to use Lisp in writing Web-based application is that you can use Lisp. When you're writing software that is only going to run on your own servers, you can use whatever language you want.“ Paul Graham

17. 2006-10-17Dyalog User Group Conference17 To write! “Morten needs a demo. I need an HTTP/1.1 web server, written in APL. APL Italiana, the company I work for, also needs one, even if they haven’t realised this yet.” Stefano Lanzavecchia

18. 2006-10-17Dyalog User Group Conference18 Components of… WildServer’06 HTTPListener HTTPRequest – Headers – Cookies – Request Variables HTTPResponse – Cookies HTML Utils Event pipeline – User Validation – Sessions Static file server Namespace as filesystem… File Handlers: – aplx files – apls files – …

19. 2006-10-17Dyalog User Group Conference19 Dyalog APL: an OO language Web server: a good test bed for the OO extensions – Abstractions: divide et impera – Inheritance: this looks a bit like that – Keyed properties: syntax tricks What is missing in the language? Is the IDE good enough?

20. 2006-10-17Dyalog User Group Conference20 HTTPListener fs← ⎕ NEW #.WildHeart.FileSystem('c:\aplserve') listener← ⎕ NEW web.HTTPListener(8080 fs) listener.Start Then point your web browser at http://localhost:8080/http://localhost:8080/ “ It ’ s a kind of magic ”

21. 2006-10-17Dyalog User Group Conference21 The trip of an HTTP request TCPSocket gets TCPRecv events HTTPRequest objects collects them, and parses the request according to the HTTP specs When the request is complete, it gets shipped to a dispatcher…

22. 2006-10-17Dyalog User Group Conference22 The dispatcher: FileSystem To me a FileSystem looks like a namespace: – Its folders are sub-namespaces – Idea: we can inject virtual folders: A virtual folder can be a folder on the HD coming from a different sub-folder A virtual folder can be… an APL namespace! Its functions would be files…

23. 2006-10-17Dyalog User Group Conference23 APLNamespace http://localhost:8080/loan/RUN executes #.loan.RUN but http://localhost:8080/loan/bullet.gif will transmit the file c:\aplserve\loan\bullet.gif fs← ⎕ NEW #.WildHeart.FileSystem('c:\aplserve') fs['loan']← ⎕ NEW web.APLNamespace(#.loan'c:\aplserve\loan') listener← ⎕ NEW web.HTTPListener(8080 fs) listener.Start

24. 2006-10-17Dyalog User Group Conference24 ZipFileSystem Now the folders and files come from a zip file. Neat, uh? In fact this presentation… fs← ⎕ NEW web.ZipFileSystem('c:\aplserve.zip?aplserve') fs['loan']← ⎕ NEW web.APLNamespace(#.loan'c:\aplserve\loan') listener← ⎕ NEW web.HTTPListener(8080 fs) listener.Start

25. 2006-10-17Dyalog User Group Conference25 FileSystem Basically one method: ProcessRequest ∇ ProcessRequest ctx;html;ns;raw;aplo :Access Public Instance Override ctx←#.WildHeart.HTTPContext.Current html←_NS ⍎ MapFileName ctx.URI r←ctx._Response r._IsBodyRaw←raw←82≠ ⎕ DR html r._Body←html :If ''≡ ⊃ r._Headers[ ⊂ 'Content-type'] r._Headers[ ⊂ 'Content-type']←u.MimeMappings['htm' 'bin'[1+raw]] :EndIf ctx.SendResponse ∇

26. 2006-10-17Dyalog User Group Conference26 Tricks: thread local storage […] ctx←#.WildHeart.HTTPContext.Current […] Thread context: wouldn’t it be nice to have an object local to a thread?

27. 2006-10-17Dyalog User Group Conference27 HTTPHandler A FileSystem delegates the handling of the requested resource to the appropriate handler: – StaticFileHandler by default – AplxHandler – AplsHandler – … more to come fs._FileHandlers[ ⊂ 'aplx']←#.WildHeart.AplxFileHandler

28. 2006-10-17Dyalog User Group Conference28 HTTPHandler One method: ProcessRequest ∇ ProcessRequest ctx;fname;tn;ext;etag :Access Public Instance Overridable fname←ctx._FileSystem.MapFileName ctx.URI tn←fname ⎕ NTIE 0 ctx._Response._IsBodyRaw←1 ctx._Response._Body← ⎕ NREAD tn,83, ⎕ NSIZE tn ⎕ NUNTIE tn ctx.SendResponse ∇

29. 2006-10-17Dyalog User Group Conference29 AplxHandler “This is only a test” Rain Graphics: Climate

30. 2006-10-17Dyalog User Group Conference30 A master page <html xmlns=“http://www.w3.org/1999/xhtml” xmlns:apl="http://wildheart.com/2006/10/ " > WildHeart.HTTPListener APL Web Server Introduction View Code …

31. 2006-10-17Dyalog User Group Conference31 The event pipeline As a request goes through the FileSystem hierarchy, events are raised: – OnEnter – OnAuthenticate – OnAuthorize – …

32. 2006-10-17Dyalog User Group Conference32 Tricks: the observer pattern :Class Event :field _Observers ∇ make name :Access Public Instance :Implements Constructor _Observers← ⍬ ⎕ DF'Event: ',name ∇ ∇ Raise(sender arg);obj;fnname :Access Public Instance :For obj fnname :In _Observers (obj ⍎ fnname)(sender arg) :EndFor ∇ ∇ Add(obj fnname) :Access Public Instance _Observers,← ⊂ obj fnname ∇ ∇ Del(obj fnname) :Access Public Instance _Observers← _Observers~ ⊂ obj fnname ∇ :EndClass

33. 2006-10-17Dyalog User Group Conference33 The observed shows what it’s got onAuthenticate← ⎕ NEW #.WildHeart.Event('Authenticate') auth← ⎕ NEW web.WebDemos.FormsAuth fs.onAuthenticate.Add auth'OnAuthenticate' onAuthenticate.Raise ⎕ THIS lurl

34. 2006-10-17Dyalog User Group Conference34 Two simple authentication modules SimpleAuth: based on HTTP challenge response: – Quick and dirty FormsAuth: based on cookies – More flexible – More work

35. 2006-10-17Dyalog User Group Conference35 Sessions HTTP is a stateless protocol People want to have conversations with peers who don’t forget instantly what they just said Have a cookie!

36. 2006-10-17Dyalog User Group Conference36 Sessions As easy as: fs.EnableSession ⎕ NULL fs.onSessionStart.Add #’DoSomething' fs.onSessionEnd.Add #’UndoSomething' In the application: :If ⎕ NULL≡ns←ctx._Session[ ⊂ 'footer'] ns← ⎕ NS'' ns.count←0 ctx._Session[ ⊂ 'footer']←ns :EndIf ns.count+←1

37. 2006-10-17Dyalog User Group Conference37 Extensibility Some of the features (ZipFileSystem, session module, authentication, aplx handler) were hacked together to prove that the server could support them Even before its completion, the WildServer has: – Had a file handler written (by Nic & Morten) – Been used to run a course in writing Web Applications using Dyalog APL

38. 2006-10-17Dyalog User Group Conference38 Open Source Open Source means: “free as in free beer” (except: is there such a thing as free beer?) No strings attached: – You get the source code and whatever documentations exists with it at the time – You get the right to do whatever you want with it (even sell it! But only a bad boy would do that…) – You get from me all the support I can give you when I can if I can (alas, I am very busy and maybe one day I’ll get a life)

39. 2006-10-17Dyalog User Group Conference39 Why Open Source? Have you looked at Ruby on Rails? Tons of contributors do a lot of work Tons of clever contributors do a lot of good work! OK, maybe that’s a good product.

40. 2006-10-17Dyalog User Group Conference40 TODO list It’s long, it’s scary, but… APL Prototypes seem to acquire a life of their own The “ducks” have started contributing

41. 2006-10-17Dyalog User Group Conference41 Thanks Peter Michael Haager: I have adapted his XML parser to produce trees of objects instead of nested arrays

42. 2006-10-17Dyalog User Group Conference42 Conclusion Initial feedback from users (Nic, Morten and Mondays course participants) is promising The WildServer is still very much ”work in progress”, and the author wants to keep control of it until about Christmas However, you are all welcome to try it now and suggest or code enhancements

43. 2006-10-17Dyalog User Group Conference43 Questions? If not… now I need some sleep. 輝く空と君の声を 感じたい

44. 2006-10-17Dyalog User Group Conference44 What is missing in the language No public operators! C# has them! No protected members! C# has them! No object serialization! C# has it! C# is a statically typed language. OK, substitute in the slide “C#” with “Ruby”