HTML/JS: SimpleHttpServer

[PGilbert]

Here is the code for a Simple HTTP Server written in APL and using .Net, it will simply get the request and reply to them.

Code: Select all

:Namespace  SimpleHttpServer
   ⍝ Simple Http Server to listen and respond to AJAX request 'GET' and 'POST' on a local machine.
   ⍝ It is using the .Net HttpListener class (https://docs.microsoft.com/en-us/dotnet/api/system.net.httplistener).
   ⍝ Will respond at one request at a time. Does not support websocket.

   ⍝ USAGE:
   ⍝ To start the server:      SimpleHttpServer.Start
   ⍝ To stop the server:       SimpleHttpServer.Stop

   ⍝ Typical AJAX request for testing:
   ⍝ req = new XMLHttpRequest()
   ⍝ req.onreadystatechange = function() {if (this.readyState == 4 && this.status == 200) {alert(this.responseText);}}
   ⍝ req.onerror = function() {if (this.readyState == 4 && this.status == 200) {alert(this.responseText);}}

   ⍝ req.open('GET','http://localhost:8080/dir1/mypage.html?key1=value1&key2=value2&key3=value3')
   ⍝ req.send()

   ⍝ req.open('POST','http://localhost:8080/dir1/mypage.html?key1=value1&key2=value2&key3=value3')
   ⍝ req.send("data for POST request goes here")


   ⍝ System variables
    (⎕IO ⎕ML ⎕WX)←1 3 3

   ⍝ .Net namespace used
    ⎕USING←'System.Net,system.dll' 'System.IO,mscorlib.dll' 'System,mscorlib.dll'

   ⍝ Global variable(s):
   ⍝ _listener

    ∇ r←Start
      :Access Public
      ⍝ To Start the server.
      :Trap 0
     
          :If (~HttpListener.IsSupported)
              r←0 'HttpListener is not supported. Windows XP SP2, Server 2003, or higher is required to use the HttpListener class.'
     
          :Else
              _listener←⎕NULL
              _listener←⎕NEW HttpListener
              _listener.AuthenticationSchemes←_listener.AuthenticationSchemes.Anonymous
     
            ⍝ Following prefix works best on non admin machine.
            ⍝ Otherwise you get Access Denied error when starting to listen.
            ⍝ Port 80/443 are reserved already for standard HTTP/HTTPS, better not used them.
              _listener.Prefixes.Add(⊂'http://localhost:8080/')
     
              _listener.Start
     
              ⍝ Wait on another thread so it does not block current thread.
              {}Listener&⍬
     
              r←1
          :End
      :Else
          r←0 ⎕EXCEPTION   ⍝ was .Message
     
      :End
    ∇


    ∇ r←Stop
      :Access Public
      ⍝ To Stop the server.
      :Trap 0
          _listener.Abort
          _listener←⎕NULL
          r←1
      :Else
          r←0
      :End
    ∇

    ∇ r←IsListening
      :Access Public
      :Trap 0
          r←_listener.IsListening
      :Else
          r←0
      :EndTrap
    ∇


    ∇ Listener arg;data_json;keyRequest
      ⎕←'Listening on ',∊⌷_listener.Prefixes
      ⎕←'Current Thread: ',⍕⎕TID
      2503⌶1 ⍝ Mark Thread as Uninterruptible. If there is an error on another thread this one will not stop.
     BEGIN:
      :While _listener.IsListening
          :Trap 0
              context←_listener.GetContext  ⍝ The GetContext method blocks while waiting for a request.
          :Else
              :If _listener≡⎕NULL
                  →0
              :ElseIf ~_listener.IsListening
                ⍝ There is no error. It was terminated by the .Stop method
                  _listener←⎕NULL
                  →0
              :Else
                ⍝ There is an error while listening.
                  SendInternalErrorResponse(⎕EXCEPTION.Message)
                  ⎕←⎕EXCEPTION.Message
                  {}Stop
                  {}Start
                  →BEGIN
              :EndIf
          :EndTrap
     
        ⍝ Get client Request and Response from the context.
          (clientRequest clientResponse)←context.(Request Response)
     
        ⍝ Common to GET and POST request.
          (length encoding inputStream)←clientRequest.(ContentLength64 ContentEncoding InputStream)
          requestText←clientRequest.Url.UnescapeDataString(⊂clientRequest.RawUrl) ⍝ URL is removed
     
        ⍝ Usefull code to be used for the business logic.
        ⍝ clientRequest.Headers.Count                  ⍝ quantity of key/value pair in header
        ⍝ clientRequest.Headers.AllKeys                ⍝ List of all keys in the header
        ⍝ clientRequest.Headers.get_Item(⊂'keyName')   ⍝ Will return ⎕NULL if it does not exist
        ⍝ (⊂'keyName') ∊ clientRequest.Headers.AllKeys ⍝ To test if a key exist
     
        ⍝ clientRequest.QueryString.Count                  ⍝ quantity of key/value pair in query string
        ⍝ clientRequest.QueryString.AllKeys                ⍝ List of all keys in the query string
        ⍝ clientRequest.QueryString.get_Item(⊂'keyName')   ⍝ Will return ⎕NULL if it does not exist
        ⍝ (⊂'keyName') ∊ clientRequest.QueryString.AllKeys ⍝ To test if a key exist
     
        ⍝ ⎕←'request Text: ',requestText
        ⍝ ⎕←'request path: ',clientRequest.Url.AbsolutePath
        ⍝ ⎕←'request Data: ',data
        ⍝ ⎕←'request Keys: ',clientRequest.QueryString.AllKeys
        ⍝ ⎕←'header Keys:  ',clientRequest.Headers.AllKeys
     
          :Select clientRequest.HttpMethod
          :Case 'POST'
            ⍝ *** POST *** request was received.
     
            ⍝ This is the data sent with the POST request
              data_json←(⎕NEW StreamReader(inputStream,encoding)).ReadToEnd
              ⎕←'POST: ',data_json
     
            ⍝ Business logic for a POST request goes here.
     
⍝              :If (⊂'api_key')∊clientRequest.QueryString.AllKeys
⍝              :AndIf '20FE4ADA4F873972E1F18F5457DF6A95'≡clientRequest.QueryString.get_Item(⊂'api_key')
     
              :If 1
     
                  :Select ↑r←#.Server.POSTRequestHandler data_json
                  :Case 200
                    ⍝ OK
                      SendTextResponse''
     
                  :Case 400
                    ⍝ The request is ill formed
                      SendBadRequestResponse data
     
                  :Case 500
                    ⍝ Bug during the APL execution
                      SendInternalErrorResponse 2⊃r
     
                  :Else
                      ∘
                  :EndSelect
     
              :Else
                  SendUnauthorizedResponse'asshole'
     
              :EndIf
     
     
          :Case 'GET'
            ⍝ *** GET *** request was received.
              ⎕←'GET: ',requestText
     
⍝              :If (⊂'api_key')∊clientRequest.QueryString.AllKeys
⍝              :AndIf '20FE4ADA4F873972E1F18F5457DF6A95'≡clientRequest.QueryString.get_Item(⊂'api_key')
⍝              :AndIf (⊂'request')∊clientRequest.QueryString.AllKeys
              :If (⊂'request')∊clientRequest.QueryString.AllKeys
     
                  keyRequest←clientRequest.QueryString.get_Item(⊂'request')
     
                  :Select ↑r←#.Server.GETRequestHandler keyRequest
                  :Case 200
                    ⍝ OK
                      SendTextResponse 2⊃r
     
                  :Case 400
                    ⍝ The request is ill formed
                      SendBadRequestResponse''
     
                  :Case 500
                    ⍝ Bug during the APL execution
                      SendInternalErrorResponse 2⊃r
     
                  :Else
                      ∘
                  :EndSelect
     
              :Else
                  SendUnauthorizedResponse'asshole'
     
              :EndIf
     
          :Else
     
              ⎕←'Don''t know what to do with: ',clientRequest.HttpMethod
              SendBadRequestResponse clientRequest.HttpMethod
     
          :EndSelect
     
      :EndWhile
     
     ⍝ Should not be here. Restarting the server.
      {}Stop
      {}Start
    ∇


    ∇ SendTextResponse text
    ⍝ Status OK 200
    ⍝ Send a text response to the client.
     
    ⍝ To allow requesting code from any origin to access the resource.
    ⍝ We are "opting out" of the "same origin policy".
      clientResponse.AppendHeader('Access-Control-Allow-Origin'(,'*'))
     
    ⍝ Prepare the response.
      clientResponse.(StatusCode StatusDescription)←HttpStatusCode.OK'OK' ⍝ req.status and req.statusText in Javascript
      clientResponse.ContentEncoding←Text.Encoding.UTF8
      clientResponse.ContentType←'text/html'    ⍝ req.getResponseHeader("content-type") is working but req.responseType is empty
     
    ⍝ Write response to the OutputStream.
      text←Text.Encoding.UTF8.GetBytes(⊂text)
      clientResponse.ContentLength64←Convert.ToInt64(⍴text)
      clientResponse.OutputStream.Write(text 0(≢text)) ⍝ was (⍬⍴⍴text)
     
    ⍝ Closing the response will send it to the client.
      clientResponse.Close ⍬
    ∇


    ∇ SendInternalErrorResponse text
     ⍝ Error 500
      clientResponse.AppendHeader('Access-Control-Allow-Origin'(,'*'))
      clientResponse.(StatusCode StatusDescription)←HttpStatusCode.InternalServerError'Internal Server Error'
      clientResponse.ContentEncoding←Text.Encoding.UTF8
      clientResponse.ContentType←'text/html'
     
    ⍝ Write response to the OutputStream.
      text←Text.Encoding.UTF8.GetBytes(⊂text)
      clientResponse.ContentLength64←Convert.ToInt64(⍴text)
      clientResponse.OutputStream.Write(text 0(≢text)) ⍝ was (⍬⍴⍴text)
             
    ⍝ Closing the response will send it to the client.
      clientResponse.Close ⍬
    ∇

    ∇ SendBadRequestResponse text
     ⍝ Error 400
      clientResponse.AppendHeader('Access-Control-Allow-Origin'(,'*'))
      clientResponse.(StatusCode StatusDescription)←HttpStatusCode.BadRequest'BadRequest'
      clientResponse.ContentEncoding←Text.Encoding.UTF8
      clientResponse.ContentType←'text/html'
     
    ⍝ Write response to the OutputStream.
      text←Text.Encoding.UTF8.GetBytes(⊂text)
      clientResponse.ContentLength64←Convert.ToInt64(⍴text)
      clientResponse.OutputStream.Write(text 0(≢text)) ⍝ was (⍬⍴⍴text)
     
    ⍝ Closing the response will send it to the client.
      clientResponse.Close ⍬
    ∇

    ∇ SendUnauthorizedResponse text
     ⍝ Error 401
      clientResponse.AppendHeader('Access-Control-Allow-Origin'(,'*'))
      clientResponse.(StatusCode StatusDescription)←HttpStatusCode.Unauthorized'Unauthorized'
      clientResponse.ContentEncoding←Text.Encoding.UTF8
      clientResponse.ContentType←'text/html'
     
    ⍝ Write response to the OutputStream.
      text←Text.Encoding.UTF8.GetBytes(⊂text)
      clientResponse.ContentLength64←Convert.ToInt64(⍴text)
      clientResponse.OutputStream.Write(text 0(≢text)) ⍝ was (⍬⍴⍴text)
     
    ⍝ Closing the response will send it to the client.
      clientResponse.Close ⍬
    ∇

:EndNamespace

Comments and suggestions for improvements are welcome.

[paulmansour]

Thanks for posting all this code.

Where does HTMLRenderer fit in to what you are doing?

I'm trying to figure out an architecture/strategy for using HTMLRenderer to enhance an existing desktop GUI (build with ⎕WC), or even completely replace it. There are a number of approaches I have thought of, and probably more I have not. In this scenario there is no need for a server listening on a port, APL is simply intercepting everything from HTMLRenderer. There is also no need for using a variety of HTTP methods, or using HTTP in any sort of appropriate (RESTful) way. And if one throws in using the new HMLRenderer.ExecuteJavaScript method, one has precluded reusing the architecture/code over the internet. In this scenario, HTMLRenderer is really just a GUI object, albeit one massively gigantic powerful one. In this scenario there are many approaches - for example, one or multiple instances of HTMLRenderer? Javascript code in the webpage run on the response to a fetch, or Javascript sent over by APL using ExecuteJavaScript, or both. Modal windows using html/css/javascript (the <dialog> element) or have APL create a new windows - or both?

Alternatively one could write everything so that it could be run over the web, using HTTP appropriately, avoiding ExecuteJavaScript, no multiple instances of HTMLRenderer, or additional Dyalog forms, etc. Of course if the purpose is to have a single page web app, then you can't assume the browser is chromium based so you would have compatability issues, or have to require a chromium based client. But doing a desktop app this is a lot of extra work, and kind of doing things with one hand tied behind your back.

My gut feeling is that trying to do both at once is not a good idea. Better to do the desktop app fully exploiting the tight integration APL with HTMLRenderer, but code in such a way that large chunks of code are "internet ready", so that if you want to move some functionality to the internet, it is easy to do.

I hope I end up going down the right path.

[PGilbert]

Hello Paul, here is our architecture and the logic behind it for our desktop application that is in current development:

1. Multipage application: We are developing a desktop application that has 10 different pages and we wanted a quick response from the GUI when changing from one page to another. What we have tried is to load the 10 pages in 10 <DIV> that are hidden on the same page and then show only one <DIV> at a time. The result is that each page appears super fast when we hide a <div> and show another one. The alternative would have been to start 10 HTMLRenderer and show only one at a time which would have been using a lot of memory. Each page is a separate file and all the files are assembled by the Main Page at start-up.

2. Popup window: We have decided not to do them in HTML/JS, we do them in APL and show them above Chromium. The popup window looks nicer when done with APL to our taste.

3. Menus: We did the menu in HTML/JS. It is not easy to do those menus in HTML/JS, but we did it anyway because it was fitting our layout on the main page better.

4. JQuery: We are using JQuery but we are trying to not be lazy and we are using JS when possible. This site is good to guide us: http://youmightnotneedjquery.com/

5. Communication with APL: We are using getJSON, setJSON, and SimpleHttpServer, an alternative could be to have no SimpleHttpServer and have HTMLRenderer intercept the requests and answer back to HTMLRender by using HMLRenderer.ExecuteJavaScript. We decided to go with a server because we needed multiple JSON files to refresh a page (on the click of a button) and that way we can use JS promises to synchronize APL and HTMLRenderer and handle the errors. Here is an example of what we do:

Code: Select all

            Promise.all([
                getJSON({Get: "_SchDiskInfo", KI: _KI}).then((objectFromServer) => _SchDiskInfo = objectFromServer),
                getJSON({Get: "_IO", KI: _KI}).then((objectFromServer) => _IO = objectFromServer)])
                .catch((error) => { // Callback when at least one of the promise has fail.
                    let errorObj = JSON.parse(error.message);
                    errorObj.sender = "F2PlusButtonClickEvent(Ready Mode)";
                    MessageBox(1000, errorObj);
                })
                .then(() => F2PlusButtonClickEventSub()) // Success of all the calls.
                .catch((error) => { // catch errors of previous .then()
                    console.error(error.stack)
                })
                .finally(() => $bnF2BtnPlus.prop("disabled", false)); // Re-enable the Plus button after success or failure

If one wants to answer a request using HTMLRender.ExecuteJavaScript it has to be done with only one call. To my knowledge, you cannot send multiple ExecuteJavascript from APL and hope they will be executed in the same order by Chromium that is multitasking them. Also in our case, the JSON files are large and we did not want to exceed, one day, the maximum character limit of ExecuteJavaScript.

I am not a professional programmer but what we do seems to be the safest way in the long run but it could be done differently if you know what you are doing for synchronizing APL and Chromium.

6.Tools: We are using Brackets to develop the single-page HTML. We like it because while you type your HTML you can see the changes immediately in Chrome. That's a unique feature not found elsewhere. For the JS and to maintain the whole project we are using WebStorm that was recommended to us by a professional programmer.

7. ZoomLevel: In our case, the customers will be using different resolution monitors or even small 4K TV to show the application with Chromium. We need the whole screen to increase in size, not just put more spaces between the elements. So we trap when the Main Window is resized and calculate the ZoomLevel and adjust it in Chromium accordingly and the result is fantastic. Since we are using only svg images, everything is scaled perfectly as vector images. At the same time, we keep the aspect ratio constant by resizing the window.

8. Chromium Engine: We are using DotNetBrowser so far because it is mature and with a lot of methods to choose from to experiment when you don't know exactly what you need. We don't mind switching to HTMLRenderer but that window is closing.

9. Printing: We need to print silently some of our pages with some control over the printer. We are waiting after DotNetBrowser that is coming with that option.

10. Number Input: We have written our own number validation routine because we did not found at the time one that was doing everything we wanted.

In conclusion, we have decided to rewrite our whole GUI in HTML/JS from scratch instead of trying to port the existing APL GUI with some tools. So far we are very satisfied with our choices but a good discussion (and some experimentation) on the architecture of such an endeavor is needed at the moment.

Thanks Paul for forcing this discussion that is helpful to the APL community.

[paulmansour]

Pierre, very informative, thanks.

Regarding #5, Communication with APL:

We are using getJSON, setJSON, and SimpleHttpServer, an alternative could be to have no SimpleHttpServer and have HTMLRenderer intercept the requests and answer back to HTMLRender by using HMLRenderer.ExecuteJavaScript.

Why is the alternative not simply intercepting and sending back a response to the request? No different conceptually than using a server - just request/response in either case?

I agree that if sending a lot of data back, you probably want to put it in the body of a response and not in a call to ExecuteJavaScript.

Interesting that if you call ExecuteJavaScript multiple times, each call may not execute in order. Of course I assume anything you can do in multiple calls can be done in one call. Right now I'm playing around with just having ExecuteJavaScript do little things like resetting the class of an element. I'll do this in the middle of a callback to the OnHTTPRequest event (say a user clicking the element), but the nice thing of having it in a sub function covering ExecuteJavaScript is that APL can initiate that little change on its own, not waiting for the HTMLRenderer to send a GET or POST.

Right now I'm not sure I'm going to miss not having direct access to the DOM in APL, or even that ExecuteJavaScript does not pass back results. It appears to me, so far in my limited experience, that fetch on the browser side combined with ExecuteJaveScript on the APL side, may go very long way.