# Narval Evolution Proposal 001

## Object

Specialize Narval as a recipe interpretor, and let services manage internet communication protocols be managed by Services more specialized than the current Socket Manager.

## Motivation

Right now, handling of internet communication protocols is made by Narval Actions which communicate with the Socket Manager (SM). This works OK when the recipe acts as a Server (and event then...), but we face problems when the recipe is a client (e.g. Jabber).

Moreover, handling the protocols requires specific code development, and prevents efficient reuse of the very rich (protocol-wise) Python Standard Library (for instance, compare module Http and SimpleHttpServer from PSL).

### Caution

This document is not about communication with the Narval interpreter itself, but with the recipes.

## Proposition

### Overall architecture

Each protocol will be handled by a ProtocolHandler. Target protocols include Http, XMLRPC, SOAP, Jabber. Other protocols can be considered as needed. The handlers can be launched either in the same process as the interpreter or in a different process. In this case communication will use the standard internal communication protocol.

At startup, Narval knows which handlers are available. It queries each handler to know its prototype using the `getPrototype` method (see handler-proto). When such an element corresponding to an activator of the handler appears in Memory, Narval activates the handler with the element. When the element disappears, narval deactivates the handler for the element.

When some data is recieved by the handler, it will use the narval handler management interface (see narval-proto) to create new elements in memory, launch a plan, or launch a plan with some elements in its context.

When Narval sees an element matching one of the inputs of the handler, it calls the handleOutgoing method of the handler. This element is generated by one of the launched plans, and generally holds an identifier taken from one of the elements created by the handler before launching the plan.

### ProtocolHandler registration

Use a config file in ``NARVAL_HOME`/data/` ? Or do automatic exploration of modules in `narval/protocol_handlers/`? I think a configuration file is better, because it means you can easily add new handlers, even if you do not have write access to the `narval/protocol_handlers/` directory.

### ProtocolHandler interface

    String
    getPrototype

- returns an XML string, valid according to the DTD given in proto-dtd.

    void
    activateElement
    activator

- Activates the handler with the given element. A handler can be activated more than once, and should behave accordingly. For instance an HTTP handler can be activated on different ports. The handler should use the eid of the element to be able to deactivate the proper service. This method can raise a HandlerAlreadyActivated exception if the activate method already has been called with a semantically identical activator.

    void
    deactivateint
    eid

- Deactivates the handler. The eid of the activation element is passed as an argument. This method can raise a HandlerNotActivated exception if the handler had not been activated with the element.

    void
    handleOutgoingstring
    arguments

- passes arguments matching one of the inputs of the handler. This method can raise a HandlerNotActivated exception if the handler has not been activated, or has not been activated to handle the input, or is not expecting any input.

### NarvalHandlerManagement interface

    void
    instanciateHandlerRecipeWithContextstring
    group.recipestring
    arguments_node

- Adds the childnodes of `arguments_node` to the memory after having checked them against the outputs declared in the handler's prototype, instanciates a plan from the recipe and adds the new elements to the context of the plan.

    void
    instanciateHandlerRecipestring
    group.recipe

- Instanciates a plan from the recipe.

    void
    addElementFromHandlerToMemorystring
    arguments_node

- Adds the childnodes of `arguments_node` to the memory after having checked them against the outputs declared in the handler's prototype.

### Client vs. Server behaviour

When the handler behaves as a server, the communication is generally initiated by a remote client once the handler has been activated. In this case, the behaviour described previously applies fully. When the handler behave as a client (for instance in the Jabber protocol), communication can be initiated by a remote client or by a local recipe.

Things behave in exactly the same way, except that the call to `handleOutgoing` can occure without a request having been seen in the first place.

### Implementation consideration

When dealing with simultaneous calls, it is necessary for a server handler to be able to give the right answer to the right client. We suggest that the handler processes each client request in a separate execution thread. The problem is that very often, the answer the client's request is computed asynchronously by a Narval recipe. The solution is to have the main thread of the handler create a Lock object and acquire it before creating the request processing thread. The processing thread reads the request, creates new elements, launches the plan and acquire the Lock, which blocks the thread. When the main thread's callback function is called, it uses some data in the argument to the callback to release the lock corresponding to the waiting thread. The waiting thread can then read the answer and send it back to the client.

## Sample sessions

### Caution

This is a draft. The triggering elements used in these samples may be different in the final implementation of the final handlers for the mentioned protocols

### HTTP Handler

1. At startup, Narval calls the getPrototype() method of the HTTPHandler. The returned value is

       <prototype>
         <description lang='en'>An HTTP server delegating operation to Narval<description>
         <activator>
           <description lang='en'>port on which the server listens, recipe to instanciate<description>
           <match>http-server[@port][@recipe]</match>
         </activator>
         <input>
           <description lang='en'>response to an HTTP request<description>
           <match>http-response[@status]/data/text()</match>
         </input>
         <output>
           <description lang='en'>HTTP request<description>
           <match>http-request[@type]</match>
           <match>http-request[@http-version]</match>
           <match>http-request/url/text()</match>
           <match>http-request/data</match>
           <match>http-request/headers/header/text()</match>
         </output>
       </prototype>

2. A new element <http-server port='7776' recipe='PDA.proxy'> appears in memory. It is assigned eid 42. The activate method of the handler is called, passing the new element as an argument.

3. The handler creates a new HttpServer listening on port 7776.

4. The server recieves a request. It calls `instanciateHandlerRecipeWithContext` to create an `http-request` element in Narval's memory with the required information and instanciate the `PDA.proxy` recipe with the element in it's context.

5. Narval launches the plan. At some point, the `http-response` element is created, and the handler `handleOutgoing` method is called.

6. The handler sends back the data to the client.

## Prototype DTD

    <!-- Future changes: -->
    <!-- Namespaces will be used -->

    <!-- The prototype element is the root element -->
    <!ELEMENT prototype (description+,(activator|input|output)*)>

    <!-- activator nodes describe elements that can activate the handler -->
    <!ELEMENT activator (description+,match+)>

    <!-- input nodes describe elements that can be passed to the handler -->
    <!ELEMENT input     (description+,match+)>

    <!-- output nodes describe elements that can be produces by the handler -->
    <!ELEMENT output    (description+,match+)>

    <!-- A match element contains an XPath describing part of the element -->
    <!-- different matches are and'ed to get the full description         -->
    <!ELEMENT match     (#PCDATA)>

    <!-- description nodes are used for documentation purpose -->
    <!-- You should at least provide a description in english -->
    <!ELEMENT description (#PCDATA)>
    <!ATTLIST description lang CDATA #REQUIRED>