NHttpLib

NHttpLib is a NewtonScript unit for making HTTP connections. It can be used on any NIE 2.0 capable Newton device.

• Version: 3.5
• Author: Eckhart Köppen
• License: LGPL 2.0
• Download via SourceForge

Usage

Overview

The package NHttpLib.pkg contains the unit %NHttpLib:40Hz% that exports one proto named protoNHttpLib. This is the declaration which needs to be in for project using NHttpLib (the major version number in the DeclareUnit statement must match the major version number of the installed NHttpLib version):

    DeclareUnit ('%NHttpLib:40Hz%, 3, 1,
        {
            protoNHttpLib: 0,
            protoHNttpLibTransport: 1
        }
    );

Code using this proto looks like this:

    httpLib := {
        _proto: UR ('%NHttpLib:40Hz%, 'protoNHttpLib);
    }

    httpLib:MGet ("www.apple.com", 80, "/index.html");

Note: When fetching an URL, the opening of the Connection Slip before making a connection can be suppressed by setting fOpenConnectionSlip to nil.

Handle Incoming Data

This code however would not do anything with the incoming data. It is also important to know that NHttpLib ‘‘does not’’ store the incoming data! To handle the different possible content types, the array fBodyInputSpecs in the frame above needs to be set. An element of this array has the form

    {
        contentTypes: ...
        spec: ...
    }

where the contentTypes slot is an array of strings representing the handled content type like “text/html” or “text/xml” as returned by the HTTP server. It is also possible to specify “*” as a wildcard for any content type. The spec frame is an inputSpec that describes how to receive the data. A simple example that just prints out incoming HTML or XML data looks like this:

    httpLib.fBodyInputSpecs := [
        {
            contentTypes: ["text/html", "text/xml"],
            spec: {
                async: true,
                reqTimeout: 30000,
                form: 'string,
                termination: {
                    endSequence: ">"
                },
                inputScript: func (ep, data, terminator, options) begin
                    print (data);
                end,
            }
         }];

Basically, an application should just have to implement the form, termination and inputScript slots of the inputSpec.

Connection Termination

To handle the successful termination of a transmission, it is also possible to specify the MConnectionClosed function which gets called after the HTTP connection has been closed:

    httpLib.MConnectionClosed := func () print ("Finished");

HTTP Headers

Before the body of the HTTP request is processed, the HTTP header fields are already available in the FSM frame. They can be accessed through the fHeaders array. It contains elements with two slots, namely name for the name of the header field and value for its value. The content type and length can also be accessed through the slots fContentType and fContentLength. The HTTP return code is available in the slot fHTTPResult. To perform additional actions before the body of the HTTP request is recevied, the MHeadersComplete function can be specified:

    httpLib.MHeadersComplete := func () begin
        print ("Got all headers, body follows");
        return true;
    end;

The return value of the MHeadersCompleteFunction determines if the download shall continue. To abort transmission after receiving the headers, nil has to be returned.

Proxy Support

The NHttpLib unit installs a new settings editor in the ‘‘Internet Setup’’ application. It can be used to add an HTTP proxy to an internet setup. The format of the proxy setting string can be any valid url, e.g. “192.168.1.1:8000”.

Cookies

Cookies are stored in the soup “NHttpLib Cookies” if fStoreCookies is set to true in the base frame. Cookies are sent if fSendCookies is set to true. By default, cookie saving and sending is enabled.

The Fetch URL Transport

NHttpLib adds a simple transport to the Inbox that allows downloading of URLs into the Inbox. After installing NHttpLib, a new menu item “Fetch URL” is added to the “Receive” picker in the Inbox. NHttpLib saves the downloaded data in the Inbox soup with this format:

    item := {
        ...
        body: {
            data: <received data in a VBO, class 'binary>,
            class: Intern (<content type of the data>)
        }
        ...
    }

This feature can be used for example by IC/VC or MAD Max to download MP3 files and import them into MAD Max or to fetch iCalendar data and import it later via IC/VC.

Implementation

Version 2.0 is a complete rewrite of the library. It is a simpler design than the previous versions which were based on protoFSM. This version explicitly hard-codes the steps needed to grab a link, bind and connect an endpoint, transfer data and close the connection. It is implemented as a proto and should be assigned to a _proto slot in a frame.

Credits

Thanks to Ken Hagler, Grant Hutchinson, Patrick Clark, Frank Gruendel and Marcus Schneider for testing!

Changes

• Version 3.5: Added support for redirect handling
• Version 3.4.2: Added some status code feedback, e.g. for redirects
• Version 3.4.1: Fixed bug in splitting URLs
• Version 3.4: Bug fixes for IC/VC
• Version 3.3: Added cookie support and an interface for fetching multiple URLs.
• Version 3.2: Added support for content encodings.
• Version 3.1: Export the Fetch URL transport via the Unit Interface.
• Version 3.0: Added Fetch URL transport, changed header InputScript to prevent lost data when switching to binary body InputScript.
• Version 2.3: Added proxy support.
• Version 2.2: Some changes to the header processing, added MHeadersComplete callback, added test program.
• Version 2.1: Added fEndpointEncoding slot and made first modifications for more tolerant header processing.
• Version 2.0: Complete rewrite, API not compatible with previous 1.x versions.

Bugs and To Do’s

Cancelling operations via the Stop button in the status view seems cause instabilties at some times (timeouts are working better). Closing a connection is slow.

Setting the fEndpointEncoding to a value different from nil causes problems (the kPCRomanEncoding is not implemented on US MessagePads).