LIBPKG(3)

libpkg - Message passing abstraction library supporting communication between cooperating processes over TCP connections

The following is a description of the libpkg routines kindly provided by Peter B. Shames on 26 Sept 1986, and updated 14 Nov 1986 by PC Dykstra to reflect BRL extensions since the first writing. It was intended that an RFC be published at some point, but at this point this document (plus the source code) is all we have. In the CAD distribution, the remote framebuffer interface (libfb/if_remote.c) and server (fbserv), communicate using this protocol.

A prototype interface that implements a fairly simple message passing abstraction between cooperating processes running over TCP connections has been developed by Mike Muuss, Charles Kennedy, and Phillip Dykstra at BRL. Some experiments have been run at STScI to implement this interface on a DECnet link with good success. A few changes have been suggested to improve network throughput for data flows that consist of short message traffic or byte-stream traffic.

This note describes the motivation for developing such an interface and describes the subroutine interfaces and functionality. The interface routine descriptions and portions of the textual descriptions have been taken directly from the source code provided by Muuss and Kennedy which has been the only available description to date.

The interface - PKG provides a simple interface for managing client/server connections in a distributed environment. A server task is initialized to wait for incoming requests from clients. Clients open a connection to the server task which executes on some host processor waiting for incoming requests. Messages with identifying type codes are passed between processes, with optional appended data blocks. The interface is non-blocking and accepts arbitrary data messages of arbitrary length.

An example - interaction with a remote data archive may require request for services, user authentication, permission granting, index hierarchy queries - with manipulation of results, archive request processing, delivery of output products - electronically or otherwise. The archive is a server that responds to service requests from clients, how many may be served at one time and whether public access and private services are supported is an implementation issue. Since no direct interactive support is required (if smart client agents are in use) the message paradigm can support the requirements.

A [counter] example - Athena’s XWindow uses a client-server model to provide screen/window management services for distributed processes. XWindow is very permissive of different window management policies and provides hooks for implementing various user interfaces. At the lowest level, XWindow requires a reliable duplex byte-stream, which can be implemented in many operating environments. Because XWindow wants to manage all process-window communications directly, it builds its own block stream protocol to manage the connection. This permits the interactive update required by some processes, but basically is the same paradigm as PKG, defined within the window system.

The PKG interface implements an client-server network connection that multiplexes synchronous and asynchronous messages across stream connections. Connection control is provided by open/ close routines, where the client authentication is handled prior to establishing the connection. Server side actions include initialization, client connection, and client request servicing. Client side actions include server request, reply processing, possible connection negotiation, and data transfer processing.

A connection is established when a client issues an open to a server, running on some host. Which hosts provide required services must be determined prior to connection requests, using some service directory. Default connection processing is stream oriented, where a process may get data as it is available on the connection. A blocking read may also be issued, waiting for any complete message, or just for messages of a specified type. The connection remains open until explicitly closed by either client or server.

Once the connection is established it may be used to process a full- duplex asynchronous byte stream, or a buffered, synchronous query/reply exchange. Message type multiplexing supports user actions based upon message type, where the multiplexing is handled in the interface. Different message type streams may operate using different flow control over the same connection. Multiplexing is handled in the interface by pre-pending a header block to the message before transmitting it to the reader. Data is transmitted as received, but the header is transmitted in Internet network byte/bit order.

An package connection may incur significant overhead if many short messages are passed between processes, so a buffered stream connection may be requested. The application calls the pkg_stream interface rather than the usual pkg_send request, and PKG builds a message buffer and passes it to the client as it gets filled. A stream buffer may be flushed with an explicit call to the pkg_flush routine.

At least two implementations of the PKG interface have been done, one based on TCP/IP connections, the other based on DECnet connections. Future releases of PKG will be layered on top of a virtual stream library known as NET.

The implementation running on 4.2BSD UNIX takes advantage of the "writev" syscall which can output the header and data in a single syscall/TCP_PUSH cycle. The advantage of the short data copy is realized only when writev can not be used, or when a pkg_flush is not required (a la PKG_STREAM).]

Further development of the lower levels of the interface will be provided so that this message passing abstraction can be provided on top of both DECnet and TCP/IP network layers, with other implementations provided as required. The program interface layer is to be as described in the body of this note, the lower layers will interface to the network layers using whatever subroutine interfaces are most appropriate for the system hosting the interface. A simple read/write/open/close abstraction may be provided at a lower layer to further isolate the network layers from the message handling layer.

Where it is necessary to cross network protocol boundaries, message relay processes will be created to reside on a Janus host that straddles the two domains. Domain routing and relay point information will be maintained above the network layer within the extended PKG interface. Clients and servers will need to determine server host id information from some available database. Server names must either carry domain information or such information must be available as part of the server database. Within the extended PKG interface routing information for domain boundary transits must be maintained, and appropriate Janus host systems connected to provide relay services.

It is not anticipated that full asynchronous I/O implementation will work particularly well across domain boundaries, but all other forms of message transfer should work quite well. As already noted, buffered stream I/O may be perfectly satisfactory for many applications where full interaction is not required. Where applicable, this mode of interface is to be preferred over full asynchronous I/O. Further development of this interface layer on other protocol families is possible, perhaps even on low level networks like BITNET and certainly on connection oriented X.25 or OSI networks.

Peter B. Shames

This software is Copyright (c) 1989-2021 by the United States Government as represented by U.S. Army Research Laboratory.

Reports of bugs or problems should be submitted via electronic mail to devs@brlcad.org