editing over a packet network

[Ed Arnold]

This item is an updated reposting of an item I posted about three months
ago, reflecting a few additional responses to my inquiry about what
usenet readers know about the problems of editing under UN*X over a
packet network.  If there is any further interest, please mail me and
I will prepare another update.

Thanks to all who have communicated on this subject:

Rich Altmaier	hplabs!oliveb!ios!richa
Sean Casey	ames!vortex!cbosgd!ukma!sean
Gilles Chartrand	ihnp4!alberta!myriasb!ggc
Brad Davis	{ihnp4,decvax,seismo}!utah-cs!b-davis
R. Drew Davis	seismo!harvard!bentley!drew
Chuck Hedrick	seismo!topaz!hedrick
Eric Kolotyluk	seismo!ubc-vision!ubcg-mts!kolotyluk	604-228-4215
Geoff Kuenning	{hplabs,ihnp4}!trwrb!desint!geoff
John Levine	ihnp4!ima!johnl
Bill Nowicki	sun!rose!nowicki
David Parter	seismo!uwvax!david
Jim Shankland	ucbvax!mtxinu!rtech!jas
Dave Sherman	ihnp4!utzoo!lsuc!dave
Maurice Suhre	ucbvax!trwrb!suhre
Chris Thomson	ihnp4!alberta!myriasb!cmt
Scott Weikart	scott at Glacier.ARPA
Jim Wilson						201-631-7219
???		ihnp4!ihtnt!jhh
???		seismo!harvard!packard!hoxna!mdm

------------------------------------------------------------------------
WHAT I HAD IN MIND
------------------------------------------------------------------------
First, an explanation of the specific situations I had in mind is probably
in order.  I didn't make it clear as to whether I was thinking of one type
of network, or was stuck with a network, or what.  The primary networks
which have to do with external access to this organization (NCAR) are
(1) Uninet, a typical packet switched network which anyone can buy time on,
and (2) remote ethernets connected to a local ethernet via TransLAN boxes
over satellite links.

Uninet is probably typical of the major public packet nets.  It has
three basic algorithms allowed in the PAD (packet assembler/disassembler):
(1) one character per packet, (2) all characters up to some terminator
(e.g. newline) per packet, or (3) a time-slice arrangement in which all
characters received within some amount of time (in units of 1/20 second,
up to 255 units) are put into a packet.  The problem with (1) is that it
is expensive.  Number (3) is a better match for UN*X editors, which
typically issue a write() call on a tty line in raw or cbreak mode, but
large writes (e.g. a total screen refresh) can end up being broken up into
more packets than is really necessary.  As pointed out by Maurice Suhre,
Tymnet implements a slight variation on these tricks: it inserts a delay
of 2.5 character times before sending out a character, to see if something
else is on the way.

Regarding ethernets, this issue really concerns anyone who has an ethernet
of any size (although it's of somewhat greater interest in a case such as
the one here, where a demo system connecting remote ethernets, called USAN
[University Satellite Network] is being put together in concert with a
number of universities).  It's obviously in the interest of your ethernet
that you don't load it with oodles of 1-character packets, although that
can be what's going on if you're using certain terminal-server products
to allow you to connect your terminals to multiple hosts.  I say "can"
because, as pointed out by Charles Hedrick, one of the best-known products
in this field does allow you to reduce the number of packets via delay,
at least on the CPU end, if you set it up that way:

 >The Bridge CS100 waits a short period to see if there are any more
 >characters.  This is only a useful option if you are using a Bridge
 >port on the CPU end.  We normally use a Bridge port for the user,
 >but the CS100 talks directly to the CPU.  We would use a Bridge on
 >both ends only if the CPU did not speak the same protocol a the
 >CS100 (TCP/IP Telnet).  In our normal configuration, the Bridge is
 >only responsible for packetizing characters that the user types.
 >The responses from the CPU are packetized by the CPU's TCP
 >implementation.  All of the implementations that I know of are
 >designed so that screen refreshes are put into large packets.

 >For machines that do not support TCP/IP, then we would use another
 >set of CS100 ports for the machine end.  At that point, the 
 >transmission delay would become significant.  The Bridge software
 >can be set to wait for a specified amount of time before sending
 >a packet, in hopes that another character will show up.  The
 >amount of delay can be set.

Note that he said "This is only a useful option if you are using a
Bridge port on the CPU end".  Presumably, the CS100 connected to the
user's terminal can wait for chars to be typed by the user, bundling
them up into groups of 2 or 3, but the need to echo each char back
from the CPU would turn this into a recreation of the old mechanical
tty.  Puke.

------------------------------------------------------------------------
WHY DO THIS AT ALL?
------------------------------------------------------------------------
I received several replies that questioned why anyone would want to do
something like this, in the first place.  There are several good reasons:

1) People are already doing this, for instance, Eric Kolotyluk.  Sites
   running IBM software with SNA are, in general, in a better position to
   support this sort of thing than UN*X, since SNA is a layered protocol
   that passes message packets (rather than sequences of characters) back
   and forth.
2) Line editing is only a substitute for screen editing if you're
   prepared to take the hit in productivity that implies.  I personally
   don't know anyone who's using "emacs", "vi", or "e", who would care to
   revert to "ed".  However, one user who replied says that he uses
   "qed" (a superset of "ed") for all his editing at 1200 (on a packet
   net) or at 19200, and that's a satisfactory solution for him.
3) When running over a relatively slow (1200 or 2400 bps link), it isn't
   always practical to transfer a complete file to a remote site for
   editing.  For instance, we have scientists who log in here to prepare
   jobs for submission to our Cray-1s, and if any of you know how many
   scientists program (no offense to the scientific community intended -
   after all, they're REAL scientists, not quiche-eating computer
   scientists :-) ), you'd know that they often generate some very
   large, non-modular pogroms, er, programs.
4) (mild flame on) Those of us who deal with UN*X frequently tend to
   forget that there are a few things it doesn't do well, and speaking
   in packets is one of them.  We have to remember that there are lots
   of IBM system managers out there who will never go for anything else,
   but they sure won't even consider going for something else if that
   something else can't do everything they're already doing.  (flame out)
5) Economics: it's probably safe to say that slow links, i.e. 4KHz voice-
   grade channels, are going to be around for a lot longer than we'd like
   them to be.

------------------------------------------------------------------------
A GENERAL SOLUTION
------------------------------------------------------------------------
There are several things needed before a general solution to this
problem emerges.

(1) There needs to be some sort of standard for packet communication
    between terminals and hosts.  If this were the case, the packet wrapper
    would make communication between terminals and hosts truly transparent,
    doing away with the ugliness of escape sequences.  Eric Kolotyluk
    mentioned that ISO/ANSI is working on a specification entitled
    "Basic Class Virtual Terminal Protocol" which would appear, at least
    from the title, to be addressing this terminal/host packet comm problem
    (info on obtaining this document is given below).  Still, it is not
    likely that there will be any standards soon in this area, as pointed
    out by jhh:

    >To make this work well and universally, a protocol is needed for
    >standardizing communications between terminals and hosts.
    >This will enable both terminal and host manufacturers to proceed
    >with development of this software independently.  This will
    >also enable the editing features to be more controlled by the
    >terminal rather than the host.  One terminal manufacturer may
    >provide an EMACS type interface, while another provides a VI
    >type interface.  Since the same terminal would work on any host,
    >the editing commands would not change from operating system to
    >operating system, but from terminal to terminal.
    >
    >This is a subject near and dear to my heart, as I work on the
    >development of a packet switch network.  To make packet switching
    >cost effective, the data must be easily packetizable, but
    >there is little effort to develop the protocols to packetize
    >things until packet switching is more readily available, which
    >won't happen until ...  The old chicken and egg problem.

    (If you are interested in the documentation mentioned near the
    start of this section, it may be obtained from ANSI - International
    Sales, 1430 Broadway, New York, NY 10018.  Ask for "Basic Class
    Virtual Terminal Protocol", which comes in two parts: part 1,
    document ISO/TC97/SC21N515, "Initial Facilities", is $35.00;
    part 2, document ISO/TC97/SC21N527, "Extended Facilities", is $24.00.
    Shipping is $6.00.  Further info may be obtained from Lisa Rajchel
    at 212-354-3346.)

(2) UN*X needs to learn how to speak in packets in a more general way.
    Sure, everyone running 4.2 has an ethernet, but I don't believe
    that can be called a general solution; there's always talk of the
    warts under the surface of 4.2 at Usenix conferences.  I suspect
    that the "streams" capability being added to System V (D.M. Ritchie,
    "A Stream Input-Output System"; BSTJ, Oct. 84) is one promising
    approach to the problem of (1) providing a true message-passing
    capability under UN*X, and (2) popping a protocol processing module
    into the path between a program and device; does anyone know of
    others?

(3) There needs to be some sort of standard for splitting of functions
    between a host and an intelligent terminal/PC during editing sessions.
    Brad Davis pointed out that there's a document that provides some
    possibilities in this area, namely: Stallman, Richard, "The SUPDUP
    Local Editing Protocol" (MIT AI Lab, 6 Aug 84).  (I have a copy of
    this, and it does not prohibit me from recopying it, so e-mail me if
    you want a copy.)

------------------------------------------------------------------------
DISTRIBUTED EDITORS
------------------------------------------------------------------------
Even without a generalized packet capability for UN*X, it's still possible
to implement distributed editors.  Several of those responding provided
comments about the general characteristics of such an editor, and a few
provided references to working products.

One of the general responses suggested that control over local echoing
in a fairly smart terminal would be a minimal approach to avoiding an
excessive number of packets.  For instance, once the user had gone into
insert mode, the terminal would have to (1) echo all characters locally,
up to a non-printing character, and (2) save up the locally-echoed
characters, outputting them to the comm server in one blast at the end.
The host would have to have tty driver mods which would prevent it from
echoing the chars echoed by the terminal itself.  Here, from Scott Weikart,
is how this idea came about (although the intent of this description was
a modification designed to prevent an editor process from waking up
repeatedly, not what is desired here):

 >Here's a piece of mail from someone proposing a way to put TOPS-20 style
 >break tables into UNIX so that the TTY driver could do most raw-mode echoing
 >rather than making the process wake up on every character.  This could
 >be generalized to have the echoing be done by a smart terminal, although
 >control keys would still be subject to packet delays.  Thus it can be viewed
 >as a minimal attempt at avoiding packet delays.
 >
 >-scott
 >-----------------
 >Date: Mon, 13 May 85 11:23:42 edt
 >From: pegasus!hansen
 >Subject: echoin, screen editors & Unix
 >
 >This is all reminiscient of facilities that were added to Multics many moons
 >ago when Stallman's Emacs was ported there by Greenberg and company. I
 >believe that they had a very general purpose mechanism which stated what to
 >do with each character: echo it or call some function or other.
 >
 >I've also made and seen several proposals through the years on doing
 >something similar within UNIX via ioctl's. There are only two major pieces
 >of information which have to be provided: how many printable characters to
 >echo before returning and which characters are to be considered printable. A
 >read would then return when that many characters had been typed or a
 >non-printable character was typed. There also needs to be a default ioctl
 >mode to return the terminal to as soon as such a special read returns.
 >
 >Adding characters at the end of the line is properly handled.  The program
 >could do inserting of characters by leaving the terminal in insert mode
 >before calling this ioctl/read pair.  And any special command could also be
 >handled properly.
 >
 >Of course, in Emacs you really have to have full control over what
 >characters cause a return. Doing automatic parenthesis matching (cursor
 >bounces back to the matching paren for a second or two) requires control
 >being returnable when typing the right paren. The bit mask suggested by Tim
 >Maroney would be suitable for this.
 >
 >	Tony Hansen
 >	pegasus!hansen

Here are responses that describe actual working editors:
------------------------------------------------------------------------
>From jhh:

 >The jim editor for the Teletype DMD 5620 works pretty much the way you
 >want.  It still lacks some maturity and features compared with emacs or
 >vi, and it is mouse based, for better or worse.  It does have two
 >parts - one running in the terminal, and one running in the host.
 >The host handles such things as context searches, and the like,
 >while the terminal handles the local editing, sending things back and
 >forth in packets.  A strategy where there is some intelligence in both
 >the terminal and the host is needed.  As the terminal-host connection
 >is usually bandwidth limited, the minimazation amount of data transfer
 >is desirable.  This requires host software to determine what is
 >the smallest amount of data that can be sent to the terminal,
 >and terminal software to make small changes and send them to the host
 >in a minimal amount of changes.

Even though jhh is talking about a specific product, his earlier comments
about the need for standards are probably a good reason to believe that
this distributed blit editor he mentioned won't be widely used because
there is not a widely-recognized standard on which it's built.
------------------------------------------------------------------------
>From Rich Altmaier:

 >But on the general question, of what to do with a slower network, I believe
 >use of a local processor ('pc') can work to advantage.  The method here
 >should not be some kludge but an editor designed to be split between
 >local processor and host.  My company built a product with a local
 >processor doing editing, communicating with a host at 1200 baud.
 >The local processor interacts with the host with a Remote Procedure Call
 >model.  A buffer of text is pulled into the local processor as needed.
 >Changes are passed back to the host.  Global operations such as text
 >searches as passed to the host.  The program written to do this is
 >not all that complicated, given an RPC facility it is quite natural
 >to design and write code knowing there is a boundary where control
 >passes from machine to machine.  Of course there are difficulties
 >with data being duplicated, insuring host operations have the latest
 >text image.  We actually didn't use a pure RPC model, local changes
 >are being sent to the host in parallel with further local editting.
 >With this model, the host program consists of a set of procedures
 >awaiting work requests from the local processor.  The host program never
 >interacts at the single character at a time level.
------------------------------------------------------------------------
>From John Levine:

 >My former employers, Interactive Systems, have for several years sold a
 >combination of an editor and a smart terminal that work amazingly well over
 >a packet network.  The terminal has gone through several incarnations, but
 >these days is typically an IBM PC running a special terminal emulation
 >program.  (It used to be a Perkin-Elmer Owl with special microcode.)  The
 >editor is a collateral descendant of the Rand "re" screen editor.

 >The editor and the terminal converse back and forth using a protocol entirely
 >encoded as lines of printable ascii characters, so that it works just fine
 >over Telenet in line mode.  The terminal does all of the editing it can
 >locally, given that it only has a 25 by 80 buffer, a historical limitation
 >due to the peculiarities of the Owl terminal.  I have used it a lot over
 >Telenet, logging into a machine in Los Angeles from my home here in Boston,
 >and at 1200 baud you hardly notice the network in between unless you're doing
 >a lot of long searches and stuff.  NCAR at least used to be an Interactive
 >customer; there are probably some of the INtext terminals lying around that
 >you might want to try out.  The editor is pretty simple compared to emacs,
 >but I suppose you can't have everything.  I wrote some rather large programs
 >with it.

Yes, my employer is a former customer of ISC, and we still run an older
version of "INed", but we don't have INtext terminals and have never had
a need for remote editing on the 11/70s it's used on, so I can't speak
from personal experience.  The problem we've had with regard to INed is that
ISC doesn't provide ports to relatively rare systems (such as Pyramid, which
is our newest system), and won't sell the source so you can do it yourself,
so obtaining/evaluating it at a reasonable cost is problematical.

------------------------------------------------------------------------
AN AT&T "END-RUN" SOLUTION
------------------------------------------------------------------------
A couple respondents (R. Drew Davis, mdm) provided information about a
solution that would be suitable if you're not already stuck with some net,
and are shopping.  This solution is in the form of the AT&T "Datakit VCS"
product, which is said to provide low delay for character-per-packet
traffic, even in the presence of other (large) packets.  That is, you
presumably don't worry about the inefficiencies of character-per-packet
traffic; you just hook VCS up to your UN*X boxes and live happily ever
after.

The primary drawback to this is that you have to buy a private net.
As I found from talking to Jim Wilson of AT&T Marketing (Morristown, NJ),
VCS is not a tariffed product which is an alternative to public nets like
Telenet.  If you are prepared to pay for & install a private net, fine; you
can get the benefits of their low (~10ms node-to-node) delay, which is the
result of special protocols and 8MB fiber LANs with 1.5MB (T1) wide-area
links.  But - there are currently no plans for the operating companies to
offer it on a public-access basis like Telenet.  And, there is always that
annoying problem with the speed of light, which means they can't use
satellites, which means you probably can't get this service to points
overseas.

------------------------------------------------------------------------
ETHERNET SOLUTIONS
------------------------------------------------------------------------
Gilles Chartrand provided some interesting food for thought for those
who have ethernets.  Although his ideas are especially useful in an
environment which links remote ethernets, they are also useful to those
who have ethernets that are getting too crowded.

The basic idea is this:

 >	We have the same problem and have come the the following solution.
 >In the case of vi, a temporary file is create on which the actual editing
 >is done.  Wich very small modifications this file can be made to exist on
 >the CURRENT system.  For example : If you are on machine A and you want
 >to edit a file on machine B you would enter the following commands on
 >machine A :
 >
 >		vi B:<file name>
 >
 >The only time the network is used would be to transfer the file at the
 >start and whenever a write is done and it would then be done in "block"
 >mode for which packet networks shine!
    .
    .
    .
 >	We looked into modifying emacs and gave up before we could find
 >any solid method.

It appears that getting the code to do this is easy:

 >	In order to get the special vi, all you have to do is relink
 >the .o files with the -lra (remote access) library.  This package will
 >allow you to prefix the files name with "<machine name>:".  If no
 >collon occures in the file name then vi (or whatever) works as before.
 >Otherwise the file on the specified machine is opened via ethernet
 >(like magic).  There is a version of the csh which allows you to do
 >things like "cd fred:" and presto, your on a new machine.  "ls
 >fred:~/bin" will give you a listing of the files in your bin on the
 >other machine.  The remote access library and the remote csh are
 >available from the Purdue University (E-mail path: ...!ihnp4!purdue!tichy)
 >They are part of a larger project called IBIS which is destined to become
 >a true, distributed file system.

Gilles also mentioned that work is being done at his site on a version
of "rlogin" which, in normal tty driver mode, echoes remotely and
doesn't send a packet until a timeout or a CR.
------------------------------------------------------------------------
In the first posting of this, I suggested that Sun's Network File System
might provide a solution for those who have ethernets linked remotely
through TransLANs or whatever (although I don't know of anyone who's
actually doing this).  This prompted a reply from Bill Nowicki of Sun
(note especially title of the PhD dissertation by R. Goldberg):

 >What you are proposing is still on the forefront of "research".
 >In fact, you might want to read Robert N. Goldberg's PhD. Dissertation
 >from Rutgers University, 1982.  I also recently finished a PhD.
 >Dissertation in this area, regarding graphical interaction over
 >networks. Mine is "Partitioning of Function in a Distributed Graphics
 >System" Computer Science Department of Stanford University 1985.
 >Goldberg's title was "Software Design Issues in the Architecture
 >and Implementation of Distributed Text Editors".

 >As far as the "real world" is going, here at Sun we have a
 >product called SunLink that you can use as a Point-to-point IP
 >level gateway.  We have it running between buildings here, and
 >are considering using it to connect to the Arpanet.  One
 >proposal we have had with Arpa was to investigate the running
 >of NFS over long-haul links.  We would just use TCP for this,
 >and investigate the tuning of the protocols necessary to make it 
 >work.  The hardest problem is not technical but administrative:
 >how to name and provide access control to the files.

 >As longer term issues, we are investigating super-computer
 >access through workstations, from several different angles.
-- 

Ed Arnold * NCAR (Nat'l Center for Atmospheric Research)
PO Box 3000 * Boulder, CO  80307-3000 * 303-497-1253
csnet: era at ncar * arpa: era%ncar at csnet-relay * uucp: ...!hao!scdpyr!era