Satellite transmission of netnews

[usenix at ucbtopaz.CC.Berkeley.ARPA]

                                   ;login:


                              USENET in the Sky
         Satellite Distribution of Netnews:  The Stargate Experiment

                                   Lou Katz

Introduction

     Several thousand computer sites in the United States, Canada, Europe and
Australia are linked together into a logical "network" which permits the
transfer of messages directly from one individual to another (mail) and the
posting of messages to be read by anyone who is interested (news).  The many
sites on this network which are involved with news transfers collectively are
called USENET.  More specifically, USENET is defined as all sites receiving
the newsgroup net.announce.  A USENET link between two sites is one that
net.announce is sent over.  Note that this is different from a uucp link, over
which mail and file transfers may occur but not necessarily news.

     As more computer sites have gained access to this network a number of
problems have arisen, in particular with respect to the communications costs
incurred in the operation of this net and to difficulty of new sites obtaining
access.  As usage increases, USENET is faced with the spectre of increased
costs possibly forcing curtailment of network activity, an eventuality which
is causing great concern in the network community.  Furthermore, the magnitude
of the load which news places on a site is so large that new sites have great
difficulty finding a site willing to feed news to them.  Many new sites wish-
ing to get such information are without connections.

     At the present time it is estimated that there are about a thousand sites
in the network, with that number growing daily!  Total network traffic is
basically proportional to the number of sites, so that traffic is growing too.

     It is vital to realize that network services, to be useful, must connect
to the machines a particular individual uses regularly and as a matter of
course.  For news and especially for mail, it doesn't work for the person to
have to make an individual special call to a different machine just to see if
there is mail or news for him/her, any more than it makes sense to walk two
miles to the post office each day just to see if there is mail, when there
often will be none.

     Note however, that USENET IS NOT A NETWORK in the formal sense!  That is,
unlike all other nets (ARPANET, CSNET, BITNET, etc), there is NO administra-
tion, no central structure, no joining, and no membership to USENET.  The net
actually represents the human and professional network of personal, technical
and business contacts, and PAIRWISE desires for groups or individuals to com-
municate and share information easily.

     It is just this pairwise organization which gives the network its vital-
ity.  Without the burden of administration, all that is required is the tele-
phone switched network, which permits any machine, anywhere to contact any
other machine DIRECTLY, subject only to administrative and software agreement
between its managers.  Some pairs of sites are linked via dedicated high speed
circuits, because of the volume of traffic between them.  This linkage is not,
however, crucial to the operation of USENET.


Volume 9, Number 6              December 1984                                1







                                   ;login:


     News forwarding often represents a massive percentage of the overall data
traffic flowing through a given USENET site.  Some sites have taken this
responsibility upon themselves for a variety of reasons, but most sites will
only receive news or forward it to very specific recipients.  Mail is treated
somewhat differently, and many sites will forward mail as a professional cour-
tesy to others, which improves overall mail performance, and helps ensure that
others will forward mail to them.

     Estimates indicate that MAIL accounts for about 15% of the network "load"
and NEWS for about 85%, although at high volume nodes or central sites which
forward both news and mail, mail may reach 50%.

     For two machines to be networked, they have to be connected in some
manner.  This connection can be a dedicated link (leased phone line, internal
wires within a site, infrared relay, fiber optics) or a shared link such as a
dialup line.  Dedicated links, except for the trivial case of running a wire
between two machines in the same room, almost always involve dealing with
large external entities such as local phone company and common carriers to get
special dedicated wire services.  These links are expensive, are rapidly grow-
ing more expensive, and can involve very long (months or longer!) waiting
periods for installation in many areas.

     The cost of network phone calls is hard to see directly.  However, if one
conservatively estimates that there is about 1 Mb of news every DAY, and if
this is transmitted at 1200 bits per second with a an error-correcting
transmission protocol, there are roughly 3 hours of transmission per day.
Current phone rates run about $.15-$.30/minute in the dead of the night (the
times usually selected for transmission, just to keep the costs down) either
interstate or intrastate.  If a site getting news initiated the call itself,
it would spend about $36/day or a little over $1000/month on such phone calls.
Unfortunately, many phone calls wind up with bad circuits, giving numerous
retries and aborted messages.  This can add up to a factor of two on call
costs.  If two sites attempt to utilize a single phone line for both a feed in
and a feed out they are likely to utilize the entire 11pm-8am nighttime rate
slot on one line.  Since there is a pyramiding effect with each site servicing
several below it, a single site could easily dedicate two or three lines just
to network service, and often wind up using more expensive evening and daytime
connections.

     If only one hundred sites have to make non-local calls for this purpose,
the national phone bill attributed to network activity would be over
$100,000/month!  This amount may very well be much too conservative, as reli-
able rumors suggest that the phone bill for one particular site is signifi-
cantly in excess of $20,000/month!

     New technology is beginning to provide us with modems capable of working
on the nationwide switched network at speeds of 2400 bps, or double the
present common speed.  The costs of these modems are much higher than the com-
mon 1200 bps hardware.  Installation of such devices could cut some phone
bills considerably (though by less than half, due to various technical fac-
tors), but only if both sides of a connection have them.  It is unfortunately
easier for many system administrators to justify rising phone bills than to
receive approval for such a specific purchase, often from a different budget


2                               December 1984               Volume 9, Number 6







                                   ;login:


category, so that faster modems may not provide any relief.  In fact, even if
phone bills COULD be cut in half in this manner, the costs would still remain
very high, and would still grow constantly worse as the network grows and news
traffic continues to increase.

A Possible Solution

     Lauren Weinstein has presented at the Summer 1984 USENIX Conference in
Salt Lake City (Cf. Conference Proceedings, p. 18) a promising technological
solution to the most pressing part of the problem, the cost of news transmis-
sion.  The idea is as follows:  portions of the video signal on TV transmis-
sion are not used for picture information, and can carry other information, in
particular, suitably encoded ASCII.  The effective bandwidth of this type of
transmission could easily exceed 65 Kbps.  It should be possible to establish
a computer system at the "headend" of a cable or satellite transmission sys-
tem, and upload such information piggyback on the TV signal.  Any site which
wished to receive the data would get a decoder and either a cable link or a
satellite receiver dish.

     The decoder would have sufficient internal memory to store a significant
fraction of a day's news transmission (e.g. 500 Kbytes), so that the local
computer could buffer and flow control the input and select and extract the
information it wanted from the decoder at its own pace.  Estimated costs for
the decoder are about $1000 each (retail) and about $1300-$1500 for a satel-
lite dish for most locations in the continental U.S. and parts of Canada, if
the channel with the information were not also carried by a local cable TV
company.

     The economics of netnews would then change radically.  No longer would a
fanout of news have to occur over the dialup network.  Rather, each item could
be transmitted ONCE to the head end distribution computer, then "broadcast"
for all to receive over the satellite system.  The TOTAL "national" phone bill
for news then decreases to about $1000/month, instead of several hundred
thousand dollars.

     The cost of the original transmission which occurs when an item is sub-
mitted (the phone call from the submitter's computer to the satellite link) is
obviously borne by the submitter.  The costs of the reception equipment and
decoders are either one-time costs to the installation, easily amortized over
a few months of phone bills, or else handled as monthly rental fees.  This
scheme does not, in any way, cut off the current mode of transmission of net-
news.  However, as more and more sites have to examine their phone budgets,
they will generate both dollars and justification for inclusion of more and
more newsgroups via satellite transmission.











Volume 9, Number 6              December 1984                                3







                                   ;login:


The Experiment

     Lauren Weinstein has secured the cooperation of several corporations and
institutions in conducting an experiment into the technical feasibility of
this mode of transmission.

     The purpose of the experiment is to test the reception quality, error
rates, flow control and system reliability and functionality.  Reception will
be tested both directly from a small reasonably priced microwave dish, and
from ordinary cable-TV service in a number of locations.

     The USENIX Association is providing support for incoming phone lines at
the transmitter site, a small microwave receiver dish to test that mode of
reception and travel to the transmission site to set up the system.  The Asso-
ciation is also providing coordination of the efforts of Lauren Weinstein and
the other participants, as well as dissemination of the results through writ-
ten articles in ;login: and, of course, over USENET, and a presentation at the
January technical meeting in Dallas.  If technical conditions permit, there
will also be a live demonstration of the system at that meeting.

     SSS (Southern Satellite Systems, Atlanta, Georgia) is supplying the
experiment with continuous use of one scan line in their broadcast signal,
with an effective baud rate of 1200 baud for a few months.  They are also pro-
viding access to the uplink encoder which will properly format the input ASCII
information and insert it into the TV signal.  There transmissions are going
out under the TV signal of WTBS, the Atlanta-based "Superstation", which is
widely available throughout the United States.  They are also providing two
sets of tuners and decoders for receiving the signal directly and extracting
the ASCII stream from the video.

     Bell Communications Research (BCR) is providing modems for the uplink
facility and other support.

     Fortune Systems Corporation (Redwood City, California) has provided the
uplink computer, a Fortune Systems XT30 UNIX system, which will receive net-
news articles from dial-in phone lines and format them for insertion into the
video signal.

     If the experiment shows that we will achieve satisfactory performance
from a technical point of view, the UNIX community at large will then be faced
with the far more difficult problem:  how to make this technology available so
that USENET will flourish.  The future organization of USENET is a more com-
plex issue.  For a stable network capable of functioning over the next few
years, a host of legal, financial and organizational issues must be faced.
How can even a modest effort be financed?  What information or news groups
could such a network distribute?  Who would be responsible for content?  These
and other considerations must be worked through if satellite transmission is
to become a viable facility.







4                               December 1984               Volume 9, Number 6