f



UDP Terminal Protocol

Many devices have had a serial port for attaching a console of some
sort, for low-level configuration/diagnostic work.  While very simple
to implement, serial ports have issues which don't need repeating.

An obvious alternative is to implement the "serial console" as a
TELNET server.  Conceptually this is trivial, not to mention well
standardized and well understood.  OTOH, this piles on a fair bit of
complexity, and requires something akin to a full TCP/IP stack.  And
if you're going to do that, you might as well just go for a web-based
admin, it's only going to be a bit more work.

OTOH, it's clear that a number of people have implemented a UDP-based
protocol for handling the sort of things often done with a serial
console attached to a device.  This has a number of advantages - you
can fake UPD over Ethernet with very little code, especially if you
define away things like fragmentation and options.  A simplified
TELNET-ish protocol on top of that would be pretty simple, you just
need some basic packet sequencing and acknowledgements. IOW, something
with a relationship to TELNET sort of like TFTPs relationship to FTP.

OTOH, despite looking, I just don't see any standards for doing that
sort of thing.  Am I just missing it?  Is there a defacto standard
even if there's no RFC?
0
Robert
12/13/2016 10:23:16 AM
comp.arch.embedded 20047 articles. 1 followers. Post Follow

34 Replies
919 Views

Similar Articles

[PageSpeed] 19

On 13.12.2016 г. 12:23, Robert Wessel wrote:
> Many devices have had a serial port for attaching a console of some
> sort, for low-level configuration/diagnostic work.  While very simple
> to implement, serial ports have issues which don't need repeating.
>
> An obvious alternative is to implement the "serial console" as a
> TELNET server.  Conceptually this is trivial, not to mention well
> standardized and well understood.  OTOH, this piles on a fair bit of
> complexity, and requires something akin to a full TCP/IP stack.  And
> if you're going to do that, you might as well just go for a web-based
> admin, it's only going to be a bit more work.
>
> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.  A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements. IOW, something
> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>
> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?
>

I suppose you have found nothing simply because doing telnet over
UDP will take reinventing tcp. tcp is very well designed, I can't
see how it can be simplified while retaining its basic functionality.
I would expect there are super minimalistic tcp implementations
which might more or less work and would do what you need but I
don't really know what is out there, I have only written the
tcp for dps which is not minimalistic.

Dimiter

------------------------------------------------------
Dimiter Popoff, TGI             http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/

0
Dimiter_Popoff
12/13/2016 3:10:44 PM
On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:

> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.  A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements.

I know of people who have implemented that on top of IP rather than
UDP.  They call it TCP. ;)

> IOW, something with a relationship to TELNET sort of like TFTPs
> relationship to FTP.

Just use a raw TCP connection. There's no need to implement the TELNET
protocol at all.  The Unix command line client will skip all the
handshaking stuff if you use a non-standard port.  Or you can use
netcat.  Or any number of terminal emulators or COM port redirectors
that support raw TCP connections.

If you're on a local Ethernet segment, just use raw UDP.  I've been
using both those approaches for close to 20 years, and have never had
any problems.

> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?

If you want something reliable that works over WANs, use TCP: it's the
only way you're going to dependably make anything work through a
firewall.

If you're going to assume local Ethernet, you can just use UDP.

-- 
Grant Edwards               grant.b.edwards        Yow! Gee, I feel kind of
                                  at               LIGHT in the head now,
                              gmail.com            knowing I can't make my
                                                   satellite dish PAYMENTS!
0
Grant
12/13/2016 3:16:12 PM
On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
<invalid@invalid.invalid> wrote:

>Just use a raw TCP connection. There's no need to implement the TELNET
>protocol at all.  The Unix command line client will skip all the
>handshaking stuff if you use a non-standard port.  Or you can use
>netcat.  Or any number of terminal emulators or COM port redirectors
>that support raw TCP connections.

Thanks for the pointer to netcat.

I tried to find out , does it also support raw ethernet frames (with
MAC addressing) so that it could work without IP-headers and ARP ?

0
upsidedown
12/13/2016 4:51:47 PM
On 12/13/2016 3:23 AM, Robert Wessel wrote:
> Many devices have had a serial port for attaching a console of some
> sort, for low-level configuration/diagnostic work.  While very simple
> to implement, serial ports have issues which don't need repeating.
>
> An obvious alternative is to implement the "serial console" as a
> TELNET server.  Conceptually this is trivial, not to mention well
> standardized and well understood.  OTOH, this piles on a fair bit of
> complexity, and requires something akin to a full TCP/IP stack.  And
> if you're going to do that, you might as well just go for a web-based
> admin, it's only going to be a bit more work.
>
> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.  A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements. IOW, something
> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>
> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?

RLOGIN is much lighter-weight than TELNET (YMMV).  But, as with TELNET,
runs atop TCP.

What criteria do you place on the operating environment?  E.g., you
can't run a bare "serial port" 1000km.  Are you just trying to exploit
an existing connector (RJ45) for a secondary purpose with a LOCAL
"console"?  I.e., do you need flow control, fragmentation/packet reassembly,
routing, etc.?  Or, will you just slap some "special device" on the
wire "in close proximity to" the device in question?

Does the interface have to be interactive?  E.g., you could leverage
something akin to TFTP to push a "screen" of data to/from a console
device (treat the screen as a "form" that is filled in instead of
as a STREAM).

You can also implement a much pared-back network stack by putting
other constraints on how you plan to USE that stack (limit number
of open sockets, window size, etc.)  E.g., my devices are intended to
talk to specific *other* devices so I don't have to maintain ARP
caches, multiple endpoints, etc.
0
Don
12/13/2016 7:53:57 PM
On 2016-12-13, upsidedown@downunder.com <upsidedown@downunder.com> wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
><invalid@invalid.invalid> wrote:
>
>>Just use a raw TCP connection. There's no need to implement the TELNET
>>protocol at all.  The Unix command line client will skip all the
>>handshaking stuff if you use a non-standard port.  Or you can use
>>netcat.  Or any number of terminal emulators or COM port redirectors
>>that support raw TCP connections.
>
> Thanks for the pointer to netcat.
>
> I tried to find out , does it also support raw ethernet frames (with
> MAC addressing) so that it could work without IP-headers and ARP ?

I don't think so.

It's trivial to write something (on Linux) in Python that bascially
does what "netcat" does but in MAC mode.  On Windows, it's way more
work...

-- 
Grant Edwards               grant.b.edwards        Yow! Will it improve my
                                  at               CASH FLOW?
                              gmail.com            
0
Grant
12/13/2016 9:13:18 PM
On 12/13/2016 08:51 AM, upsidedown@downunder.com wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
>
>> Just use a raw TCP connection. There's no need to implement the TELNET
>> protocol at all.  The Unix command line client will skip all the
>> handshaking stuff if you use a non-standard port.  Or you can use
>> netcat.  Or any number of terminal emulators or COM port redirectors
>> that support raw TCP connections.
>
> Thanks for the pointer to netcat.
>
> I tried to find out , does it also support raw ethernet frames (with
> MAC addressing) so that it could work without IP-headers and ARP ?
>

There's a Linux program called socat that will do that type of thing. 
It's not installed by default but should be easy to get.
0
Rob
12/13/2016 9:23:28 PM
On 14/12/16 08:13, Grant Edwards wrote:
> On 2016-12-13, upsidedown@downunder.com <upsidedown@downunder.com> wrote:
>> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
>> <invalid@invalid.invalid> wrote:
>>
>>> Just use a raw TCP connection. There's no need to implement the TELNET
>>> protocol at all.  The Unix command line client will skip all the
>>> handshaking stuff if you use a non-standard port.  Or you can use
>>> netcat.  Or any number of terminal emulators or COM port redirectors
>>> that support raw TCP connections.
>>
>> Thanks for the pointer to netcat.
>>
>> I tried to find out , does it also support raw ethernet frames (with
>> MAC addressing) so that it could work without IP-headers and ARP ?
>
> I don't think so.
>
> It's trivial to write something (on Linux) in Python that bascially
> does what "netcat" does but in MAC mode.  On Windows, it's way more
> work...
>

libpcap has pcap_sendpacket, which I think does what you want,
including on Windoze
0
Clifford
12/13/2016 10:05:50 PM
On Tue, 13 Dec 2016 17:10:44 +0200, Dimiter_Popoff <dp@tgi-sci.com>
wrote:

>On 13.12.2016 ?. 12:23, Robert Wessel wrote:
>> Many devices have had a serial port for attaching a console of some
>> sort, for low-level configuration/diagnostic work.  While very simple
>> to implement, serial ports have issues which don't need repeating.
>>
>> An obvious alternative is to implement the "serial console" as a
>> TELNET server.  Conceptually this is trivial, not to mention well
>> standardized and well understood.  OTOH, this piles on a fair bit of
>> complexity, and requires something akin to a full TCP/IP stack.  And
>> if you're going to do that, you might as well just go for a web-based
>> admin, it's only going to be a bit more work.
>>
>> OTOH, it's clear that a number of people have implemented a UDP-based
>> protocol for handling the sort of things often done with a serial
>> console attached to a device.  This has a number of advantages - you
>> can fake UPD over Ethernet with very little code, especially if you
>> define away things like fragmentation and options.  A simplified
>> TELNET-ish protocol on top of that would be pretty simple, you just
>> need some basic packet sequencing and acknowledgements. IOW, something
>> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>>
>> OTOH, despite looking, I just don't see any standards for doing that
>> sort of thing.  Am I just missing it?  Is there a defacto standard
>> even if there's no RFC?
>>
>
>I suppose you have found nothing simply because doing telnet over
>UDP will take reinventing tcp. tcp is very well designed, I can't
>see how it can be simplified while retaining its basic functionality.
>I would expect there are super minimalistic tcp implementations
>which might more or less work and would do what you need but I
>don't really know what is out there, I have only written the
>tcp for dps which is not minimalistic.


Sure, doing TCP *well* requires pretty much re-inventing TCP.  But the
requirements here are much lower than that.  Performance requirements
are minimal, and operation over WANs (especially efficient operation
over WANs) is not really an issue either.  And it's already been done.
A lot.  I just want to know how.  Again, TFTP is an example of a
solution to a very similar problem.
0
Robert
12/14/2016 1:01:01 AM
On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
<invalid@invalid.invalid> wrote:

>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>
>> OTOH, it's clear that a number of people have implemented a UDP-based
>> protocol for handling the sort of things often done with a serial
>> console attached to a device.  This has a number of advantages - you
>> can fake UPD over Ethernet with very little code, especially if you
>> define away things like fragmentation and options.  A simplified
>> TELNET-ish protocol on top of that would be pretty simple, you just
>> need some basic packet sequencing and acknowledgements.
>
>I know of people who have implemented that on top of IP rather than
>UDP.  They call it TCP. ;)


Well, yes, but the point is that TCP is a fair chunk of code, and for
this function it has to go in a spot where it's less than convenient
to update.  Hence the desire for maximum simplicity.  I used TFTP as
an example, since it solves a similar problem (and you could
certatinly trivially implemented a TCP version of TFTP), with similar
constraints.


>> IOW, something with a relationship to TELNET sort of like TFTPs
>> relationship to FTP.
>
>Just use a raw TCP connection. There's no need to implement the TELNET
>protocol at all.  The Unix command line client will skip all the
>handshaking stuff if you use a non-standard port.  Or you can use
>netcat.  Or any number of terminal emulators or COM port redirectors
>that support raw TCP connections.


Given how little TELNET actually does (or at least, needs to do), if
you have TCP, there would be no really issue implementing that.  Raw
TCP is obviously an option, but again, the incremental cost to do
actual TELNET is low.  And as I said, if you have TCP, why not just do
this with HTTP, and really make things convenient for people.


>If you're on a local Ethernet segment, just use raw UDP.  I've been
>using both those approaches for close to 20 years, and have never had
>any problems.


Yet what exactly is the protocol?  How many characters can you put in
a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
on or off?  With UDP (and a bit of consideration at the protocol
layer), you can even avoid any IP configuration requirements (at least
on a single LAN) by doing everything with broadcasts.

I've simply never implemented anything with UDP that didn't take error
handling into account - is the common practice just to assume that you
will not lose/reorder/duplicate/mangle any packing so long as you're
on a local segment?  Again, TFTP addresses those issues, at least as
they apply to its requirements.


>> OTOH, despite looking, I just don't see any standards for doing that
>> sort of thing.  Am I just missing it?  Is there a defacto standard
>> even if there's no RFC?
>
>If you want something reliable that works over WANs, use TCP: it's the
>only way you're going to dependably make anything work through a
>firewall.


Sure, no disagreement there.  And no, operation over WANs is not a
prime consideration, although it certainly wouldn't hurt if it could,
even if the user had to deal with firewall issues, or set up a relay
server of some sort.  But the scope would be within a data-center, or
perhaps campus (assuming at least real IP address assignment - without
that, local LAN).  Remember the idea here is to replace a serial
console port.


>If you're going to assume local Ethernet, you can just use UDP.
0
Robert
12/14/2016 7:28:21 PM
On Tue, 13 Dec 2016 12:53:57 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 12/13/2016 3:23 AM, Robert Wessel wrote:
>> Many devices have had a serial port for attaching a console of some
>> sort, for low-level configuration/diagnostic work.  While very simple
>> to implement, serial ports have issues which don't need repeating.
>>
>> An obvious alternative is to implement the "serial console" as a
>> TELNET server.  Conceptually this is trivial, not to mention well
>> standardized and well understood.  OTOH, this piles on a fair bit of
>> complexity, and requires something akin to a full TCP/IP stack.  And
>> if you're going to do that, you might as well just go for a web-based
>> admin, it's only going to be a bit more work.
>>
>> OTOH, it's clear that a number of people have implemented a UDP-based
>> protocol for handling the sort of things often done with a serial
>> console attached to a device.  This has a number of advantages - you
>> can fake UPD over Ethernet with very little code, especially if you
>> define away things like fragmentation and options.  A simplified
>> TELNET-ish protocol on top of that would be pretty simple, you just
>> need some basic packet sequencing and acknowledgements. IOW, something
>> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>>
>> OTOH, despite looking, I just don't see any standards for doing that
>> sort of thing.  Am I just missing it?  Is there a defacto standard
>> even if there's no RFC?
>
>RLOGIN is much lighter-weight than TELNET (YMMV).  But, as with TELNET,
>runs atop TCP.


Not really sure I'd agree with that, but both are pretty simple.  If I
had TCP, doing either TELNET or RLOGIN (or HTTP) is not much of an
issue.  OTOH, there doesn't really seem to be much advantage for doing
RLOGIN over TELNET.


>What criteria do you place on the operating environment?  E.g., you
>can't run a bare "serial port" 1000km.


No, but one would hope to do something with a bit more range than a
serial port (which is what's there now), and a bit more convenient
infrastructure-wise.  Design "range" should be something on the
data-center to campus scale.  One of the prime considerations is the
ability to manage many devices without the complexity of attaching to
many serial ports scattered around the data center.


>  Are you just trying to exploit
>an existing connector (RJ45) for a secondary purpose with a LOCAL
>"console"?  I.e., do you need flow control, fragmentation/packet reassembly,
>routing, etc.?  Or, will you just slap some "special device" on the
>wire "in close proximity to" the device in question?


I don't really want a special device, although a PC with special
software is reasonable, but at worst case you'd want that PC to need
to be no closer than somewhere on the local LAN (data center or
campus, as mentioned above, would be better).  If you didn't mind a
special device, there are RS-232-to-TELNET devices you could just plug
into the serial port, and define the whole problem away.  As to flow
control and whatnot, those issue should be kept to a minimum (again
TFTP is a parallel), but since this has been done before, I'm trying
to figure out if there's existing practice to piggyback on.  If, for
example, there's a semi-standard way to do this, there are likely
terminal emulators that already support that.

The big advantage to going "whole hog" and doing TCP and TELNET (or
TCP and HTTP), is that you open this up to the whole Internet, and an
untold number of existing clients for any conceivable platform.

Physically, the idea would be to replace the serial port with an
Ethernet port.  It would be nice to be able to still connect without
any network infrastructure at all (beyond, possibly, a crossover
Ethernet cable).



>Does the interface have to be interactive?  E.g., you could leverage
>something akin to TFTP to push a "screen" of data to/from a console
>device (treat the screen as a "form" that is filled in instead of
>as a STREAM).


The model under consideration is a replacement of a serial console
assumed to be a VT-100 compatible device.  I don't think a hack like
you're suggesting is viable.  At that point we're talking about custom
software on both ends, so a "proper" TELNET-ish* protocol over UDP
would be simpler.  Developing such a (new) protocol is no problem.


*And let's not get too hung up on the specifics of TELNET here - some
bits of handshake information probably need to be exchanged at the top
of the session, and then it's just a packaging of the bi-directional
serial stream.


>You can also implement a much pared-back network stack by putting
>other constraints on how you plan to USE that stack (limit number
>of open sockets, window size, etc.)  E.g., my devices are intended to
>talk to specific *other* devices so I don't have to maintain ARP
>caches, multiple endpoints, etc.


Sure: I'd like to pare-back the network stack by not implementing TCP.
;-)  Limited UDP-only on top of Ethernet is pretty trivial.
0
Robert
12/14/2016 8:03:59 PM
W dniu wtorek, 13 grudnia 2016 11:22:52 UTC+1 u=C5=BCytkownik robert...@yah=
oo.com napisa=C5=82:
> Many devices have had a serial port for attaching a console of some
> sort, for low-level configuration/diagnostic work.  While very simple
> to implement, serial ports have issues which don't need repeating.
>=20
> An obvious alternative is to implement the "serial console" as a
> TELNET server.  Conceptually this is trivial, not to mention well
> standardized and well understood.  OTOH, this piles on a fair bit of
> complexity, and requires something akin to a full TCP/IP stack.  And
> if you're going to do that, you might as well just go for a web-based
> admin, it's only going to be a bit more work.
>=20
> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.  A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements. IOW, something
> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>=20
> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?

If you are interested in control and diagnosis of FPGA based systems, then =
you may take a look at IPbus: https://svnweb.cern.ch/trac/cactus
However, as this is comp.arch.embedded, not comp.arch.fpga, that probably w=
on't help you too much.

Regards,
Wojtek
0
wzab01
12/14/2016 9:16:57 PM
On 12/14/2016 1:03 PM, Robert Wessel wrote:
> On Tue, 13 Dec 2016 12:53:57 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>>> OTOH, despite looking, I just don't see any standards for doing that
>>> sort of thing.  Am I just missing it?  Is there a defacto standard
>>> even if there's no RFC?
>>
>> RLOGIN is much lighter-weight than TELNET (YMMV).  But, as with TELNET,
>> runs atop TCP.
>
> Not really sure I'd agree with that, but both are pretty simple.  If I
> had TCP, doing either TELNET or RLOGIN (or HTTP) is not much of an
> issue.  OTOH, there doesn't really seem to be much advantage for doing
> RLOGIN over TELNET.

Just to make sure there's no misunderstanding, RLOGIN runs over TCP;
you'd use it in lieu of TELNET.  (a full TELNET implementation is
many thousands of lines of code -- much of which you probably won't
need if you just want to emulate a serial port)

>> What criteria do you place on the operating environment?  E.g., you
>> can't run a bare "serial port" 1000km.
>
> No, but one would hope to do something with a bit more range than a
> serial port (which is what's there now), and a bit more convenient
> infrastructure-wise.  Design "range" should be something on the
> data-center to campus scale.  One of the prime considerations is the
> ability to manage many devices without the complexity of attaching to
> many serial ports scattered around the data center.

Once you "leave the room", things typically get complicated (unless
you're willing to live with a 100m cable limitation).  And, once you
start trying to talk to existing fabric, you have even less leeway
in terms of the corners you can cut (e.g., what happens if some
host tries to ping your node?  open an FTP connection?  etc -- even
if wrong in attempting to do so).

Many years ago (10+) I put together a proposal for a low cost "console server"
(essentially a one-port terminal server) to facilitate LOM in a datacenter
environment.  At the time, I could enhance security and cut cost by
bastardizing the network protocols with the caveat:  all of these devices will
reside on an isolated internet (so they didn't have to contend with the
variety of traffic they might encounter on a "live" internet).

Nowadays, SoC's have far more resources so even a complete stack isn't
going to break the bank (unless you're *really* scraping the bottom
of the cost/commodity barrel).

>>  Are you just trying to exploit
>> an existing connector (RJ45) for a secondary purpose with a LOCAL
>> "console"?  I.e., do you need flow control, fragmentation/packet reassembly,
>> routing, etc.?  Or, will you just slap some "special device" on the
>> wire "in close proximity to" the device in question?
>
> I don't really want a special device, although a PC with special

The "PC" was the intent of my reference  :>  The point being, it's not
an OTS solution (you still need to "develop" that end of the link).

> software is reasonable, but at worst case you'd want that PC to need
> to be no closer than somewhere on the local LAN (data center or
> campus, as mentioned above, would be better).  If you didn't mind a
> special device, there are RS-232-to-TELNET devices you could just plug
> into the serial port, and define the whole problem away.  As to flow
> control and whatnot, those issue should be kept to a minimum (again
> TFTP is a parallel), but since this has been done before, I'm trying
> to figure out if there's existing practice to piggyback on.  If, for
> example, there's a semi-standard way to do this, there are likely
> terminal emulators that already support that.

There have been lots of attempts at making "console servers" (my
reference above).  HP had a "Secure Web Console" that presented
a "terminal emulator" in a web browser window that was tethered
to the remote device's serial port.

[It was a poorly designed product, though -- particularly the
laughable "secure" aspect of it!  :> ]

You can buy single port terminal servers in a package not
much larger than a DB25 backshell (the proposal I outlined above
called for a DB9 backshell package as most servers now have 9 pin
serial ports so you don't have room for a "box" behind the server)

> The big advantage to going "whole hog" and doing TCP and TELNET (or
> TCP and HTTP), is that you open this up to the whole Internet, and an
> untold number of existing clients for any conceivable platform.

As Devil's Advocate, I'd point out that you now present a larger
attack surface and a (potentially) bigger adversary pool; one
thing with which a simple serial port didn't have to contend!

> Physically, the idea would be to replace the serial port with an
> Ethernet port.  It would be nice to be able to still connect without
> any network infrastructure at all (beyond, possibly, a crossover
> Ethernet cable).

It might be interesting to consider putting serial port *and* ethernet
on the same connector (I'd have to consider the hardware consequences
to see if there's a "cheap" way of doing so).

>> Does the interface have to be interactive?  E.g., you could leverage
>> something akin to TFTP to push a "screen" of data to/from a console
>> device (treat the screen as a "form" that is filled in instead of
>> as a STREAM).
>
> The model under consideration is a replacement of a serial console
> assumed to be a VT-100 compatible device.  I don't think a hack like
> you're suggesting is viable.  At that point we're talking about custom
> software on both ends, so a "proper" TELNET-ish* protocol over UDP
> would be simpler.  Developing such a (new) protocol is no problem.

If you were willing to make the "special device" a PC, then this
just changes the nature of the software running in the PC (given
that the "device end" is already negotiable).

I offer it only because I've frequently designed "screen oriented"
user interfaces (think: menu/dialog system atop curses(1)) so the
UI is always a ~2KW buffer in the device's memory that just gets
painted (via ANSI escape sequences) onto the hardware TTY.  Pushing
those 4KB out through a set of ~8 UDP packets (on demand any time
user asks for the screen to be repainted) would be a piece of cake;
much simpler than the curses stack residing in the device -- or the
ANSI emulator in the PC!

[The Sun Ray operated in a similar fashion; and, AFAICT, the Pano
cubes (no CPU -- entirely FPGA based) use a similar approach to
"painting a screen"]

> *And let's not get too hung up on the specifics of TELNET here - some
> bits of handshake information probably need to be exchanged at the top
> of the session, and then it's just a packaging of the bi-directional
> serial stream.

If you want to interoperate with other devices, you may have to implement
bits of the protocol that you'd hoped you could elide.  E.g., MS's
TELNET client is a PITA for me to use, here -- so I install third-party
TELNET clients that "work".

>> You can also implement a much pared-back network stack by putting
>> other constraints on how you plan to USE that stack (limit number
>> of open sockets, window size, etc.)  E.g., my devices are intended to
>> talk to specific *other* devices so I don't have to maintain ARP
>> caches, multiple endpoints, etc.
>
> Sure: I'd like to pare-back the network stack by not implementing TCP.
> ;-)  Limited UDP-only on top of Ethernet is pretty trivial.

But, even with TCP, there are economies available.  If you try to
cheap out and port an existing stack (FOSS), you will typically end
up with more resources (and features) than you might otherwise need.

IMO, the biggest downside is you'll typically insulate yourself from
the inner workings of the stack and, thus, not be aware of the sorts of
exploits that lie in wait.  Or, the costs (space/performance) that you
are bearing.

Thereafter, you have to track new releases of that codebase to see
if/when to fold them into one of your "updates".  This could result in
your taking on additional functionality/resource requirements that
you might not want.

OTOH, if you "take ownership" of such a codebase (i.e., start tweeking
it to *fit* your needs), then you're still stuck watching the releases
and now have to merge "pertinent" changes into your codebase.

[I.e., you quickly start to feel like the dog's *tail*!]

If you haven't, already, Comer's XINU (vol2) is a lightweight introduction
to the stacks; followed by his _Internetworking with TCP/IP_ series.
With those behind you, Stevens' _TCP/IP Illustrated_ is by far the best
reference.

[Though Stevens' texts tend to describe more efficient (from the
standpoint of performance) implementations; Comer's tend to be much
easier to visualize -- more *discrete* layers (and, thus, less
efficient)]

And, watch the CERT advisories to see how profoundly varied attacks
on stacks can be, in practice!
0
Don
12/14/2016 10:09:13 PM
On 12/14/2016 12:28 PM, Robert Wessel wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
>
>> On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>
>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>> protocol for handling the sort of things often done with a serial
>>> console attached to a device.  This has a number of advantages - you
>>> can fake UPD over Ethernet with very little code, especially if you
>>> define away things like fragmentation and options.  A simplified
>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>> need some basic packet sequencing and acknowledgements.
>>
>> I know of people who have implemented that on top of IP rather than
>> UDP.  They call it TCP. ;)
>
> Well, yes, but the point is that TCP is a fair chunk of code, and for
> this function it has to go in a spot where it's less than convenient
> to update.  Hence the desire for maximum simplicity.  I used TFTP as
> an example, since it solves a similar problem (and you could
> certatinly trivially implemented a TCP version of TFTP), with similar
> constraints.

TFTP runs using UDP datagrams.  It's simplicity lies in the fact
that it has very few options (that need to be negotiated) and
a window size of 1 packet:  next packet isn't sent until
previous is ACK'd

[lie:  newer RFC's extend TFTP to support multiple outstanding
packets; but, this means a more complicated implementation]

>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>> relationship to FTP.
>>
>> Just use a raw TCP connection. There's no need to implement the TELNET
>> protocol at all.  The Unix command line client will skip all the
>> handshaking stuff if you use a non-standard port.  Or you can use
>> netcat.  Or any number of terminal emulators or COM port redirectors
>> that support raw TCP connections.
>
> Given how little TELNET actually does (or at least, needs to do), if
> you have TCP, there would be no really issue implementing that.  Raw
> TCP is obviously an option, but again, the incremental cost to do
> actual TELNET is low.  And as I said, if you have TCP, why not just do
> this with HTTP, and really make things convenient for people.

Have you looked at an *actual* TELNETd implementation?  You're looking
at several KLoC's.  You can pare this back by deciding to only
advertise certain capabilities.  But, it's not a "weekend job"!

[I assume the TELNET client is something you'd offload to the user's
responsibility]

And, with a "virtual serial port", other issues crop up that
you may not have anticipated (if you're in a "hardware serial port"
mindset).  E.g., what happens if TWO clients try to connect to your
TELNETd?  Is that a *feature*?  Have you anticipated locks on
the various structs/parameters that the user will be modifying
via these interfaces?  Have you addressed the possibility of a
TELNET session "hanging" and the held locks never being released?

Etc.

[All of this is obvious -- in hindsight!  :> ]

>> If you're on a local Ethernet segment, just use raw UDP.  I've been
>> using both those approaches for close to 20 years, and have never had
>> any problems.
>
> Yet what exactly is the protocol?  How many characters can you put in
> a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
> on or off?  With UDP (and a bit of consideration at the protocol
> layer), you can even avoid any IP configuration requirements (at least
> on a single LAN) by doing everything with broadcasts.

If you're trying to emulate a serial port, you probably won't *need*
to pack many characters into a datagram.  Figure 500 bytes is "always
safe" and don't even bother trying to negotiate a larger datagram
size.

> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?  Again, TFTP addresses those issues, at least as
> they apply to its requirements.

TFTP without the "windowsize" option handles the problem just by requesting
retransmits until you've "got" the next packet IN SEQUENCE.  Support
for the windowsize forces the client to keep track of *which* packets
it's received -- and tolerate the possibility that it might accumulate
them out-of-order in how it frames its acknowledgements.

"No free lunch"

As you're *probably* not looking for maximum performance, minimum latency,
etc. ("It's a serial port!"), you can probably deliberately tone down the
implementation so it doesn't tax the fabric *nor* the clients involved
(i.e., if you give a client plenty of time to process packet 1 before
trying to send packet 2, there's a better chance that it won't DROP
packet 2 and, thus, need to re-request it).

[Historically, there have been NICs that would "not see" packets
appearing too close to other packets on the wire -- think of it
as "hardware bugs" necessitating software work-arounds]
0
Don
12/14/2016 10:29:47 PM
Robert Wessel <robertwessel2@yahoo.com> writes:
> Well, yes, but the point is that TCP is a fair chunk of code, and for
> this function it has to go in a spot where it's less than convenient
> to update.

I still don't see the problem: TCP has been around for decades and there
are some tiny implementations, especially if you don't need multiple
packets in the air at the same time etc.  Why not just use one?
0
Paul
12/14/2016 11:22:42 PM
On Wed, 14 Dec 2016 14:03:59 -0600, Robert Wessel
<robertwessel2@yahoo.com> wrote:


>Sure: I'd like to pare-back the network stack by not implementing TCP.
>;-)  Limited UDP-only on top of Ethernet is pretty trivial.

I guess that you are aiming of some single chip microcontrollers with
internal RAM only but with on-chip Ethernet interface. 

Of course, you will need two 1500 byte buffers for the Ethernet Rx and
Tx frames,  Raw Ethernet will need only marginally more and the UDP
implementation doesn't meed much more for multiple port number
support.

0
upsidedown
12/15/2016 1:01:01 AM
Robert Wessel <robertwessel2@yahoo.com> wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
> 
> >On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
> >
> >> OTOH, it's clear that a number of people have implemented a UDP-based
> >> protocol for handling the sort of things often done with a serial
> >> console attached to a device.  This has a number of advantages - you
> >> can fake UPD over Ethernet with very little code, especially if you
> >> define away things like fragmentation and options.  A simplified
> >> TELNET-ish protocol on top of that would be pretty simple, you just
> >> need some basic packet sequencing and acknowledgements.
> >
> >I know of people who have implemented that on top of IP rather than
> >UDP.  They call it TCP. ;)
> 
> 
> Well, yes, but the point is that TCP is a fair chunk of code, and for
> this function it has to go in a spot where it's less than convenient
> to update.  Hence the desire for maximum simplicity.  I used TFTP as
> an example, since it solves a similar problem (and you could
> certatinly trivially implemented a TCP version of TFTP), with similar
> constraints.

My experience with TFTP in early nineties was that on our
network attempting to send file larger than 1MB will usually
stall.  We had several segments of thin Etehrent connected
via a repeater and used cheap equipement.  TCP run with
quite resonable performance, but TFTP was essentially
unusable.  Apparently TFTP specification was extended
to get better reliability (or maybe our implemetation
dropped optional parts), but it seems that using raw UDP
will give you _more_ reliability than I got from TFTP.
More preciesely, IIRC both at Ehernet level and IP level
you get a checksum for each packet.  You are talkong
about low performance, so presumably no need to throttle
transmission.  At low speed in local network there
should be no problem with reordering packets.  As my
expample shows in practice TFTP was unable to retransmit
missing packets.  Together this means that acknowledgements
buy you nothing => you are end up at raw UDP level.

> 
> >> IOW, something with a relationship to TELNET sort of like TFTPs
> >> relationship to FTP.
> >
> >Just use a raw TCP connection. There's no need to implement the TELNET
> >protocol at all.  The Unix command line client will skip all the
> >handshaking stuff if you use a non-standard port.  Or you can use
> >netcat.  Or any number of terminal emulators or COM port redirectors
> >that support raw TCP connections.
> 
> 
> Given how little TELNET actually does (or at least, needs to do), if
> you have TCP, there would be no really issue implementing that.  Raw
> TCP is obviously an option, but again, the incremental cost to do
> actual TELNET is low.  And as I said, if you have TCP, why not just do
> this with HTTP, and really make things convenient for people.
> 
> 
> >If you're on a local Ethernet segment, just use raw UDP.  I've been
> >using both those approaches for close to 20 years, and have never had
> >any problems.
> 
> 
> Yet what exactly is the protocol?  How many characters can you put in
> a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
> on or off?  With UDP (and a bit of consideration at the protocol
> layer), you can even avoid any IP configuration requirements (at least
> on a single LAN) by doing everything with broadcasts.
> 
> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?  Again, TFTP addresses those issues, at least as
> they apply to its requirements.

Well, maybe modern TFTP is better, but IME TFTP did _not_ address
the isssues.

I think you look at your problem from wrong point of view.  Once
you want decent capability for transmitting streams of data you
end up with something very similar to TCP.  So better question
is: "what parts of typical TCP implementation can be safely
skipped and which existing implementation has good support
for doing so".  Partial answer is: TCP implementation can be
simplified quite a lot and still work reasonably well.
Commer and Stevens in their 1994 book described "complete"
implementation of Internet protocls, their code was simple
but they tried to cover all required parts.
They wrote that they needed about 3000 LOC for TCP
(and slightly larger amount for mamdatory lower level parts).
If you limit their implementation to TCP only (+ needed lower
level parts) it will by probably 60% bigger than UDP only (again
UDP + needed lower level parts).  Compared to 1994 TCP
was extended, but it seems safe to assume that even in
modern network you should be able to use implementation
having 6000 LOC (for TCP and lower layers, but exluding
hardware handling).  This 6000 LOC-s is likely to
produce no more than 8-16 KB of machine code.  RAM
overhead is of order few hundreds bytes per connection
plus needed buffering (likely few kilobytes).  If
you want reliable transmission you need buffering
(to be able to retransmit) so extra requirements
seem to be quite modest.  And if you have enough free
RAM you probly also have enough program space (typical
MCU-s seem to have much more program space than RAM).
Of course, you may use tiny MCU with external
Ethernet (or Wifi) chip providing buffering, but
if you add such a chip to existing system maybe
it is better to add a "communication processor"
that handles all network aspects (ESP8266 will
do this for Wifi).

The TCP/IP stacks that I know about seem to be larger
than estimate I gave above.  But they have options
to omit parts, so maybe you can trim them down.

Significant part of complexity of TCP comes from
handling long streams.  If you can live with small
HTTP pages than there is very small implementation
at tuxgraphics.org.  The implementation is quite
small but they do it via limiting HTTP page to
a single Ethernet packet.  This implementation
violates several requirements of TCP/IP, but
seem to work quite well in practice.


-- 
                              Waldek Hebisch
0
antispam
12/15/2016 1:49:38 AM
On 2016-12-14, Robert Wessel <robertwessel2@yahoo.com> wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
><invalid@invalid.invalid> wrote:
>
>>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>
>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>> protocol for handling the sort of things often done with a serial
>>> console attached to a device.  This has a number of advantages - you
>>> can fake UPD over Ethernet with very little code, especially if you
>>> define away things like fragmentation and options.  A simplified
>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>> need some basic packet sequencing and acknowledgements.
>>
>> I know of people who have implemented that on top of IP rather than
>> UDP.  They call it TCP. ;)
>
> Well, yes, but the point is that TCP is a fair chunk of code,

By the time you do "packet sequencing and acknowledgements" for both
target and host side, test it, revisit the bad protocol design
decisions, and then re-implement it for both sides you've spent more
time than you would have just doing a simple TCP implementation.

And in the end you've got something that's non-standard, non-portable,
probably still doesn't work right, and for which Wireshark doesn't
have a packet dissector.

Don't underestimate the importance of that last one.

>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>> relationship to FTP.

What features of TELNET do you need?

>>Just use a raw TCP connection. There's no need to implement the TELNET
>>protocol at all.  The Unix command line client will skip all the
>>handshaking stuff if you use a non-standard port.  Or you can use
>>netcat.  Or any number of terminal emulators or COM port redirectors
>>that support raw TCP connections.
>
> Given how little TELNET actually does (or at least, needs to do), if
> you have TCP, there would be no really issue implementing that.  Raw
> TCP is obviously an option, but again, the incremental cost to do
> actual TELNET is low.

In my experinece the incremental value of Telnet is negative, since
various Telnet clients behave it vastly different and bizarre ways.

YMMV.

> And as I said, if you have TCP, why not just do this with HTTP, and
> really make things convenient for people.

OK.

>>If you're on a local Ethernet segment, just use raw UDP.  I've been
>>using both those approaches for close to 20 years, and have never had
>>any problems.
>
> Yet what exactly is the protocol?  

Um... UDP.

> How many characters can you put in a packet?

UDP says 65536 bytes per datagram.  UDP will fragment that and
reassemble it.  If you want to do a stripped-down UDP iplimentation,
limit it to about 1200.

> Any sequencing or recovery?

Nope.

> Acknowledgements?

Nope.

> Checksums on or off?

For a local Ethernte segment, you let the MAC layer worry about that.

> With UDP (and a bit of consideration at the protocol layer), you can
> even avoid any IP configuration requirements (at least on a single
> LAN) by doing everything with broadcasts.

Some IP stacks won't receive UDP broadcast packets with a
non-reachable source IP.

> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?

Unless your Ethernet segment is pretty heavily loaded, I find you can
ignore all that.

If you can't, then you need the equivalent of TCP.

-- 
Grant



0
Grant
12/15/2016 2:06:18 AM
Grant Edwards <invalid@invalid.invalid> writes:
> you've spent more time than you would have just doing a simple TCP
> implementation.

I still don't understand why the OP would want to "do" (i.e. write) a
simple TCP implementation instead of downloading one.  There are lots,
e.g.:

https://en.wikipedia.org/wiki/UIP_(micro_IP)
0
Paul
12/15/2016 5:09:01 AM
On Thu, 15 Dec 2016 02:06:18 +0000 (UTC), Grant Edwards
<invalid@invalid.invalid> wrote:

>On 2016-12-14, Robert Wessel <robertwessel2@yahoo.com> wrote:
>> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
>><invalid@invalid.invalid> wrote:
>>
>>>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>>
>>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>>> protocol for handling the sort of things often done with a serial
>>>> console attached to a device.  This has a number of advantages - you
>>>> can fake UPD over Ethernet with very little code, especially if you
>>>> define away things like fragmentation and options.  A simplified
>>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>>> need some basic packet sequencing and acknowledgements.
>>>
>>> I know of people who have implemented that on top of IP rather than
>>> UDP.  They call it TCP. ;)
>>
>> Well, yes, but the point is that TCP is a fair chunk of code,
>
>By the time you do "packet sequencing and acknowledgements" for both
>target and host side, test it, revisit the bad protocol design
>decisions, and then re-implement it for both sides you've spent more
>time than you would have just doing a simple TCP implementation.


Not if this is limited, as is (basic) TFTP, to a single outstanding
packet (in each direction).  A simple sequence number is all you need,
and the ability to retransmit one packet, plus a timer for detecting
missing acks.  I've done all that before, more than once.


>And in the end you've got something that's non-standard, non-portable,
>probably still doesn't work right, and for which Wireshark doesn't
>have a packet dissector.
>
>Don't underestimate the importance of that last one.


While I've never written a decode for Wireshark, I've written several
for other packet monitors, I suspect I'd actually enjoy that little
task.  ;-)  OTOH, given the simplicity of the protocol, it's value
would be moderate.


>>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>>> relationship to FTP.
>
>What features of TELNET do you need?


None really.  The idea would be to assume a simple bidirectional
stream to a VT-100 (or heck, even a TTY in many cases).  Just enough
of TELNET to allow standard clients to be used.  If this were a custom
protocol, I'd at least have a version exchange somewhere at session
setup.


>>>Just use a raw TCP connection. There's no need to implement the TELNET
>>>protocol at all.  The Unix command line client will skip all the
>>>handshaking stuff if you use a non-standard port.  Or you can use
>>>netcat.  Or any number of terminal emulators or COM port redirectors
>>>that support raw TCP connections.
>>
>> Given how little TELNET actually does (or at least, needs to do), if
>> you have TCP, there would be no really issue implementing that.  Raw
>> TCP is obviously an option, but again, the incremental cost to do
>> actual TELNET is low.
>
>In my experinece the incremental value of Telnet is negative, since
>various Telnet clients behave it vastly different and bizarre ways.
>
>YMMV.


Well, most of them at least manage a basic VT-100 connection, and most
of their (TELNET) responses, odd or not, could be mostly ignored in
this implementation.  I've been reconsidering the notion of a raw-TCP
stream support, this appears to be a rather more common option in
TELNET clients than I had expected.  Still doing a minimalist TELNETD
would not be much work, and if it significantly improved
compatibility, would probably be worth it.


>> And as I said, if you have TCP, why not just do this with HTTP, and
>> really make things convenient for people.
>
>OK.
>
>>>If you're on a local Ethernet segment, just use raw UDP.  I've been
>>>using both those approaches for close to 20 years, and have never had
>>>any problems.
>>
>> Yet what exactly is the protocol?  
>
>Um... UDP.
>
>> How many characters can you put in a packet?
>
>UDP says 65536 bytes per datagram.  UDP will fragment that and
>reassemble it.  If you want to do a stripped-down UDP iplimentation,
>limit it to about 1200.
>
>> Any sequencing or recovery?
>
>Nope.
>
>> Acknowledgements?
>
>Nope.
>
>> Checksums on or off?
>
>For a local Ethernte segment, you let the MAC layer worry about that.


Well, if that's what people have been doing, then I think the answer
to my original question is:  no, nothing usable is standard(-ish).  I
don't think any usable UDP-based protocol can avoid some error
handling (unless it's a protocol for something where data loss is
tolerable - media streams, for example).  Which doesn't mean that for
some protocols it's not utterly trivial: Didn't get your DNS response?
Send the query again.

I suspect we'll probably end up with a proper TCP, the advantages of
not having to deal with the client are significant, despite the
considerable extra complexity on the device.

Remember that the starting point here is little more than the
equivalent of fgets() or puts() support to a serial port (hardcoded
9600-N-8-1, no flow control).  You can do that (less any actual
command implemented) in under 500 LOCs. The *fancy* version assumes a
VT-100.  Going the TCP route with the Ethernet chip driver, IP, UDP,
DHCP, ICMP, and TCP, and you're talking well over 10KLOCs, with over
half that being in TCP.


>> With UDP (and a bit of consideration at the protocol layer), you can
>> even avoid any IP configuration requirements (at least on a single
>> LAN) by doing everything with broadcasts.
>
>Some IP stacks won't receive UDP broadcast packets with a
>non-reachable source IP.


The whole what-do-you-do-if-you-don't-have-an-IP-config thing is a
PITA for IPv4.  Hope for DHCP, then try a link-local address, then try
a fixed local address, then...  But that's an issue for any
implementation, but UDP offers some additional possibilities (like
broadcasts).  But nothing works everywhere.


>> I've simply never implemented anything with UDP that didn't take error
>> handling into account - is the common practice just to assume that you
>> will not lose/reorder/duplicate/mangle any packing so long as you're
>> on a local segment?
>
>Unless your Ethernet segment is pretty heavily loaded, I find you can
>ignore all that.
>
>If you can't, then you need the equivalent of TCP.
0
Robert
12/15/2016 5:58:31 AM
On Wed, 14 Dec 2016 15:22:42 -0800, Paul Rubin
<no.email@nospam.invalid> wrote:

>Robert Wessel <robertwessel2@yahoo.com> writes:
>> Well, yes, but the point is that TCP is a fair chunk of code, and for
>> this function it has to go in a spot where it's less than convenient
>> to update.
>
>I still don't see the problem: TCP has been around for decades and there
>are some tiny implementations, especially if you don't need multiple
>packets in the air at the same time etc.  Why not just use one?


That may well be the way we go, but given that more than a few
UPD-only implementations have been done, I was trying to determine if
a lighter-weight alternative was feasible.
0
Robert
12/15/2016 6:00:03 AM
On Thu, 15 Dec 2016 01:49:38 +0000 (UTC), antispam@math.uni.wroc.pl
wrote:

>Robert Wessel <robertwessel2@yahoo.com> wrote:
>> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
>> <invalid@invalid.invalid> wrote:
>> 
>> >On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>> >
>> >> OTOH, it's clear that a number of people have implemented a UDP-based
>> >> protocol for handling the sort of things often done with a serial
>> >> console attached to a device.  This has a number of advantages - you
>> >> can fake UPD over Ethernet with very little code, especially if you
>> >> define away things like fragmentation and options.  A simplified
>> >> TELNET-ish protocol on top of that would be pretty simple, you just
>> >> need some basic packet sequencing and acknowledgements.
>> >
>> >I know of people who have implemented that on top of IP rather than
>> >UDP.  They call it TCP. ;)
>> 
>> 
>> Well, yes, but the point is that TCP is a fair chunk of code, and for
>> this function it has to go in a spot where it's less than convenient
>> to update.  Hence the desire for maximum simplicity.  I used TFTP as
>> an example, since it solves a similar problem (and you could
>> certatinly trivially implemented a TCP version of TFTP), with similar
>> constraints.
>
>My experience with TFTP in early nineties was that on our
>network attempting to send file larger than 1MB will usually
>stall.  We had several segments of thin Etehrent connected
>via a repeater and used cheap equipement.  TCP run with
>quite resonable performance, but TFTP was essentially
>unusable.  Apparently TFTP specification was extended
>to get better reliability (or maybe our implemetation
>dropped optional parts), but it seems that using raw UDP
>will give you _more_ reliability than I got from TFTP.
>More preciesely, IIRC both at Ehernet level and IP level
>you get a checksum for each packet.  You are talkong
>about low performance, so presumably no need to throttle
>transmission.  At low speed in local network there
>should be no problem with reordering packets.  As my
>expample shows in practice TFTP was unable to retransmit
>missing packets.  Together this means that acknowledgements
>buy you nothing => you are end up at raw UDP level.
>
>> 
>> >> IOW, something with a relationship to TELNET sort of like TFTPs
>> >> relationship to FTP.
>> >
>> >Just use a raw TCP connection. There's no need to implement the TELNET
>> >protocol at all.  The Unix command line client will skip all the
>> >handshaking stuff if you use a non-standard port.  Or you can use
>> >netcat.  Or any number of terminal emulators or COM port redirectors
>> >that support raw TCP connections.
>> 
>> 
>> Given how little TELNET actually does (or at least, needs to do), if
>> you have TCP, there would be no really issue implementing that.  Raw
>> TCP is obviously an option, but again, the incremental cost to do
>> actual TELNET is low.  And as I said, if you have TCP, why not just do
>> this with HTTP, and really make things convenient for people.
>> 
>> 
>> >If you're on a local Ethernet segment, just use raw UDP.  I've been
>> >using both those approaches for close to 20 years, and have never had
>> >any problems.
>> 
>> 
>> Yet what exactly is the protocol?  How many characters can you put in
>> a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
>> on or off?  With UDP (and a bit of consideration at the protocol
>> layer), you can even avoid any IP configuration requirements (at least
>> on a single LAN) by doing everything with broadcasts.
>> 
>> I've simply never implemented anything with UDP that didn't take error
>> handling into account - is the common practice just to assume that you
>> will not lose/reorder/duplicate/mangle any packing so long as you're
>> on a local segment?  Again, TFTP addresses those issues, at least as
>> they apply to its requirements.
>
>Well, maybe modern TFTP is better, but IME TFTP did _not_ address
>the isssues.


Certainly bad TFTP implementations exist, but there's really nothing
in the protocol that should permit behavior as bad as you describe.


>I think you look at your problem from wrong point of view.  Once
>you want decent capability for transmitting streams of data you
>end up with something very similar to TCP.  So better question
>is: "what parts of typical TCP implementation can be safely
>skipped and which existing implementation has good support
>for doing so".  Partial answer is: TCP implementation can be
>simplified quite a lot and still work reasonably well.
>Commer and Stevens in their 1994 book described "complete"
>implementation of Internet protocls, their code was simple
>but they tried to cover all required parts.
>They wrote that they needed about 3000 LOC for TCP
>(and slightly larger amount for mamdatory lower level parts).
>If you limit their implementation to TCP only (+ needed lower
>level parts) it will by probably 60% bigger than UDP only (again
>UDP + needed lower level parts).  Compared to 1994 TCP
>was extended, but it seems safe to assume that even in
>modern network you should be able to use implementation
>having 6000 LOC (for TCP and lower layers, but exluding
>hardware handling).  This 6000 LOC-s is likely to
>produce no more than 8-16 KB of machine code.  RAM
>overhead is of order few hundreds bytes per connection
>plus needed buffering (likely few kilobytes).  If
>you want reliable transmission you need buffering
>(to be able to retransmit) so extra requirements
>seem to be quite modest.  And if you have enough free
>RAM you probly also have enough program space (typical
>MCU-s seem to have much more program space than RAM).
>Of course, you may use tiny MCU with external
>Ethernet (or Wifi) chip providing buffering, but
>if you add such a chip to existing system maybe
>it is better to add a "communication processor"
>that handles all network aspects (ESP8266 will
>do this for Wifi).


All at least roughly true.  As an aside, though, the LOC count is
wrong in the 1994 edition.  The pie chart is correct, but they didn't
update the LOCs from the 1991 edition, despite adding OSPF and some
other stuff to the implementation.  So from the 1991 edition, you had
26% of the 15KLOC base in TCP (or ~4K, which is about the right number
for their implementation), not 20% of 15KLOCs as in the 1994 edition
(where the count should actually be been ~19KLOCs).  The 4KLOCs in
their SNMP and RIP implementations are omitable.


>The TCP/IP stacks that I know about seem to be larger
>than estimate I gave above.  But they have options
>to omit parts, so maybe you can trim them down.


Yes, in my experience they've all been bigger than that.


>Significant part of complexity of TCP comes from
>handling long streams.  If you can live with small
>HTTP pages than there is very small implementation
>at tuxgraphics.org.  The implementation is quite
>small but they do it via limiting HTTP page to
>a single Ethernet packet.  This implementation
>violates several requirements of TCP/IP, but
>seem to work quite well in practice.


Not really an option, one packet is just going to be too small for
HTTP.
0
Robert
12/15/2016 6:17:15 AM
On Wed, 14 Dec 2016 15:29:47 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>On 12/14/2016 12:28 PM, Robert Wessel wrote:
>> Well, yes, but the point is that TCP is a fair chunk of code, and for
>> this function it has to go in a spot where it's less than convenient
>> to update.  Hence the desire for maximum simplicity.  I used TFTP as
>> an example, since it solves a similar problem (and you could
>> certatinly trivially implemented a TCP version of TFTP), with similar
>> constraints.
>
>TFTP runs using UDP datagrams.  It's simplicity lies in the fact
>that it has very few options (that need to be negotiated) and
>a window size of 1 packet:  next packet isn't sent until
>previous is ACK'd


Yes, that's why I mentioned it as an example of a protocol solving a
similar problem.


>> Given how little TELNET actually does (or at least, needs to do), if
>> you have TCP, there would be no really issue implementing that.  Raw
>> TCP is obviously an option, but again, the incremental cost to do
>> actual TELNET is low.  And as I said, if you have TCP, why not just do
>> this with HTTP, and really make things convenient for people.
>
>Have you looked at an *actual* TELNETd implementation?  You're looking
>at several KLoC's.  You can pare this back by deciding to only
>advertise certain capabilities.  But, it's not a "weekend job"!


Yes, and I've actually done a couple.  And yes, if you just handle the
common TELNET commands (which you can mostly ignore, or handle
trivially), support a minimal number of options (ECHO, SGA, TERMTYPE,
and BINARY (maybe)?) and maybe accept and TERMINAL-TYPE you get from
an IAC/SB/TERMTYPE negotiation, it's pretty small.


>As you're *probably* not looking for maximum performance, minimum latency,
>etc. ("It's a serial port!"), you can probably deliberately tone down the
>implementation so it doesn't tax the fabric *nor* the clients involved
>(i.e., if you give a client plenty of time to process packet 1 before
>trying to send packet 2, there's a better chance that it won't DROP
>packet 2 and, thus, need to re-request it).


That would be the assumed protocol.


>[Historically, there have been NICs that would "not see" packets
>appearing too close to other packets on the wire -- think of it
>as "hardware bugs" necessitating software work-arounds]


Thank you for reminding me of 3c501s.  I had put that piece of crap
out of mind decades ago...  The 3c501 was special in that it had only
one I/O buffer, and it needed to be set to either transmit or receive
mode.  So if you sent a packet, you'd set it to transmit mode, push
the packet into the buffer, tell the NIC to send it, and then switch
the NIC into receive mode.  Unfortunately the switch was slow enough
that if whoever you sent the packet to was really fast to respond
(that required a better NIC than a 3c501 at the other end, of course),
it was not actually possible to get the local NIC into receive mode
fast enough!  So the "software workaround" was a delay at the other
end.

Of course the 3c501 had issues with too many consecutive receives as
well.
0
Robert
12/15/2016 6:34:10 AM
On Wed, 14 Dec 2016 15:09:13 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

>> Physically, the idea would be to replace the serial port with an
>> Ethernet port.  It would be nice to be able to still connect without
>> any network infrastructure at all (beyond, possibly, a crossover
>> Ethernet cable).
>
>It might be interesting to consider putting serial port *and* ethernet
>on the same connector (I'd have to consider the hardware consequences
>to see if there's a "cheap" way of doing so).


It somewhat depends on the connector.  RS-232 and Ethernet signals
won't damage each other's ports, so some incorrect connections are not
harmful.  If you stuck with a standard RJ-45, and 10/100 Ethernet, you
have four unused pins, which is likely enough for the RS-232 side. You
could then use a standard twisted pair Ethernet patch cable for the
Ethernet connection, and a special cable for the serial side. While
the RS-232 pins would be connected to the far device (typically the
switch) in the Ethernet scenario, the extra pins should not get used,
since the device won't negotiate anything faster than 100Mb, so the
remote side should ignore anything it sees on the other four wires.

A problem with that is that usually the inexpensive way to get the
RJ-45 connector onto your device is to get a jack that integrates the
magnetics.  And those tend not to make the extra pins available.  So
you'd probably have to use discrete magnetics.

If you were willing to go to a less standard connector (perhaps an
RJ-50 - aka 10P10C), you'd have enough signals for handshaking on the
serial port as well, but you'd need custom cables for both connection
types.  And of course you'd need separate magnetics.
0
Robert
12/15/2016 6:50:52 AM
On 12/14/2016 11:34 PM, Robert Wessel wrote:
>>> Given how little TELNET actually does (or at least, needs to do), if
>>> you have TCP, there would be no really issue implementing that.  Raw
>>> TCP is obviously an option, but again, the incremental cost to do
>>> actual TELNET is low.  And as I said, if you have TCP, why not just do
>>> this with HTTP, and really make things convenient for people.
>>
>> Have you looked at an *actual* TELNETd implementation?  You're looking
>> at several KLoC's.  You can pare this back by deciding to only
>> advertise certain capabilities.  But, it's not a "weekend job"!
>
> Yes, and I've actually done a couple.  And yes, if you just handle the
> common TELNET commands (which you can mostly ignore, or handle
> trivially), support a minimal number of options (ECHO, SGA, TERMTYPE,
> and BINARY (maybe)?) and maybe accept and TERMINAL-TYPE you get from
> an IAC/SB/TERMTYPE negotiation, it's pretty small.

Given that you're talking about a "console" in a "data center" (as
one application domain), I'd also consider support for encryption (not a
"standard" option in many telnet clients).  Else you risk someone
eavesdropping on the connection(s) and harvesting (console) passwords,
traffic, etc.

I.e., you might find ssh/sshd better (and more modern) alternatives.

Of course, this brings key management into the picture, etc.  Tug
on a loose thread and you never know where it will lead!  :-/

If "campus" is a generic term (i.e., not specifically tied to
academic institutions), then you're faced with a network that
potentially spans multiple buildings/floors/departments/etc.
This suggests internal firewalls, routers, etc.  (which makes
a UDP solution more problematic -- unless you station a "special
host" in each such network segment and let *them* talk to each
otehr).

You'll also have some sort of IT department with which to contend.
And, "local policies".  I.e., certain ports/protocols might be
blocked as a matter of course.

This suggests you may need to offer additional configuration
options for each node.  Which begs the issue of how you maintain
and distribute that configuration information.

It might lead to using something like a BOOTPd -- or DHCPd as the
more modern/flexible alternative.  If you're expecting *many* of these
at a site, then maintenance approaches that lend themselves to
applying a template to a whole set of devices (e.g., a DHCP "class")
can have considerable appeal to the bloke charged with keeping
them all running, NOT having to read MACs printed in 3pt type
off of labels affixed to the underside of devices, etc.

The point of all this is that the cost of TCP might be a small
fraction of the total "consequences" of adopting an ethernet
solution (over a more straight-forward "serial port" approach).

So, instead, you might try approaching the problem with a different
set of assumptions:  assume you *have* a full stack (with some
particular set of capabilities, yet to be quantified).  With that,
what might you do differently?  And, why?  Is it still appropriate
to think of it as providing a "serial port for interactive use"?
Or, might there be other features for which you can leverage this
capability?

>> As you're *probably* not looking for maximum performance, minimum latency,
>> etc. ("It's a serial port!"), you can probably deliberately tone down the
>> implementation so it doesn't tax the fabric *nor* the clients involved
>> (i.e., if you give a client plenty of time to process packet 1 before
>> trying to send packet 2, there's a better chance that it won't DROP
>> packet 2 and, thus, need to re-request it).
>
> That would be the assumed protocol.

That is how RLOGIN works (but, over TCP)

>> [Historically, there have been NICs that would "not see" packets
>> appearing too close to other packets on the wire -- think of it
>> as "hardware bugs" necessitating software work-arounds]
>
> Thank you for reminding me of 3c501s.  I had put that piece of crap
> out of mind decades ago...  The 3c501 was special in that it had only
> one I/O buffer, and it needed to be set to either transmit or receive
> mode.  So if you sent a packet, you'd set it to transmit mode, push
> the packet into the buffer, tell the NIC to send it, and then switch
> the NIC into receive mode.  Unfortunately the switch was slow enough
> that if whoever you sent the packet to was really fast to respond
> (that required a better NIC than a 3c501 at the other end, of course),
> it was not actually possible to get the local NIC into receive mode
> fast enough!  So the "software workaround" was a delay at the other
> end.
>
> Of course the 3c501 had issues with too many consecutive receives as
> well.

Actually, I was thinking at the component level -- specifically,
the LANCE.

But, my point was for you to fold your hardware considerations
into the design of the stack -- instead of trying to work-around them
outside its scope.  I'm guessing you're targeting a SoC implementation...?
A peek at any outstanding "issues" for the chosen target might save
you hassles, later.
0
Don
12/15/2016 8:10:09 AM
On Tue, 13 Dec 2016 04:23:16 -0600, Robert Wessel
<robertwessel2@yahoo.com> wrote:

>Many devices have had a serial port for attaching a console of some
>sort, for low-level configuration/diagnostic work.  While very simple
>to implement, serial ports have issues which don't need repeating.
>
>An obvious alternative is to implement the "serial console" as a
>TELNET server.  Conceptually this is trivial, not to mention well
>standardized and well understood.  OTOH, this piles on a fair bit of
>complexity, and requires something akin to a full TCP/IP stack.  And
>if you're going to do that, you might as well just go for a web-based
>admin, it's only going to be a bit more work.

If you just need a primitive command line debugger for a small device
with ethernet port but no serial port, the simplest would be to hack
the printf() in the small device with a routine that sends a raw
Ethernet package or an UDP packet. 

On the bigger client machine, you would need a routine that reads a
command line (e.g. to facilitate command line editing) and when the
user hits the CR key, a raw ethernet or UDP frame is sent and
processed by the small server. 

An even simpler case, setup Wireshark capture filter to listen for the
MAC address of the small device. In this case, the only thing that the
"telnet" needs to do is to read a command line and send it out as a
raw ethernet or UDP frame. 

If there is only one connection point, raw Ethernet with MAC addresses
should be enough, but if multiple command points are needed in the
future, including the IP header with port numbers will help, but then
you would have to implement the ARP o the small device.

0
upsidedown
12/15/2016 10:56:59 AM
On 2016-12-15, Paul Rubin <no.email@nospam.invalid> wrote:
> Grant Edwards <invalid@invalid.invalid> writes:
>> you've spent more time than you would have just doing a simple TCP
>> implementation.
>
> I still don't understand why the OP would want to "do" (i.e. write) a
> simple TCP implementation instead of downloading one.

The OP was complaining about how much work it would be to implement
TCP, so I was comparing that to what he was proposing.  Using an
open-source TCP/IP implementation makes the comparison come out even
more in favor of TCP.

> There are lots,
> e.g.:
>
> https://en.wikipedia.org/wiki/UIP_(micro_IP)

I've not used uIP, but it's by Adam Dunkels so it gets a few points
right away in favor.  I've used his protothreads library for stackless
multitasking on a uController with only a couple hundred bytes of RAM,
and it was very cool, very well done, and very useful.

-- 
Grant Edwards               grant.b.edwards        Yow! Why are these athletic
                                  at               shoe salesmen following
                              gmail.com            me??
0
Grant
12/15/2016 5:38:23 PM
On 12/14/2016 11:50 PM, Robert Wessel wrote:
> On Wed, 14 Dec 2016 15:09:13 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>>> Physically, the idea would be to replace the serial port with an
>>> Ethernet port.  It would be nice to be able to still connect without
>>> any network infrastructure at all (beyond, possibly, a crossover
>>> Ethernet cable).
>>
>> It might be interesting to consider putting serial port *and* ethernet
>> on the same connector (I'd have to consider the hardware consequences
>> to see if there's a "cheap" way of doing so).
>
> It somewhat depends on the connector.  RS-232 and Ethernet signals
> won't damage each other's ports, so some incorrect connections are not
> harmful.  If you stuck with a standard RJ-45, and 10/100 Ethernet, you
> have four unused pins, which is likely enough for the RS-232 side. You

I was thinking more in terms of *sharing* conductors between interfaces.
See my previous '"On-line" 100BaseTX hardware diagnostics' post as
its along the same lines -- using a second I/F to leverage the existing
*wiring* of the first in the event of a failure of the first.  I.e.,
ethernet has failed but can I still exploit the existing WIRING
to talk to other nodes -- albeit at a crippled rate.

The flip side of this is allowing either interface to appear on
a single connector, moving the "user's choice" into the
cable with which he connects to your device (one terminated in
a DB9/25, the other in an RJ45)

> could then use a standard twisted pair Ethernet patch cable for the
> Ethernet connection, and a special cable for the serial side. While
> the RS-232 pins would be connected to the far device (typically the
> switch) in the Ethernet scenario, the extra pins should not get used,
> since the device won't negotiate anything faster than 100Mb, so the
> remote side should ignore anything it sees on the other four wires.
>
> A problem with that is that usually the inexpensive way to get the
> RJ-45 connector onto your device is to get a jack that integrates the
> magnetics.  And those tend not to make the extra pins available.  So
> you'd probably have to use discrete magnetics.
>
> If you were willing to go to a less standard connector (perhaps an
> RJ-50 - aka 10P10C), you'd have enough signals for handshaking on the
> serial port as well, but you'd need custom cables for both connection
> types.  And of course you'd need separate magnetics.
>

0
Don
12/15/2016 6:01:22 PM
On 12/15/2016 3:56 AM, upsidedown@downunder.com wrote:
> On Tue, 13 Dec 2016 04:23:16 -0600, Robert Wessel
> <robertwessel2@yahoo.com> wrote:
>
>> Many devices have had a serial port for attaching a console of some
>> sort, for low-level configuration/diagnostic work.  While very simple
>> to implement, serial ports have issues which don't need repeating.
>>
>> An obvious alternative is to implement the "serial console" as a
>> TELNET server.  Conceptually this is trivial, not to mention well
>> standardized and well understood.  OTOH, this piles on a fair bit of
>> complexity, and requires something akin to a full TCP/IP stack.  And
>> if you're going to do that, you might as well just go for a web-based
>> admin, it's only going to be a bit more work.
>
> If you just need a primitive command line debugger for a small device
> with ethernet port but no serial port, the simplest would be to hack
> the printf() in the small device with a routine that sends a raw
> Ethernet package or an UDP packet.

My read of the OP's comments was that he wanted a means of talking with
the "command/control console" of his device, remotely, using network
fabric instead of a dedicated "serial" link.

E.g., in a data center environment, this would be used for LOM -- in
lieu of a "display and keyboard".

Old server farms were littered with KVM's and rack-mounted "terminals"
(fold-away LCD monitor plus keyboard wired to the KVM).  But, that required
users to be present in the cold aisle to "manage" the servers.

> On the bigger client machine, you would need a routine that reads a
> command line (e.g. to facilitate command line editing) and when the
> user hits the CR key, a raw ethernet or UDP frame is sent and
> processed by the small server.
>
> An even simpler case, setup Wireshark capture filter to listen for the
> MAC address of the small device. In this case, the only thing that the
> "telnet" needs to do is to read a command line and send it out as a
> raw ethernet or UDP frame.
>
> If there is only one connection point, raw Ethernet with MAC addresses
> should be enough, but if multiple command points are needed in the
> future, including the IP header with port numbers will help, but then
> you would have to implement the ARP o the small device.
>

0
Don
12/15/2016 6:06:19 PM
Robert Wessel wrote:
> Many devices have had a serial port for attaching a console of some
> sort, for low-level configuration/diagnostic work.  While very simple
> to implement, serial ports have issues which don't need repeating.
>
> An obvious alternative is to implement the "serial console" as a
> TELNET server.  Conceptually this is trivial, not to mention well
> standardized and well understood.  OTOH, this piles on a fair bit of
> complexity, and requires something akin to a full TCP/IP stack.  And
> if you're going to do that, you might as well just go for a web-based
> admin, it's only going to be a bit more work.
>

A bit??? I think you left a "quite" out. :) If the target
is Linux, then it's less trouble, but it's still somewhat involved.

> OTOH, it's clear that a number of people have implemented a UDP-based
> protocol for handling the sort of things often done with a serial
> console attached to a device.  This has a number of advantages - you
> can fake UPD over Ethernet with very little code, especially if you
> define away things like fragmentation and options.

Easily done.

> A simplified
> TELNET-ish protocol on top of that would be pretty simple, you just
> need some basic packet sequencing and acknowledgements.

I am not sure you even really need that. SNMP doesn't need it; it has
the "as if simultaneous" rule; when multiple PDus are used to define
an essentially atomic operation, they are to be treated as if they all
got there the same time. It's not assumed that the sender sent them in
order or without some other operation concurrently.

It is worth defining use cases carefully for such a protocol to keep
half-done things and race type[1] conditions to a minimum.

[1] unless you wish to assume only one session at a time.

Then again, there's nothing wrong with sequence numbers, but it's
sort of non-Telnet-ish if you do that.

> IOW, something
> with a relationship to TELNET sort of like TFTPs relationship to FTP.
>
> OTOH, despite looking, I just don't see any standards for doing that
> sort of thing.  Am I just missing it?  Is there a defacto standard
> even if there's no RFC?
>

There isn't one that I know of. But it is essentially quite easy.

-- 
Les Cargill
0
Les
12/16/2016 3:42:42 AM
Robert Wessel wrote:
> On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
> <invalid@invalid.invalid> wrote:
<snip>
>
> I've simply never implemented anything with UDP that didn't take error
> handling into account - is the common practice just to assume that you
> will not lose/reorder/duplicate/mangle any packing so long as you're
> on a local segment?  Again, TFTP addresses those issues, at least as
> they apply to its requirements.
>
>

If you are using UDP on a local segment, you're reasonably safe assuming 
things are in order and not fragmented unless you try to
push the envelope with large PDUs.

Over a router of unknown configuration? Not so much.

>>> OTOH, despite looking, I just don't see any standards for doing that
>>> sort of thing.  Am I just missing it?  Is there a defacto standard
>>> even if there's no RFC?
>>
>> If you want something reliable that works over WANs, use TCP: it's the
>> only way you're going to dependably make anything work through a
>> firewall.
>
>
> Sure, no disagreement there.  And no, operation over WANs is not a
> prime consideration, although it certainly wouldn't hurt if it could,
> even if the user had to deal with firewall issues, or set up a relay
> server of some sort.  But the scope would be within a data-center, or
> perhaps campus (assuming at least real IP address assignment - without
> that, local LAN).  Remember the idea here is to replace a serial
> console port.
>

Serial ports don't change the order of arrival or fragment :)

>
>> If you're going to assume local Ethernet, you can just use UDP.


-- 
Les Cargill
0
Les
12/16/2016 3:49:20 AM
Don Y wrote:
> On 12/14/2016 12:28 PM, Robert Wessel wrote:
<snip>
> TFTP without the "windowsize" option handles the problem just by requesting
> retransmits until you've "got" the next packet IN SEQUENCE.  Support
> for the windowsize forces the client to keep track of *which* packets
> it's received -- and tolerate the possibility that it might accumulate
> them out-of-order in how it frames its acknowledgements.
>

I am pretty sure bootp only works on local segments. Not 100% sure about 
TFTP, but I would be surprised if it survived fragmentation or
changes in sequencing.

<snip>

-- 
Les Cargill
0
Les
12/16/2016 3:52:42 AM
On 12/15/2016 8:52 PM, Les Cargill wrote:
> Don Y wrote:
>> On 12/14/2016 12:28 PM, Robert Wessel wrote:
> <snip>
>> TFTP without the "windowsize" option handles the problem just by requesting
>> retransmits until you've "got" the next packet IN SEQUENCE.  Support
>> for the windowsize forces the client to keep track of *which* packets
>> it's received -- and tolerate the possibility that it might accumulate
>> them out-of-order in how it frames its acknowledgements.
>
> I am pretty sure bootp only works on local segments. Not 100% sure about TFTP,
> but I would be surprised if it survived fragmentation or
> changes in sequencing.

Classic TFTP only allows one outstanding packet.  Later revisions
allow a window size to be negotiated -- essentially allowing
multiple packets to be on the wire (from the standpoint of the
client) simultaneously.  Client-side code decides which packets
to acknowledge and when.  So, if a packet 5 (of 1-8 in this window)
arrives later than the client "planned", 1-4 are acknowledged
causing 5-12 to be sent in the next window... even if the "first" 5
arrives just after the ack has been sent.

You can workaround fragmentation by negotiating a suitable packet
size (yet another extension to the original TFT protocol).

I.e., you can conceivably configure TFTP to bridge routers
*if* that's a deliberate goal.

BOOTP only addresses the idea of centralizing the devices'
configurations (instead of having to push them into the
device at "install time").  DHCP can achieve similar results
(in a more modern approach).

[Of course, you can also bastardize any number of protocols
to come up with your own approach -- "downloading" configuration
"files" that are named (by convention) with the MAC address
of the device in question, etc.  But, you want to be cognizant of
typical "best practices" so you don't pose a system configuration
headache for the network administrator(s)]
0
Don
12/16/2016 5:00:43 AM
In article <qs535ctbk361kav29rjd236dat22mo199e@4ax.com>,
Robert Wessel  <robertwessel2@yahoo.com> wrote:
>On Tue, 13 Dec 2016 15:16:12 +0000 (UTC), Grant Edwards
><invalid@invalid.invalid> wrote:
>
>>On 2016-12-13, Robert Wessel <robertwessel2@yahoo.com> wrote:
>>
>>> OTOH, it's clear that a number of people have implemented a UDP-based
>>> protocol for handling the sort of things often done with a serial
>>> console attached to a device.  This has a number of advantages - you
>>> can fake UPD over Ethernet with very little code, especially if you
>>> define away things like fragmentation and options.  A simplified
>>> TELNET-ish protocol on top of that would be pretty simple, you just
>>> need some basic packet sequencing and acknowledgements.
>>
>>I know of people who have implemented that on top of IP rather than
>>UDP.  They call it TCP. ;)
>
>
>Well, yes, but the point is that TCP is a fair chunk of code, and for
>this function it has to go in a spot where it's less than convenient
>to update.  Hence the desire for maximum simplicity.  I used TFTP as
>an example, since it solves a similar problem (and you could
>certatinly trivially implemented a TCP version of TFTP), with similar
>constraints.

A rudimentary "simple-tcp" that is not quite compliant with
TCP but will interoperate with one that is is not much code.
You will have to handle a connection or maybe a handful, but not
very many. You will have to do retransmissions, but you can back
off to a constant, semi-high value (like 6 seconds) on the second
attempt. You can deny all options, and stay with a small window
of a few k, and send 512 bytes or less.

That should reduce the requirements to a few hundred lines of
code beyond implementing UDP.

You can also have a very rudimentary telnet negotiation, basically
enforcing a standard set of options and accepting nothing else.

>>> IOW, something with a relationship to TELNET sort of like TFTPs
>>> relationship to FTP.
>>
>>Just use a raw TCP connection. There's no need to implement the TELNET
>>protocol at all.  The Unix command line client will skip all the
>>handshaking stuff if you use a non-standard port.  Or you can use
>>netcat.  Or any number of terminal emulators or COM port redirectors
>>that support raw TCP connections.
>
>
>Given how little TELNET actually does (or at least, needs to do), if
>you have TCP, there would be no really issue implementing that.  Raw
>TCP is obviously an option, but again, the incremental cost to do
>actual TELNET is low.  And as I said, if you have TCP, why not just do
>this with HTTP, and really make things convenient for people.
>
>
>>If you're on a local Ethernet segment, just use raw UDP.  I've been
>>using both those approaches for close to 20 years, and have never had
>>any problems.
>
>
>Yet what exactly is the protocol?  How many characters can you put in
>a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
>on or off?  With UDP (and a bit of consideration at the protocol
>layer), you can even avoid any IP configuration requirements (at least
>on a single LAN) by doing everything with broadcasts.

You probably have to have some default IP address, which you can
change. Making a simple udp one-shot protocol to set such options
should be easy.

>I've simply never implemented anything with UDP that didn't take error
>handling into account - is the common practice just to assume that you
>will not lose/reorder/duplicate/mangle any packing so long as you're
>on a local segment?  Again, TFTP addresses those issues, at least as
>they apply to its requirements.
>
>
>>> OTOH, despite looking, I just don't see any standards for doing that
>>> sort of thing.  Am I just missing it?  Is there a defacto standard
>>> even if there's no RFC?
>>
>>If you want something reliable that works over WANs, use TCP: it's the
>>only way you're going to dependably make anything work through a
>>firewall.
>
>
>Sure, no disagreement there.  And no, operation over WANs is not a
>prime consideration, although it certainly wouldn't hurt if it could,
>even if the user had to deal with firewall issues, or set up a relay
>server of some sort.  But the scope would be within a data-center, or
>perhaps campus (assuming at least real IP address assignment - without
>that, local LAN).  Remember the idea here is to replace a serial
>console port.

A "simple-TCP" will handle this, and you could make a udp simple
config-setter like the ones digium does for the iaxys should be
an afternoon's work.

>>If you're going to assume local Ethernet, you can just use UDP.

You will be amazed on how much chatter, duplications and other
stuff I have found on ethernets.

-- mrr



0
Morten
12/24/2016 3:10:34 PM
On 2016-12-24, Morten Reistad <first@last.name.invalid> wrote:

> You can also have a very rudimentary telnet negotiation, basically
> enforcing a standard set of options and accepting nothing else.

That turns out to be surprisingly difficult.  It's not a lot of code,
but no two clients behave the same, and the first time you trie a new
client, you find a new problem.

>>Yet what exactly is the protocol?  How many characters can you put in
>>a packet?  Any sequencing or recovery?  Acknowledgements?  Checksums
>>on or off?  With UDP (and a bit of consideration at the protocol
>>layer), you can even avoid any IP configuration requirements (at least
>>on a single LAN) by doing everything with broadcasts.
>
> You probably have to have some default IP address, which you can
> change. Making a simple udp one-shot protocol to set such options
> should be easy.

One clever method for establishing an initial IP address that I've
seen is that the embedded device watches for unicast IP packets sent
to its MAC address.  When it sees one, it temporarily adopts whatever
destination IP address was in the packet. That way, to initially
configure the device's IP, all you have to do is add an entry to your
host's ARP cache with the desired IP address and then ping that IP
address. IIRC, the device in question only did this if the device was
at factory defaults.

-- 
Grant
0
Grant
12/24/2016 6:53:28 PM
Reply: