Unix Programming - terminal/serial programming question, seem to be dropping data

Author

terminal/serial programming question, seem to be dropping data

af300wsm@gmail.com

2006-08-23, 7:27 pm

Hi,

I'm working on a program that "blasts" 2500 packets over a serial wire
to a remote host that is simply waiting to receive the pattern data.
Each packet is defined to be 255 bytes in length.

In debugging my program, I have the received packets dumped to a file
for inspection, and all is well until about packet 2496. At this
packet number, for some reason, all the data read from the serial port
are zeros. What could be causing this?

I have the termios structure configured for non-canonical mode. I'm
doing this via information obtained from:

http://www.freestandards.org/spec/r...fmakeraw-3.html

using the following operations:

ioPort.c_iflag &= ~(IGNBRK|BRKINT|PARMRK|ISTRIP
|INLCR|IGNCR|ICRNL|IXON);
ioPort.c_oflag &= ~OPOST;
ioPort.c_lflag &= ~(ECHO|ECHONL|ICANON|ISIG|IEXTEN);
ioPort.c_cflag &= ~(CSIZE|PARENB);
ioPort.c_cflag |= CS8;

Also, I'm setting VMIN = 255 and VTIME = 0, and I'm setting the
hardware flow control flag to on using the CRTSCTS macro. (Just to
make sure I'm not missing something obvious, if hardware flow control
is turned on, that means that the hardware will take care of flow
control entirely and there's nothing I must do ... correct?)

Any ideas as to why I would be seeing this?

Thanks,
Andy

David Schwartz

2006-08-24, 7:26 am

af300wsm@gmail.com wrote:

> I'm working on a program that "blasts" 2500 packets over a serial wire
> to a remote host that is simply waiting to receive the pattern data.
> Each packet is defined to be 255 bytes in length.
>
> In debugging my program, I have the received packets dumped to a file
> for inspection, and all is well until about packet 2496. At this
> packet number, for some reason, all the data read from the serial port
> are zeros. What could be causing this?

Do an experiment. Change the code to blast 5000 packets and see if it
still fails at packet 2496 or at packet 4996. Do you keep all the
packets in memory on the receiver or do you just keep the latest for an
instant before you write it out?

I'm thinking it's something like a buffer overflow where you actually
only allocate enough space for just less than 5,000 packets,
'read/recv' returns EFAULT, but you don't check correctly.

DS

af300wsm@gmail.com

2006-08-24, 1:38 pm

David Schwartz wrote:
> af300wsm@gmail.com wrote:
>
>
> Do an experiment. Change the code to blast 5000 packets and see if it
> still fails at packet 2496 or at packet 4996. Do you keep all the
> packets in memory on the receiver or do you just keep the latest for an
> instant before you write it out?

While developing, it came up that I might be wasting too much time
inspecting each packet as it arrives. So, I now stuff all packets into
memory and after the final packet is received, I inspect and write it
to disk. This may not be a good idea, huh?

>
> I'm thinking it's something like a buffer overflow where you actually
> only allocate enough space for just less than 5,000 packets,
> 'read/recv' returns EFAULT, but you don't check correctly.

I will look into this. By the way, is my assumption correct regarding
hardware flow control? Sorry to ask such a simple question, but I have
very little experience writing for serial devices.

Thanks,
Andy

David Schwartz

2006-08-24, 7:26 pm

af300wsm@gmail.com wrote:

> While developing, it came up that I might be wasting too much time
> inspecting each packet as it arrives. So, I now stuff all packets into
> memory and after the final packet is received, I inspect and write it
> to disk. This may not be a good idea, huh?

There's nothing inherently wrong with either approach. They just would
tend to have different types of typical programming errors. So I was
asking to try to narrow the search field for your problem, not to
advise one approach over the other.

[vbcol=seagreen]
> I will look into this. By the way, is my assumption correct regarding
> hardware flow control? Sorry to ask such a simple question, but I have
> very little experience writing for serial devices.

Yes. However, serial devices do not have any error checking or error
correction, and errors are not unheard of, especially at higher speeds.
You should not assume the serial link will be 100% reliable -- data
could be corrupted in transit.

DS

Eric Sosman

2006-08-24, 7:26 pm

David Schwartz wrote On 08/24/06 14:47,:
>
> Yes. However, serial devices do not have any error checking or error
> correction, and errors are not unheard of, especially at higher speeds.
> You should not assume the serial link will be 100% reliable -- data
> could be corrupted in transit.

Actually, serial devices have the ability to detect
parity errors (except most people run them without parity
bits) and framing errors, and most can also detect data
overrun errors.

But David's advice about the need for higher-level
error countermeasures is good. There's a reason people
feel they don't need parity-checking even on noisy lines,
and the reason is that they've found parity-checking to
be too weak a mechanism. They've therefore implemented
fancier schemes higher up the protocol stack, things like
dividing the data streams into packets with checksums.
Since the higher-level stuff provides better protection
than parity would, they then choose not to spend 9% of
the line's capacity sending useless parity bits!

--
Eric.Sosman@sun.com

af300wsm@gmail.com

2006-08-25, 1:26 pm

Eric Sosman wrote:
> David Schwartz wrote On 08/24/06 14:47,:
>
> Actually, serial devices have the ability to detect
> parity errors (except most people run them without parity
> bits) and framing errors, and most can also detect data
> overrun errors.
>
> But David's advice about the need for higher-level
> error countermeasures is good. There's a reason people
> feel they don't need parity-checking even on noisy lines,
> and the reason is that they've found parity-checking to
> be too weak a mechanism. They've therefore implemented
> fancier schemes higher up the protocol stack, things like
> dividing the data streams into packets with checksums.
> Since the higher-level stuff provides better protection
> than parity would, they then choose not to spend 9% of
> the line's capacity sending useless parity bits!
>

This makes sense. I now have two further questions.

1) How would I go about implementing such error detection scheme's?

2) Unrelated to the topic of question one, I know have my program
working between FreeBSD and Linux (although I'm still missing some
packets when Linux sends to FreeBSD). However, the target for this
work on is LynxOS on one side all the time and either FreeBSD or Linux
on the other. Currently, from Lynx to FreeBSD or Linux, I can receive
packets on the Lynx side, but I'm unable to send anything. That is,
when I send from FreeBSD or Linux, the LynxOS is getting data, but
neither FreeBSD or Linux receive anything even though Lynx believe's
it's transmitting. Any thoughts would be helpful.

Andy

Eric Sosman

2006-08-25, 7:28 pm

af300wsm@gmail.com wrote On 08/25/06 13:48,:
> Eric Sosman wrote:
>
> This makes sense. I now have two further questions.
>
> 1) How would I go about implementing such error detection scheme's?

If I were you, I'd use something already in existence
rather than trying to invent my own brand-new protocol. At
one extreme, you could run PPP and the IP suite over the serial
lines, much the way a lot of dial-up Internet connections
work. Definitely a heavyweight solution, but resistant to
errors -- and the pieces may be lying around on your Linux
machnes already.

Even if you decide to do your own implementation of a
lighter-weight solution, I'd suggest using a protocol with
some kind of track record. The old ancient original Xmodem
protocol made terribly inefficient use of the line and gave
only modest protection against errors, but it didn't take
much code to implement (it was designed, after all, for
machines too small to run enormous amounts of code). There
may be libraries you can use to implement Xmodem or one of
its improved successors like Zmodem, or if you're really a
do-it-yourself fan you could probably implement Xmodem from
the specs in a fairly short time.

--
Eric.Sosman@sun.com