Serial Port Fun

(last updated 2018-Sep-16)

Background

If there is ever a civilization-ending catastrophe, among the things that will survive are cockroaches and serial ports. You can never really get rid of them.

On my first programming job in 1994 the system we were working on had a main CPU board running pSOS on a 68040, and a few controller boards with slightly dumber 68000 chips connected to the main board with "silver satin" RS232 cables. I had my hands in the serial port code on that project.

I've made numerous 3-wire TX/RX/GND connectors and cables over the years for different embedded boards, soldered and breadboarded a few MAX232 level shifter circuits, and had daisy chains of weird adapters to match male/female and 9/25 pin cables and ports.

As recently as 2017 I was working on a product on which we added programmable LED strip lights for visual effects, and our supplier provided a simple controller with... a serial port interface.

A new project

A recent hobby project of mine used a Raspberry PI as a controller, and I had a few USB ports left available, so I wondered if I could do something with them. In my collection of assorted hardware I found a DIGI Watchport/T temperature sensor. I mounted it on the wall near the RPi,

It connects via USB, and appears to the operating system as a serial port.

[    6.516777] usb 1-1.5: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    6.516809] usb 1-1.5: SerialNumber: V32412288-0
[    6.686527] usbcore: registered new interface driver usbserial
[    6.686715] usbcore: registered new interface driver usbserial_generic
[    6.686836] usbserial: USB Serial support registered for generic
[    6.708544] usbserial: USB Serial support registered for Edgeport TI 1 port adapter

Like many USB devices, it required some proprietary firmware to be downloaded, in this case,

/lib/firmware/edgeport/down3.bin

Once I got that in place, it appeared reliably on boot, and I tested the simple command/respose protocol using Minicom,

TF (get temperature in Fahrenheit)
+74.9375F

Great! So I wrote a small program called "iotemp" to open the serial port and poll the temperature every 5 seconds. I have plans of tracking this data over time (weeks, months, years), because I'm curious to see how the basement temperature tracks or lags behind outside temperature. This might be a good opportunity to learn a few AWS services, or I'll just roll something of my own. But I have to start simple at the source, reading the temperature data reliably.

I was mildly disappointed, but not surprised, when I rebooted the RPi and found that my little program did not work. I'd get a single temperature reading, then the device would respond with "Invalid Command" thereafter.

Serial ports (and the serial port layer in Linux or any OS) are weird, finicky things with lots of history behind them and corresponding arcane settings. You don't just open up the device and read/write, you have to consider things like baud rate, control flow (hardware or software), and newline character translation. Before the internet and widespread TCP/IP networks, most communication was over serial ports and modems, and instead of computers there were "dumb" terminals that just provided a keyboard and a screen with enough memory to hold 80x25 characters (I used a "terminal emulator" program called ProComm to connect to the CS department's VAX 3300 to do my CS101 homework in 1990). Because of lack of standardization, allowances had to be made for terminals with different vendor-specific behaviors. Every OS with serial port support has a programming interface for dealing with these behaviors, and on Linux it is called the "terminal i/o subsystem" and is documented in the termios man page. Here is a sample of some of the dozens of flags that can be turned on and off,

   INLCR  Translate NL to CR on input.
   IGNCR  Ignore carriage return on input.
   ICRNL  Translate carriage return to newline on input (unless IGNCR is set).
   IUCLC  (not in POSIX) Map uppercase characters to lowercase on input.
   ONLCR  (XSI) Map NL to CR-NL on output.
   ONOCR  Don't output CR at column 0.
   TOSTOP Send  the SIGTTOU signal to the process group of a background process which
          tries to write to its controlling terminal.
   CSIZE  Character size mask.  Values are CS5, CS6, CS7, or CS8.

And look at this one,

   PARMRK If  this  bit  is set, input bytes with parity or framing errors are marked
          when passed to the program.  This bit is meaningful only when INPCK is  set
          and IGNPAR is not set.  The way erroneous bytes are marked is with two pre‐
          ceding bytes, \377 and \0.  Thus, the program actually  reads  three  bytes
          for one erroneous byte received from the terminal.  If a valid byte has the
          value \377, and ISTRIP (see below) is not set, the program might confuse it
          with the prefix that marks a parity error.  Therefore, a valid byte \377 is
          passed to the program as two bytes, \377 \377, in this case.

          If neither IGNPAR nor PARMRK is set, read a character with a  parity  error
          or framing error as \0.

Just, wow.

Meanwhile, back in my little program that didn't work after a reboot, I remembered from past experience that Minicom usually set things up in a good way and left them there even after exit, and running it once would allow simpler programs like mine to "just work", at least until the system was rebooted. I confirmed this by starting and exiting Minicom, and indeed that fixed the problem, "iotemp" could then read the temperature over and over again. But I didn't want to have to run minicom every time I rebooted, so I decided to dig down and see if I could figure out what Minicom was doing and replicate it in my own code. I stripped "iotemp" down to a bare minimum main function,

int main()
{
  struct termios settings;

  int fd = open("/dev/serial/by-id/usb-Inside_Out_Networks_Watchport_T_V32412288-0-if00-port0", O_RDWR);
  if (fd == -1) {
    fprintf(stderr, "Could not open serial port\n");
    return 1;
  }

  if (tcgetattr(fd, &settings) != 0) {
    perror("tcgetattr");
  }
  else {
    tdump(&settings);
  }
  close(fd);
  return 0;
}

And added a function to dump every value in the "struct termios" bitfield variables, which include,

       tcflag_t c_iflag;      /* input modes */
       tcflag_t c_oflag;      /* output modes */
       tcflag_t c_cflag;      /* control modes */
       tcflag_t c_lflag;      /* local modes */

where tcflag_t is an unsigned int.

#define d(field, f) do { printf("%s %s=%08x\n", #field, #f, settings->field & f ); } while (0);
void tdump(struct termios * settings)
{
  // c_iflag
  d(c_iflag, IGNBRK);
  d(c_iflag, BRKINT);
  d(c_iflag, IGNPAR);
  d(c_iflag, PARMRK);
  d(c_iflag, INPCK);
  d(c_iflag, ISTRIP);
  d(c_iflag, INLCR);
  d(c_iflag, IGNCR);
  d(c_iflag, ICRNL);
  d(c_iflag, IUCLC);
  d(c_iflag, IXON);
  d(c_iflag, IXANY);
  d(c_iflag, IXOFF);
  d(c_iflag, IMAXBEL);
  d(c_iflag, IUTF8);
  ...
  // c_oflag
  ...
  // c_cflag
  ...
  // c_lflag
  ...
}

I started learning Golang recently, and as much as I like it, I still love and marvel at C's ability to get things done. I don't think I could have done the above in so little code without C's wonky preprocessor.

So I ran this program before and after running Minicom, capturing the results with shell redirection, and compared the results,

$ ./dump-terminal-settings > ts.0
$ minicom
(exit minicom)
$ ./dump-terminal-settings > ts.1
$ diff -y ts.0 ts.1 | grep '|' (compare results side-by-side)
c_iflag IGNBRK=00000000   | c_iflag IGNBRK=00000001
c_iflag ICRNL=00000100    | c_iflag ICRNL=00000000
c_iflag IXON=00000400     | c_iflag IXON=00000000
c_oflag OPOST=00000001    | c_oflag OPOST=00000000
c_oflag ONLCR=00000004    | c_oflag ONLCR=00000000
c_cflag HUPCL=00000400    | c_cflag HUPCL=00000000
c_lflag ISIG=00000001     | c_lflag ISIG=00000000
c_lflag ICANON=00000002   | c_lflag ICANON=00000000
c_lflag ECHO=00000008     | c_lflag ECHO=00000000
c_lflag ECHOE=00000010    | c_lflag ECHOE=00000000
c_lflag ECHOK=00000020    | c_lflag ECHOK=00000000
c_lflag ECHOCTL=00000200  | c_lflag ECHOCTL=00000000
c_lflag ECHOKE=00000800   | c_lflag ECHOKE=00000000

And there we have it. Minicom turns IGNBRK on, and 12 other flags off. In the iotemp.c source, I was able to effect the same settings using the classic "design pattern" of read/copy/modify/write,

  if (tcgetattr(fd, &saved_settings) != 0) { perror("tcgetattr"); }
  new_settings = saved_settings;
  new_settings.c_iflag |= (IGNBRK);
  new_settings.c_iflag &= ~(ICRNL|IXON);
  new_settings.c_oflag &= ~(OPOST|ONLCR);
  new_settings.c_cflag &= ~(HUPCL);
  new_settings.c_lflag &= ~(ISIG|ICANON|ECHO|ECHOE|ECHOK|ECHOCTL|ECHOKE|IEXTEN);
  if (tcsetattr(fd, TCSANOW, &new_settings) != 0) { perror("tcsetattr"); }

And now iotemp works as expected after a reboot!

$ ./iotemp
+75.0625F
+75.0625F
+75.0625F
+75.0625F

I hope to do something with long-term logging later, that might be a subject for another tech note.

Side story, device name by id

Linux enumerates devices and creates device nodes dynamically, so if you have more than one of the same or similar devices, you might end up with a non-deterministic ordering of /dev nodes, for example,

/dev/ttyUSB0
/dev/ttyUSB1
/dev/ttyUSB2
...

For devices with unique serial numbers, Linux generates symlinks so that if you know the serial number you can always find and open a specific device. In this case, my Watchport/T sensor (assuming it is plugged in) always appears at,

/dev/serial/by-id/usb-Inside_Out_Networks_Watchport_T_V32412288-0-if00-port0

And I didn't have to do any magic to find that path, simply plug it in and do,

$ ls /dev/serial/by-id/

then copy/paste the path.