Thursday, July 6, 2017

Computer Contest Logging in 1977

As this year's IARU Radiosport contest approaches my thoughts turn back to the very first one. It was in 1977, 40 years ago. As I recall how it came to be the ARRL had such great success with the one time running of the Bicentennial contest in July 1976 that they saw an appetite for a summertime international contest. They must have been right since it's still going strong.

Back in 1977 I was in university working on my B.Sc. in Computer Science and on the cusp of exiting my teen years. I was an enthusiastic contester with a small home station and I was a member of club station VE4UM at the University of Manitoba. Since 1977 was the university's centennial we got a most unusual special call sign: VC9UM. At the time there was no 9th call district in Canada; it is now assigned to New Brunswick.

Dupe sheet from my 1970s files: contest unknown; the other side has US 6 to 0 and VE; one sheet per band
Every contester who predates K1EA's CT has painful memories of paper logging and dupe sheets. It was easily the worst aspect of contesting. There were a talented few who could simultaneously operate a keyer and write the log and use a dupe sheet. The rest of us cursed as the rate picked up and we scrambled to manually check the dupe sheet, fill it in (or chase away the caller) and log the QSO. Big gun multi-ops often had one person on the radio while another managed the log and dupe sheets. Unlike today dupes typically involved a penalty, and you didn't want that.

Imsai 8080: Not good enough, unfortunately (photo credit)
Personal computers at the time were wholly unsuited to the task of contest logging and dupe checking. Hams were experimenting with the simple home computers then available, such as the IMSAI 8080, using teletype machines as terminals and paper tape for storage. Soon enough the PC (including an updated IMSAI) would have the capability but in 1977 it was still a few short years in the future. The major impediments included:
  • No operating system: All I/O and other core tasks had to be manually coded.
  • No high level programming languages: It was all machine code or symbolic (assembly) machine language, or cross-compilers that ran on mainframes and minicomputers and downloaded to microcomputers.
  • No direct access to mass storage: Even floppy disks were not yet available, let alone anything larger. Programs were typically stored on and loaded from paper tape, either on the teletype terminal or the recently available electronic paper tape readers.
  • Insufficient RAM: Directly accessible semiconductor was typically no more than 4 KB. Try to fit a contest log and the logging software into that!
  • It cost money and we didn't have any. We were poor students barely scraping by.
Nevertheless computer logging was being done by a few. The early baby boomer contesters were by that time in their late 20s and working in industry. Many were engineers or an early generation of computer programmers and worked in large companies or other organizations with mainframes and minicomputers. An enterprising few were able to negotiate personal use of those machines over contest weekends.

This was no small accomplishment since those computers filled machine rooms and were expensive to power and required salaried operators to keep them running smoothly. It was typical for these computers to only be run during regular business hours except when dedicated to supporting clients with longer hours.

An idea is born

A friend and I talked it over. He -- Derrick VE4VV (SK) -- was also a fervent contester and a fellow student one year behind. We were too young and lacked influence to convince anyone with any power to grant us use of the university IBM S/370 mainframe or one of the several departmental PDP-11 minicomputers scattered across campus. I did try. The furthest I got was polite interest. The needs of graduate students and faculty would not be adjusted to suit my requirements.

From my personal library -- I enjoy hanging on to a few mementos
Since it was summer break I was working for a government agency as a systems and application programmer. I regularly used the a government owned S/370 and we had our own PDP-11/45 minicomputer. My attention focussed on that minicomputer. Like any machine that size it was turned off at the end of every workday. Could I get it left on for the weekend? That was my challenge. Software development was the least of my worries.

I shared my desire with my supervisor. Although he had no power to grant my request he became a strong ally. He was not a ham but had an interest in it. He also liked the idea of getting management to do something unconventional, to kick them out of their comfort zone so to speak. He talked it over with his boss. It took some persuasion to at least turn him neutral on the project. But it was his boss who would have to make the decision and he was a stereotypical senior government bureaucrat. He was not a mean person but one with a very narrow and well-defined comfort zone.

Of course he refused. Not only would it involve an expense for no sensible reason he could fathom it was a valuable government resource he was in effect turning over to a couple of teenagers. He saw it as risking unwelcome public exposure for him and his political masters should something go wrong. It was pointed out to him that I was in position every hour of every day to do anything at all with that computer, including access to confidential data and an ability to severely disrupt operations.

It took a few days when to my surprise and delight I received grudging approval. I never found out what was said to finally clinch the deal and at the time I didn't much care. What I did care about was that in fact we had no plan, no software and no idea how it was all supposed to happen in the few days we had until the contest started.

Putting a plan together

We had access to a minicomputer, but that was it. Every other problem was entirely up to us to deal with by dint of hard work. We needed remote access to the PDP-11/45 since downtown Winnipeg and the university are 10 km apart. The only possibility was 300 bps dial-up telephone modems of which we had two on the minicomputer for staff use.

Acoustic modems I've used (Columbia University)
If you've never heard of these contraptions they involved dialing the computer modem on a telephone then inserting the handset into the "ears" of the modem when you heard the carrier tone. If all went well the modem would detect the carrier and establish the data link. They were terribly unreliable and often dropped calls or failed to connect on the first attempt. At least we were only calling across town. Long distance calls and especially overseas calls often experienced phase shifting (it was almost all analogue transmission at the time) which was not tolerated well by the modems.

The terminal itself was a challenge. After exploring a couple of alternatives we borrowed a VT52 from my office and transported it to the club station. This wasn't easy since these "dumb" terminals were large, heavy and fragile. I didn't want to think what would happen to me if I dropped it or bumped into a wall as my friend raced ahead opening doors for me and guiding me around obstacles on the downtown streets towards his car.

VT52 from Wikipedia
It was remarkable that we got the terminal and communications link working at all. For the duration of the contest Derrick only experienced two line drops.

Software constraints and features

As already mentioned, when I got approval I had not yet written a single line of code. There was only a little over a week to develop a contest logging application from scratch. I didn't even have a clear idea of how to do it, what the user interface would look like and how I would deal with bugs and other disasters. All of this needed to be fleshed out, and quickly.

People are often surprised that many poets do not feel constrained by the need to fit their thoughts and emotions into a strict meter and rhyme template. Although a challenge it limits the range of possibilities to a solution space that is more tractable. My dilemma was much the same. I was limited by an interface that could paint no more than 30 characters per second on the screen, a couple of bulky hard disk drives and 32 KB of address space (the computer had 80 KB of magnetic core RAM, but each process was more limited by the system architecture). It simplified the problem by limiting me to what would fit these constraints.

The first thing to recall is that there was no such thing as computer control of rigs. Back then all transceivers were dumb. Neither were there integrated keyers. Well, that's not entirely true but very uncommon for contest software at the time since PCs were still terribly inadequate. There was no way a simple 300 bps ASCII connection to a dumb terminal in the shack was going to do anything like that. So the rig, keyer and microphone were manually operated.

All we were going to do was log and dupe. That's it. But that's a lot. If you're too young to remember those days it may not register just how difficult that was to do manually while operating, especially for a single op.

The software we decided would do the following, and nothing more:
  • Manually set the band and mode; Radiosport is a multi-mode contest.
  • Check for a dupe when a call is entered, and report log details for the earlier contact.
  • Enter an exchange and log the contact. The computer fills the time.
  • Check the band and mode, just in case.
  • Simple real time reports of contacts and zones logged.
  • Ability to back fill QSOs in case of temporary computer or communications outage.
  • Print log and dupe sheets and calculate score.
That's it. But as already said that's a lot.

The programming language was FORTRAN. Only that and assembly code were available. The DEC compiler for the RSX-11D operating system on the PDP-11/45 had libraries to support use of the data management services on the various storage devices. For our purpose these were two disk drives of 20 MB storage that were the size and weight of large washing machines. I forget the model number but they were roughly equivalent to the IBM 3330. Access time was not fast; that was a critical consideration if we were to achieve acceptable performance.

The VT52 has a graphics mode whereby the 24x80 character matrix could be directly manipulated. I eschewed that option since it added complexity and didn't seem to add much value for our limited feature set. I stuck with a scrolling text screen to keep it simple. It worked surprisingly well for the operator.

Development phase 1

With the clock running down I soon realized not all the planned features could be developed in time. Yet I was able to get the core functionality working within a few days, designing and coding in my spare time and, to be honest, during work hours. My supervisor knew what I was up to but chose to say nothing since my work assignments were progressing on schedule. More days were consumed in tweaking the software and data management to boost performance.

My design choices were greatly limited by the technology available to me. That a program could only be 32 KB, including code and dynamic data storage forced my hand. Think about the length of a call sign. Let's assume 16 bytes per log entry for a call sign, exchange data and time stamp. A log with 1,000 QSOs requires 16 KB, or half the address space. I had to support more than twice that number of QSOs and still have room for the program and the library routines that the compiler would include.

I investigate a variety of data compression algorithms. Unfortunately all involved substantial complexity. Then I discovered the program was consuming more space than I'd hoped for. I thus opted to put the log on the hard disk and optimize access with extensive hash tables and other techniques.

Happily it worked quite well during testing. Derrick came in one evening and I had him log random QSOs for a couple of hours to test the data base process. Afterward I analyzed the tables and adjusted the hash algorithm. Performance depended heavily on the QSOs being reasonably evenly distributed by a randomized hash key derived from the call sign.

For disasters in which the log data base was lost due to system or program failure I stored a plain text journal file on the second disk drive, writing to it after a QSO was logged. Although we never needed it there was comfort in knowing it was available. The journal file could only be used for post-contest log recovery since I had not yet developed code to recreate the log from the journal file.

User interface

The user interface was extremely simple, and surprisingly effective despite the simplicity. After connecting to the minicomputer the user signs in and executes the logging program. There was no setup to be done since only the one contest was supported. In case of a crash you run it again and carry on logging.

Whether running or S & P the UI was the same: enter a call sign and hit return. If it isn't a dupe nothing happens except for another prompt. You then enter the exchange, signal report optional. A dupe was flagged with log details and the sound of the terminal buzzer. It might look something like this (I forget the details so this is not precisely as it was). Plain text (not bold) is user input:
>=b20
>=c
>k3lr
>k3fr

>8
0334
>g3aa
>57927
0336
>f6abc
DUPE: *** F6ABC *** 1/2315 20 CW
>=s
>ve1xx
>9
0341
The first two commands (user input with the "=" prefix) set the band to 20 meters and mode to CW. Then a call sign is entered. The operator either made a mistake or didn't work the station and enters a new call sign. This action automatically erases the earlier entry.

The subsequent entry of an exchange (zone number) causes the call to be logged. Since no report was entered it defaults to 599 in the log. The time of the QSO is the only feedback given. Recall that we're working at 300 bps and brevity is important.

For the next call entered -- G3AA -- a signal report other than 599 is received. It is entered before the zone, just as it is sent by the the other station. The software parses the line to extract both items. The software assumes that 59 (or 599) is always sent so there is no way to log a different sent report.

The next call entered is a dupe. The log details are presented. The "1/" in front of the time indicates the day of the contest -- could be 1 or 2 since the first Radiosport was a 48 hour contest. The other data is parroted since the operator could have quite easily forgotten to change band or mode or typed the call incorrectly. Either way the operator proceeds to the next QSO. If it was a typo the correct call is entered followed by an exchange in the usual sequence.

Next, the operator changed mode to SSB and successfully logs an SSB contact.

The software did not fully validate call signs since the variety was too great and I didn't want to cause problems for the operator should the program improperly flag a valid call sign. Only simple errors such as a call beginning with a "0", no numeral present, trailing numeral and a few others were flagged. It was up to the operator to retype the call or ignore the warning and proceed to enter an exchange and log it. Although simple it worked well enough that we only found one badly formed call signed in the log after the contest.

There were a few other commands available. There was "=q" to report the number of QSOs in the log and "=z" to report the list of zones worked on the current band. Before entering an exchange a QSO time could be entered with something like "/20404" to override the computer clock and log the time as 0404Z on day 2 of the contest. This was needed in case of system outage so that manually written log entries could be put into the computer log during a break or after the contest.

I had a couple of debugging commands in there as well. These were of no use to the operator so I didn't even both to tell Derrick about them. If needed I could have walked him through their use over the phone.

The contest weekend

Late Friday afternoon Derrick drove over to my office to pick up me and the VT52. We had already acquired a modem from a helpful professor at the university -- it was summer and there was unused equipment available. I was given a key to the building front door and computer room by my boss. As the office closed for the weekend the operator shut down the computer per his procedures.

I restarted the minicomputer and ensured that it and the telephone ports were operational. I created a user account for Derrick to use. Then I shut the lights, locked the door and crossed my fingers. Off we went to the university to get him set up for the contest. I took a bus across town to my own home and station. Derrick was doing a single op, as was I from my own station. But I was on call in case disaster struck and I had to get downtown in a hurry to fix things. My attention was split between the contest and worrying that the phone would ring.

The phone never rang. That is, not until a few minutes after the contest ended. Derrick gave a enthusiastic report of the logging software. He couldn't stop laughing because it was the easiest and most relaxing contest he'd ever done. Apart from typing all he had to do was talk into the mic or press buttons on the memory keyer. Duping was a suddenly a relic of the past.

Since we were both tired from the contest we wrapped up and met the next day. In 1977 computer contest logging was a rarity. No one I personally knew had done it so I had no direct knowledge of how big a difference it would make. The difference was big, very big. We both realized right then that contesting would never be the same after the introduction of computers in the shack.

We got down to a serious post mortem discussion. Since he'd already practiced with it there was no learning curve to deal with during the contest. The limited feature set was itself a feature. He made a crib sheet with the commands written on it and found he didn't need it. We began talking of new features we could add.

A few problems did occur during the weekend though nothing too inconvenient. One was the expected line drops. Bell 103 modems with those bunny ears were not exceptionally reliable. We were lucky the problem only occurred twice. Each time he was able to redial and get back to business in a couple of minutes. He never had to resort to paper logging.

A bigger problem was that dupe checking became noticably slower with more than 1,000 QSOs in the log. Although the delay never got to be much more than 1 second in the heat of the contest it can seem an eternity. A few minutes of forensic analysis after the contest uncovered the problem. It had an easy solution. I counted myself lucky that nothing worse happened. The problem wouldn't have appeared if used at my small low power station since I made only 625 contacts in that first Radiosport (I actually found a copy of my old log).

The most delightful experience Derrick reported was the amazement from other hams, particularly when operating SSB, when he could not only tell stations they were dupes but the exact time of the QSO. Time and again they'd be suspicious and then astounded that the QSO was in their logs right where he said it was. Some even called back minutes later just to tell him he was right.

What they imagined was going on they likely never figured out and Derrick never told anyone on the air. Recall that this was 1977 and even technically savvy hams typically had zero experience and knowledge of computers. This was great fun!

Development phase 2

At the end of the contest we had a log and nothing more. I made backup tapes of the data base and software to protect our investment. Our next task was to print and score the log so that the VC9UM entry could be submitted. Those were features that I had deferred since they were not needed during the contest.

The log sheets were easy enough to format to look similar to the official ARRL/IARU log sheets. The one significant task was to parse the call signs in the log to determine the country and combine that with the zone (continent?) to calculate points for each QSO. At least that's the way I remember it. My memory of the scoring in that first Radiosport contest is hazy.

Dupe sheets were a problem. Of course there were no dupes since the software had already taken care of that. I went ahead and developed the features to score and print the log while Derrick made a few calls to the folks in Newington CT. There was an unanticipated difficulty.

Bureaucracy and the computer

The ARRL contest desk of 1977 refused to accept computer generated logs. They also demanded handwritten dupe sheets. This is despite the fact that the printed log sheets were nearly identical to the official sheets and there were no dupes. Both demands were nonsensical. Nevertheless they were unmoved by his protestations.

When it was pointed out to them that neither added any value, they would only point to the rules that declared entries must use the official log sheets (or copies of same) and that logs of over 200(?) QSOs must include dupe sheets. Flexibility was not in their vocabulary. That rigid compliance with the rules involved a substantial amount of work to manually transcribe the log and generate dupe sheets moved them not at all.

Derrick gave up the fight and spent a summer weekend doing what they demanded. It would be many years yet until the ARRL looked upon computer logs as helpful and even desirable. We were ahead of our time.

Aftermath

With a great success in our pockets we wanted to do it again. CQ WW was coming up in the fall and we set that as our goal. We had many ideas for improvements and new features that I was eager to build. This time it would be a multi-op effort so that I would not be left out of the fun.

It was not to be. Once I was back in school and no longer employed by the government the same senior bureaucrat was adamant that access to the PDP-11/45 was absolutely and categorically out of the question. My friends on the inside tried their best to no avail.

I made another play for the university department's PDP-11. It was even more impossible than before because the fall semester was in full swing. While there was still interest in what I was doing, and nothing sells better than proven success, they told me that, unfortunately, academic priorities would not and could not be set aside for even that one weekend.

That was the end of my foray into computer contest logging. I never did it again although I thought of it often.

Not long after that initial experiment I was working towards a post-graduate degree, and after that I moved across the country to start my professional career. PCs were by then very capable of contest logging and that's what many were doing. In my files I located notes from early 1980s of a far more sophisticated computer logging system. Those plans predated CT by a year. But I had no time for it and not even a station of my own to use it with.

When CT came out I was intrigued and thought of what might have been had I stayed with it. But by then my contesting activities were entering a lull as I put what little radio time I had into DXing and fooling around with antennas and modelling software. I did use CT a couple of times but that was for me and contest software for many years to come.

I missed a few generations of contest software and the computerization of radio equipment while I was absent from hobby between 1992 and 2013. When I made a serious return to contesting in 2014 I adopted N1MM Logger as my contest software from among the many excellent alternatives available. Three years on it remains my software of choice.

There is no reason at all for me to think about developing my own contesting software. I am more than happy to use the superb software that others have developed. I am content to be a user and put my energies into station building and operating.

No comments:

Post a Comment

All comments are moderated, and should appear within one day of submission.