Engineering Radio

My former employer thinks he knows better than anyone what to do in every given situation.  ”Mister,” as he is “affectionately” known, has a legendary cheap streak.  When I worked for the company, every year there would be a debate on whether we should plow and maintain the road to a certain transmitter site.  Mind you, this is not just any transmitter site, but the transmitter site of the number one billing station of the entire group of 35 stations.  It is located in the wealthy suburban setting of market Number One and bills more than most of their other markets combined.

Naturally, when I was there, I put up a stiff fight to make sure the road got patched and plowed.  Oh they would scream and nash their teeth about how unfair it was, and can’t we do this or do that, etc.  This went on every year for the entire ten years I worked for the company.  For my part, I just ignored it.  Back in October of last year, when I was first starting to see the handwriting on the wall, there was this clandestine meeting with the other residents on the road which I was not invited to.  You see, the lower half of the road has houses on it.  Mister thought that the residents of the road should chip in for the road plowing.  When they refused (because they were already plowing the lower part of the road themselves) he said we would absolutely, positively ,100% not be plowing the road this winter.

I departed the company in January.  Since then, the upper part of the road was not touched.  Then came last week’s blizzard.  Prior to the blizzard, the generator fuel tank was 9/10 full.  The power went out on Wednesday during the first storm.  I called the General Manager for the radio station on Friday and told him that the generator would need fuel soon and asked if the road had been plowed.  He said they were working on it and it should be done on Saturday.  I told him that we needed a fuel truck up there ASAP otherwise they would be going off the air.  He said he was on it.

Sunday morning at 6:30 am, the generator ran out of fuel.  Naturally, my phone rings.  I begin calling around all the fuel oil companies in the area to see if I can get a Sunday delivery.  I finally arrange something and we also get a 4WD pickup with a 100 gallon day tank to meet us there.  When I arrive at the site at 9:30 am, a backhoe was just starting to clear the upper part of the road.  The snow is knee deep and there is a layer of ice under it.  It took until about 1:30pm to get the road cleared enough to get the 4WD pickup, with tire chains near the generator to transfer fuel.  Then, because the fuel pump sucked air, we had to bleed the injectors, reset the faults, etc.  We finally got the generator started around 2:00pm.

So, let us compare costs:

Plowing the road cost about $800.00-$900.00 per storm.  This year, there were five to six storms where the road needed to be plowed. Total $4,500.00

Last Sunday, the station was down for about 7 hours.  I’d say that station likely bills $150.00 per unit on a Sunday morning, 10 units an hour so they lost $10,500.00 by being off the air.  Then there is the backhoe needed to clear the road.  A backhoe was needed because there was so much snow on the road that a regular snow plow could not move it, especially plowing up hill.  That cost $1,500.00.  Then there is my overtime and the guy with the 4WD pickup, another $1,440.00.  Total cost to plow the road and get the station back on the air, somewhere in the neighborhood of $13,440.00.

So, yeah, Mister is really saving money.  How’s that working out for you, a$$hat?

As promised, here is the AM transmitter site maintenance check list.  This is for a generic directional AM station with a backup transmitter, generator and an RF STL.

Broadcast Electronics AM6A transmitter

Usual disclaimers apply.

AM site Maintenance checklist

Weekly Maintenance:

A.  Visit site, Check following:

1. Check critical transmitter values against last logged value
2. Check forward/reflected power on main transmitter
3. Check and reset any overloads
4. Check signal strength on STL against last logged value
5. Check generator fuel level
6. General check of building, look in all rooms, inspect for damage from vandalism, Leaking roofs, obvious signs of trouble, take steps to correct.

Monthly Maintenance:

B.  Visit site, Check following:

1. Do a full multi-meter log, (includes tower phase angles, loop currents), run backup transmitter into dummy load.
2. Start and run generator for 5 minutes, check block heater, hoses, belts, oil and antifreeze levels
3. Calibrate remote control meters with transmitter meters, log it*
4. Check all tower fences for integrity and locked gates*
5. Complete Items 3, 4 and 5 under weekly maintenance.

Quarterly Maintenance:

C.  Visit site, Check following:

1. Complete 1 through 5 under monthly maintenance.
2. Check all air filters, clean or replace as needed.
3. Check frequencies of all transmitters, STL receiver, and log.
4. Complete quarterly tower lighting and painting inspection*

Bi-yearly Maintenance:

D.  Visit site, Check Following:

1. Complete 1 through 5 under quarterly maintenance.
2. Conduct monitor point readings for all directional antenna patterns*
3. Check base current readings for day/night towers.  Ratio.*
4. Clean backup transmitter
5. Place backup transmitter on air and clean main transmitter.

Yearly Maintenance:

E.  Check all licenses and authorizations for accuracy. Make sure that all renewal cards etc are in public file and are posted at control point.*

F.  Visit site, Check following

1. Complete 1 through 5 under Bi-yearly maintenance
2. Equipment performance measurements (NRSC, Harmonics, frequency)*
3. Complete service of generator
4. Complete Inspection of towers, check for vertical and plumb, check guy wire tensions, retension as needed.
5. Check property for anything out of the ordinary
6. Repair driveway as needed

General maintenance that is completed on an as needed basis

1. Re-fill fuel generator fuel tank when drops below 50 percent
2. Empty trash, sweep floors, dust.
3. Cut/remove vegetation inside tower fences, spray herbicide as needed
4. Water proof tower fences every 2 years
5. Paint exterior of building
6. Replace tower lights*
7. Paint towers*

*These are FCC inspection items, pay close attention if you do not want a fine.

That is it, a .pdf version of this file can be downloaded here.

Almost all radio stations use a tower of some sort to support their transmitting antennas.  These towers need maintenance from time to time and only qualified people should perform maintenance on towers.  Hence, the tower company is formed.

405 foot guyed tower with ERI FM antennas

Over my years of experience, I have dealt with many different tower companies, from one man operations to big corporations that have multiple crews out in the field on any given day.  I have discovered that not all tower companies are created equal.  Not only do tower climbers need to be in good physical shape and be trained correctly in all tower climbing safety procedures,  they also need to be good mechanics so they can actually repair things on the tower.   Climbing a 470 foot tower to repair a strobe light is all well and good.  Once the climber gets to the strobe light, he needs to be able to disassemble it without dropping parts or breaking things, trouble shoot if needed, install new parts and re-assemble the unit, again without dropping or breaking anything.

Applying a RF connectors, installing a FM antenna or STL antenna, repairing light fixtures or conduit all require some amount of manual dexterity and concentration.  Assembling high powered antenna requires close attention to detail.  Any pinched O rings, cross threaded bolts, bent bullets and the antenna will have problems, likely at the worst possible time.

The sign of a bad tower company is if it’s climbers cannot carry out those tasks with one or at most two climbs.  I have a situation on a tower where our FM station is a tenant.  The tower has a strobe light failure near the top of the tower where our FM antenna is located.  They have climbed the tower no less than four times to repair this, and it is still not fixed yet.  Each time they climb, the station has  to reduce power to protect the tower climbers from excessive RF exposure.  Each climb it takes them several hours longer than anticipated to finish their work.

A good rule of thumb, If the defective part cannot fixed in the first two climbs, then the entire strobe unit should be replaced on the third climb.  Even though the strobe units are expensive, by the time they get done paying for all this tower work, they could have bought two new strobes.  Today will be the fifth climb and there is no guarantee that it will be fixed.

I advised the tower owner that they should be looking around for another tower company because these guys aren’t exactly setting the world on fire.

Nautel V-10 Power Supply

Gone is the day when the radio station engineer had to trouble shoot down to the component level, often crawling in and out of transmitters to get at the suspected part.  I for one, spent many a long night at a transmitter site chasing some weird combination of symptoms down to the $0.34 1N914 diode in the directional coupler (see previous post about the MW-50).

It is a skill set now mostly confined to manufactures’ repair departments, for which they charge a pretty penny.  Nowadays, the technician simply slides out one module or circuit card and slides in another.  If that doesn’t fix it, panic ensues.  I know of several class C FM radio stations that are now relying on the computer guy to fix transmitters, because, you know, it’s cheaper.

To be fair, most engineers are contractors and many of those simply do not have the time to trouble shoot to the component level.  So, they ship everything back to the factory then pass the cost on to their client.

Then of course, most circuit boards these days are surface mount systems, which are hard to work on if you don’t have the right tools.  Normally an expensive temperature controlled soldering station is required, as well as a magnifying glass.

All of these things combine make circuit board work something to be outsourced.  Unfortunately, a night spent trouble shooting was often a great learning experience.  I have done some of my best work when my back was up against a wall and I was out of options.

I make the attempt to fix things locally, unless the transmitter or other item is under warranty or not having a spare/attempting to trouble shoot will take the station off the air.  I think it is important to keep abreast of technology and keep my trouble shooting skills up to par.  Besides, I find it gratifying that at least I can still fix things.

In the never ending saga of things I have not yet seen, we had a 350 kcml 3 phase underground feeder burn out yesterday at our AM transmitter site.

Actually, it happened the day before, around noon.  I received a call from the remote control that we were on the backup generator.  Upon arriving at the site, I found several trees down on the three phase primary down the street.  I figured that was the cause.  After checking the generator fuel supply, oil pressure, temperature, phase volts/amps, I decided that everything was okay and the power company would be along shortly to restore power.  I then continued up the road to our FM site to do weekly maintenance.

Upon returning to the office several hours later, I looked at the utility company website.  They have a pretty cool interactive map application that shows all outages and give restoration times.  The area around our transmitter site showed no outages, therefore I figured it had been cleared.

I committed two errors here:

1. Not calling the utility company myself to ensure that the outage was reported.  I assumed that the tree across the three phase primary was the cause, it was not.
2. Not calling the transmitter site remote control to check the generator status after I checked the website.

To be honest, I don’t know if I am coming or going these days.  With seven radio stations, each with it’s own transmitter site, three of them Directional AM stations, and three studio locations spread out over a 75 mile stretch, it is difficult to keep up with the small details.  Did I mention that I am solo, the engineering assistant position was cut two years ago.  But, I am not here to make excuses…

The net result is the generator ran all night long.  The next day, when I checked the transmitter in the morning, I was surprised to find the generator still running.  Unfortunately, I had an FM station on low power (see post below) that needed to be taken care of first.  When I finished replacing the RF module in the FM transmitter, I made my way to the AM site.

I called the power company then checked the generator fuel, the propane tank was down to 10%.  Yikes, better get this taken care of fast.  I will say the power company showed up pretty quickly.  After some measurements with a hand held meter, it was determined that the under ground feeder was open between the pole and the transmitter building.

The line man was not at all surprized, in fact, he called it before he even went up in the bucket truck.  After some back and forth with his supervisor, who came out in a pick up truck, it was decided that they would run a temporary overhead feed to the meter can.

temporary overhead electrical feeder

They also did some research in there records and discovered that they (the utility company) own the underground cable and therefore they would dig it up and fix it.  That’s nice because otherwise it seems like it would be an expensive repair job.  On a station that makes not a lot of money.

We have a Harris Z5-CD transmitter for one of our FM stations.  Brand H is not my preferred make, however, it was already installed when we bought the station, so I have to live with it.

This particular site gets hit by lightning strikes often.  Normally, it does not affect anything  until the transmitter gets turned off for maintenance.  Then, almost invariably, when turning the transmitter back on one of the modules will fail.  Most often this is manifest when one of the two power supplies shut down causing the transmitter to run no more than 20% power.

The way this is trouble shot is to slide each module out and turn the transmitter back on.  When the power supply stays on, the bad module has been located.  A confirmation test is to check the MOSFET for a short circuit between Drain and Source.  This short circuit condition puts a direct short on the power supply causing it to crow bar and turn off.

So, once the bad module has been located, and the spare module is installed in the transmitter, then what?  Most engineers call Harris and ship the module back for repair.  Most engineers don’t want to mess with unsoldering a surface mount MOSFET and soldering a new one in.   I find it moderately entertaining to fix things myself, so I do not do what most engineers do.

NXP BLF177 MOSFETS

The MOSFET in this particular module is the BLF177, made by NXP.  Harris will sell you one for quite a bit of money.  You can also buy one from Mouser for about half the cost.

Harris FM Z series transmitter PA module with cover removed

Once the parts are obtained, the worst part of the entire job is unsoldering the old MOSFET.  This takes some patience and skill.  What I found works best is to melt some solder on the foil leads and get them good and hot.  Since this MOSFET is already destroyed, we don’t have to worry about heat etc.  The one thing you do not want to do it actually break the MOSFET open.  That is because it contains beryllium oxide, a known carcinogen.  Once all the solder is liquid, carefully pry the foil up with a small screw driver.  There are several components that have to be moved to work on this.

Harris Z series PA module with MOSFETS removed

After the old MOSFET is removed, clean up the solder pad with a solder pump and solder wick.  I like to use a little liquid flux on the solder wick, it makes things go faster.

Once all the old solder is cleaned off the solder pads, I brush a light coat of liquid flux in the pad.  Again, this makes things go faster.

Harris Z series FM transmitter module new MOSFETs waiting to be soldered

The new MOSFETS are very sensitive to static discharge, so I always use a static drain wrist band when handling.  I place both MOSFETs on to the circuit board.  I then solder them on using as little heat as possible from the soldering iron.  Again, the MOSFETs are sensitive to heat and one can easily be destroyed if it gets too hot.

Harris Z FM series PA module repaired

This is the module with the new MOSFETs soldered in. I use defluxing compound to remove all the extra flux. Once it cools off, I test the new module with a DVM:

Harris Z series FM PA circuit board under test, resistance is 3.3 Mohm

If the MOSFETS are good, they will have an internal resistance of around 3.3 MΩ.  If the module is bad the MOSFETS will read only  a few ohms if shorted:

Harris Z series FM PA module under test, DVM reads 1.6 ohms

That is how you do it.  I think Harris charges $775.00 per module to repair.  I fixed this one for $240.00, but that is not the reason I did it.  I did it for the fun that was in it.

Every now and then something goes wrong.  One of the nicer features of a solid state transmitter is a soft failure mode.  For example, the loss of a single RF module may bring the transmitter down to 95% power vs 100% power.  In a tube transmitter, the failure of the tube would mean 0% power.

This happened recently when a transmitter was turned off for tower maintenance.  Upon restart, an RF module failed.  Unfortunately, the spare RF module had already been used due to a lightning strike in early July.  So we were down a few percent on the output meter until another RF module was ordered and installed.  The station was running at 94% power according to the external watt meter.  That equates to about half a dB power reduction over normal operations, which is really insignificant.

Naturally, the fact that the transmitter was at low power gave the program director another excuse to pile on.  First I received this e-mail:

Getting reports out of both XXX and XXX that it’s nothing but the The (competing station) on 1xx.1 – the tropo is going pretty good right now.  I’ll monitor on the ride in but needless to say we can’t fix this soon enough. We’ve been bad in XXX County for the last two weeks and I just assumed tropo and stayed quiet – could this module have been out for a while?

Where are we on a software controller  so we can log in and monitor stuff like this?

To which I responded:

The module problem arose after the transmitter was turned back on, so no, this has not been a problem for the last two weeks, it has only been a problem since Sunday Morning at 11:42 am.

As I said below, the new module was ordered and as soon as it arrives, it will be put back in.

I then received this e-mail:

With all the shadowing in our contour and the short-spacing, we just can’t afford to loose 1db without tangible effects.  We need every nanowatt possible everyday – especially in the summer.

So, Mr. Smarty paints there thinks that 0.00000001 watt makes a difference.  The absurdity of that statement is un-measurable.  Why not a femto watt or a yacto watt?  Here was my politically correct response:

1. I understand you want the transmitter fixed.
2. I have done everything humanly possible to effect repairs including calling Harris on my day off to order parts and have them shipped.
3. e-mails of this type do not make things go any faster, and are in fact, counter productive.

I hope one day, he reads this blog.

Tell them no.

In my early engineering days, I was tasked with all sorts of non-engineering things.  Squeaky hinges, broken chairs, changing light bulbs, etc.  I drew the line in a few areas, things like unclogging a toilet for example, required a visit from the plumber.

As a part of these building managment janator services, maintaining the septic tank at an acceptable level so the toilets worked fell under my perview.  Every so often, I would call the honey dipper to come and haul all that crap away.  Then one day the leach field completely failed.  It was only a matter of time, the thing had been in service since 1947.  So, the decision was made to connect up to the city sewer line that had been installed a few years previous.  Why they didn’t just connect to it when the town was installing it is beyond me, however, that all happened before my employment.

It took a while to line up the permits, get a contractor, get a plumber, hire an excavator etc.  Of course, the toilets were still being used, because people need to poop after all, at work.  It is there right.  Under the constitution. Therefore, I hired the honey dipper to come every few days and pump the tank, there were no other options.  Until the general manager had an idea, we could just pump the raw sewage out onto the field by the antenna towers.

No, I said, that is illegal.  Besides, there is a swamp right there and the sewage would get into the swamp.

His reply was “Deer shit in the woods, I’m telling you to pump out the tank.  I don’t want to pay for the shit pumper anymore.”

No.

Raising his voice another few dB “PUMP OUT THAT TANK OR ELSE.”

I very calmly said “Or else what?”

He stomped off to his office to sulk.  I never did pump out that septic tank into the field and it took several more weeks to make the sewer connection.  Every three to four days, that septic service truck would be out there, sucking all the shit out of the tank, in full view of his office.  I would snicker, I wish I could have rigged up a camera to tape his reaction.

What if I had caved and did as he asked.  Then one day, the health department stops by because the neighbors complained about the smell.  I would have been held responsible, even though I was doing what my manager asked me to.  That is the moral of the story, even if you are ordered to do something that you know is illegal, it is ultimately you who will be held responsible.  It is better to go job hunting that to face potential legal consequences, not to mention the loss of reputation, this is a very small business after all.

As radio engineers, we have a whole host of rules that must be adhered to, FCC rules, building codes, fire codes, FAA rules, OSHA rules, etc.   It is up to you to know them and ensure that they are followed within your department.

A few pictures from my last trip to one of our FM transmitter sites.  This is a mountain top site, in as much as a medium sized hill is a mountain around here.  This site has a 2.3 mile road through the woods that is almost impassable 3-4 months out of the year.

Previous engineers have walked up the hill with a tool box.  I can say this with all honesty; not me.  In the past they have also rented a helicopter, used a snow cat, snowmobiles or an ATV with snow tracks.  I’d do those things provided they are safe and insured.  As I get older (and wiser), I realize that the only person who going to look after my well being is me.

Anyway, the trip starts here, at the gate:

Gate to transmitter road

Then it goes up the hill:

Transmitter site access road

Some sections are worse than others:

washed out road

Along the way there are some nice views:

City reservior near transmitter site

Finally, the gate to the tower farm:

Access gate to transmitter site

There are two digital TV stations, several cell phone carriers, some government two way gear, some FM translators, Media Flow, and us at this site.  There are also some Ham radio repeaters off to the side in another building.  All in all, a pretty RF intense site.

The view from the top:

view looking north

The reason why we came:

Transmitter room

That is a 24 year old BE FM5B transmitter.  The back up is a Gates FM5G, which aren’t we glad we have a solid reliable transmitter selection for such a remote site.  Actually, we were supposed to put in a new Nautel V-10 here last year, but the money was spent on computers instead.  Oh well, good thing there will be no computer crashes when we go off the air.

A standard maintenance trip consists of meter readings, comparing the reading to the last set of readings, changing the air filters, checking the remote control and calibrating it to the transmitter, checking the tower light sensor, etc.

Normally, the backup transmitter would be run into the dummy load, but the backup transmitter no longer works.  Parts are not available to fix it, so we operate without a net.  One of the previous general managers asked if that keeps me awake at night, to which my answer was no, not at all.

You know it is going to be a bad day when:

a farmer mowing grass took a wrong turn on June 16, KFEQ AM lost a guy wire and eventually one of its four towers came toppling down

From Above Ground Level magazine.

Is hiring the farmer down the road to come and mow the AM field a smart thing to do?  It depends, I suppose, on whether or not your towers will be standing afterward.  Hopefully the guy had some insurance, if not then the station is basically screwed.  The article did not mention that, although it did state that “The station is wieghing it’s options,” Which does not sound good.

The good news is at least they were doing the maintenance.  Most AM stations these days don’t even bother to mow the fields.  Look at this picture:

Tall grass at an AM transmitter site. Owner says don't cut it.

It is not that I don’t want the grass cut, I do.  However, I am not going to pay for it out of my own pocket, that is ridiculous.  So, it grows.