The roof is on FIRE!

We don’t need no water, let the… oh, wait… The actual roof is actually on fire you say?

YES: Ahh! Time to run around like crazy people!

Carrier HVAC unit damaged by fire
Carrier HVAC unit damaged by fire

This happened over the weekend at one of our clients in NY. The back story is this; over the last two weeks, the area has received almost three feet of snow. This roof is pitched slightly toward the back of the building. The roofing material is some type of PVC, which is very slippery when wet. Thus, at some point the snow/ice pack shifted towards the back of the building, it broke the natural gas pipe off where it entered the unit:

Broken gas pipe, HVAC unit 1
Broken gas pipe, HVAC unit 1

The next time the HVAC unit cycled on, there was giant torch on the roof with flames reportedly eight feet high.  A local fire fighter just happened to be driving down the road and spotted the fire, thus likely saving the building from major damage.  The fire department came and cut off the gas and electric.  The building was evacuated for about 20 minutes while they overhauled and checked for internal fires.

Carrier HVAC unit damaged by fire
Carrier HVAC unit damaged by fire

A second unit suffered the same fate, only with less damage:

Carrier HVAC unit damaged by fire
Carrier HVAC unit damaged by fire

The fire in this unit was contained to the controller area.  Same situation with the gas pipe, only it looks like the pipe was not broken all the way off:

HVAC unit broken gas pipe
HVAC unit broken gas pipe

The other two units are shut off while the gas pipes are dug out of the snow pack and checked for damage. At some point, they will be turned back on so that the heat can be restored to the second floor sales bullpen. Meanwhile, the sales people; they are complaining.

We threw a tarp over the unit with the cover ripped off because more snow is on the way:

Carrier HVAC unit tarped
Carrier HVAC unit tarped

Secure everything, assume nothing, take nothing for granted

The big lesson learned from Sandy is take nothing for granted. For several days prior to the storm’s arrival, we checked everything; refueled and started every generator, checked the oil, water and battery electrolyte, set up fuel deliveries ahead of time for the worst case scenario, checked all the backup transmitters and STL’s, and so on.  The one thing that I didn’t consider was a storm surge so high that the propane tank would float away.  After all, those tanks are heavy.

However, a brief examination of elementary physics reveals that even when full, a propane tank will float:

One gallon of water weighs 8.3 pounds if it is fresh water or approximately 8.55 pounds for salt water (depending on where it is from).

One gallon of liquid propane weighs approximately 4.1 pounds, thus it is about half as dense as water.

Most propane tanks are not full, being at most 80% liquid volume. It is always the thing that you didn’t think of.

We seem to be suffering a 500 year storm about once a year or so around these parts.  I expect that things will only get worse.  With that in mind, it is perhaps time to re-think our disaster preparedness and recovery plans to incorporate every worst case scenario we can imagine.  Everyone knows, but it bears repeating: Radio is the last link that people have when all other technology fails.  Thus, when it comes to storm preparation, there is no such thing as too much.  Thus:

  • Secure everything
  • Assume nothing
  • Take nothing for granted

Our assumptions about power utility and telephone network reliability and restoration may be wrong.  Our assumptions about access to remote sites, our ability to use vehicles, availability of gasoline and other fuels may be over optimistic.  Our assumptions that basic food stuffs, clean water and secure resting areas may also be wrong.  Get those items wrong and it does not matter how much equipment redundancy is built into facilities.

For remote transmitter sites, access can be a major problem after a storm.  In low lying coastal areas, flooding will be an issue.  In those situations, having backup transmitter sites would be a key feature of any disaster plan.

All good disaster plans also have the human component; clean water, food and safe, secure resting areas for the staff.  As always, when the SHTF and there are no options and no ideas, there is the Bear Grylls survival method:

Bear Grylls
Bear Grylls

Which we really, really don’t want to do (from the TV show Man vs. Wild on Discovery).

A day in Pictures

The aftermath:

Long beach
Long beach, this used to be an isthums, now it is a sand bar
A set of old stairs
A set of old stairs on the beach where the cottages used to be located.
Surf
Surf
100 lb propane tanks
Found the reason why the generator is not running
Propane tanks adrift
Propane tanks adrift from storm surge. There was a strong propane smell around these tanks, I secured all the valves.
WICC propane tank pad
Where the propane tanks should be
debris washed ashore during storm surge
Debris washed ashore during storm surge around north tower, including a section of dock
Second high tide after Hurricane Sandy, noon on Tuesday
Second high tide after Hurricane Sandy, noon on Tuesday, flooding ground system
three phase power line down
Three phase power line down due to wind
Three phase power line down
More wind damaged power lines
Telco wires taken down by trees
Telco wires taken down by trees
Generator room water level, as seen on the side of the battery
Generator room water level, as seen on the side of the battery

More work here tomorrow.

Update: Took longer than anticipated, but the station is back on the air with generator power as of 8:15 am, Thursday (11/1).  Commercial power restoration is not expected until Monday or Tuesday at the earliest.

Update: Commercial power restored on Thursday, 11/8 for a total outage of 10 days.  One good thing about incidents like this, I now have a fresh set of contacts for all the important people connected to servicing this site.

Fire! Fire! Fire!

Class Charlie fire in the transmitter room electrical panel.  Away fire party from repair locker forward.  Set condition ZEBRA throughout the ship, this is not a drill.

Or something like that.  If you were driving around Albany, NY this afternoon and noticed WDCD-FM was off the air, this is the reason why.

WDCD AM/FM main distribution panel
WDCD AM/FM 480 volt 3 phase AC main distribution panel

A little after noon time, the 480 volt main distribution panel at WDCD AM/FM caught fire, taking the FM station off the air.

WDCD conference room clock, time of power outage noted
WDCD conference room clock, time of power outage noted

According to this clock, it happened at 12:19 pm, when there was a loud bang and the lights in the studio flickered several times, followed by the building fire alarm going off.  Thankfully, a quick response by the station staff and the Town of Colonie fire department limited the damage to the interior of the distribution panel.  Other than the dry chemical fire extinguisher residue all over the place, the building is none the worse for wear.

WDCD distribution panel burned parts
WDCD distribution panel burned parts

The 480 Volt three phase electrical distribution panel was installed in 1947 when the original building was constructed.  The power company cut the power to the building and an electrician was able to re-route the distribution for the dry step down transformers that power the studios and equipment racks.  The original 480 volt service was installed due to the 50 KW AM transmitter for WPTR (WDCD-AM).   Currently, WDCD-AM is silent, pending programming decisions by the owner, Crawford Broadcasting.

WDCD burned electrical distribution panel parts
WDCD burned electrical distribution panel parts

So, we spent the late afternoon vacuuming the NextGen computers and UPS out, wiping down the equipment and making sure to clean out the power supplies and other nooks and crannies.  Then, we powered everything back up, one at a time and to our pleasant surprise, all came back up without error.  Total off air time for the FM station was about 6 hours.