South Pole Log - Stanford Universitynova.stanford.edu/~vlf/hardware/fieldsites/southpole/trip_reports... · South Pole Log South Pole Log ... 2010, 2036 UT), battery banks 1 and 2

South Pole Log

South Pole Log Written during my trip to repair the South Pole Beacon from Jan 4 to 14, 2010, NZDT By Daniel Golden ([email protected])

1. Todo List2. December 28 PST3. December 30 NZDT4. January 15. Monday, January 4: South Pole Arrival6. Tuesday, January 57. Wednesday, January 68. Thursday, January 79. Friday, January 8

10. Saturday, January 911. Sunday, January 1012. Monday, January 1113. Tuesday, January 1214. Wednesday, January 1315. Thursday, January 14: South Pole Departure

All times are NZDT unless otherwise specified

Todo List

1. Figure out what's wrong with charge/discharge of batteries 2. (Turn beacon back on) 3. Connect shipped amplifier 4. Fix amplifier 5

December 28 PST

● Departed San Francisco

December 30 NZDT

● Arrived Christchurch, NZ ● Screwed around

January 1

● Departed Christchurch ● Arrived McMurdo ● Screwed around

https://docs.google.com/View?id=dfxbxqmq_841d6wfmchn (1 of 14) [8/20/2010 9:41:51 AM]

South Pole Log

Monday, January 4: South Pole Arrival

● Departed McMurdo ● Arrived South Pole ● Met Ethan Good (Science Tech) and Al Baker (Science Manager) ● Al drove Ethan and I out to the MAPO building in a Pistin Bully. Went on tour of VIPER control

room. Ethan enabled the beacon, and demonstrated its operation for one beacon minute (approx 3 pm). We then turned it back off via the software “kill switch.” Al found the shipped amplifier and we unpacked it.

● I measured the temperatures of the battery banks (which Ethan had already done) using a Fluke temperature sensor from the VIPER control room. Banks on both levels were around 61 degrees F, +/- 1 degree

● I walked the 1 km back to station, despite Ethan suggesting that I don't exert myself while I'm getting accustomed to the altitude. Now I have a headache.

● I made a few calls and sent a few e-mails before the TDRS-5 comms satellite went out of view around 5:13 pm. Tomorrow, GOES-3 will be in view from 0336 to 1035 NZDT and TDRS-5 will be in view from 1334 to 1649 NZDT.

● Tomorrow, I'll be in touch with Ev and we'll start on the batteries.

Tuesday, January 5

● House mouse today at 4:00 pm - bathroom cleaning time! ● I'm working from the B2 science lab right now (next to Ethan's desk). I think this area is also

called the CUSP lab. ● At around 9:30 am I asked Ethan to enable the previously-disabled beacon. I'm going to watch it

for at least two charge-discharge cycles to get a feel for how it's performing, and whether the problem of discharging too low is persisting. Also, by running a few cycles, we'll see how many cycles it takes before the batteries discharge too low, which may give an idea of where the problem is.

�❍ According to Ev, the batteries shouldn't discharge significantly below 24 V at the end of the transmision

�❍ On the first discharge after idle (Jan 4, 2010, 2036 UT), battery banks 1 and 2 dip to slightly over 23 V, and battery banks 3 and 4 dip to slightly over 24 V. All four banks started around 27 V, and all four banks returned to 27 V, although bank 2 leveled out about 0.2 V less than the others.

�❍ I left the beacon on for now, because the battery charge and discharge cycles are consistently the same.

● One of Ev's open questions was why Ethan was able to measure a total current in Bank 2 during discharge of 225 A, but the amperage discharging from the upper and lower batteries in bank 2 only added up to ~45 A.

�❍ I'm going to go down to the beacon control room now, and take some current measurements

● Scratch that - I've decided that the charge/discharge curves look OK to me, so I'm going to skip debugging battery problems.

● Instead, I'm going to devote some time to getting the shipped amplifier installed and working. This will involve, among other things, installing a new bootstrap capacitor into its driver card. Then I'll look at the HF oscillation problem for amplifier 5.

● Just because it was easy to do, I measured the temperatures at the batteries in the room (the beacon has now been active all morning). They're now around 74 degrees F, which is 13 degrees warmer than yesterday, when the beacon was inactive.

● I then set to work on replacing the broken voltage meter for amplifier two. The new meter was very similar, but not exactly the same as the old meter. For one, the faceplate was the same


South Pole Log

shape, but had different anchoring holes. I had to entreat Bob, the surly master of the next-door machine shop, to drill me some new holes. I then filed them to fit. Also, the holes in the circuit board on the back were slightly too close together for the screws in the back of the meter, so I had to file those as well. In the end, I got everything fixed up. All that remains is for Ev to reply to my e-mail regarding whether is should be calibrated for 60 Hz or 19 kHz, and whether it should be peak-to-peak or RMS voltage.

● After lunch and "house mouse" (a euphemism for scrubbing the bathroom), I went back to the control room and repeated some of Ethan's voltage and current measurements to see if I could find any persisting problems with battery bank 2.

�❍ I did find that the upper bank of batteries discharges significantly more current than the lower bank (136 A vs. 87 A). This might indicate a bad battery or connection on the lower bank.

�❍ I also tested the voltage of each battery while the bank was absorbtion charging. All the batteries were steady around 6.9-7.1 volts, except for one battery that was at 6.6 volts. This is a very small difference, but may be indicative of an underlying problem in that battery. I e-mailed Ev to ask if I should try replacing that battery (or if I should do more tests).

● Finally, I opened up the case of the shipped amplifier, and was contemplating changing the bootstrap capacitor (to reduce the current spike when the amplifier turned on), before I realized that I didn't know the polarity in which to install the capacitor. I e-mailed Ev to ask him about it.

�❍ In retrospect, I could look at George's old e-mails �❍ And then I figured out the polarity myself; the positive terminal should be between R16

and R17

Wednesday, January 6

● In the morning, I arrived at the VIPER lab, intending to replace the broken voltage gauge. The new voltage gauge was ready to go. However, I realized that I'd have to disconnect and unmount Amplifier 2 to do that, which seemed like a lot of work.

● Instead, I set to work replacing the bootstrap capacitor (C8) on the driver card of Amplifier 3 (the shipped amplifier). After trying for 15 minutes to solder in the new electrolytic capacitor, I gave up, and I'll instead bring it to the science lab for soldering. They have better irons, vises, and a big magnifying glass.

● Presumably, I'll only be able to do the most elemental soldering in the VIPER lab, which is disappointing. That means a lot of bringing components back and forth, which always has the potential for damage. But I guess I'm even more likely to damage them while wrestling with them in the VIPER lab, so that's that.

● So I will go ahead and replace the broken gauge while I'm here. ● With respect to the batteries:

�❍ At Ev's suggestion, I diddled the "battery type selector" on inverter 2, and made sure that the temperature sensor was secure. Neither seemed to do anything.

�❍ Next, I tried swapping the temperature sensors for inverters 2 and 4 (although note that inverter 4 is only powering one amplifier). But then I started screwing with amplifier 2, so I don't know know if it worked or not. I'll have to try again when I put amplifier 2 back.

● I killed the beacon and disconnected amplifier 2 at 10:54 NZDT to install the new voltage gauge. ● I resurrected the beacon at 11:07 NZDT with amplifier 2 removed; six amplifiers total ● Note the steps to removing an amplifier:

�❍ Kill the beacon �❍ Turn off the AC breaker boxes for the amplifier in question (just to be safe) �❍ Remove the 0V wire from the next amplifier, and pass through the 0V connection from the

current amplifier �❍ Remove the grounding wire �❍ Unscrew the face plate from the rack and don't lose the screws�❍ Remove the amplifier, check the connections, and re-enable the beacon


South Pole Log

● I carried Amplifier 2 - with some difficulty - over to the workbench, and began the harrowing process of installing the new voltage gauge. This required a lot of cramped work, attempting to get various nuts and bolts on and tight. I gave up on one of the four bolts securing the gauge to the case because I didn't have enough room, but three will be plenty.

● While I had the amplifier open, I decided to replace George's large 500 uF bootstrap capacitor with one of the 330 uF capacitors that Ev had given me. However, attempts to turn on the amplifier immediately tripped the circuit breaker.

● George explained that all of his testing was done through the inverters, so that's how I'll have to do my testing as well. This means I might only get a certain amount of time before the battery voltage drops too far (23 V, say) and I have to power everything off to let the batteries recharge.

�❍ The beacon was killed again at 1356 NZDT in preparation for testing an amplifier off of the inverters.

● Side note: the mouse on the beacon computer is screwed up and drives me crazy. We should send another one down when we get a chance.

● I fired up amplifier 2, and after a few mistakes (e.g., connecting the "sync" output of the signal generator to the amplifier input instead of the "output"... bad idea) I got a maximum of 22 V RMS output on the Keithly AC multimeter before the signal started clipping. In theory, if VRMS =

(0.707/2)*Vpk-pk, then this is way too low; it would indicate only 62 Vpk-pk output, when the

amplifier should be capable of outputting 80 Vpk-pk without clipping.

�❍ However, here are the wacky results when I connect up the Tektronix oscilloscope and multimeter to the signal generator. The signal generator claims to be outputting peak-peak voltage, the multimeter makes no claims, and the 'scope does both.

�❍ For 1 Vpp signal generator output, I get ■ Multimeter: 0.708 V ■ Scope: 0.712 V RMS, 2.08 V pk-pk

�❍ For 10 Vpp signal generator output, I get ■ Multimeter: 7.08 V ■ Scope: 7.07 V RMS, 20.6 V pk-pk

�❍ Conclusion: the signal generator display is actually the peak voltage, not the peak-to-peak voltage. This despite the fact that the bloody display says "VPP."

● With a 1.8 Vpeak input voltage, amplifier 2 generates 76 Vpk-pk output before becoming distorted.

● I fired up Amplifier 3 (which had been shipped), and checked the DC voltages, which were fine, and the clipping amplitude, which was fine, at 80 Vpk-pk, with 1.8 Vpk input.

�❍ I did notice that wacky high frequency stuff at low amplitudes, and after the signal had begun to clip. It does seem to disappear at "nominal power" though (when the input is 1.8 Vpk). I agree with Ev that we should try to do something to reduce that.

● For now, I reinstalled Amplifier 2 into the rack, with some difficulty. I don't know exactly how much these things weigh, but I bet it's pretty close to how much I weigh. Amplifier 2 is the one that was previously installed and functional, which I removed to replace the voltage gauge and change out George's blue 500 uF bootstrap capacitor for one of the 330 uF capacitors I brought. Hopefully it will work.

● The beacon was resurrected at 1658 NZDT. I'll leave it running while I attempt to troubleshoot the 4.7 MHz oscillations in the filter card of amplifier 5. I'll install amplifier 3 and uninstall amplifier 5 when I confirm that the beacon works with my upgrades to amplifier 2.

● Yep, it all works. I'll now remove amplifier 5. I hope I don't hurt myself. ● I removed amplifier 5 without killing myself, and passed amplifier 2 through to amplifier 6,

bypassing the blank space where amplifier 5 was. This required a bit of jury rigging, since the 0V - 112 V wire wasn't long enough. I spliced two wires together using the old voltage meter from amplifier 2 in a manner in which I'm not proud. Let's leave it at that. Luckily the grounding wire was long enough.

● I resurrected the beacon, again with six amplifiers, at 1727 NZDT. ● I'm now going to make sure that the beacon works with all six amplifiers, then I'll head to the


South Pole Log

science lab to start making ferrite windings. Also I have to pee, and there's no bathroom here. Yikes.

● Yep, it's working just fine. Back to the science lab with me! ● I started making some ferrite cores. I'll make ~four, solder two onto the driver board from

amplifier 3, and see if the 4.7 MHz oscillations are cured (which, on that amplifier, only occur when it's saturated or has low input). If not, I'll try something else.

● I got to go on an Ice Cube tour today! I saw a really deep hole with electronics balls in it. Wow, science.

Thursday, January 7

● Today, after breakfast, I set to work making some more ferrite cores. I made four, and then soldered two onto the driver card from amplifier 3.

�❍ Unfortunately, I forgot to bring alligator clips for a control - first bypassing the ferrite to see the magnitude of the 4.7 MHz oscillation, then removing the bypass and letting the ferrite do its thing.

�❍ So, instead, I'll just install the driver card and see whether the 4.7 MHz oscillation is gone. �❍ I've been routinely melting the insulation on the alligator clips that are attached to the

dummy loads, probably via radiated heat, so let's see if I can test an amplifier without doing that this time.

● I killed the beacon at 1022 NZDT to prepare for testing amplifier 3 with the modified driver card. ● Note that one of the rear case fans of amplifier 3 is broken; it was damaged in shipment. ● The signal generator frequency was set to 19 kHz in all cases, unless otherwise specified ● On amplifier 3, with the ferrite cores installed between R23 and R24, and R25 and R24, with two

loops of wire (three passes through the core) �❍ At 1.3 Vpk input, output sine wave is 57 Vpk-pk and 4.7 MHz oscillation is 23 Vpk-pk

�❍ At 1.9 Vpk input, the output is distorted, and the output sine wave is 82 Vpk-pk and 4.7

MHz oscillation is 24 Vpk-pk

�❍ At 1.8 Vpk input, output sine wave is 79 Vpk-pk and there is no 4.7 MHz oscillation. This is

the maximum input before distortion. ● Next, I'll test amplifier 5. Then I'll swap amplifier 5 and amplifier 3's driver cards and test them

again. ● On amplifier 5, with the unmodified filter card

�❍ At 1.3 Vpk input, output sine wave is 61 Vpk-pk and 4.7 MHz oscillation is 22 Vpk-pk

�❍ At 1.9 Vpk input, output sine wave is 84 Vpk-pk and 4.7 MHz oscillation is 24 Vpk-pk

�❍ At 1.7 Vpk input, output sine wave is 76 Vpk-pk and there is no 4.7 MHz oscillation

�❍ At 1.8 Vpk input, the output is distorted, and the 4.7 MHz oscillation is approximately 18 V.

�❍ I tested the usual DC voltages (+60 V, output module drains +/- 50 V) and they were fine.

● I measured the beacon input: amplifiers are nominally fed at 3.6 Vpk-pk, or 1.8 Vpk.

● Now, I'll swap the driver cards and try again. ● With amplifier 3's ferrite-filled driver card, here are the results for amplifier 5

�❍ At 1.7 Vpk input, the output is undistorted, but now there's 8 Vpk-pk of 4.7 MHz noise that

I don't think used to be there �❍ At 1.8 Vpk input, the output is distorted, and there's 16 Vpk-pk of 4.7 MHz noise, about the

same as before ● With amplifier 5's ferrite-free driver card, here are the results for amplifier 3

�❍ No visible 4.7 MHz oscillation at 1.7-1.8 Vpk input, but oscillation below and distortion with

oscillation above


South Pole Log

● Thus, I tentatively concluded that the problem with amplifier 5 was not in its driver card. ● After lunch, I came back and tested out a larger value of C11. This was our "second choice"

option after the ferrite failed to work. The original value for C11 is 33 pF. �❍ While amplifier 5's driver card was installed in amplifier 3, I tested up to 200 pF in parallel

with C11. Nothing seemed to work. �❍ When I repeated the test on amplifier 5, once it had its original driver card back in, the

4.7 MHz oscillations at input voltage of 1.8 Vpk were reduced from 16 V to 10 V.

�❍ The oscillations at 1.3 Vpk input were not reduced, and held steady around 26 V.

�❍ Note that the oscillations are riding on top of a much slower sine wave, so their actual amplitude is hard to measure. I estimate in a slightly different way each time, but the relative changes should be accurate.

● On a lark, I replaced amplifier 5's original filter card. Voila! No 4.7 MHz oscillation at all, at any imput amplitude! The output was around 75 Vpk-pk with a 1.8 Vpk input. I also got the input up

to 1.9 Vpk with 80 Vpk-pk output. At 1.8 Vpk input, the output was around 76 Vpk-pk. There's only

a slight bit of waveform distortion, a little hump on the downgoing portion of the waveform, but it doesn't look like oscillation. This strongly suggests that there's something wrong with the new filter cards, which is a predicament, because supposedly the new cards are "better" than the old ones. But why?

● Now I'll test amplifier 3 with, and without the ferrite. In the latter case, I'll bypass it with alligator clips.

�❍ Conclusion: it makes absolutely no difference, both at 1.3 Vpk input, and at 1.9 Vpk input

(at which point the output is distorted and the oscillations return). There are no oscillations anyway at 1.8 Vpk input for amplifier 5.

● I'll be trying the following things, in order, to attempt to figure out what's wrong with amplifier 5's spurious 4.7-MHz oscillations:

�❍ Swap all output modules between amplifier 3 and 5, to see if it's a problem with amplifier 5's output modules

�❍ Put ferrite beads on the output modules' input (total of eight ferrite beads per amplifier) �❍ Give up and replace the driver card with the old driver card

● Removing the output modules requires carefully removing a million screws. Here's a list of screws for removing the output modules:

�❍ Medium brass flathead: secures output module casing to bottom of amplifier �❍ Very long thin flathead: +45 V on case of output module �❍ Small steel philips: secures output modules to output module case �❍ Black washers: between very long thing flathead and fins of output module heatsink (first

thing under screwhole) �❍ Small brass flathead: temperature sensor between facing output modules

● The beacon was resurrected at 1802 NZDT because I'm done using the inverters for now ● I successfully (?) removed all of the output modules from amplifier 3. Tomorrow, I'll remove the

ones from amplifier 5, and make the swap. ● I just had a look at amplifier 2's output waveform while the beacon was running and it looks

whacked out. I killed the beacon at 1824 NZDT, and I'll have to have a closer look at it tomorrow.

Friday, January 8

● Since I may need it later, here's how to measure inductance (from http://www.sxlist.com/techref/inductor/measure.htm)

�❍ Get a sine wave oscillator, and put the inductor and a resistor in series across the output. I generally start with about 100 ohms. Adjust the oscillator frequency until the voltage across the inductor and the resistor is equal. Since they are 90 degrees out of phase, each will be 0.707 times the oscillator voltage. At this frequency, the inductor has an impedance of 100 ohms, and the inductance can be calculated from Z = 2 * pi * f * L.


South Pole Log

�❍ I just tried measuring the inductance of the ferrite core using this method, using a 100-ohm resistor.

�❍ 1 volt, 100 kHz, 384 mV across resistor, 52 mV across ferrite. Therefore, impedance of ferrite is 0.135 that of the resistor, or 13.5 ohms. So the inductance is:

. This also means that the apparent resistance to a 4.7

MHz signal would be . Not very impressive! For our 19.4 kHz beacon signal, the impedance would be 2.6 ohms.

�❍ I got fed up with the crappy multimeter and redid the measurements with a scope output. At 1.74 MHz, the voltages across the resistor and ferrite matched. This gives an inductance of 9.15x10-6 H, even lower than I measured before. Just to be safe, I'll repeat these measurements with a different ferrite.

�❍ Done, the other ferrites are the same. ■ NOTE: I later determined that my measurement method was incorrect. I was

shorting the scope to ground. The correct measurement is at the end of this day. �❍ As a control, I measured the inductance of a single piece of wire. The voltages equalized

at 7.4 MHz, so a single piece of wire has an inductance of 2.15x10-6 H. �❍ So, for perspective, at 4.7 MHz, the impedance of one of my ferrite cores is 270 ohms,

and the impedance of a piece of wire is 63 ohms. At 19.4 kHz, the impedance of a ferrite core is 1.11 ohms and the impedance of a piece of wire is 0.26 ohms

● I'm back in the VIPER control room now, with the amplifiers. I thought yesterday that there was something wrong with amplifier 2, so I hooked up the dummy load to the system for testing.

�❍ The problems that I was seeing seem to have disappeared for the moment, but now it looks like there's something wrong with the dummy load. It doesn't light up when six amplifiers are on, even though the amplifiers are obviously running and outputting their usual voltage. When I measure the line voltage and current on the scope (100V/V and 2A/V), I get 6.6 V and 3.3 V RMS, corresponding to line voltage of 660 V, and line current of 6.6 A.

�❍ According to Ev's documentation VLF Transmitter Reference 2005.doc, the output line voltage should be the sum of output voltages (i.e., 112 V * 6), which it is, and the amperage should be 15.3 A, which it isn't, maybe due to the low dummy load impedance.

�❍ In any event, the frickin' dummy load isn't lighting up and I don't know why. �❍ On the assumption that the impedance is just too high, I hooked the amplifiers back up to

the antenna and resurrected the beacon at 1052 NZDT. I'll be sure to watch the amplifiers and the line voltage and current during the next transmission.

�❍ With the beacon on, the output line voltage is 650 V, output current is 2.32 A rms. ● I then turned my attention back to swapping the output modules in amplifiers 3 and 5. In three

hours, I removed the output modules from amplifier 3 and put amplifier 5's output modules in their place. Three hours!!!! Most of this is turning screws, removing inputs, trying to remember where they go, wiping thermal paste off of my hands, etc. I see no obvious way to speed up the process. I'll have to just try and keep output module swaps as infrequent as possible.

● After lunch, I returned to the beacon control room and made some more attempts to figure out why the dummy load wasn't working.

�❍ First, I just waited for a longer time. Nothing happened, and it still wasn't particularly hot. �❍ Then I measured the DC impedance again, which is 96 ohms, like I measured before.

That's 16 times more than the antenna DC impedance of 6 ohms. �❍ Then I gave up

● After another hour or so, I finished the swap of output modules between amplifiers 3 and 5. ● After looking more closely at the antenna waveforms during beacon transmission, I've come to

the conclusion that the current is WAY too low. I measured 2.3 A rms, but it should be closer to 13 A rms.

�❍ Maybe amplifier 2 is misbehaving? I'll try bypassing it. �❍ Whoops, that didn't work at all. With amplifier 2 bypassed, I got no output! �❍ Presumably this current thing explains why the dummy loads weren't heating up. I'm at a


South Pole Log

loss for how to debug it. �❍ The problem started some time near the end of my work last night and the beginning this

morning. It has something to do with either the weird amplifier 2 performance from last night, or the slight burning smell I was getting when I first started using the dummy load this morning. All the amplifiers appear to be working fine with respect to their output voltage waveforms and amplitudes (all around 26 V rms). It's just that the current is too low.

�❍ I still don't understand why the dummy load doesn't heat up if it's burning 660 V * 6.6 A = 4.3 kW. I hope that power isn't going somewhere weird and unexpected.

�❍ This problem is over my head; my only solution is to unrack all the amplifiers and test them individually. Since I really don't want to do this, I'll keep working on amplifier 5 and hope that Ev responds with a super solution soon.

�❍ Using the archived current data, I determined that the problem started between 2010-01-07 05:20 (last known good transmission) and 2010-01-07 20:50 (first known bad transmission). That's between 2010-01-07 18:20 and 2010-01-08 09:50 NZDT.

�❍ According to my notes, here's the sequence of events, in UT. ■ 2010-01-07 05:02 - resurrected the beacon. I had killed it because I was using the

inverters to test some other amplifiers. ■ 2010-01-07 05:20 - during the transmission, it looked like the output of amplifier 2

- which is the one on which I'd changed the voltage gauge - looked screwed up (it looked like some kind of periodic capacitive decay, maybe indicative of a short). Also, its voltage gauge stayed near zero during transmission. I didn't look at any of the other amplifiers' outputs because their voltage gauges looked fine. I killed the beacon for the night, intending to come back for it in the morning. The waveforms for this transmission look fine.

■ 2010-01-07 20:00 (approx) - hooked dummy load up to the system and turned it on. Amplifier 2's misbehaving did not recur. The elements didn't light up, but they got warm. I thought I smelled something burning (uh oh), at which point I turned everything off. I didn't see any smoke or anything burned anywhere, so I figured I was just smelling the dummy load making its usual smell. I was getting 660 V and 6.6 A rms into the dummy load according to the o-scope hooked to the "line voltage" and "line current" outputs. On the assumption that the dummy load was just malfunctioning (or wasn't supposed to glow), I hooked the amplifiers back up to the antenna and resurrected the beacon without incident. The waveforms looked sinusoidal enough, so I left the beacon running; I didn't notice that the current was too low until this afternoon.

�❍ It seems likely that the problem is in amplifier 2, so that's the first one I'll test. ● Results of tests of amplifiers 3 and 5 with swapped output modules. Both amplifiers are using

their own new driver cards. ● On amplifier 5 (using amplifier 3's output modules)

�❍ The waveform is stable at 1.8 Vpk input, and distorts at 1.9 Vpk input.

�❍ At 1.8 Vpk input, the 4.7 MHz oscillation is present, with an amplitude of up to 12 Vpk-pk.

However, it's more "evanescent" than before, with rapidly changing amplitude. �❍ At 1.7 Vpk input, the 4.7 MHz oscillations are worse.

● On amplifier 3 (using amplifier 5's output modules) �❍ The waveform is stable at 1.7 Vpk input and distorts slightly at 1.8 Vpk input

�❍ At 1.7 Vpk input, the 4.7 MHz oscillation is present, with an amplitude of up to 5 Vpk-pk.

This is of the same "evanescent" type as in amplifier 5. �❍ At 1.8 Vpk input, distortion is slight at the top of the waveform, and the 4.7 MHz oscillation

is present, with an amplitude of up to 8 Vpk-pk.

● Conclusion: swapping the output modules just made things confusing. One thing is for certain, though, which is that it's amplifier 5's output modules that are responsible for the early


South Pole Log

distortion. There's probably a bad transistor on there somewhere, but I'll be damned if I can find it.

● I added ferrite cores to the driver boards of amplifier 5. They seemed to eliminate the 4.7 MHz oscillation at input 1.7 Vpk, but it's still up to 10 Vpk-pk at 1.8 Vpk input

● Next I'll try adding ferrite with more turns, keeping it under, say, 5 ohms impedance at 19.4 kHz. I'll try to make those after dinner tonight.

● I'm trying to make some different inductors. The procedure for measuring inductance is very confusing to me. When I connect the resistor in series with the inductor and start increasing the frequency, the voltage drop across the inductor first starts to increase, but then it starts to decrease! I guess this is some capacitive effect, possibly in my rat's nest of measurement wires. Also, I find that having two probes hooked up - one across the resistor, one across the inductor - messes things up, possibly because the black probe is shorted to ground inside the 'scope or something.

�❍ Instead, I'm resigned to measuring the input voltage and the inductor voltage, with one side of the inductor grounded, and the probed on either end of the resistor. I can't directly measure the voltage drop across the resistor this way, but I can infer it... assuming that the 90-degree phase shift of the inductor does result in 0.707 of the voltage being dropped across both the resistor and the inductor when ther voltage drops are equal. To look for 100-ohm impedance of the inductor, I screw with stuff until I have one volt (peak-peak) input and 0.707 volts (peak-peak) across the inductor. More-or-less.

�❍ Using this method, when I measure one of the small inductors, I get 0.707 of the input voltage across the inductor when f = 560 kHz -> L = 2.84e-5 H -> Z19.4kHz = 3.46 ohms,

Z4.7MHz = 838 ohms.

�❍ When I measure an inductor with five turns (five loops, 6 passes through ferrite) - which is close to as many as I can cram in there - I get 0.707 of the input voltage across the inductor when f = 130 kHz -> L = 1.22e-4 H -> Z19.4kHz = 14.9 ohms, Z4.7MHz = 3601

ohms. This inductance (and the associated impedances) is about four times higher than the 2-turn version. It's possible that the tigher wrapping of the 5-turn inductor helps too.

�❍ I'd like to make a few of these and try them tomorrow

Saturday, January 9

● I made four more 5-turn inductors, and I brought them to the beacon control room. ● Incidentally, did you know that the former "Inan shack" is now the "Naked lady shack"? Is this

one of the many fine examples of entropy at the SouthPole? ● I soldered in the 5-turn inductors on amplifier 5's filter card. Remember that amplifier 5 is still

using amplifier 3's output modules. ● I'll try two back-to-back tests: one with the inductors bypassed, and one with them in the

circuit. ● On amplifier 5, with the inductors bypassed:

�❍ At 1.8 Vpk input (max before distortion), there is no 4.7 MHz oscillation. Output amplitude

is 83 Vpk-pk

�❍ There is also no oscillation at 1.7 Vpk input

�❍ At 1.3 Vpk input, the oscillation is about 23 V

● On amplifier 5, with the 5-turn inductors in the circuit, I get a nasty 24 Vpk-pk 240 kHz oscillation

at all input voltages. Bah! �❍ After experimenting some more with different inductors, I've found that 3-turn inductors

still introduce 240 kHz oscillations, and 2-turn inductors do absolutely nothing. The FFT function of the 'scope helps here.

�❍ It now looks like, without the 2-turn inductors, the 4.7 MHz oscillations are gone at input


South Pole Log

of 1.7 Vpk.

�❍ Near as I can tell, the 2-turn inductors do absolutely nothing beneficial or detrimental. �❍ I unsoldered the 2-turn inductors from amplifier 5's filter card and soldered the resistors

back to their original configuration. Alas! I guess my next step is to try the 2-turn inductors on the inputs of the

● In about an hour, I soldered the eight 2-turn inductors onto the inputs of the output modules of amplifier 5 (remember, these are originally amplifier 3's output modules). Unfortunately, I had to cut the lead of R1/R8 where it connects to the input, because there's no way I could have gotten to the back side of the board to unsolder it without great difficulty. Now I'll test it before I go to lunch. Eek.

● Kiss my grits! They worked! The 4.7 MHz oscillations are totally gone! Hurrah! ● The good news: I found a solution that works. The bad news: it takes me over an hour per

amplifier to implement it (and I leave in five days). I wonder if there's somewhere else the inductors can go?

● Unless I hear from Ev in the mean time, I'll start making the same changes to amplifier 3 after lunch. And I'm going to leave amplifier 3 and 5's output modules swapped.

● After I test that, if it works, I'm going to start pulling amplifiers off the racks and testing their output, to figure out why the beacon current is too low.

● Luckily, Ev e-mailed back, and suggested that it was a burnt surge arrestor on the transformer. I asked Ethan to urgently take me there (which would spare me the indignity of pulling amplifiers off the racks) and he showed up shortly later in a pisten bully.

�❍ One of the surge arrestors had most certainly failed. Unfortunately, I couldn't get the current arrestors out of the little metal tabs via which they were connected to the transformer inputs, so I couldn't replace them without finding new tabs. Since Ethan had to get back, we took another trip out to the transformer location and covered the exposed high voltage cable (from the amplifiers) with a plastic bag, to weatherproof it until we could replace the transformer.

● Now that I knew that the transformer was the source of the low current problem, I set out to light up the bloody dummy load with my six racked amplifiers. Which I did... after about 30 seconds with the lights off. They just light up a little. But I guess it's working.

● After discussions with Ev, we determined that it would be possible to put ferrite only on the output of the driver card instead of on the inputs to all of the output modules. This required cutting the original output trace on the driver card and soldering the ferrite in its place.

�❍ Lo and behold, it worked great! This is great news, because it would have been a royal pain to solder eight ferrites onto amplifier 3's output modules. I hate extracting those fricking ouput modules. I bet George does too.

�❍ Unless Ev tells me otherwise, I'll leave the ferrite on the output modules of amplifier 5, and on the driver card of amplifier 3. Yeah, they're not symmetrical, but so what. As long as they both work.

�❍ The final status of amplifiers 3 and 5 is ■ They both display no 4.7 MHz oscillations, at any input amplitude ■ They both operate with no distortion at 1.7 Vpk input

■ Amplifier 5 displays very slight distortion at 1.8 Vpk input

■ Amplifier 3 displays significant distortion at 1.8 Vpk input.

● My next task will be replacing the burnt surge arrestor in the transformer. If I'm really clever, I'll rig something up so they can be replaced in the field. That means I'll need three or four sockets of some form, as well as three metal tabs. Let's see what I can find in the science lab...

�❍ Apparently, the "metal tabs" are properly known as "solder lugs." �❍ I found some solder lugs in the science lab, but I couldn't find any sockets. So I'll just

solder the surge arrestors directly to the lugs. �❍ Ev suggested using the 4500 V surge arrestors, so that's what I'll do. �❍ Someone stole the soldering iron that I was using from the science lab, so I'll have to

solder from the beacon control room tomorrow.


South Pole Log

�❍ Maybe I'll ski out to the antenna and replace the transformer tomorrow! That would be fun (as long as I can carry the transformer somehow). I'll have to leave early so all the bloody rec skiiers don't take my skis and boots.

Sunday, January 10

● I got up around 7 am, and searched again, in vain for the soldering iron. Eventually, I found a note on the bench saying that someone had taken it to the greenhouse for god knows what reason. Al Baker stormed off there and retreived it, saying bad things about people who take equipment from the lab without asking.

● Now that I had the soldering iron, soldered some new surge arrestors onto some solder lugs. The solder logs are surprisingly solder-phobic, and it look longer than I was expecting. But I got it done.

● Then, I implemented my cross-country skiing plan. I got a radio from comms and some skiis from the ski shack, and skied to the mapo building. There, I managed to cram the transformer box and all necessary tools into my backpack and I set off. It's about 1.4 miles from the mapo building (home to the beacon control room) to the antenna, and it took me around 30 minutes to ski it. I was using the same wacky telemark skis from last time.

● In a rare case of unbridled success, it turned out that I had all the tools I needed, and I was able to remount the transformer and reconnect everything in short order. The whole process took less than 10 minutes. My hands got cold on the way back, and both of my thumbs went numb (I was wearing mittens), so I had to stop for a few minutes and stick my hands in my armpits to warm them up. Otherwise, it was an absolutely lovely ski excursion.

● I'm not back in the beacon control room, and I'm going to try firing up the antenna again. I hope it works!

● I resurrected the beacon at 1326 NZDT. ● For whatever reason, sometimes the transmission waveforms look fine, and sometimes they look

clipped. I turned the gain down from 1350 to 1280 (at 2010-01-10 0135), and I'll leave it there. ● One of the fans has been broken on amplifier 3 since it got here (or since it got to Stanford).

According to George, there are replacement fans already somewhere in the beacon control room. I'll look for it tomorrow.

● Ev recommended that I install the new bootstrap capacitor and the ferrite inductors into the driver cards of the six currently operational amplifiers. I proposed to make sure the beacon operates at full power, and then, if I have time, make those changes.

● I just noticed that the two inverters that are powering two amplifiers (inverters 2 and 3) are drawing 212 amps during transmission. According to Ev's "VLF Transmitter Reference 2005" document, they shouldn't draw more than 193 A during transmission. I lowered the gain to 1150, so that inverters 2 and 3 are now drawing 193 A during transmission. Alas, according to the beacon diagnostics' "Transmitted Power" page, we're now down to a paltry 5.6 kW of transmitted power (vs 7.8 kW when the gain was up way too high). Obviously, the relationship between transmitted power and gain is not linear.

Monday, January 11

● With Ethan's help this morning, I re-racked amplifiers 3 and 5, and we hooked them up to the dummy load for testing. This seemed to work, but it wasn't very gratifying; the dummy load glow could still only be seen with the lights off. This is presumably because I've turned the gain down to 1150 (from 1350) to prevent excessive current draw from the batteries.

● Satisfied, we hooked the amplifiers up to the antenna, resurrected the beacon, and waited for deliverance.

● We got a couple of good transmissions, but there seems to be a problem where inverter four is not turning on at the appropriate time. It just stays off while the other inverters turn on and attempt to transmit; but because inverter 4's amplifiers are off, no signal is transmitted (and,


South Pole Log

hopefully, nothing is fried). Each time this happened, the a bad sound would come out of the inverter, and the green light would flicker. When I pressed the power button on the inverter, it would go back into bulk charge mode (solid orange light). Then, generally, it would work once and then screw up again. I never noticed this happen when only one amplifier was plugged into inverter 4.

● The waveforms of the successful transmission look good, but their amplitude is only the same as when George left. This is also likely the result of my turning the gain way down. So, except for inverter 4's issue, I've increased reliability without increasing power output. Not as much fun, but I guess it's still a good thing.

● After much debugging, involving plugging every permutation of inverter 2 and 4's amplifiers into each inverter, I've determined that there's only a problem when amplifier 3 and 2 are plugged into the same inverter. This suggests a potentially too-low bootstrap capacitor; I've replaced george's 500 uF bootstrap capacitor on both amplifiers 2 and 3 with a 330 uF version. Maybe 330 uF is too low? Golly. I have 470 uF capacitors with me as well; maybe I'll replace the 330 uF bootstrap caps on amplifiers 2 and 3 with those.

● Amplifier 2 again did its thing where it didn't transmit with the others (it turned on, but its voltage meter stayed at 0). This is a very odd and disturbing intermittent problem. I'll monitor its output on the oscilloscope during the next transmission, and kick it if I have to.

● During the next transmission, the oscilloscope output got all crazy, and a terrible buzzing sound came out of the amplifiers (or inverters). This seemed similar to what George was seeing which he attributed to arcing, but it didn't look quite the same. Amplifier 2, in particular, looked again like its output was all wonky.

● I tested everything into the dummy load, and nothing remarkable happened as usual. ● Then I tested everything into the antenna and nothing remarkable happened. ● Now I'll wait for the next synoptic minute to see if the transmission goes crazy again. ● It worked fine. ● I managed to extract amplifier 2's driver card while it was still racked. I installed the ferrite and

the 470 uF bootstrap capacitor and reinstalled it. I hooked it back up, and hooked the output of all the amplifiers to the dummy load, turned it on, and... amplifier 2 did the same bloody thing that it's been doing (the funky spikes and no transmission). Bah! I don't know what to do at this point except wait for Ev's response.

● Lunch time! Unracking amplifiers to follow! ● After lunch, with Ethan's help, I unracked all the amplifiers, and I'll be modifying all of their

driver cards with ferrite on the output, and my 100 V, 470 uF bootstrap capacitors to replace George's Big Blue 50 V, 500 uF capacitors.

● Before I start, here's a reminder of the status of all amplifiers: �❍ Amplifier 1: Untouched �❍ Amplifier 2: Ferrite on driver card, 470 uF bootstrap cap, unexplained misbehaving �❍ Amplifier 3: Ferrite on driver card, 330 uF bootstrap cap, max input 1.7 Vpk

�❍ Amplifier 4: Untouched �❍ Amplifier 5: Ferrite on output modules, 330 uF bootstrap cap �❍ Amplifier 6: Untouched �❍ Amplifier 7: Untouched �❍ Amplifier 8: Untouched

● Installed 470 uF cap in amplifier 5. 20 A turn on current. Tested fine. ● Amplifiers 2 and 5 exhibit the same turn-on problem as amplifiers 2 and 3 did, even with the

470 uF caps. �❍ I tested all permutations of two amplifiers of amplifiers 2, 5 and 1 turning on

simultaneously. Of those, only amplifiers 2 and 5 exhibited problems turning on twice in a row, and both of those have 470 uF bootstrap capacitors and ferrite. So it's either my bootstrap capacitors, or the ferrite, that prevents two amplifiers from turning on in a row.

�❍ Interestingly, amplifiers 2, 3 and 5 all have problems turning on with one another, and those are all the ones in which I installed my own bootstrap capacitors and ferrite inductors. So it's either the bootstrap capacitor or the ferrite that is creating this problem.


South Pole Log

�❍ I bypassed the inductors in amplifier 3, and tested turning it on with amplifier 5. Problem solved; they turned on together, twice in a row, with no problems. So you can't start two amplifiers together twice in a row if they both have ferrite inductors on their driver cards. That's crazy.

�❍ But amplifiers 2 and 3 were having the turn-on problem before I installed ferrite in amplifier 2 (but after I installed a 330 uF bootstrap capacitor).

�❍ So I don't know if it's my bootstrap capacitors, the ferrite or what. George never had that problem, but then again, George never had amplifier 3.

�❍ I guess I'll just leave all the cards as-is and re-rack the amplifiers tomorrow. Hopefully, Ev will respond with some advice about amplifier 2 before then. If not, maybe I'll just re-rack it and hope for the best. Or leave it off. Or who knows. Problems that can't be replicated stink.

● I went back after dinner to try to debug amplifier 2, but I wasn't able to replicate the problem. I tried poking all the wires while it was on, but nothing happened. I'll just try to rerack it tomorrow and hope it works!

Tuesday, January 12

● With Ethan's help, I reracked all the amplifiers this morning. As before, I used the 98 V taps on the amplifiers, and left the gain at 1150.

● We first tested them into the dummy load with no problems. Amplifier 5's voltage meter didn't go on, but when I tested the output, it was fine; so now amplifier 5 has a broken voltage meter (and amplifier 2 seems to be working fine). It can be fixed by tapping on it.

● The amplifiers also seemed to work into the antenna with no problems. ● I did a quick check on gain levels:

�❍ 1150 is the maximum gain for the inverters to draw 193 A (inverter 2 draws 196 A, all the others draw less)

�❍ 1350 is the maximum gain for which the output is undistorted (at which point the inverters draw 212 A).

● I resurrected the beacon at 0916 NZDT, and it seems to be performing just fine. Hurrah! ● Next, I'll solder those zener diodes onto the old driver cards, and then screw around for the rest

of my time here (while simultaneously monitoring the beacon, of course) ● In the afternoon, I took advantage of some free time to replace diodes on the old driver cards. I

replaced diodes D4 and D6 on old cards 2, 4, 5, 6, 7 and 8 with zener diodes in the same orientation. That should prevent impedance mismatch problems from nuking the output modules if these cards are used in the amplifiers.

● I was just reviewing some old e-mails, and realized that Ev had suggested measuring inverter 2's battery voltages while the beacon was transmitting. I guess I forgot to do this; I had only measured the voltages while they were absorbtion charging (orange light blinking). Inverter 2's battery voltages have improved, but they're still not as good as the other inverters'. So now I'm back in the beacon control room, and I'll make that measurement.

�❍ During transmission, all the voltages were around 6 volts, within 0.2 V of each other. All of the voltages dropped during transmission, so a direct comparison was difficult, but nothing stood out.

Wednesday, January 13

● Ev e-mailed me to tell me that there's nothing special about drawing 193 A from the inverters, so I came down to the beacon control room and cranked the gain up from 1150 to 1350. The waveform began clipping, so I reduced the gain to 1300, which seemed OK.

● At this gain level, the inverters draw about bit less than 225 A each. Battery bank 2's discharge currents are still quite asymmetrical, with the top and bottom banks drawing about 137 and 87 amps, respectively. That's 61% and 39% of the total current draw.


South Pole Log

�❍ Since we have four spare batteries, and since I couldn't figure out any problems with individual batteries, I asked Ev whether I shouldn't just try replacing all of the batteries on the lower shelf of bank 2. I'm waiting for a response.

�❍ I also measured the current during bulk charge; at first, the upper batteries in bank 2 draw 12 A, and the lower batteries draw 17 A. During the latter half of bulk charge, they start to even out.

● I left the gain at 1300 and left the beacon running. I now pronounce this operation a success! Unless, of course, Ev thinks I should change out the lower four batteries from bank 2.

Thursday, January 14: South Pole Departure

● Ev responded about the batteries, recommending I check the connection tightness one more time, and that the longer battery cable was on the upper row.

�❍ I killed the beacon at 0849 to do the tests, and resurrected it at 0911. �❍ The connections were all plenty tight (I got one or two turns out of a couple), and the long

battery cable was where it was supposed to be. �❍ I also measured the voltage drop across the inter-battery connections, but these were all

zero. �❍ Finally, I wiggled the connection on the upper battery bank when the two banks were

connected. It didn't do anything. �❍ So I gave up. Hopefully the problem magically fixes itself, but I wouldn't hold out too

much hope. We'll have to keep a close watch on the batteries, because once the voltages start going downhill, they do so in a hurry.

● My flight departed for McMurdo at 12:30 pm! So long, Pole! We had some good times!

Edit this page (if you have permission) |

Google Docs -- Web word processing, presentations and spreadsheets.


https://docs.google.com/Doc?tab=edit&dr=true&id=dfxbxqmq_841d6wfmchn

https://docs.google.com/

https://docs.google.com/

Documents

South Pole Log - Stanford Universitynova.stanford.edu/~vlf/hardware/fieldsites/southpole/trip_reports... · South Pole Log South Pole Log ... 2010, 2036 UT), battery banks 1 and 2