Computers shut down again for more troubleshooting
BY WILLIAM HARWOOD
STORY WRITTEN FOR CBS NEWS "SPACE PLACE" & USED WITH PERMISSION
Posted: June 15, 2007
Russian computers aboard the international space station failed to boot up properly early today even though they were cut off from U.S. solar array power. Engineers had speculated that some subtle change in the station's shared power grid, caused by the installation this week of a new solar array, might have triggered the Russian computer crashes that have crippled the space station. But analysis of the electricity flowing from the new array into the Russian segment of the lab complex did not reveal any obvious "smoking gun" and when the circuit was unplugged, the computers behaved much as they did Thursday.
The station's main command-and-control computer system is made up of three redundant machines that can operate in stand-alone mode or as synchronized, fully redundant "lanes." The station's guidance and navigation system is known as the terminal computer, also made up of three redundant lanes. The computers normally draw power from the station's shared electrical grid, which includes electricity from U.S. solar arrays. The new array channel that was routed to the Russian segment earlier this week, roughly when the computer problems began, is known as power channel 3A.
Russian engineers "performed some more troubleshooting over the last two ground sites and basically, kind of a repeat of some troubleshooting they did yesterday," said space station flight director Holly Ridings. "If you remember yesterday, they'd been successful in bringing up the central computer and talking to it and getting a command down to the FGB (Zarya module) to get some power over from the U.S. segment to the Russian segment. Unfortunately ... that computer, the central computer, went off line again.
"And so today's troubleshooting was kind of a repeat of what they did yesterday, trying to turn on the central computer and then the terminal computer. And unfortunately, they did get power to both of those computers and get good feedback that they were receiving power, briefly had some what they call 'availability,' kind of like a heartbeat, on one of the lanes. There are three of them, of the central computer and the computer below it, the terminal computer, but were unable to communicate with it properly.
"So on the next ground site after they'd left the power to those computers on for about an hour and a half, they decided they would turn the power back off again and turn what we call an SNT (a Russian acronym pronounced ess-en-tay), which is how we send power from the U.S. power system to the Russian segment, back on. So we ended up in the configuration that we started out the day in, which was, unfortunately not having a central computer or a terminal computer. They're going to let the crew get some sleep ... and put together a forward troubleshooting plan."
The computer system has been acting up ever since the Atlantis astronauts attached a new set of solar arrays to the right side of the station's main power truss Monday. The terminal computer lanes initially crashed. Then, during a programmed reboot of both the terminal and central computer lanes, the entire system hung up.
The navigation system computers are required to fire Russian maneuvering jets to make major changes in the station's orientation. Minor adjustments are made with U.S. control moment gyroscopes, but that cannot make major changes and the system periodically has to be reset using rocket control. The station cannot safely operate without full orientation control to ensure its solar arrays stay face-on to the sun and to prevent sensitive systems from getting too hot or too cold.
The problem is not serious as long as the shuttle Atlantis remains docked because the orbiter's thrusters can be used, when needed, to make adjustments that are beyond the ability of the gyros or when the gyro system needs to be reset. But the shuttle will undock and return to Earth next week and the Russians have been working around the clock to get the computer glitches resolved before the orbiter departs.
Engineers theorized that the new S4 solar array, or components in the circuitry delivering that power to the Russian segment of the station, triggered some subtle change in the lab's electrical grid. The central and terminal computers, built in Germany by Daimler-Benz in Germany, are known to be sensitive to "noisy" power.
Late Thursday, outgoing station flier Sunita Williams, her replacement Clay Anderson and flight engineer Oleg Kotov used a signal analyzer to characterize the power flowing from the U.S. to the Russian segment. After that data was shared with the ground, the system was physically disconnected to make sure no ground path could be causing problems even with power shut down.
"One of the theories about why this is going on with the computers is that there's some interference from the new pieces of the power system that we put in with the S4 truss and possibly some electromagnetic interference or some other issue with the power that we're sending them across," Ridings said.
"So we asked the crew to go in and do some testing with the scope meter on the actual power cables that come from the new truss and power channel 3-alpha to those SNTs, the boxes that send the U.S. power over to the Russian segment. So they put the scope meter on those cables and looked at the waves ... so all our engineers could go off and assess.
"The engineers looked at that data, they did not find anything that was grossly off nominal, nothing huge jumped out at them immediately," Ridings said. "It would have been nice to find a smoking gun, but that's usually not how these things work."