New Hubble trouble stalls telescope reactivation
BY WILLIAM HARWOOD
STORY WRITTEN FOR CBS NEWS "SPACE PLACE" & USED WITH PERMISSION
Posted: October 17, 2008;
Updated after news conference
Work to switch the Hubble Space Telescope to a backup science data management system after a component failed last month has been interrupted by a pair of on-board glitches during the restart process, officials said today. Engineers do not yet know if the anomalies are related, whether any actual hardware failures are involved or even whether the data management reconfiguration played a role. But they are hopeful analysis of telemetry and computer logs will help them resolve the issue and resume normal science operations.
"The operations team is working diligently to understand the cause and options for proceeding," said Jon Morse, director of the astrophysics division at NASA headquarters. "We remain optimistic at this time for recovering full science operations. But even the best laid plans can encounter some unanticipated difficulties."
The original failure occurred Sept. 27 as NASA was preparing to launch the shuttle Atlantis Oct. 14 on a long-awaited servicing mission. Channel A of the telescope's control unit science data formatter, or CU/SDF-A, began acting erratically and the telescope's main flight computer, following pre-programmed instructions, "safed" the payload computer and science instruments.
An attempt by ground controllers to reset the formatter was not successful and engineers quickly determined the box had suffered a "hard" failure. With the formatter out of action, data from Hubble's operational science instruments could not be relayed to the ground.
The shuttle flight was delayed to mid February at the earliest to give engineers time to flight qualify a spare science instrument control and data handling system, a box that contains both A- and B-side electronics. In the meantime, engineers at the Goddard Space Flight Center in Greenbelt, Md., decided to switch Hubble over to its current B-side data management system to restore science operations. Those components have not been powered up since launch in 1990.
The switchover began Wednesday. Telemetry from the telescope indicated the initial transition went smoothly. Wednesday night, engineers turned on the B-side science instrument control and data handling system, which includes the B-side data formatter. They then confirmed the Wide Field Planetary Camera 2, the one operational channel of the Advanced Camera for Surveys and the Near Infrared Camera and Multi-Object Spectrometer, or NICMOS, were able to communicate with the B-side SI C&DH system. The instruments then were put back in safe mode pending commands to switch control to the SIC&DH.
Thursday, engineers tried to bring the science instruments back on line. That's when the first of two problems developed.
"On Wednesday, engineers at the Goddard Space Flight Center reconfigured six components of the Hubble data management system and five components in the science instrument command and data handling system to use their redundant, or what we sometimes call the B side," said Art Whipple, manager of the Hubble Systems Management Office at Goddard. "This work was to work around a failure that occurred on Sept. 27 in the side-A science data formatter, which is a part of that SIC&DH system. It resulted in a cessation of all science observations except for astrometry, which are done with the fine guidance sensors that don't go through the SIC&DH.
"The reconfiguration proceeded nominally and the Hubble resumed the science timeline at noon Eastern time on Thursday. The first activities out of that onboard science timeline were the commanding of the science instruments from their 'safe' to 'operate' mode. This occurred nominally for the Wide Field Planetary Camera 2 and for the Near Infrared Camera and Multi-Object Spectrometer (NICMOS). However, an anomaly occurred during the last steps of the commanding to the Advanced Camera for Surveys."
The Advanced Camera for Surveys, installed in 2002, suffered an earlier failure that knocked out its visible light wide-field and high-resolution cameras. Only its solar blind channel, which is sensitive to ultraviolet light, is still in operation. The Atlantis astronauts hope to repair the ACS during the upcoming servicing mission.
"At 1:30 p.m. Eastern (Thursday), when the low-voltage power supply to the ACS solar blind channel was commanded on, software that was running in the microprocessor in ACS detected an incorrect voltage level in the SBC and suspended ACS," Whipple said. "Then, at 5:14 p.m., the Hubble spacecraft computer, the 486, sensed the loss of a keep-alive signal from the NASA standard spacecraft computer in the C&DH and correctly responded by safing the SIC&DH and the science instruments.
"At this time, it's not known if these two events were related. The investigation into both anomalies is underway. All the data's been collected and it's being analyzed here at Goddard. The science instruments will remain in safe mode until the SIC&DH issue is resolved. All other systems on the spacecraft are performing nominally."
Asked if the telemetry had shed any light on what sort of problem - hardware failure, commanding error or some sort of misconfiguration - might be responsible, Whipple said "we're in the early stages of going through a mountain of data that has been downloaded over the last 24 hours."
"At this point, we are fairly certain, although nothing's been 100 percent ruled out, but we're fairly certain it is not a configuration or a commanding error," he said. "We are not to the point where we can rule out either transient issues or, for that matter, hard failure. We're just not there yet."
The initial fault in the Advanced Camera for Surveys occurred when a computer inside the instrument failed to detect the required 8 volts from the low-voltage power supply. Several hours later, the 486 flight computer, which constantly monitors the status of the payload computer, detected an apparently loss of keep-alive power in the B-side SIC&DH computer.
Whipple said the actual transition to Hubble's B-side science data management system went smoothly, with no apparent problems.
"There really are no changes that we're seeing in any of our other telemetry," he said. "The behavior of those six data management system components we switched over has been absolutely perfect so far."
But, he added, "we changed a number of things in the configuration of the spacecraft. It was not unexpected that there might be issues. This is, in fact, one of the contingency cases we thought a great deal about ahead of time and we're not totally unprepared for. ... We expect we will work through it, we will be back up and doing science between now and the servicing mission. The team is doing well. I can't tell you they'll get the entire weekend off, but we're cognizant of the fact that this is a marathon, not a sprint. It's important that we do things right, rather than fast."