Shuttle managers outline plans to monitor fuel sensors
BY WILLIAM HARWOOD
STORY WRITTEN FOR CBS NEWS "SPACE PLACE" & USED WITH PERMISSION
Posted: July 24, 2005
NASA managers today cleared the shuttle Discovery for another launch try Tuesday, weather permitting, on a critical flight to service and resupply the international space station. The decision came after senior managers agreed on a strategy that would permit blastoff even if - and only if - the shuttle experiences a fuel sensor problem like the one that grounded the ship July 13.
Asked if he was concerned about any public perception NASA had "launch fever" and was not giving the sensor issue the attention it might deserve, Griffin said "I think what you want coming out of the Columbia accident and the loss of Columbia and the soul-searching examination that NASA has undertaken since then, what you want of NASA is that we make the right technical decisions, that we do the right thing to the extent that we can figure that out. Which is hard.
"We can't restrict the range of our options to those things which are going to present well. We have to figure out the right thing and try to do that and then work hard to try to explain to you why it is the right thing. And these are rather arcane matters, I would admit. They're rather difficult. And sometimes they don't always present well. (But) you want us doing what's right, not what's necessarily obvious or popular."
Wayne Hale, chairman of NASA's mission management team, put it like this: "I wake up every day and I ask myself are we pushing too hard, are we doing this thoroughly, have we done the right technical things, have we asked the right people, have we built the tests properly? I am committed, and I think the whole team is committed, to doing this in a safe manner.
"I think we're all still struggling a little bit with the ghost of
Columbia and therefore we want to make sure we do it right. This has been a
very vexing problem. ... I would tell you that based on the last 10 days
worth of effort, the huge number of people and the tremendous number of
hours that have been spent in testing and analysis, I think we're coming to
the right place."
But the real question mark is how Discovery's fuel sensors will operate when the shuttle's external tank is loaded with supercold rocket fuel early Tuesday.
Despite exhaustive, around-the-clock tests, engineers were never able to find the problem that caused hydrogen main engine cutoff - ECO - sensor No. 2 to "fail wet" during the July 13 launch attempt. The ECO sensors are part of a backup system that ensures the shuttle's main engines shut down normally before the tank runs out of fuel. NASA launch rules require all four ECO sensors to be operating before a countdown can proceed.
But in a routine pre-launch test during Discovery's first launch try, ECO sensor No. 2 continued indicating it was wet after computer commands were sent to simulate a dry tank. The countdown was called off and engineers mounted a major effort to track down the cause of the problem.
They initially were hopeful that electrical interference caused by subtle grounding problems might explain why sensor No. 2 failed to respond properly. The grounding problems were fixed, but troubleshooters were never able to duplicate the unexpected sensor behavior.
Even so, they believe the testing completed to date proves the problem, whatever the cause, is not a generic defect and as such, they are confident it will not affect the other sensors.
"We have literally run every check that we can think of that people could suggest to us to try to find this problem and so far, no repeat," said Hale. "So we have developed a plan that says we have to go to cryogenic temperature to find out what's going on next.
"We also need to turn on all the equipment in the orbiter and the launch pad area to see if there's any electromagnetic interference that we could not check piece wise earlier. So both of those things together say we are ready to go the launch countdown configuration to see our next level of checking."
Engineers plan to begin pumping liquid oxygen and hydrogen rocket fuel into Discovery's external tank around 12:30 a.m. Tuesday. Two minutes after the hydrogen ECO sensors are submerged in super-cold hydrogen, engineers will begin sending commands to simulate a dry tank to make sure none of the sensors have "failed wet."
Then, just before the crew straps in around 7:19 a.m., the commanding will stop and the main countdown computer will carry out a series of checks to verify the sensors change state, from wet to dry and back again, on command. A final check is planned about a half-hour before liftoff.
"We are going to continuously check the validity of the signal from the sensor all the way to the orbiter computers during the tanking (process) so if at any time any one of these circuits fails, we will know about it, we'll be able to isolate time-wise when it happened, we'll be able to know what was going on in the ship, either electrically or thermally, so that will give us a clue to where the problem might lie," Hale explained.
To help isolate the problem during fuel loading, engineers swapped the wiring between ECO sensors 2 and 4. If a problem shows up during fueling with sensor No. 4, engineers will have high confidence the problem is in the sensor itself or somewhere in the wiring between the sensor and the point sensor box. If sensor No. 2 misbehaves, they will have high confidence the problem is in the point sensor box.
Because electrical interference could still be a contributing factor, engineers will conduct checks during the final hours of the countdown to isolate any such signatures.
Here's the ECO sensor launch strategy at a glance:
Because of the wiring swap, "if the problem recurs, it will give us an indication of whether the problem is in our famous point sensor black box or in the wiring or the sensor itself," Hale said. "So we'll know that. And we've defined a very rigid set of requirements and tests to be done if this problem re occurs. We also know that we've done a lot of moving around in the aft, we've mated and demated connectors, we've wiggled a lot of wiring, we've tested the point sensor box extensively and it is possible that we have caused whatever the problem was to go away.
"So if the problem doesn't recur, we feel we have good redundancy to go fly and that's why we are prepared, following the tests to ensure all these sensors are working right ... to load the crew up and go fly."
But if sensors 2 or 4 act up, "then we're going to do some more tests just to make sure we understand what's causing that to happen," Hale said. "And if we're comfortable that we have a good understanding of the cause, then we can go fly for those specific two cases. If anything else happens - if we have any of the liquid oxygen sensors fail, if we have a hydrogen sensor that fails to a dry state instead of the wet state, if we have a sensor that fails on channel 1 or 3 or multiple sensor failures, anything like that happens - we're going to stop because that says we really need to do more testing."
But launching with three operational sensors would violate NASA's launch commit criterion requiring all four to be working at launch.
The agency's original launch rule required three operational ECO sensors for a countdown to proceed. But in the wake of the 1986 Challenger disaster, the LCC was amended to four-of-four because of concerns two sensors could be knocked out by a single failure in an upstream electronic black box known as a multiplexer-demultiplexer. The single-point failure was corrected during Discovery's last overhaul, but the four-of-four launch rule remained on the books.
And that rule will be in place Tuesday. But the mission management team is prepared to sign an exception to the rule - permitting Discovery to launch with three of four ECO sensors - depending on what happens after the tank is loaded with fuel.
Detailed background on how the sensors work and a chronology of sensor-related problems affecting Discovery's launch processing flow, is available on the CBS News/Spaceflight Now ECO sensor page.
See the Status Center for full play-by-play coverage.