Shuttle launch delayed to July 1 for tank sensor swap
BY WILLIAM HARWOOD
STORY WRITTEN FOR CBS NEWS "SPACE PLACE" & USED WITH PERMISSION
Posted: March 14, 2006
The shuttle Discovery's launch on the second post-Columbia mission has been delayed to at least July 1 because of work to replace suspect engine cutoff - ECO - sensors in the ship's external tank. Shuttle program manager Wayne Hale made the decision Tuesday, after two days of detailed engineering discussions, even though the issue was not an open-and-shut case and even though the sensor in question may be good enough to fly. In the end, Hale decided to err on the side of caution in a bid to resolve, once and for all, questions about the sensors that have lingered since Discovery's launch on the first post-Columbia mission last July.
"This was not an easy decision," Hale told reporters late today. "We had quite an interesting debate, pros and cons, looked at every possible way around this and finally concluded it was far smarter for us to be conservative and to take the safe route and replace the sensors that are in the tank.
"That will take us about three weeks of work and that, of course, will move us out of the May launch window for STS-121. So today, we are proposing that the no-earlier-than launch date, the earliest possible launch date, would be July 1."
It was the loss of a large piece of insulation that doomed Columbia in 2003 and the loss of another large piece of foam during Discovery's launch last July that has held up subsequent flights. NASA currently is conducting wind tunnel tests to make sure changes to the tank's insulation are safe and Hale said today he does not yet have enough data to make a final decision.
But assuming the foam modifications are good to go, Discovery must be launched in daylight for photo-documentation to verify the insulation's performance. Because the shuttle is flying to the space station, it must launch into the plane of the lab's orbit and that, coupled with a daylight launch constraint and other factors, limits when NASA can make a launch attempt. The next "window" opens July 1 and closes July 19.
"We, in fact, will be ready, we think, with the vehicle before July 1, but we are dedicated to launching in the daylight so we can watch what happens to the external tank and the rest of the flight vehicle during the daylight for at least two more flights, STS-121 and the subsequent STS-115," Hale said. "So we are aiming now for July 1."
The new launch date will give NASA time to resolve a variety of other challenging issues, including analysis of the foam modifications, what to do about main propulsion system (MPS ) contamination, what to do about MPS seals that may not meet specifications and recent damage to the shuttle's robot arm that could force NASA to use one taken from Endeavour. Given the launch delay, however, NASA may be able to fix Discovery's arm in time for launch in July.
The ECO sensors are located at the base of the hydrogen section of the shuttle's huge external tank. The sensors are part of a backup system intended to make sure the shuttle's three main engines shut down before they completely drain the tank. Running the tank dry could cause powerful turbopumps to cavitate and fail, with potentially catastrophic results.
The sensors use platinum wires whose electrical resistance depends on temperature. When the sensors are submerged in ultra-cold liquid hydrogen, resistance is extremely low. As the fuel level drops and the sensors become exposed, the temperature - and resistance - goes up. The change in resistance is monitored by a so-called point sensor box that, in turn, sends data to the shuttle's flight computers.
Sensors can "fail wet" or "fail dry." Failing in the wet state is considered relatively benign because the shuttle is launched with more propellant than it actually needs and because the sensors are, in essence, a backup system. If sensors fail wet, the engines would continue to run and the shuttle's computer system presumably would shut them down on time based on meeting orbital requirements. But if two sensors failed in a dry state, the computers would be misled into believing the tank was nearly empty and the main engines would be ordered to shut down early. Unless it happened extremely late in the climb to space, that could trigger a dangerous, untried abort scenario.
"Just like your car, you don't want to run it out of gas, that's not a good thing," Hale said. "You'd like to get where you want to go before the tank is completely empty. So the normal planning for a mission allows us to achieve the right orbital conditions - altitude, speed, direction of travel - without running out of propellant. In fact, we load extra fuel on board to make sure that even if we have small variations in the performance of the vehicle during launch, a small reserve is there to make sure we get to that point in the sky without running out of gas.
"The sensors are there in case we have some kind of performance problem, which we have had twice in the history of the program."
In one case - the launch of Challenger on mission STS-51F in July 1985 - a main engine shut down early because of an engine sensor problem, triggering an abort to a lower-than-planned orbit. As a result, the two remaining engines had burn longer than usual to make up the shortfall and the shuttle essentially ran out of gas below the desired altitude. The ECO sensors performed normally and forced engine shut down before the tank was completely dry.
The only other instance of ECO sensors playing a role in a flight came in 1999 during launch of the Chandra X-ray Observatory aboard the shuttle Columbia. Because of a hydrogen leak and an electrical short circuit, Columbia's tank was nearly drained and the ECO sensors, once again, came to the rescue.
"In both cases, the sensors, through the on-board computers, correctly told us that the tank was dry, we were out of fuel and we should shut the engines down," Hale said. "You like to shut the engines down with just a little bit of gas left in the lines to make sure those pumps, that pump the hydrogen and the oxygen into the engines, don't cavitate as they spin down. That's not good for the engines and it can lead to a number of problems, so we have in place these sensors on both the fuel side and the oxygen side."
During Discovery's launch campaign last year, however, the ECO sensor system experienced a variety of subtle problems. Engineers were never able to trace the issue to an obvious fault and in the end, shuttle managers opted to replace the fuel tank. Even then, problems remained, but engineers believed the only likely worst-case result would be one sensor failing wet. After a detailed analysis, they approved a plan to launch Discovery with three of four working sensors for a limited set of circumstances. As it turned out, all four sensors worked normally on launch day and Discovery's ascent was uneventful from an ECO sensor standpoint.
But NASA launched an extensive engineering evaluation that ultimately made a tentative connection between changes in the resistance of a sensor and possibly loose wiring leading to the detectors. And as it turned out, one sensor in Discovery's current tank - ECO-3 - showed a small two-ohm resistance shift during a test prior to shipment to the Kennedy Space Center.
The tank was shipped anyway because a sensor swap-out, if one was ordered, would be easier to carry out with the tank in a vertical position, something not possible at Lockheed Martin's Michoud Assembly Facility near New Orleans.
"During the course of this investigation over the last several months, they found that there may be a problem in the manufacturing of these sensors and that problem is in the way the wires are attached to these low-level sensors," Hale said. "There is a place that the wires attach to the sensors called a swage fitting. That swage fitting in some sensors that have been removed some time back in the history of the program have been noted to be a little loose and that's caused intermittent readings, varying resistance, in the sensor, which of course is how the sensor tells you whether it's reading a dry or a wet signal.
"Last year, when we prepared to launch STS-114, we had a high degree of confidence the sensors would only fail, if they were to fail, in the wet reading condition. There is now some body of evidence that would indicate it's possible for the sensors to read erroneously dry when the tank is not, in fact, dry. A predecessor indication that something may be going on in the sensors is a shift in the resistance over time, particularly after the tank has been transported, vibrated, rattled around a little bit.
"We, in fact, have a sensor in the external tank slated for the next flight that's showing a very small shift in its resistance reading, well within any previously established specification. However, because of the new knowledge we have this year and the ongoing engineering work that's looking at how these sensors work and how they may potentially have problems, we are taking the step of removing the external tank sensors from the bottom of the liquid hydrogen tank, from the tank we're going to use for the next space shuttle flight."
The current ECO sensors were manufactured in 1996. The new sensors were built in 2002 and are believed to be immune to the loose wiring/swage fitting phenomenon. Ultimately, Hale said, the ECO sensor decision was driven by safety and the need for engineering forensics, not schedule.
"This is what we call a criticality 1 (system), life-or-death kind of situation that you want those sensors to work properly, either way," he said. "They can prevent bad things from happening if they work properly and certainly, if they work badly they can cause bad things to happen. So we need to have a good set."
To replace the sensors, engineers from the Michoud Assembly Facility, working in the Kennedy Space Center's Vehicle Assembly Building, will remove foam insulation from the very bottom of the tank, unbolt a large manhole cover and get inside the cavernous hydrogen section. Once the new sensors are in place, the manhole cover will be bolted back in place and, after lowering the tank to a horizontal orientation, foam insulation will be re-applied.
"We hope to take the four sensors that we take out and put them in extensive tests," Hale said. "We want to see if the one sensor that's got this slightly elevated resistance reading really has this problem that the engineering tests say it could potentially have and then, of course, we will look at the other three sensors which were manufactured about the same time in the same facility.
"We have a number of these sensors in tanks that are still slated to fly. The sensor in question was made 10 years ago, in 1996, and passed all its acceptance tests."
The additional six weeks of work "should provide us plenty of time" to wrap up ongoing analysis of foam modifications, as well as a variety of other issues, including what to do about contamination on a liquid oxygen filter screen leading to one of the ship's main engines and questions about critical seals between the engines and the main propulsion system plumbing.
"I think we'll be in good shape to look forward to a launch about the first of July," Hale said. "I remain optimistic we'll still be able to get three missions in this year, but I don't have the details on where we will fly the next two missions. Later in the fall, I'm sure."