PG NewsPG delivery
Pittsburgh Post-Gazette Home Page
PG News: Nation and World, Region and State, Neighborhoods, Business, Sports, Health and Science, Magazine, Forum
Sports: Headlines, Steelers, Pirates, Penguins, Collegiate, Scholastic
Lifestyle: Columnists, Food, Homes, Restaurants, Gardening, Travel, SEEN, Consumer, Pets
Arts and Entertainment: Movies, TV, Music, Books, Crossword, Lottery
Photo Journal: Post-Gazette photos
AP Wire: News and sports from the Associated Press
Business: Business: Business and Technology News, Personal Business, Consumer, Interact, Stock Quotes, PG Benchmarks, PG on Wheels
Classifieds: Jobs, Real Estate, Automotive, Celebrations and other Post-Gazette Classifieds
Web Extras: Marketplace, Bridal, Headlines by Email, Postcards
Weather: AccuWeather Forecast, Conditions, National Weather, Almanac
Health & Science: Health, Science and Environment
Search: Search by keyword or date
PG Store: Pittsburgh Post-Gazette merchandise
PG Delivery: Home Delivery, Back Copies, Mail Subscriptions

Headlines by E-mail

Headlines Region & State Neighborhoods Business
Sports Health & Science Magazine Forum

Wiring, air-cooling systems go in as assembly of Terascale approaches

Setting the stage for the supercomputer

Monday, April 16, 2001

By Byron Spice, Science Editor, Post-Gazette

Lynn Layman spent 2 1/2 hours in a conference call last week, discussing the length of computer cables. It was just as dull as it sounds.

But for Layman, manager of the Pittsburgh Supercomputing Center's computer facility in Monroeville, it's also a subject of critical importance. Engineers are working out the final configuration of the center's new computer, the Terascale Computing System, and the length of cables is one of those niggling details that could have an out-size effect on its performance.

At the Westinghouse Energy Center in Monroeville, 8-inch cooling pipes and wiring by the yard have been installed above the ceiling in preparation for assembly of the Pittsburgh Supercomputing Center's Terascale computer. (Steve Mellon/Post-Gazette)

"It could really drive the entire system," Layman said.

In designing the fastest computers in the world, computer engineers constantly fight a battle against the speed of light. It takes electrical signals a billionth of a second to travel a meter, so each extra meter of cable represents a delay of a billionth of a second.

And when you're building a computer such as the Terascale, which is supposed to be capable of 6 trillion calculations per second, a delay of a few billionths of a second can seem like forever.

The Terascale, or TCS-1, will be the fastest computer available to researchers outside the U.S. defense program when it is finished this September. The National Science Foundation awarded the Pittsburgh Supercomputing Center $45 million for the three-year project last year.

Investigators already are clamoring to use the new machine, said Ralph Roskies, one of the center's scientific directors. The first pieces of the TCS-1 won't arrive until the first week of June, but a smaller, initial configuration, dubbed TCSini, is up and running. As of April 1, it began its "production mode," which means it is available for use by scientists and engineers nationwide.

A hint of things to come

TCSini's performance is less than 5 percent of that of the final Terascale system, but "it is no slouch of a machine," Roskies said. It already is the 70th-fastest computer in the world, capable of up to 342 billion calculations per second. When the National Science Foundation allotted researchers time on the new machine this spring, "there was way more demand than we could satisfy," he added.

Even before TCSini entered production mode, it had been banging away for months, performing calculations for a select group of "friendly users" willing to put up with occasional disruptions.

"This is really a dream for the biomedical researchers today," said one of the friendly users, biophysicist Klaus Schulten of the University of Illinois. He used the computer to simulate "mechanosensitive channels," proteins involved in translating stress applied to cells into the nerve impulses that are perceived as the senses of touch and hearing.

Simulating the action of this protein -- in this case, a bacterial form -- requires calculating the behavior of 100,000 or more atoms. On previous computers, this would take years, an impractical amount of time. But using the TCSini, Schulten completed his study in two months. "It was a wonderful result," confirming behavior seen in cell studies, he said. His report on the work will appear next month in Biophysics Journal.

"This is going to open up a new era of understanding of cells," Schulten predicted.

Likewise, Doyle Knight, an aerospace engineer at Rutgers University, used TCSini to simulate the turbulence that occurs along aircraft surfaces at supersonic speeds. It's a fundamental problem that has baffled engineers since the 1940s and, as a result, has forced them to "over-engineer" the wings and other structures to make sure they don't break apart at supersonic speeds.

The calculations now possible with TCSini -- and ultimately, the full Terascale machine -- should enable aerodynamicists to design more efficient wings without relying on trial and error, he said. He will be presenting his findings at an aerospace meeting in England this fall.

Schulten already is looking forward to using the bigger machine, planning a one-million-atom simulation of the protein-making molecule called the ribosome. Knight, too, has plans for more detailed calculations.

"It's not too difficult to make the problem more difficult," Knight added.

The Terascale Computing System eventually will have a footprint bigger than a basketball court. But until the components begin arriving in June, two truckloads a week, workmen have been busy preparing the computer room in the basement of the Westinghouse Energy Center.

The same room once housed three of the Pittsburgh Supercomputing Center's big Cray supercomputers and several smaller machines, as well as Westinghouse's own array of IBM machines. But the TCS-1 is a very different animal than those previous machines, necessitating $750,000 in renovations to the space and the removal of all of Westinghouse's IBMs.

The last of the Crays, a T3E model nicknamed Jagomir, continues to crank away in one corner. Like other Crays, it is notable for its small size. Its 512 Alpha processors are tightly packed into two 6-foot-high, 6-foot-wide, 3-foot-deep boxes to keep cable lengths to an absolute minimum. All of the processors are immersed in a liquid coolant to keep the circuits from frying themselves.

This sort of highly integrated machine was possible when the Cold War was at its height and the big-budgeted nuclear weapons laboratories were demanding faster computers for simulating nuclear explosions. But that approach is no longer economically practical for either the defense labs or the academic community. So the next generation of machines will rely on linking large numbers of off-the-shelf, commercially available components.

Supercomputers, once compact, now are sprawling. The Terascale system will use 750 of Compaq's latest generation of AlphaServer computer, each of which contains four computer processors. (The 256-processor TCSini was built with the previous generation of AlphaServer and will be dismantled.)

Layman said the AlphaServers will be housed in 150 8-foot-high cabinets, each about 3 feet wide and 4 feet deep. To minimize the length of interconnecting cables, the cabinets will be arranged in concentric circles, with switching equipment in the center.

As of last week, engineers had arranged the boxes so that the longest cable would measure 39 meters, Layman said, but are hoping to find a way to reduce that length.

While the Crays relied on internal, liquid coolants, the AlphaServers will be air-cooled, just like a personal computer. To carry away the heat generated by 3,000 computer processors, Westinghouse has installed 12 air handlers -- air conditioning units with a combined cooling capacity equivalent to that obtained from melting 360 tons of ice per hour.

Those air handlers, in turn, required the installation of new 8-inch-diameter pipes that can circulate up to 900 gallons of chilled water per minute.

The preparations also are being made with an eye to expanding the Terascale machine, Layman said. It's likely the National Science Foundation will eventually decide to boost the machine's power, adding more processors to push its performance up to 20 or 25 trillion calculations per second.

As Compaq technicians begin setting up the new AlphaServers in June, the Pittsburgh center will start putting them to work, harnessing them together to do supercomputing. "Very rapidly, perhaps within a month, we will 'break the speed of sound,' " said Michael Levine, the center's other co-scientific director, reaching computing speeds not previously clocked.

"This is terra incognita," he added. In the past, new Cray systems were built and tested by Cray Research before delivery, but the Terascale machine has never been assembled before, so engineers at the center will be faced with testing a new machine for the first time.

The project also has required a lot of interaction between the center and the Compaq engineers to make sure that components originally designed for business and engineering applications are capable of meeting the demands of high-performance scientific computing. "We're not designing parts, but we are integrating them in a novel manner," Roskies said.

The experience to date with TCSini, however, has been encouraging. "It has behaved extremely well," Roskies said. "It has proven very stable."

bottom navigation bar Terms of Use  Privacy Policy