Posted on 02/21/2003 10:07:39 PM PST by Noumenon
A computer whodunnit A batch of seriously bad components is disabling computers around the world
RACHEL ROSS TECHNOLOGY REPORTER
Thomas Geery turned on his computer one morning and heard a small explosion, like the distant sound of a child's cap gun.
Pop!
But Gerry, a computer technician from Herndon Va., knew it wasn't child's play. With more than 25 years experience on the job, he knew his computer had just blown a capacitor.
It's a scenario that has played out frequently over the past year, as a rash of faulty capacitors — battery-like energy-storing components vital to computers — have leaked or exploded in homes and workplaces around the world.
Smaller than a triple-A battery, these tiny electronic components aren't likely to cause serious injury to computer users. But a faulty capacitor can render a computer useless without warning.
And they are — by the thousands, if not millions. Capacitors in all kinds of electronic circuits seem to be missing some critical ingredients that keep them from exploding. It's unclear how this bad batch of capacitors came to be. Some say it's a case of bungled corporate espionage. Industry analysts say that something like this was bound to happen in such a highly competitive market.
Gary Headlee has replaced some 40,000 leaky capacitors on almost 1,500 computer motherboards over the past year, including Geery's. The electronics technician in Midvale, Utah, first encountered the problem when he found leaky capacitors on his own computer motherboard.
He told a couple of people about the repair and they asked him if he'd fix their boards too. Though none of the motherboard makers was really talking about it, it seemed a lot of people were having the same problem. Word spread quickly about Headlee's service and by the summer of 2002, the boards were pouring in.
"Over 10 (motherboards) a day come in at times," Headlee said. "And they just keep on coming."
The bad motherboards arrive from companies and consumers as far away as Greenland, Finland, Norway, Australia. Headlee's negotiating with a company, that he declined to name, which has 300 problem computers that have leaky capacitors.
"I'm seeing an epidemic of this," said Carey Holzman, who builds and sells custom PCs in Glendale, Ariz. "(The manufacturers) are saying, `We don't have that problem. We don't know what you're talking about.'"
But Holzman insists that this is well beyond the normal failure rates for such a critical and ubiquitous component.
Capacitors, like batteries, are found in many electronic devices. The two components both store electricity. But a capacitor can dump all its energy in a less than a second, while batteries take far longer to discharge.
Aluminum electrolytic capacitors — the kind that have been failing in record numbers lately — are just one kind of capacitor found in computer motherboards, camcorders, VCRs, televisions and car stereos. They are made, as the name suggests, with strips of aluminum and conductive fluid known as electrolyte. A piece of paper is dipped into the electrolyte and rolled up between sheets of aluminum, much like a jelly roll. The roll is put in what looks like a tiny cylindrical can with two prongs on the bottom.
The current problem, it seems, is with the electrolyte found in capacitors made by a variety of manufacturers. Headlee said he's replaced faulty capacitors made by at least a half dozen or so different firms.
How did bad electrolyte wind up in so many different places? The origins of the problem are as sketchy as they are intriguing. In the February issue of the trade magazine The Institute Of Electrical And Electronics Engineers' Spectrum, unnamed sources cited a bizarre tale of corporate espionage gone awry.
One version of the story says a scientist stole an electrolyte formula from his employer in Japan "and began using it himself at the Chinese branch of a Taiwanese electrolyte manufacturer."
That formula, according to this version, was later sold to an electrolyte maker in Taiwan which, in turn, sold electrolyte to capacitor makers from Taiwan and, possibly, elsewhere.
But something was wrong with the stolen formula. It didn't have the right additives. And that, according to industry analyst Dennis Zogbi, is what made the parts faulty.
Normally, an additive in the electrolyte prevents the formation of hydrogen gas inside a capacitor. If the additive isn't there, then hydrogen gas builds up inside the can.
"The capacitor essentially becomes a pressure cooker," said Bill Drake, a computer systems integrator from Vancouver, B.C. "Fortunately, there's a little seal on the capacitor which prevents it from turning into a grenade."
As the gas builds up, the top of the little can bulges up and sometimes the seal on the bottom pops off. Then electrolyte oozes out on to the circuit board, the capacitor starts to dry up, and the computer starts to fail.
"The board stalls and won't start up properly," Drake said.
Sometimes when a capacitor blows, it makes a popping noise, as the seal breaks off the bottom. Sometimes the whole computer starts to smell like fish. It's the effect of the leaking electrolyte and hydrogen gas.
Yet it's difficult for the average person to tell the difference between a problem with software and a problem with capacitors. That leaves many consumers looking for answers in all the wrong places, calling up software technical support hotlines and re-installing their operating system.
"Some people just think it's a software issue," said Geery, the Virginia technician. "They try to do operating system repairs and before they know it they can't even boot up their computer."
That may be the reason why it's taken more than a year for word of the problem to reach the masses: few people have the time and know-how to find the source of the problem.
A half-dozen or so computer repair shops in Toronto indicated last week that they weren't aware of any problem with aluminum electrolytic capacitors. All later admitted, however, that they don't actually look at the motherboard's components. If a motherboard isn't working, they just replace the whole thing instead of bothering to identify and replace faulty components. Their time is expensive relative to a computer motherboard, which generally costs between $150 and $300.
The other reason many people are still in the dark about a problem that's been going on for years, is because computer and computer component makers have been hesitant to talk.
Only two companies have admitted to the problem: IBM Corp. and motherboard maker ABIT Computer Corp. Both companies said they've since switched to other brands of capacitors and will replace faulty boards that are still under warranty.
But that wouldn't help people like James Laronde, a police officer in Brooks, Alta., who bought an ABIT. Laronde's capacitors failed about 10 days after his one-year warranty expired.
The 36-year-old said he didn't know there was a larger issue with capacitors on ABIT motherboards at the time, so he went ahead and bought another one.
"If I had known that, I would have gone with another brand," Laronde said.
Now he's having problems with the newer motherboard too, which is approaching the end of its warranty period.
Holzman thinks the company should have done more to notify consumers about the problem with the aluminum electrolytics.
"They still haven't formed a recall," he said. "I just don't think it's fair."
Jeremy Smith, public relations manager for ABIT U.S.A., said there's no way of knowing which of their customers might have the bad capacitors.
"There is a large pool of parts that come in, and (parts) are ordered throughout a year of production, so that we could not say that `lot xyz' or production runs of a certain board are bad," Smith said.
Representatives for both ABIT and IBM also pointed out that many other companies likely used the defective capacitors too.
"Although we have documented the aluminum electrolyte capacitor failure in a limited number of IBM desktop models, this component is common throughout the industry and used by several major U.S.-based PC manufacturers," IBM spokesperson Raymond Gorman said.
Analysts agree the problem is more widespread than manufacturers are letting on.
"I think anybody who uses contract manufacturers that outsource to Taiwan was affected by the problem," said Zogbi, who is also president of the Paumanok Group research firm of Cary, N.C.
Headlee said he's replaced leaking capacitors on just about every kind of computer motherboard "except Asus," including motherboards made by Micro Star International (MSI) and Gigabyte Technology Co. Ltd.
At press time, Gigabyte had not responded to questions concerning the leaky capacitors. A spokesperson for MSI in California denied its motherboards were affected by any kind of leaking-capacitor problem.
Headlee said not only were the faulty "caps" on several different kinds of boards, they also appear to have been made by several different capacitor companies, including Teapo Electronic Corp., Jackcon Capacitor Electronics Co., Lelon Electronics Corp. and Luxon Electronics Corp. But as the exciting tale of corporate espionage suggests, the problem may be further down the supply chain with one of the companies that makes the electrolyte fluid.
According to Spectrum magazine, a Taiwan company called Lien Yan has been accused of buying the stolen electrolyte formula and selling the faulty fluid to a variety of capacitor companies. Spectrum reports the company denies those allegations.
Two of the biggest Taiwanese capacitor firms — Lelon and Luxon — issued new releases lalast year stating they have never bought the kind of electrolyte in question from Lien Yan.
"Luxon definitely understands that the electrolyte is one of the most important materials for aluminum electrolytic capacitors," read the Luxon release.
"In order to ensure our reliability and innovative technology, Luxon always develops the electrolyte by ourselves."
Regardless of whether the Lien Yan rumours are true, the company's reputation has already been damaged.
Many companies, such as ABIT, have switched to capacitors from Japan. A spokesperson for Lien Yan told Spectrum orders were down 30 per cent since the problem was discovered.
Martin Reynolds, a research fellow for the Gartner Group in San Jose, Calif., said the problem is "almost certainly the result of cost cutting." As profit margins shrink and computer sales slow, manufacturers are forced to look for cheaper prices for component parts.
The fundamental problem is that brand-name companies are continually outsourcing to contract manufacturers who are forced, in turn, to find the cheapest suppliers, Zogbi said.
"Unfortunately, that does not equate to quality."
Why bother getting a better price on such a cheap component? With a couple of dozen aluminum electrolytic capacitors on each board, the cost of those small parts add up.
A recent issue of Zogbi's Passive Component Industry magazine estimated the global aluminum electrolytic capacitor market's worth at approximately $3 billion to $3.5 billion (U.S.) worldwide last year.
Zogbi believes this latest capacitor problem is just the tip of the iceberg when it comes to problems with the high-tech supply chain because motherboard manufacturers aren't the only part of the electronics industry eager to cut costs.
Headlee, who also repairs home entertainment systems, claims he's seen a significant increase in faulty aluminum electrolytic capacitors in Mitsubishi televisions with picture-in-picture modules, JVC VCRs and camcorders made by Sony.
Sony didn't respond to a written request for an interview, but JVC and Mitsubishi both said they were unaware of any such problems.
A spokesperson for JVC said its customer service centre wasn't reporting unusually high rates of complaints for their VCRs.
Mitsubishi pointed out that several of the televisions people discussed online as having problematic picture-in-picture modules were far too old to have been affected by the recent batch of bad capacitors.
Cayce Blanchard, director of corporate communication for Mitsubishi Electric & Electronics U.S.A., Inc., said some of the models discussed were far past their normal life expectancy of seven or eight years.
At that point, she said, it's to be expected that a capacitor might fail. A recent issue of Passive Component Industry magazine points to other systems that might have been affected by the bad electrolyte.
Zogbi said that, after writing his initial article outlining the problem, a company that makes power supplies for aircraft called to tell him they were going to start testing the aluminum capacitors they had bought from Taiwanese manufacturers.
Zogbi said he was also contacted by the U.S. Federal Aviation Administration "which noted they were conducting an inquiry into the aircraft electronics supply chain."
Zogbi noted that he wasn't aware of any problems so far and that the FAA was likely just being "cautious" by gathering information on the subject. A spokesperson for the FAA told the Star there was no indication such an inquiry was underway.
Despite the fact that some of the electronics makers have switched suppliers, the problem is far from over. It will likely play out in court in the years to come.
"We're investigating possible litigation in Taiwan with our parent company," said Smith of ABIT Computer.
Meanwhile, the story seems likely to continue playing out in Headlee's Utah repair shop. While several motherboard makers have switched capacitor brands, much of the bad batch might still be on the market or in homes around the world just waiting to pop.
Headlee said he's so confident the problem with persist, he's ordering replacement parts by the hundreds.
I've seen dozens of dead motherboards, but never one with any visible defect.
The "kaboom" of a failing cap can be quite invigorating. My big 12 VDC power supply had a 30,000 mfd electrolytic cap. I had it mounted with the terminals upward. A brownish insulating cap held the terminals and a small black cork in the center. One of the silicon diodes failed and applied the AC current from the transformer to the cap. I head a loud pop and immediately unplugged the power supply. The black cork was gone and the insulator was cracked upward with 3 big cracks. I found the black cork embedded in the ceiling.
You're right.
I also work in electronics. We build medical devices. The consequences of a failed capacitor can be much more serious than a flaky mother board.
Would like a good reliable MB for a Pentium 1.5GHz processor 512MB max RAM at least. Sounds easy, but I don't want to buy a motherboard only to find out it as slow I/O or something odd. Less than $100 would be nice as well.
didn't read the entire post. Question- 40K/year equals >100 replacements/day. Does this seem reasonable? Maybe it is, I don't know, please don't firebomb my house. Just asking.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.