year. Written by a member of their Trustworthy Computing group, this report exhibits more spin than the teacup ride at Disney Land.Posted on 01/30/2008 5:15:59 AM PST by ShadowAce
Setting the Stage
Microsoft has released a self-analysis of Vista that proclaims among other things that Vista has less found vulnerabilities than any other OS in their first year. Written by a member of their Trustworthy Computing group, this report exhibits more spin than the teacup ride at Disney Land.
It’s an interesting way to get a favorable number — to say that the code base is brand new, thereby resetting all the expectations. Wrong way, guys, newer code always has more bugs in it, you just counteracted the whole point. Never mind the fact that Vista’s feature set could arguably (and kindly) be called XP Service Pack 4, and totally ignoring the fact that it took 5 years to come up with and release this new set of code/features/functionality
We’re Not as Bad and Getting Better
Even more amusingly is the continuing notion that a lesser number of found flaws somehow equates to a more secure code base. This ignores some basic software concepts, such as the fact that every code base has flaws, it’s just how many flaws there are and when/if they are found that’s the difference.
The “Security through Obscurity”, or “Our code sucks less” mentality isn’t new, but it’s about as accurate a measurement as body counts in Vietnam were, and about as good an indicator of winning the war on bugs and flaws.
In order to make a good comparison, let’s see what the security research community has to say about numbers of bugs in lines of code:
Commercial software typically has 20 to 30 bugs for every 1,000 lines of code, according to Carnegie Mellon University’s CyLab Sustainable Computing Consortium.
Doing a Number(s) on Windows
There exists an excellent whitepaper and discussion of Security Vulnerabilities in Software Systems from Colorado State by Alhazmi, Malaiya and Ray that clearly lays out a formulaic method for determining bugs in code, which is very helpful in understanding the issues of bugs in code. Wordpress isn’t the easiest to do mathmetical notations in, so I’ll spare you the nifty formulas and just use a calculator and some common terms to explain the deal.
Note: LOC = Lines of Code, SLOC = Source LOC, KLOC = 1,000 LOC and MLOC = Million LOC and BKLOC = My own notation for Bugs per KLOC.
According to published sources, Microsoft XP contains 40,000,000 lines of code, so let’s do the math to see how many possible bugs in XP total:
40,000,000 / 1,000 (chunks of code) = 40,000
x 25 (bugs per chunk) = 1,000,000
Even taking the number of possible bugs down to the lowest approximation (20) nets you almost 800,000 possible bugs in Windows XP.
Since the report is about Vista, let’s do the same analysis for the 50,000,000 lines of code in Vista to see how many possible bugs total:
50,000,000 / 1,000 (chunks of code) = 50,000
x 25 (bugs per chunk) = 1,250,000
Let’s take Microsoft’s word and assume (very generously) that they’ve improved their security dramatically, so for the sake of argument we can cut the number of possible bugs in half, getting:
50,000,000 / 1,000 (chunks of code) = 50,000
x 12.5 = 625,000
Jeffrey Jones’ report shows us that there have been 66 flaws reported in the first 6 months of Vista’s life, so extrapolating that out to 112 in a year, it’s going to take a while to get significant buggage out of Vista.
Doing a Number(s) on Linux/OSS
The aforementioned Stanford University study comparing security between proprietary and open source code found:
According to a four-year analysis of the 5.7 million lines of Linux source code conducted by five Stanford University computer science researchers, the Linux kernel programming code is better and more secure than the programming code of most proprietary software.
The number of possible vulnerabilities per 1,000 lines of code for the Linux Kernel?
The study identified 0.17 bugs per 1,000 lines of code in the Linux kernel.
The latest Kernel 2.6 project numbers from the Ohloh.net project pages puts the number of lines of Kernel Source code at about 10,000,000, so let’s do the numbers:
10,000,000 / 1,000 (chunks of code) = 10,000
x .17 (bugs per chunk) = 1,700
Ah, but you may say we aren’t comparing the same number of lines of code, and that’s true, so let’s do just that:
50,000,000 / 1,000 (chunks of code) = 50,000
x .17 (bugs per chunk) = 8,500
There you have it, even if the Linux Kernel were 5x it’s size, there would still be less than 10,000 bugs.
Other Comparisons
It’s not just the Linux Kernel that’s been analyzed for security, the LAMP (Linux, Apache, MySQL, PHP) stack has undergone scrutiny for bugs per lines of code, and Coverity recently did a source code analysis of 32 Open Source projects with an average bug density of .43 bugs per thousand lines of code, with the LAMP stack scoring better than average when compared to the other projects.
Let’s not forget the Great-Grand-Daddy of all code comparisons, “More than a Gigabuck: Estimating GNU/Linux’s Size” by David Wheeler. This paper and it’s conclusions did a lot to help convince and demonstrate the value of the Open Source Community and development process.
Summary
Let’s call it what it is: Vista, even if it’s only 1/2 as buggy as XP is still vastly more buggy than the comparable number of lines of Linux Kernel or Open Source code. Microsoft has admittedly come a long way with their security improvements, but the numbers tell us that there is quite a distance to go before Vista can be proclaimed as being more secure or less buggy than Open Source code.
I have a tangential vent:
My working-class son bought a Sony Vaio laptop from Best Buy, which I regard as his 1st mistake. He did not consult with me or his older brother (2nd mistake), and ended up getting a Vista OS— which is all that Best Buy offers, whether laptop or PC.
The laptop is 2 months old and piled up with crap. [I never use that word here, but there is no other that adequately describes the situation.] It came with several protective programs, but still picked up a virus somewhere.
He has been consulting Geek Squad out of Best Buy (3rd big mistake).
I am having the blinking laptop reformatted to XP and have explained to my son that companies such as BB are interested solely in acquiring a customer’s money, not the customer’s satisfaction, that entities like Geek Squad function to alleviate the unknowing customer of any remaining money in the wallet.
In addition to all of this, the Vaio’s 2 month-old hard drive was defective, but, thank goodness, that’s all on Sony. So, my son’s college-attending wife is without a laptop for two weeks.
Total dollars spent: $1,200+
A pox on them all.
It's just harder when that "someone" is a close family member.
You nailed Geek Squad squarely on the head. They are not technical people, and conversely, should someone with competent technical abilities somehow get hired on to the Geek Squad, their lives will be made so miserable that they end up quitting. Geek Squad is merely a sales force trained in technical jargon. I always get a good laugh out of conversations with them.
Your vent is well-founded and can be extrapolated out to the nth degree in many other instances. Perfect example:
I built a family friend a system from random parts in my box-o-stuff. She was running a 333 MHz PC with 192 MB of RAM. It was time for an upgrade. I scrambled an Intel mobo, some Micron RAM and a 128 MB video card for a 2 GHz system with 2 GB of RAM. She was happy as a pig in excrement, to say the least!
She decides she wants to get another machine for her ex-husband, and she finds HP selling a system complete with everything (monitor, etc) for $600 at Circuit City. I explained to her that we could just reset the system I built for her to default and she can take the new system for herself. She was content with that idea.
Well when I got that HP, wow... Vista, 2 GB of bloatware on the 500 GB HDD, etc. I said, “To heck with this,” and busted out the XP CD for a fresh install. Unbeknownst to me, HP is only doing Vista and only supporting Vista from their site for all desktop systems. We’re an HP shop on the server side, but retail desktops have been out of my life since the mid-90s.
Long story short, I get XP running and have to pore through documentation on HPs site to find out the chipset information, etc. Moral of the story: consult a techie in the tech world, not a retail tech nerd who spends 99% of his time working to pay for an overpriced Alienware machine on which to play the latest shooter.
My last laptop was a Sony I bought from Best Buy (won’t do that again), and it also had a hard drive failure. But (after Best Buy refused to honor their scam warranty plan) Sony sent a technician to my office the next day to replace the hard drive. They even gave me two new batteries because mine wasn’t holding a charge as long as it should.
Your son should ignore Best Buy and deal directly with Sony. They’ll get him up and running a lot faster.
Thanks for the suggestion. I’ll pass it along.
We were going to reformat the laptop ourselves. But laptops are tough. The XP disk couldn’t find the hard drive— turns out always to be the case when backing up to XP. So you have to go into BIOS and change some setting. Then you have to go out and find all of the XP drivers for all of your hardware and burn them to a disk. Since there’s no floppy drive, you have to “slipstream” the drivers onto the hard disk. That’s where I drew the line, and why I’m letting Geek Squad do this. They foul it up, they fix it.
Nertz to Microsoft and Vista! Up with Kubuntu!
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.