If You Haven't Donated Yet
Please Help To End The FReepathon
By Clicking here!!
Posted on 07/07/2011 8:55:49 PM PDT by TenthAmendmentChampion
According to database pioneer Michael Stonebraker, Facebook is operating a huge, complex MySQL implementation equivalent to “a fate worse than death,” and the only way out is “bite the bullet and rewrite everything.”
Not that it’s necessarily Facebook’s fault, though. Stonebraker says the social network’s predicament is all too common among web startups that start small and grow to epic proportions.
During an interview this week, Stonebraker explained to me that Facebook has split its MySQL database into 4,000 shards in order to handle the site’s massive data volume, and is running 9,000 instances of memcached in order to keep up with the number of transactions the database must serve. I’m checking with Facebook to verify the accuracy of those numbers, but Facebook’s history with MySQL is no mystery.
The oft-quoted statistic from 2008 is that the site had 1,800 servers dedicated to MySQL and 805 servers dedicated to memcached, although multiple MySQL shards and memcached instances can run on a single server. Facebook even maintains a MySQL at Facebook page dedicated to updating readers on the progress of its extensive work to make the database scale along with the site...
(Excerpt) Read more at gigaom.com ...
Shoking. NOT. No replacement for Oracle.
Translation?
PostgreSQL.
Flame-retardant equipment deployed here for what will be an incendiary thread. And it's ess-que-ell not sequel. :-) Harumph!
I think it means spaghetti code.
But in query form.
I run mysql and have dealt a little with memcahed stuff for a ticketing database. The problems facebook might have depends of which data storage engine they run, likely innodb, which are huge relational glop files. I run myisam tables for speed and because I’m only dealing with a gigabyte of data.
But yes, switching to another database would require a very bug prone rewrite with many database quirks. The language (SQL) is not consistently implemented from database maker to database maker and the connectors are inconsistent. They’d also be stuck with an oracle or sqlserver license fee out the wazoo.
Basically yes but a bit more complex. Easily translated: you do not run REALLY large operations on freeware. You get what you pay for. Oracle is the best. Become one with the Borg.
IOW....it would be a ‘hacker’s delight’???
So rather than update, patches are applied to the output to make changes when served. And it's working ok for the moment, but it is swiftly reaching the horizon of functionality. At a growth rate of 52,000 items a year, we've at most two years left of the database before it becomes impossibly bogged down, without throwing extensive hardware at the problem.
And we're a tiny company of just ten people.
ROFL!!!! That alone is contentious enough to get us through several pages of debate/flame wars!
For the record, I've always referred to it as 'sequel'. Not passionate about it, just seems to roll off the tongue a bit easier spoken that way.
The problem is common, and has nothing to do with MySQL, ProgSQL, or Oracle.
It has to do with lazy barstids that didn't do the work up front and became frantic barstids, trying to keep the thing working.
It's a common engineering problem. Systems Engineering, not just for aerospace.
Someone has to ask (and answer) "what is the growth path?"
I'm doing engineering on a start-up now. And we're good on growth path until I sell out or die. And then I don't care.
/johnny
What's the backend RDBMS here on FR?
Of course, Perl is the glue but I wonder if the database access is via the DBI layer or if it uses the native APIs?
Facebook started their enterprise using a database called MySQL. (SQL is an an acronym meaning Structured Query Language)
They didn't see the potential of their enterprise and stuck with MySQL and when the thing expanded exponentially they were caught unawares. Call it a lack of vision.
Now, their database isn't beefy enough (supposedly) to handle all the daily transactions it must deal with. They are breaking the thing down into smaller chunks so it doesn't slow to a crawl and piss off the kiddies who simply MUST get their Facebook fix every 5 minutes.
Or something like that.
The should have migrated to Oracle a long time ago.
I manage a Informix database of up to 1.5 million items, with large associated blob items, and it runs like a charm on a modest HP Unix server.
Makes me wonder about the tiny little MySQL stuff I play with on the side.
Two words.
Z196
DB2
Nuff said.
Heh! Just wait until we start on brace and indentation styles (K&R vs. Allman)! And of course, there's always vi vs. emacs -- but that battle has been won since vim came along. Perl vs. Python. C# vs. Java...
:-) * 10e6
Model204.
Stonebraker explained to me that Facebook has split its MySQL database into 4,000 shards in order to handle the site's massive data volume, and is running 9,000 instances of memcached in order to keep up with the number of transactions the database must serve.Over a year ago I read that FB had (at that time) 30,000 servers, and hosted more pix than all other sites combined. As the old saying goes, it's not that the dancing bear dances well, it's that he dances at all.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.