'Blue Screen of Death' on Oil Rig's Computer ( Deepwater Horizon Oilspill )

HardOCP ^ | Saturday July 24, 2010 | Al

[Ernest_at_the_Beach]

Testifying at a federal hearing on Friday, the chief electronics technician told of numerous instances of a "Blue Screen of Death" on the computer system responsible for monitoring and controlling drilling. The largest oil spill in American history may be due to a simple computer glitch.

The machine had been locking up for months, Williams said, producing what he and others on the crew called a "blue screen of death." "It would just turn blue. You'd have no data coming through," Williams said today, according to the New York Times' story. With the computer frozen, the driller would not have access to crucial data about what was going on in the well.

[Ernest_at_the_Beach]

They seemed to have a number of problems included the one of the article.

Not sure how crucial the computer was in their doing their job.

[Gene Eric]

The ol’ blame-the-computer card.

[Ernest_at_the_Beach]

They are blaming a lot of stuff...not sure where the computer falls in the rankings....

Just posted this:

As federal panel probes oil spill, picture emerges of a series of iffy decisions

[mvpel]

Unix OS’s don’t crash like that. The software the UNIX OS is currently running might crash, but usually the OS stays up.

That's generally true when you're running well-behaved applications in an uncomplicated envornment. But, this list is the patch history of PHNE_40123 for HP-UX 11.11's ONC and NFS subsystem, which has a defect description document that's around 100 pages long:

PHNE_41023: ABORT
PHNE_39167: ABORT
PHNE_37568: HANG ABORT
PHNE_37110: HANG ABORT
PHNE_36168: HANG
PHNE_35418: ABORT
PHNE_34938: PANIC ABORT
PHNE_34662: HANG ABORT CORRUPTION
PHNE_34293: ABORT PANIC
PHNE_33971: ABORT HANG PANIC
PHNE_33315: HANG ABORT MEMORY_LEAK
PHNE_32811: CORRUPTION ABORT
PHNE_32477: ABORT HANG PANIC MEMORY_LEAK
PHNE_31097: HANG ABORT PANIC
PHNE_30661: ABORT HANG PANIC
PHNE_30380: ABORT
PHNE_30378: HANG ABORT
PHNE_29883: ABORT PANIC
PHNE_29303: PANIC HANG
PHNE_28983: ABORT PANIC MEMORY_LEAK HANG
PHNE_28137: ABORT PANIC CORRUPTION MEMORY_LEAK HANG
PHNE_28103: HANG
PHNE_27218: PANIC MEMORY_LEAK CORRUPTION HANG
PHNE_26388: PANIC
PHNE_25627: ABORT PANIC MEMORY_LEAK HANG
PHNE_25625: ABORT
PHNE_24910: ABORT PANIC CORRUPTION HANG
PHNE_24035: ABORT CORRUPTION HANG
PHNE_23502: ABORT PANIC HANG
PHNE_22878: PANIC HANG

[central_va]

Without getting too technical UNIX OS's give applications a defined memory space to operate in. The UNIX OS doesn't care what the app does AS LONG AS IT STAYS IN ITS RUN-TIME AREA. There is a wall between the OS and the app. I am not so sure it is like that in Windows.

Moreover if the app crashes and it is mission critical, automation(cron) scripts can take over and re-spawn the app, thereby minimizing down time. This is what I do for a living....

[mvpel]

I do this for a living too. While it’s true that it’s difficult for a typical application to crash a UNIX system, it still happens if the app trips over problems in the OS itself:

---

( SR:8606361473 CR:JAGaf22163 )
While performing an inode truncation operation on a VxFS filesystem, the number of subfunctions generated for the transaction overflows the internal transaction table [i.e., kernel stack overflow]. This overflow corrupts the information in the common transaction area. In particular, the inode information is corrupted which results in a panic.

---

( QX:QXCR1000557861 SR:8606427857 CR:JAGaf87338 )
An NFS client panics with the panic string “Data page fault” while performing an nfsreaddirplus operation.

---

( SR:8606392946 CR:JAGaf53027 )
The system panics during I/O on an NFS mounted file. The panic string is “crfree: freeing free credential struct”. The panic may also occur in crhold().

---

( QX:QXCR1000544021 SR:8606390486 CR:JAGaf50632 )
An NFS client panics when control messages from the TCP transport are received in the wrong sequence.

---

While the above are all straightened out now, this sort of thing is why you sometimes hear of unfortunate sysadmins at companies still running HP-UX 9.05 or 10.20. Nobody was willing to take the risk of upgrading the control systems and becoming the first ones to identify a new bug in the OS.

[lentulusgracchus]

It also sure seemed that a lot of “Southern Engineering” was going on regarding alarms, BOP and drilling in general.

Is that by any chance an infelicitous euphemism for "n****** engineering"?

As if that sort of jury-rigging was the norm in the South, and nobody had ever heard of, say, NASA, whose engineering is sometimes called "rocket science"?

[lentulusgracchus]

Instantly stolen.

Tks 8)

[lentulusgracchus]

Never heard that one before ......

I trust our AEGIS cruisers and destroyers are running newer, more stable versions of Windows? Or preferably SOMETHING ELSE?

Like, oh, say, Linux? Or FreeBSD?

Land of the Freeware, home of the brave?

[dusttoyou]

You got it! Personally never called nor heard anyone else routinely call it “Southern Engineering” just heard it, but one has to be very very careful EVEN HERE.

The Oil Patch ain’t NASA (or what we used to have called NASA)and lots of “..... engineering” goes on by the hands just to work around the nutty stuff flowing like the stuff we see on the ROV cams, from the Engineers AND to get the job done.

[TheBattman]

So - with the kind of financial investment that these drilling operations represent, why run outdated and flawed computers/software, particularly when it comes to drilling safety?