Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

[LINUX TECH] Experience with btrfs?
2014-05-28 19:08:22 UTC | /dev/null

Posted on 05/28/2014 12:14:45 PM PDT by re_nortex

Sure there are technical forums that discuss btrfs, the shiny new filesystem for Linux but my experience has shown that FReepers meet or exceed the technical expertise anywhere on the net. The two cutting-edge filesystems that are (supposedly) impervious to bitrot are btrfs and zfs (which has a Linux implementation, older than what Oracle has now closed off).


TOPICS: Computers/Internet
KEYWORDS: beadm; bitflip; bitrot; btrfs; computer; filesystems; flippedbit; ide; linux; llnlgov; lvm; oracle; sata; solaris; torvalds; zfs
If anyone has used either zfs or btrfs, what are the pros and cons? Some smart people are evidently involved with both of these so-called next generation filesystems. I'm more familiar with zfs on Solaris and have found that zfs on Linux understandably lags behind that of Oracle. Apparently btrfs emerges as the one that might get the most traction.

My main interests aside from data integrity are snapshots and the ability to boot from such a snapshot. That's afforded by beadm on Solaris which uses a zfs snapshot behind the scenes.

Any thoughts, opinions or comments based on experience with btrfs and/or zfs on a Linux platform?

1 posted on 05/28/2014 12:14:45 PM PDT by re_nortex
[ Post Reply | Private Reply | View Replies]

To: rdb3; Calvinist_Dark_Lord; JosephW; Only1choice____Freedom; amigatec; Still Thinking; ...

2 posted on 05/28/2014 12:15:48 PM PDT by ShadowAce (Linux -- The Ultimate Windows Service Pack)
[ Post Reply | Private Reply | To 1 | View Replies]

To: re_nortex
I run btrfs on both a physical laptop and a VM.

The biggest pro I see so far (only been a few months) is that fsck takes virtually no time at all.

Another thing I've noticed (and this seems odd to me) is that my custom partitioning schema usually involves a separate partition for /,/home,etc. but "df -h" typically shows both / and /home as sharing the same device/partition. Not sure yet what that is all about.

3 posted on 05/28/2014 12:18:27 PM PDT by ShadowAce (Linux -- The Ultimate Windows Service Pack)
[ Post Reply | Private Reply | To 1 | View Replies]

To: re_nortex

If you plan on using zfs I have found by painful experience you need to have an enormous amount of memory. I am currently running with zfs and it takes 20G of memory to really support it well.


4 posted on 05/28/2014 12:31:15 PM PDT by fulltlt
[ Post Reply | Private Reply | To 1 | View Replies]

To: re_nortex

bookmark for later


5 posted on 05/28/2014 12:34:19 PM PDT by dadfly
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce
I run btrfs on both a physical laptop and a VM.

Thanks for the speedy response and for the ping to all the experts here on Free Republic.

I'm just now getting acquainted with btrfs and, so far, am confining my still-early learning to a VM. As far as fsck is concerned, it appears that best practice is to set the pass to 0 in /etc/fstab. From what I gather (and there's a lot of information on the 'net, some contradictory) is that btrfs doesn't need a filesystem check. That's similar to zfs in that the only manual maintenance required is the occasional scrub operation.

I hope I'm not making the mistake of transferring my fairly good knowledge of zfs into the new realm of btrfs. They are different but for a n00b like me, similar enough to be a tad confusing. :)

6 posted on 05/28/2014 12:35:08 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 3 | View Replies]

To: fulltlt
If you plan on using zfs I have found by painful experience you need to have an enormous amount of memory.

I can confirm that's the certainly the case on Linux. The process table has a lot of "z.*" items running for sure when using the zfs filesystem. Since Oracle has ceased open source development on zfs, that's why I've started exploring btrfs as an alternative.

I really like the ease of handling boot environments in Solaris with the beadm tool (written in Python, by the way) and am seeking something roughly akin to that with btrfs in Linux land.

While the zfs on Linux crew is doing a laudable job, their last release is from August 2013 whereas btrfs is more recent, albeit a fast-moving target.

7 posted on 05/28/2014 12:44:01 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 4 | View Replies]

To: re_nortex

Both have strengths. ZFS has very nice snap shot technology but the draw back is zfs is not native to the kernel (this could have changed, it’s been a long while since I’ve read the commits) so it takes a good chunk of memory since it is “user space”. btrfs has an excellent journal type recovery method and is well suited for VERY large filesystems.
If it’s just your local system and a couple TB, ext4 still fits a good mold.


8 posted on 05/28/2014 12:46:36 PM PDT by Ghost of SVR4 (So many are so hopelessly dependent on the government that they will fight to protect it.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Ghost of SVR4

Was just about to ask about comparison to ext4... you pretty much answered though ;^)


9 posted on 05/28/2014 12:51:55 PM PDT by Bikkuri (Molon Labe)
[ Post Reply | Private Reply | To 8 | View Replies]

To: re_nortex

Good reading material ( nerd porn )...:)

http://www.dhtusa.com/media/IOPerfCMG09.pdf


10 posted on 05/28/2014 1:00:55 PM PDT by ImJustAnotherOkie (zerogottago)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Ghost of SVR4; Bikkuri
If it’s just your local system and a couple TB, ext4 still fits a good mold.

Thanks for the reply. In fact, ext4 is still my day-to-day default choice for every filesystem on bare metal. I do layer it atop LVM for snapshots -- which has saved my bacon countless time. Most recently, a careless rm -rf /usr/lib (when I meant to type rm -rf /usr/libx32, a no-longer used directory on my system) would have been disaster without the snapshot.

That said, if btrfs affords such capability without the LVM underpinnings, I may move that way. For now, I'm just exploring it in the anything-goes playpen afforded by a VM.

11 posted on 05/28/2014 1:04:23 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 8 | View Replies]

To: ImJustAnotherOkie
Much appreciated! I'm dizzy from reading every known link about zfs vs. btrfs, much of it the usual "works for me" as well as those pushing an agenda one way or another. Obviously, raising the matter here on FR was beneficial since, frankly, more intelligent people tend to congregate here than on the 'net as a whole. Such is the nature of Conservatives.

That paper that you linked to is one I hadn't encountered. It's a cut above the usual stuff and is being printed to some dead trees for thorough reading, yellow highlighting and my marginal notes. So thanks again.

12 posted on 05/28/2014 1:12:16 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 10 | View Replies]

To: re_nortex
Tsk, tsk, tsk.

You should always run 'rm' and 'mv'with a following '-i'. This will prompt you with an "are you sure?" question before they execute the command.

My startup scripts have aliases in them for powerful commands;

alias remove='rm -i'
alias move='mv -i'

But you probably already know this. I learned the hard way too.

13 posted on 05/28/2014 1:44:38 PM PDT by Bloody Sam Roberts (Only Liberals can look at an amendment that says "shall not be infringed" and see blank parchment.)
[ Post Reply | Private Reply | To 11 | View Replies]

To: Bloody Sam Roberts
You should always run 'rm' and 'mv' with a following '-i'.

I've got to admit that snapshotting has made me work without a net, as it were. Knowing that a fat-fingered mishap is so readily recovered allows me to live life on the edge. I now laugh in the face of danger!

Fine print: Or at least on my own machines. On customer systems, I'm a coward. :)

14 posted on 05/28/2014 2:18:44 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 13 | View Replies]

To: re_nortex

Agreed there.. even with LVM (encrypt)... STILL need back-up.. learned that the hard way... power outage while I was writing to encrypted HD (whole HD, not file or folders).. Screwed up the encrypted HD.. spent 6 months trying to find a way to read it again.. gave up last week.. and did low lvl format :/ (and then partitioned with ext4)...

(Was not home or boot HD..)


15 posted on 05/28/2014 2:38:19 PM PDT by Bikkuri (Molon Labe)
[ Post Reply | Private Reply | To 11 | View Replies]

To: re_nortex

Yes.. more help here than even on Mint site :/ Takes weeks (if mot months) to get an answer there :/

At least many here will tell us if they have no idea what to do :p


16 posted on 05/28/2014 2:41:21 PM PDT by Bikkuri (Molon Labe)
[ Post Reply | Private Reply | To 12 | View Replies]

To: ShadowAce
I run btrfs on both a physical laptop and a VM.


Any links about btrfs? Rings a bell, but not familiar with it yet..
17 posted on 05/28/2014 2:44:16 PM PDT by Bikkuri (Molon Labe)
[ Post Reply | Private Reply | To 3 | View Replies]

To: re_nortex

Definitely smarter than the lefty stuff.

Yes, that document is pretty stout.


18 posted on 05/28/2014 2:50:59 PM PDT by ImJustAnotherOkie (zerogottago)
[ Post Reply | Private Reply | To 12 | View Replies]

To: re_nortex
"experience has shown that FReepers meet or exceed the technical expertise anywhere on the net"

Allow me to disabuse you of the notion: "Are you logged in?"

19 posted on 05/28/2014 3:10:32 PM PDT by Dalberg-Acton
[ Post Reply | Private Reply | To 1 | View Replies]

To: Bikkuri
I'd start with the Wikipedia article, which has 78 footnotes and two external links.

If looking for discussions, etc., I'd start with Hacker News threads, e.g. btrfs or btrfs vs zfs. Or sample StackOverflow questions.

20 posted on 05/28/2014 3:26:03 PM PDT by cynwoody
[ Post Reply | Private Reply | To 17 | View Replies]

To: cynwoody; Bikkuri; ShadowAce
What piqued my interest in btrfs was the article, Bitrot and atomic COWs: Inside “next-gen” filesystems. The "graphic gone wrong" with the JPG shown in the article is a vivid example of a flipped bit.

I encountered such a thing a few years ago while still using the ext3 filesystem and an IDE (remember those?) drive. Here's what happened. I extracted a largish tarball to build QT-4 from source. All was going well until the compile failed on a C++ file since it yelped about a missing header. I examined the source file and saw this:

#include <uniste.h>
Yes, one bit was flipped (should be unistd.h, of course). I edited the errant file and happily finished the compile. Curious as to what went amiss, I re-extracted the tarball into a partition on another drive (also IDE but on a different controller) and it was correct. That was readily visible and I caught it but I can't help but wonder how much silent data corruption has gone on through the decades on my various systems.
21 posted on 05/28/2014 3:55:15 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 20 | View Replies]

To: re_nortex
Fine print: Or at least on my own machines. On customer systems, I'm a coward. :)

Ha! I know what you mean.

22 posted on 05/28/2014 3:58:44 PM PDT by Bloody Sam Roberts (Only Liberals can look at an amendment that says "shall not be infringed" and see blank parchment.)
[ Post Reply | Private Reply | To 14 | View Replies]

To: Bikkuri

I don’t have any handy. It was an option on Fedora 20.


23 posted on 05/28/2014 5:21:48 PM PDT by ShadowAce (Linux -- The Ultimate Windows Service Pack)
[ Post Reply | Private Reply | To 17 | View Replies]

To: Ghost of SVR4
If it’s just your local system and a couple TB, ext4 still fits a good mold.

That was my first thought. Zfs is freaking cool because of the huge sizes it can support. If you're going to have an exobyte of storage, zfs is a great option. I haven't really read enough about btrfs to get a good feel for what it's purpose is. For me, at least until I get to 100TB, I figure ext4 will work fine. I haven't have any problem with it at all. I'm not seeing the bit rot people are talking about.

It seems to work well, and I don't reboot often enough to have even hit a forced check through my file system.

 

24 posted on 05/28/2014 6:01:32 PM PDT by zeugma (I have never seen anyone cross the street to avoid a black man in a suit.)
[ Post Reply | Private Reply | To 8 | View Replies]

To: re_nortex
Yes, one bit was flipped (should be unistd.h, of course). I edited the errant file and happily finished the compile. Curious as to what went amiss, I re-extracted the tarball into a partition on another drive (also IDE but on a different controller) and it was correct. That was readily visible and I caught it but I can't help but wonder how much silent data corruption has gone on through the decades on my various systems.

I got to wondering how noticeable flipped bits in binaries might be. So, I wrote a little Ruby program to flip a random bit in the cat command and see if the corrupted binary still runs:

#!/usr/bin/env ruby
require 'tmpdir'
require 'open3'
$outPath = "#{Dir.tmpdir}/catmandu"
$data = File.new(`which cat`.chomp, "rb").read
$bits = $data.length * 8
$me = IO.read __FILE__
puts "The cat command is #{$bits} bits and #{$data.length} bytes long."

def flipBit
  byte, bit = (rand*$bits).to_i.divmod 8
  d = $data.clone
  d[byte] = (d[byte].ord ^ (1<<bit)).chr
  File.open($outPath, 'wb') do |f|
    f.write d
    f.chmod 0700
  end
  return byte, bit
end

def runTest trial
  byte, bit = flipBit
  begin
    check, err, status = Open3.capture3("#{$outPath} #{__FILE__}")
  rescue => e
  end
  if check == $me && status.to_i == 0 && e == nil
    result = 'pass'
    trouble = ''
  else
    result = 'fail'
    err = e if e
    err = err.to_s.force_encoding("ASCII-8bit").tr("\n", "/")[0, 60]
    trouble = " status = #{status}: #{err}"
  end
  puts "#{'%4s' % trial}: #{result} : byte #{byte} #{bit}#{trouble}"
  result
end

if __FILE__ == $0
  trials = (ARGV[0] || 50).to_i
  failures = 0
  trials.times do |trial|
    failures += 1 if runTest(trial) == 'fail'
  end
  puts "#{failures} out of #{trials} failed."
end

I found a much lower error rate than I expected. In one run of 1000 trials, it failed only 38 times. There was a little excitement, however. About half-way through, one of the failures took the form of an output loop, requiring me to kill the corrupted child process to allow the program to complete. Most of the failures are stuff like seg faults and load errors due to symbols not found or faulty binary file format.

I have a hunch bit rot in source code and script files probably shows up more readily than in most binaries. Although a flipped bit in a comment will probably not cause a failure, a similar error elsewhere is likely to cause a syntax error or some sort of not-found condition, as in your example.

25 posted on 05/28/2014 8:09:06 PM PDT by cynwoody
[ Post Reply | Private Reply | To 21 | View Replies]

To: cynwoody
I wrote a little Ruby program to flip a random bit in the cat command and see if the corrupted binary still runs...

Great stuff and very nice, readable code (as expected from the denizens of FReeperdom). Here's the tail end of one my runs, just for grins:

2494: fail : byte 6880 7 status = pid 17827 SIGSEGV (signal 11): 
2495: pass : byte 16316 0
2496: pass : byte 35390 7
2497: pass : byte 32143 6
2498: fail : byte 4651 1 status = pid 17843 SIGSEGV (signal 11): 
2499: pass : byte 1744 2
174 out of 2500 failed.
And yes, there were some runs where it wedged and needed a surekill. I ain't gonna try it on the kernel though!

My experience with the flipped bit in QT goes back further than I thought. It was March 2006 with QT-4.1.1 and I even documented the drive I was using at the time: a Maxtor 6Y060L0. And the kernel was 2.16.15.2 on a FIC FB11 mainboard. How's that for being anal? :) Considering how long ago it was and the creaky old hardware, just about anything may have caused that one-time-only bitflip.

Seriesly, whenever something goes amiss, I keep a log of such things just for archival purposes so that if an oddity recurs, it won't be such a surprise the second time around.

26 posted on 05/28/2014 10:59:14 PM PDT by re_nortex (DP - that's what I like about Texas)
[ Post Reply | Private Reply | To 25 | View Replies]

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson