At Tue, 5 Apr 2011 09:55:00 +0100,
Chris Webb wrote:
>
> Hi. I'm doing some testing with an initial version of our infrastructure
> ported to run over Sheepdog 0.2.2. I'm seeing some performance problems I
> don't remember seeing when I tested before, a few months back, and wondered
> whether I'm doing something obviously wrong!
>
> I've set up a single host with six SATA drives, made a btrfs on each drive,
> mounted in /sheep/{0,1,2,3,4,5} with default mount options, and run sheep on
> ports 7000 -> 70005 for each of these mount points. These drives are
> reasonably fast (80MB/s or so), and independent of one another---this isn't
> the obviously bad configuration of six store directories all on the same
> backing devices!
It should be the ideal environment to test multiple sheep daemons on
one server.
>
> A 10GB file in (say) /sheep/0/ used as a raw drive image with
>
> -drive file=/sheep/0/,if=ide,index=0,cache=none
>
> gets reasonable performance of around 52MB/s doing a simple
>
> dd if=/dev/zero of=/tmp/test bs=1M count=500 oflag=direct
>
> test from within the guest. However, if I create a sheepdog drive by
>
> collie cluster format [default x3 data replication]
> qemu-img convert /sheep/0/test.img sheepdog:test [takes hours]
>
> and then start qemu with
>
> -drive file=sheepdog:test,if=ide,index=0
>
> I'm only getting around 5MB/s with the same write test.
>
> I see similarly poor (perhaps a little better) performance with ext4 +
> user_xattr.
>
> Should I be mounting the filesystems with options other than the defaults,
> or is there a bottleneck I'm not aware of with multiple sheep on a single
> host, despite the independent drives? Is there a good way to find out what
> the bottleneck really is here?
I'm not familiar with btrfs mount options, but if a raw image show a
good performance on the same file system, I think this is a problem of
Sheepdog. To be honest, I don't have the slightest idea why Sheepdog
shows such a bad results in your environment.
Would you test the following settings on the same host?
1) 1 sheep daemon and no data replication (add "-c 1" to format options)
2) 6 sheep daemons and no data replication
If 2) shows a good performance, there should be problems in Sheepdog
replication flow. If 1) shows a good performance but 2) is not, I
think there would be unknown problems to run multiple daemons on the
same host (or something goes wrong on one of your disks, perhaps).
> PS Is btrfs still the recommended configuration for reliable recovery, or is
> that recommendation no longer applicable? The code and documentation no
> longer mentions it, but I remember at one stage it was needed for atomic
> filesystem transactions.
Currently, we don't need btrfs for Sheepdog. Sheepdog supports
journaling internally, so we can safely update vdi objects.
Thanks,
Kazutaka
|
|