[sheepdog-users] Low performance (low IOPS) on local node

[Marcin Mirosław]

Hi!
I've launched 4 sheep on 4 physical HDD. Every sheep has different zone.
I formated cluster using ECC "-c 2:1".
# dog vdi list
    Name        Id    Size    Used  Shared    Creation time   VDI id
Copies  Tag   Block Size Shift
    testowy      0   10 GB  3.2 GB  0.0 MB 2015-05-25 15:38   cac836
2:1                22


Next I mounted it using sheepfs (via loop device). pg_test_fsync shows:
# pg_test_fsync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
        open_datasync                        58.589 ops/sec   17068 usecs/op
        fdatasync                            55.243 ops/sec   18102 usecs/op
        fsync                                21.172 ops/sec   47233 usecs/op
        fsync_writethrough                              n/a
        open_sync                            19.980 ops/sec   50050 usecs/op

When I start sheep with "-n" option I'm getting:
# pg_test_fsync
5 seconds per test
O_DIRECT supported on this platform for open_datasync and open_sync.

Compare file sync methods using one 8kB write:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
        open_datasync                      1166.246 ops/sec     857 usecs/op
        fdatasync                          1229.497 ops/sec     813 usecs/op
        fsync                               252.289 ops/sec    3964 usecs/op
        fsync_writethrough                              n/a
        open_sync                           514.587 ops/sec    1943 usecs/op


Every sheep keep files on ext4 (hdd - 7200 rpm), also ext4 is used on
top of vdi. All disk are saturated with write operations (iostat shows
>200 tps).

Shouldn't it be a little faster?
Marcin