[sheepdog-users] 50.000 iops per VM

[Stefan Priebe]

Hi Christoph,

thanks for your reply.

Am 09.07.2012 16:49, schrieb Christoph Hellwig:
> Before I would like to let you try 29bfdbd6a95fdf8d827e177046dbab12ee342611
> or earlier?
I already tried that. It gives me the same results.

>>> Can you also do a perf record -g on both a storage node and the
>>> kvm box to see if there's anything interesting on them?
>>
>> Option -g is not known by my perf command?
>
> perf record -g records the callgraph, and it's been there for a long
> time.  Did you try that above line or something like perf -g record?
*argh* i really tried -g record ;-( i'm on holiday right now so it will 
take some days. What is the expected output of this command? Should i 
attach it?

>> is a perf record sleep 10 enough? Should i upload then the data file
>> somewhere?
>
> Would be nice to get a slightly long run.
How long

>> Snapshot of perf top from KVM host:
>>   14.96%  [kernel]                     [k] _raw_spin_lock
>
> With the callchains we could expand what spinlock we hammer here.
?!

>>    8.13%  kvm                          [.] 0x00000000001d8084
>
> Also if kvm/qemu is self-build can you build it with -g to get debug
> info?  If not see if there is a qemu-dbg/kvm-dbg or similar package for
> your distribution.

It is self build - i'll do a debug.

>> Snapshot of perf top from sheep node (not acting as the gateway /
>> target for kvm):
>>    2,78%  libgcc_s.so.1         [.] 0x000000000000e72b
>>    2,21%  [kernel]              [k] __schedule
>>    2,14%  [kernel]              [k] ahci_port_intr
>>    2,08%  [kernel]              [k] _raw_spin_lock
>>    1,77%  [kernel]              [k] _raw_spin_lock_irqsave
>>    1,15%  [kernel]              [k] ahci_interrupt
>>    1,11%  [kernel]              [k] ahci_scr_read
>>    0,94%  [kernel]              [k] kmem_cache_alloc
>>    0,90%  [kernel]              [k] _raw_spin_lock_irq
>>    0,81%  [kernel]              [k] menu_select
>>    0,76%  [kernel]              [k] _raw_spin_unlock_irqrestore
>>    0,73%  libpthread-2.11.3.so  [.] pthread_mutex_lock
>>    0,70%  libc-2.11.3.so        [.] vfprintf
>
> Mostly not spending any noticable CPU time, which is quite interesting.

Yes the sheep acting as gateway is taking 20x times more CPU than the 
node sheep processess.

Stefan