[sheepdog-users] High cpu usage by sheep (1/5)

Hitoshi Mitake mitake.hitoshi at gmail.com
Thu Jan 16 07:06:35 CET 2014


At Sun, 12 Jan 2014 22:29:44 +0100,
Marcin Mirosław wrote:
> 
> [1  <text/plain; ISO-8859-2 (7bit)>]
> Hi!
> Today I run `dog cluster check` and I saw high cpu utilization on both
> cores, both by kernel thread and userland. It took 30seconds to check
> 5GB of data, it gives about 170MB/s.
> Next i tested `dog vdi check testowy`, again speed was limited by CPU.
> It took 17s so it gives speed about 300MB/s.

Hi Marcin, sorry for my late reply.

Currently I cannot say something about the above result (it is too
slow or not). But basically vdi checking is not light operation
because it detect majority of every replica of every object of
VDI. Basic scheme is like below:

1. read sha1 value of every replica on an index (if copies ==3, 3
   hashed values are read)
2. compare and decide majority. if there are broken replicas, dog
   recovers them with correct one
3. increment the index, goto 1, until every replica is checked

> 
> So I was wonder where is bottleneck? I used tool called "perf", I didn't
> ever use this tool. I hope i use it correctly. I'll attach output of
> perf on the end of email. I'd like to ask you, is there place to do some
> optimization or not?

I think caliculation of sha1 value can be optimized. get_buffer_sha1()
in lib/sha1.c is the function for it. In the above checking scheme,
copies * (a number of objects) of sha1 caliculation is
done (exception: read only objects would have their own static value,
so we need to caliculate for them only once at first
time). Previously, we have an optimized function but it was buggy so
we replaced it with safe but naive one.

If you can write an optimized version with modern instruction set, it
is definitely welcome :)

Thanks,
Hitoshi


> I created vdi using `dog vdi create -P testowy 5G`, it looks it creates
> some kind of spare files so I/O was completly idle. I tested on two
> cores cpu Intel(R) Core(TM)2 CPU 4300  @ 1.80GHz, sheepdog was taken
> from master branch. Kernel is 3.10.x.
> 
> Marcin
> [2 vdi_read_perf.txt <text/plain; windows-1250 (7bit)>]
> # ========
> # captured on: Sun Jan 12 21:07:18 2014
> # hostname : zserwerek
> # os release : 3.10.25-gentoo
> # perf version : 3.12
> # arch : x86_64
> # nrcpus online : 2
> # nrcpus avail : 2
> # cpudesc : Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz
> # cpuid : GenuineIntel,6,15,2
> # total memory : 1529164 kB
> # cmdline : /usr/sbin/perf record -u sheepdog -asgv -o vdi_read 
> # event : name = cycles, type = 0, config = 0x0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host = 0, excl_guest = 1, precise_ip = 0, attr_mmap2 = 0, attr_mmap  = 1, attr_mmap_data = 0
> # HEADER_CPU_TOPOLOGY info available, use -I to display
> # pmu mappings: cpu = 4, software = 1, breakpoint = 5
> # ========
> #
> # Samples: 140K of event 'cycles'
> # Event count (approx.): 59300926918
> #
> # Overhead  Command
> # ........  .......
> #
>    100.00%    sheep
>               |          
>               |--39.74%-- copy_user_generic_string
>               |          |          
>               |          |--87.92%-- inet_sendmsg
>               |          |          sock_sendmsg
>               |          |          ___sys_sendmsg
>               |          |          __sys_sendmsg
>               |          |          sys_sendmsg
>               |          |          system_call_fastpath
>               |          |          sendmsg
>               |          |          |          
>               |          |          |--99.95%-- 0x40000000000000
>               |          |           --0.05%-- [...]
>               |          |          
>               |          |--12.06%-- generic_file_aio_read
>               |          |          do_sync_read
>               |          |          vfs_read
>               |          |          sys_pread64
>               |          |          system_call_fastpath
>               |          |          pread64
>               |           --0.01%-- [...]
>               |          
>               |--10.59%-- memset
>               |          |          
>               |          |--99.96%-- mpage_readpages
>               |          |          ext4_readpages
>               |          |          __do_page_cache_readahead
>               |          |          ra_submit
>               |          |          |          
>               |          |          |--99.99%-- ondemand_readahead
>               |          |          |          |          
>               |          |          |          |--94.96%-- page_cache_async_readahead
>               |          |          |          |          |          
>               |          |          |          |          |--99.99%-- generic_file_aio_read
>               |          |          |          |          |          do_sync_read
>               |          |          |          |          |          vfs_read
>               |          |          |          |          |          sys_pread64
>               |          |          |          |          |          system_call_fastpath
>               |          |          |          |          |          pread64
>               |          |          |          |           --0.01%-- [...]
>               |          |          |          |          
>               |          |          |           --5.04%-- page_cache_sync_readahead
>               |          |          |                     generic_file_aio_read
>               |          |          |                     do_sync_read
>               |          |          |                     vfs_read
>               |          |          |                     sys_pread64
>               |          |          |                     system_call_fastpath
>               |          |          |                     pread64
>               |          |           --0.01%-- [...]
>               |           --0.04%-- [...]
>               |          
>               |--10.48%-- clear_page_c
>               |          __alloc_pages_nodemask
>               |          |          
>               |          |--99.82%-- handle_pte_fault
>               |          |          handle_mm_fault
>               |          |          __do_page_fault
>               |          |          do_page_fault
>               |          |          page_fault
>               |          |          |          
>               |          |          |--99.91%-- generic_file_aio_read
>               |          |          |          do_sync_read
>               |          |          |          vfs_read
>               |          |          |          sys_pread64
>               |          |          |          system_call_fastpath
>               |          |          |          pread64
>               |          |           --0.09%-- [...]
>               |           --0.18%-- [...]
>               |          
>               |--2.98%-- page_fault
>               |          |          
>               |          |--99.88%-- generic_file_aio_read
>               |          |          do_sync_read
>               |          |          vfs_read
>               |          |          sys_pread64
>               |          |          system_call_fastpath
>               |          |          pread64
>               |           --0.12%-- [...]
>               |          
>               |--2.48%-- do_mpage_readpage
>               |          |          
>               |          |--99.60%-- mpage_readpages
>               |          |          ext4_readpages
>               |          |          __do_page_cache_readahead
>               |          |          ra_submit
>               |          |          ondemand_readahead
>               |          |          |          
>               |          |          |--93.98%-- page_cache_async_readahead
>               |          |          |          generic_file_aio_read
>               |          |          |          do_sync_read
>               |          |          |          vfs_read
>               |          |          |          sys_pread64
>               |          |          |          system_call_fastpath
>               |          |          |          pread64
>               |          |          |          
>               |          |           --6.02%-- page_cache_sync_readahead
>               |          |                     generic_file_aio_read
>               |          |                     do_sync_read
>               |          |                     vfs_read
>               |          |                     sys_pread64
>               |          |                     system_call_fastpath
>               |          |                     pread64
>               |           --0.40%-- [...]
>               |          
>               |--2.04%-- get_page_from_freelist
>               |          |          
>               |          |--98.49%-- __alloc_pages_nodemask
>               |          |          |          
>               |          |          |--42.63%-- handle_pte_fault
>               |          |          |          handle_mm_fault
>               |          |          |          __do_page_fault
>               |          |          |          do_page_fault
>               |          |          |          page_fault
>               |          |          |          |          
>               |          |          |          |--99.65%-- generic_file_aio_read
>               |          |          |          |          do_sync_read
>               |          |          |          |          vfs_read
>               |          |          |          |          sys_pread64
>               |          |          |          |          system_call_fastpath
>               |          |          |          |          pread64
>               |          |          |           --0.35%-- [...]
>               |          |          |          
>               |          |          |--32.43%-- __do_page_cache_readahead
>               |          |          |          ra_submit
>               |          |          |          ondemand_readahead
>               |          |          |          |          
>               |          |          |          |--93.36%-- page_cache_async_readahead
>               |          |          |          |          generic_file_aio_read
>               |          |          |          |          do_sync_read
>               |          |          |          |          vfs_read
>               |          |          |          |          sys_pread64
>               |          |          |          |          system_call_fastpath
>               |          |          |          |          pread64
>               |          |          |          |          
>               |          |          |           --6.64%-- page_cache_sync_readahead
>               |          |          |                     generic_file_aio_read
>               |          |          |                     do_sync_read
>               |          |          |                     vfs_read
>               |          |          |                     sys_pread64
>               |          |          |                     system_call_fastpath
>               |          |          |                     pread64
>               |          |          |          
>               |          |          |--24.04%-- sk_page_frag_refill
>               |          |          |          tcp_sendmsg
>               |          |          |          inet_sendmsg
>               |          |          |          sock_sendmsg
>               |          |          |          ___sys_sendmsg
>               |          |          |          __sys_sendmsg
>               |          |          |          sys_sendmsg
>               |          |          |          system_call_fastpath
>               |          |          |          sendmsg
>               |          |          |          0x40000000000000
>               |          |           --0.90%-- [...]
>               |          |          
>               |          |--0.66%-- __do_page_cache_readahead
>               |          |          ra_submit
>               |          |          ondemand_readahead
>               |          |          page_cache_async_readahead
>               |          |          generic_file_aio_read
>               |          |          do_sync_read
>               |          |          vfs_read
>               |          |          sys_pread64
>               |          |          system_call_fastpath
>               |          |          pread64
>               |          |          
>               |          |--0.56%-- handle_pte_fault
>               |          |          handle_mm_fault
>               |          |          __do_page_fault
>               |          |          do_page_fault
>               |          |          page_fault
>               |          |          generic_file_aio_read
>               |          |          do_sync_read
>               |          |          vfs_read
>               |          |          sys_pread64
>               |          |          system_call_fastpath
>               |          |          pread64
>               |           --0.29%-- [...]
>               |          
>               |--1.60%-- __rmqueue
>               |          |          
>               |          |--98.20%-- get_page_from_freelist
>               |          |          __alloc_pages_nodemask
>               |          |          |          
>               |          |          |--44.82%-- __do_page_cache_readahead
>               |          |          |          ra_submit
>               |          |          |          ondemand_readahead
>               |          |          |          |          
>               |          |          |          |--96.53%-- page_cache_async_readahead
>               |          |          |          |          |          
>               |          |          |          |          |--99.93%-- generic_file_aio_read
>               |          |          |          |          |          do_sync_read
>               |          |          |          |          |          vfs_read
>               |          |          |          |          |          sys_pread64
>               |          |          |          |          |          system_call_fastpath
>               |          |          |          |          |          pread64
>               |          |          |          |           --0.07%-- [...]
>               |          |          |          |          
>               |          |          |           --3.47%-- page_cache_sync_readahead
>               |          |          |                     generic_file_aio_read
>               |          |          |                     do_sync_read
>               |          |          |                     vfs_read
>               |          |          |                     sys_pread64
>               |          |          |                     system_call_fastpath
>               |          |          |                     pread64
>               |          |          |          
>               |          |          |--44.57%-- handle_pte_fault
>               |          |          |          handle_mm_fault
>               |          |          |          __do_page_fault
>               |          |          |          do_page_fault
>               |          |          |          page_fault
>               |          |          |          generic_file_aio_read
>               |          |          |          do_sync_read
>               |          |          |          vfs_read
>               |          |          |          sys_pread64
>               |          |          |          system_call_fastpath
>               |          |          |          pread64
>               |          |          |          
>               |          |          |--9.99%-- sk_page_frag_refill
>               |          |          |          tcp_sendmsg
>               |          |          |          inet_sendmsg
>               |          |          |          sock_sendmsg
>               |          |          |          ___sys_sendmsg
>               |          |          |          __sys_sendmsg
>               |          |          |          sys_sendmsg
>               |          |          |          system_call_fastpath
>               |          |          |          sendmsg
>               |          |          |          0x40000000000000
>               |          |           --0.62%-- [...]
>               |          |          
>               |           --1.80%-- __alloc_pages_nodemask
>               |                     |          
>               |                     |--46.59%-- __do_page_cache_readahead
>               |                     |          ra_submit
>               |                     |          ondemand_readahead
>               |                     |          page_cache_async_readahead
>               |                     |          generic_file_aio_read
>               |                     |          do_sync_read
>               |                     |          vfs_read
>               |                     |          sys_pread64
>               |                     |          system_call_fastpath
>               |                     |          pread64
>               |                     |          
>               |                     |--40.97%-- handle_pte_fault
>               |                     |          handle_mm_fault
>               |                     |          __do_page_fault
>               |                     |          do_page_fault
>               |                     |          page_fault
>               |                     |          generic_file_aio_read
>               |                     |          do_sync_read
>               |                     |          vfs_read
>               |                     |          sys_pread64
>               |                     |          system_call_fastpath
>               |                     |          pread64
>               |                     |          
>               |                     |--10.72%-- sk_page_frag_refill
>               |                     |          tcp_sendmsg
>               |                     |          inet_sendmsg
>               |                     |          sock_sendmsg
>               |                     |          ___sys_sendmsg
>               |                     |          __sys_sendmsg
>               |                     |          sys_sendmsg
>               |                     |          system_call_fastpath
>               |                     |          sendmsg
>               |                     |          0x40000000000000
>               |                     |          
>               |                      --1.71%-- __get_free_pages
>               |                                __tlb_remove_page
>               |                                unmap_single_vma
>               |                                unmap_vmas
>               |                                unmap_region
>               |                                do_munmap
>               |                                vm_munmap
>               |                                sys_munmap
>               |                                system_call_fastpath
>               |                                munmap
>               |                                0x40000000000000
>               |          
>               |--1.55%-- handle_pte_fault
>               |          |          
>               |          |--98.77%-- handle_mm_fault
>               |          |          __do_page_fault
>               |          |          do_page_fault
>               |          |          page_fault
>               |          |          |          
>               |          |          |--99.57%-- generic_file_aio_read
>               |          |          |          do_sync_read
>               |          |          |          vfs_read
>               |          |          |          sys_pread64
>               |          |          |          system_call_fastpath
>               |          |          |          pread64
>               |          |           --0.43%-- [...]
>               |          |          
>               |           --1.23%-- __do_page_fault
>               |                     do_page_fault
>               |                     page_fault
>               |                     |          
>               |                     |--84.42%-- generic_file_aio_read
>               |                     |          do_sync_read
>               |                     |          vfs_read
>               |                     |          sys_pread64
>               |                     |          system_call_fastpath
>               |                     |          pread64
>               |                     |          
>               |                      --15.58%-- 0x7fd12a5c9c2c
>               |          
>               |--1.53%-- free_pcppages_bulk
>               |          |          
>               |          |--79.52%-- free_hot_cold_page
>               |          |          |          
>               |          |          |--99.78%-- free_hot_cold_page_list
>               |          |          |          |          
>               |          |          |          |--99.88%-- release_pages
>               |          |          |          |          free_pages_and_swap_cache
>               |          |          |          |          tlb_flush_mmu
>               |          |          |          |          tlb_finish_mmu
>               |          |          |          |          unmap_region
>               |          |          |          |          do_munmap
>               |          |          |          |          vm_munmap
>               |          |          |          |          sys_munmap
>               |          |          |          |          system_call_fastpath
>               |          |          |          |          munmap
>               |          |          |          |          0x40000000000000
>               |          |          |           --0.12%-- [...]
>               |          |           --0.22%-- [...]
>               |          |          
>               |           --20.48%-- drain_pages
>               |                     drain_local_pages
>               |                     |          
>               |                     |--99.60%-- on_each_cpu_mask
>               |                     |          drain_all_pages
>               |                     |          __alloc_pages_nodemask
>               |                     |          sk_page_frag_refill
>               |                     |          tcp_sendmsg
>               |                     |          inet_sendmsg
>               |                     |          sock_sendmsg
>               |                     |          ___sys_sendmsg
>               |                     |          __sys_sendmsg
>               |                     |          sys_sendmsg
>               |                     |          system_call_fastpath
>               |                     |          sendmsg
>               |                     |          0x40000000000000
>               |                      --0.40%-- [...]
>               |          
>               |--1.37%-- file_read_actor
>               |          |          
>               |          |--98.90%-- generic_file_aio_read
>               |          |          do_sync_read
>               |          |          vfs_read
>               |          |          sys_pread64
>               |          |          system_call_fastpath
>               |          |          pread64
>               |          |          
>               |           --1.10%-- do_sync_read
>               |                     vfs_read
>               |                     sys_pread64
>               |                     system_call_fastpath
>               |                     pread64
>               |          
>               |--1.33%-- _raw_spin_lock
>               |          |          
>               |          |--62.01%-- handle_pte_fault
>               |          |          handle_mm_fault
>               |          |          __do_page_fault
>               |          |          do_page_fault
>               |          |          page_fault
>               |          |          |          
>               |          |          |--99.91%-- generic_file_aio_read
>               |          |          |          do_sync_read
>               |          |          |          vfs_read
>               |          |          |          sys_pread64
>               |          |          |          system_call_fastpath
>               |          |          |          pread64
>               |          |           --0.09%-- [...]
>               |          |          
>               |          |--9.55%-- ext4_es_lru_add
>               |          |          ext4_es_lookup_extent
>               |          |          |          
>               |          |          |--99.58%-- ext4_map_blocks
>               |          |          |          _ext4_get_block
>               |          |          |          ext4_get_block
>               |          |          |          do_mpage_readpage
>               |          |          |          mpage_readpages
>               |          |          |          ext4_readpages
>               |          |          |          __do_page_cache_readahead
>               |          |          |          ra_submit
>               |          |          |          ondemand_readahead
>               |          |          |          |          
>               |          |          |          |--89.83%-- page_cache_async_readahead
>               |          |          |          |          generic_file_aio_read
>               |          |          |          |          do_sync_read
>               |          |          |          |          vfs_read
>               |          |          |          |          sys_pread64
>               |          |          |          |          system_call_fastpath
>               |          |          |          |          pread64
>               |          |          |          |          
>               |          |          |           --10.17%-- page_cache_sync_readahead
>               |          |          |                     generic_file_aio_read
>               |          |          |                     do_sync_read
>               |          |          |                     vfs_read
>               |          |          |                     sys_pread64
>               |          |          |                     system_call_fastpath
>               |          |          |                     pread64
>               |          |           --0.42%-- [...]
>               |          |          
>               |          |--4.49%-- grab_super_passive
>               |          |          prune_super
>               |          |          shrink_slab
>               |          |          do_try_to_free_pages
>               |          |          try_to_free_pages
>               |          |          __alloc_pages_nodemask
>               |          |          sk_page_frag_refill
>               |          |          tcp_sendmsg
>               |          |          inet_sendmsg
>               |          |          sock_sendmsg
>               |          |          ___sys_sendmsg
>               |          |          __sys_sendmsg
>               |          |          sys_sendmsg
>               |          |          system_call_fastpath
>               |          |          sendmsg
>               |          |          0x40000000000000
>               |          |          
>               |          |--4.35%-- ext4_es_lookup_extent
>               |          |          ext4_map_blocks
>               |          |          _ext4_get_block
>               |          |          ext4_get_block
>               |          |          do_mpage_readpage
>               |          |          mpage_readpages
>               |          |          ext4_readpages
>               |          |          __do_page_cache_readahead
>               |          |          ra_submit
>               |          |          ondemand_readahead
>               |          |          |          
>               |          |          |--97.40%-- page_cache_async_readahead
>               |          |          |          generic_file_aio_read
>               |          |          |          do_sync_read
>               |          |          |          vfs_read
>               |          |          |          sys_pread64
>               |          |          |          system_call_fastpath
>               |          |          |          pread64
>               |          |          |          
>               |          |           --2.60%-- page_cache_sync_readahead
>               |          |                     generic_file_aio_read
>               |          |                     do_sync_read
>               |          |                     vfs_read
>               |          |                     sys_pread64
>               |          |                     system_call_fastpath
>               |          |                     pread64
>               |          |          
>               |          |--3.41%-- handle_mm_fault
>               |          |          __do_page_fault
>               |          |          do_page_fault
>               |          |          page_fault
>               |          |          generic_file_aio_read
>               |          |          do_sync_read
>               |          |          vfs_read
>               |          |          sys_pread64
>               |          |          system_call_fastpath
>               |          |          pread64
>               |          |          
>               |          |--1.96%-- __free_pages_ok.part.81
>               |          |          free_compound_page
>               |          |          __put_compound_page
>               |          |          put_compound_page
>               |          |          put_page
>               |          |          skb_release_data
>               |          |          skb_release_all
>               |          |          __kfree_skb
>               |          |          tcp_ack
>               |          |          tcp_rcv_established
>               |          |          tcp_v4_do_rcv
>               |          |          release_sock
>               |          |          sk_stream_wait_memory
>               |          |          tcp_sendmsg
>               |          |          inet_sendmsg
>               |          |          sock_sendmsg
>               |          |          ___sys_sendmsg
>               |          |          __sys_sendmsg
>               |          |          sys_sendmsg
>               |          |          system_call_fastpath
>               |          |          sendmsg
>               |          |          0x40000000000000
>               |          |          
>               |          |--1.31%-- mod_timer_pending
>               |          |          __nf_ct_refresh_acct
>               |          |          tcp_packet
>               |          |          nf_conntrack_in
>               |          |          ipv4_conntrack_local
>               |          |          nf_iterate
>               |          |          nf_hook_slow
>               |          |          __ip_local_out
>               |          |          ip_local_out
>               |          |          ip_queue_xmit
>               |          |          tcp_transmit_skb
>               |          |          tcp_write_xmit
>               |          |          |          
>               |          |          |--91.62%-- tcp_push_one
>               |          |          |          tcp_sendmsg
>               |          |          |          inet_sendmsg
>               |          |          |          sock_sendmsg
>               |          |          |          ___sys_sendmsg
>               |          |          |          __sys_sendmsg
>               |          |          |          sys_sendmsg
>               |          |          |          system_call_fastpath
>               |          |          |          sendmsg
>               |          |          |          0x40000000000000
>               |          |          |          
>               |          |           --8.38%-- __tcp_push_pending_frames
>               |          |                     tcp_rcv_established
>               |          |                     tcp_v4_do_rcv
>               |          |                     release_sock
>               |          |                     sk_stream_wait_memory
>               |          |                     tcp_sendmsg
>               |          |                     inet_sendmsg
>               |          |                     sock_sendmsg
>               |          |                     ___sys_sendmsg
>               |          |                     __sys_sendmsg
>               |          |                     sys_sendmsg
>               |          |                     system_call_fastpath
>               |          |                     sendmsg
>               |          |                     0x40000000000000
>               |          |          
>               |          |--1.13%-- put_super
>               |          |          drop_super
>               |          |          prune_super
>               |          |          shrink_slab
>               |          |          do_try_to_free_pages
>               |          |          try_to_free_pages
>               |          |          __alloc_pages_nodemask
>               |          |          sk_page_frag_refill
>               |          |          tcp_sendmsg
>               |          |          inet_sendmsg
>               |          |          sock_sendmsg
>               |          |          ___sys_sendmsg
>               |          |          __sys_sendmsg
>               |          |          sys_sendmsg
>               |          |          system_call_fastpath
>               |          |          sendmsg
>               |          |          0x40000000000000
>               |          |          
>               |          |--1.02%-- enqueue_to_backlog
>               |          |          netif_rx
>               |          |          loopback_xmit
>               |          |          dev_hard_start_xmit
>               |          |          dev_queue_xmit
>               |          |          ip_finish_output
>               |          |          ip_output
>               |          |          ip_local_out
>               |          |          ip_queue_xmit
>               |          |          tcp_transmit_skb
>               |          |          tcp_write_xmit
>               |          |          |          
>               |          |          |--83.60%-- tcp_push_one
>               |          |          |          tcp_sendmsg
>               |          |          |          inet_sendmsg
>               |          |          |          sock_sendmsg
>               |          |          |          ___sys_sendmsg
>               |          |          |          __sys_sendmsg
>               |          |          |          sys_sendmsg
>               |          |          |          system_call_fastpath
>               |          |          |          sendmsg
>               |          |          |          0x40000000000000
>               |          |          |          
>               |          |           --16.40%-- __tcp_push_pending_frames
>               |          |                     tcp_rcv_established
>               |          |                     tcp_v4_do_rcv
>               |          |                     release_sock
>               |          |                     sk_stream_wait_memory
>               |          |                     tcp_sendmsg
>               |          |                     inet_sendmsg
>               |          |                     sock_sendmsg
>               |          |                     ___sys_sendmsg
>               |          |                     __sys_sendmsg
>               |          |                     sys_sendmsg
>               |          |                     system_call_fastpath
>               |          |                     sendmsg
>               |          |                     0x40000000000000
>               |          |          
>               |          |--0.95%-- tcp_v4_rcv
>               |          |          ip_local_deliver_finish
>               |          |          ip_local_deliver
>               |          |          ip_rcv_finish
>               |          |          ip_rcv
>               |          |          __netif_receive_skb_core
>               |          |          __netif_receive_skb
>               |          |          process_backlog
>               |          |          net_rx_action
>               |          |          __do_softirq
>               |          |          call_softirq
>               |          |          do_softirq
>               |          |          |          
>               |          |          |--82.44%-- local_bh_enable
>               |          |          |          ip_finish_output
>               |          |          |          ip_output
>               |          |          |          ip_local_out
>               |          |          |          ip_queue_xmit
>               |          |          |          tcp_transmit_skb
>               |          |          |          tcp_write_xmit
>               |          |          |          |          
>               |          |          |          |--92.81%-- tcp_push_one
>               |          |          |          |          tcp_sendmsg
>               |          |          |          |          inet_sendmsg
>               |          |          |          |          sock_sendmsg
>               |          |          |          |          ___sys_sendmsg
>               |          |          |          |          __sys_sendmsg
>               |          |          |          |          sys_sendmsg
>               |          |          |          |          system_call_fastpath
>               |          |          |          |          sendmsg
>               |          |          |          |          0x40000000000000
>               |          |          |          |          
>               |          |          |           --7.19%-- __tcp_push_pending_frames
>               |          |          |                     tcp_sendmsg
>               |          |          |                     inet_sendmsg
>               |          |          |                     sock_sendmsg
>               |          |          |                     ___sys_sendmsg
>               |          |          |                     __sys_sendmsg
>               |          |          |                     sys_sendmsg
>               |          |          |                     system_call_fastpath
>               |          |          |                     sendmsg
>               |          |          |                     0x40000000000000
>               |          |          |          
>               |          |           --17.56%-- local_bh_enable_ip
>               |          |                     _raw_spin_unlock_bh
>               |          |                     release_sock
>               |          |                     sk_stream_wait_memory
>               |          |                     tcp_sendmsg
>               |          |                     inet_sendmsg
>               |          |                     sock_sendmsg
>               |          |                     ___sys_sendmsg
>               |          |                     __sys_sendmsg
>               |          |                     sys_sendmsg
>               |          |                     system_call_fastpath
>               |          |                     sendmsg
>               |          |                     0x40000000000000
>               |          |          
>               |          |--0.88%-- get_page_from_freelist
>               |          |          __alloc_pages_nodemask
>               |          |          |          
>               |          |          |--50.11%-- __do_page_cache_readahead
>               |          |          |          ra_submit
>               |          |          |          ondemand_readahead
>               |          |          |          |          
>               |          |          |          |--86.17%-- page_cache_async_readahead
>               |          |          |          |          generic_file_aio_read
>               |          |          |          |          do_sync_read
>               |          |          |          |          vfs_read
>               |          |          |          |          sys_pread64
>               |          |          |          |          system_call_fastpath
>               |          |          |          |          pread64
>               |          |          |          |          
>               |          |          |           --13.83%-- page_cache_sync_readahead
>               |          |          |                     generic_file_aio_read
>               |          |          |                     do_sync_read
>               |          |          |                     vfs_read
>               |          |          |                     sys_pread64
>               |          |          |                     system_call_fastpath
>               |          |          |                     pread64
>               |          |          |          
>               |          |           --49.89%-- handle_pte_fault
>               |          |                     handle_mm_fault
>               |          |                     __do_page_fault
>               |          |                     do_page_fault
>               |          |                     page_fault
>               |          |                     generic_file_aio_read
>               |          |                     do_sync_read
>               |          |                     vfs_read
>               |          |                     sys_pread64
>               |          |                     system_call_fastpath
>               |          |                     pread64
>               |          |          
>               |          |--0.76%-- free_pcppages_bulk
>               |          |          |          
>               |          |          |--62.97%-- free_hot_cold_page
>               |          |          |          free_hot_cold_page_list
>               |          |          |          release_pages
>               |          |          |          free_pages_and_swap_cache
>               |          |          |          tlb_flush_mmu
>               |          |          |          tlb_finish_mmu
>               |          |          |          unmap_region
>               |          |          |          do_munmap
>               |          |          |          vm_munmap
>               |          |          |          sys_munmap
>               |          |          |          system_call_fastpath
>               |          |          |          munmap
>               |          |          |          0x40000000000000
>               |          |          |          
>               |          |           --37.03%-- drain_pages
>               |          |                     drain_local_pages
>               |          |                     on_each_cpu_mask
>               |          |                     drain_all_pages
>               |          |                     __alloc_pages_nodemask
>               |          |                     sk_page_frag_refill
>               |          |                     tcp_sendmsg
>               |          |                     inet_sendmsg
>               |          |                     sock_sendmsg
>               |          |                     ___sys_sendmsg
>               |          |                     __sys_sendmsg
>               |          |                     sys_sendmsg
>               |          |                     system_call_fastpath
>               |          |                     sendmsg
>               |          |                     0x40000000000000
>               |          |          
>               |          |--0.71%-- try_to_wake_up
>               |          |          |          
>               |          |          |--94.21%-- default_wake_function
>               |          |          |          |          
>               |          |          |          |--74.70%-- autoremove_wake_function
>               |          |          |          |          __wake_up_common
>               |          |          |          |          |          
>               |          |          |          |          |--66.55%-- __wake_up_sync_key
>               |          |          |          |          |          tcp_prequeue
>               |          |          |          |          |          tcp_v4_rcv
>               |          |          |          |          |          ip_local_deliver_finish
>               |          |          |          |          |          ip_local_deliver
>               |          |          |          |          |          ip_rcv_finish
>               |          |          |          |          |          ip_rcv
>               |          |          |          |          |          __netif_receive_skb_core
>               |          |          |          |          |          __netif_receive_skb
>               |          |          |          |          |          process_backlog
>               |          |          |          |          |          net_rx_action
>               |          |          |          |          |          __do_softirq
>               |          |          |          |          |          call_softirq
>               |          |          |          |          |          do_softirq
>               |          |          |          |          |          local_bh_enable
>               |          |          |          |          |          ip_finish_output
>               |          |          |          |          |          ip_output
>               |          |          |          |          |          ip_local_out
>               |          |          |          |          |          ip_queue_xmit
>               |          |          |          |          |          tcp_transmit_skb
>               |          |          |          |          |          tcp_write_xmit
>               |          |          |          |          |          tcp_push_one
>               |          |          |          |          |          tcp_sendmsg
>               |          |          |          |          |          inet_sendmsg
>               |          |          |          |          |          sock_sendmsg
>               |          |          |          |          |          ___sys_sendmsg
>               |          |          |          |          |          __sys_sendmsg
>               |          |          |          |          |          sys_sendmsg
>               |          |          |          |          |          system_call_fastpath
>               |          |          |          |          |          sendmsg
>               |          |          |          |          |          0x40000000000000
>               |          |          |          |          |          
>               |          |          |          |           --33.45%-- __wake_up
>               |          |          |          |                     sk_stream_write_space
>               |          |          |          |                     tcp_new_space
>               |          |          |          |                     tcp_rcv_established
>               |          |          |          |                     tcp_v4_do_rcv
>               |          |          |          |                     release_sock
>               |          |          |          |                     sk_stream_wait_memory
>               |          |          |          |                     tcp_sendmsg
>               |          |          |          |                     inet_sendmsg
>               |          |          |          |                     sock_sendmsg
>               |          |          |          |                     ___sys_sendmsg
>               |          |          |          |                     __sys_sendmsg
>               |          |          |          |                     sys_sendmsg
>               |          |          |          |                     system_call_fastpath
>               |          |          |          |                     sendmsg
>               |          |          |          |                     0x40000000000000
>               |          |          |          |          
>               |          |          |           --25.30%-- __wake_up_common
>               |          |          |                     __wake_up_locked
>               |          |          |                     ep_poll_callback
>               |          |          |                     __wake_up_common
>               |          |          |                     __wake_up_locked_key
>               |          |          |                     eventfd_write
>               |          |          |                     vfs_write
>               |          |          |                     sys_write
>               |          |          |                     system_call_fastpath
>               |          |          |                     write
>               |          |          |          
>               |          |           --5.79%-- wake_up_state
>               |          |                     wake_futex
>               |          |                     do_futex
>               |          |                     sys_futex
>               |          |                     system_call_fastpath
>               |          |                     pthread_cond_signal
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |                     0x7fd12bf0d910
>               |          |                     0x7fd12bf08188
>               |          |          
>               |          |--0.51%-- mb_cache_shrink_fn
>               |          |          shrink_slab
>               |          |          do_try_to_free_pages
>               |          |          try_to_free_pages
>               |          |          __alloc_pages_nodemask
>               |          |          sk_page_frag_refill
>               |          |          tcp_sendmsg
>               |          |          inet_sendmsg
>               |          |          sock_sendmsg
>               |          |          ___sys_sendmsg
>               |          |          __sys_sendmsg
>               |          |          sys_sendmsg
>               |          |          system_call_fastpath
>               |          |          sendmsg
>               |          |          0x40000000000000
>               |          |          
>               |          |--0.50%-- prune_super
>               |          |          shrink_slab
>               |          |          do_try_to_free_pages



More information about the sheepdog-users mailing list