[sheepdog-users] dog {cluster,vdi} check needs too much ram
Hitoshi Mitake
mitake.hitoshi at gmail.com
Sat May 31 17:05:18 CEST 2014
At Sat, 31 May 2014 00:33:38 +0200,
Marcin Mirosław wrote:
Hi Marcin,
>
> Hi!
> I'm locally launching three sheeps on three disks, next:
> # dog vdi create -P -c 2:1 -v test 1T
> 0.9 % [>
> ] 9.2 GB / 1.0 TB ^C
> and I'm hitting ctrl+c to stop command. Now I can see:
> # dog vdi list
> Name Id Size Used Shared Creation time VDI id
> Copies Tag
> test 0 1.0 TB 9.2 GB 0.0 MB 2014-05-31 00:19 7c2b25 2:1
>
> and I started wondering if vdi isn't corrupted:
>
> # dog vdi check test
> 7.3 % [========>
> ] 75 GB / 1.0 TB Killed
>
> # dog cluster check
> fix vdi test
> 3.4 % [===>
> ] 34 GB / 1.0 TB Killed
>
> In dmesg there is:
> > [ 2577.036700] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
> > [ 2577.036711] [ 546] 0 546 6461 120 14 0 -1000 udevd
> > [ 2577.036716] [ 1040] 0 1040 120489 193 34 0 0 rsyslogd
> > [ 2577.036719] [ 1062] 0 1062 1062 35 5 0 0 acpid
> > [ 2577.036723] [ 1100] 0 1100 4979 148 14 0 0 crond
> > [ 2577.036726] [ 1364] 0 1364 579 17 3 0 0 busybox
> > [ 2577.036730] [ 1439] 40 1439 49058 4903 28 0 0 named
> > [ 2577.036733] [ 1474] 123 1474 7298 79 18 0 0 ntpd
> > [ 2577.036736] [ 1491] 0 1491 2253 44 9 0 0 rsync
> > [ 2577.036740] [ 1509] 0 1509 11638 163 26 0 0 smbd
> > [ 2577.036743] [ 1514] 0 1514 7924 109 19 0 0 nmbd
> > [ 2577.036746] [ 1515] 0 1515 7843 94 19 0 0 nmbd
> > [ 2577.036750] [ 1529] 0 1529 11638 163 25 0 0 smbd
> > [ 2577.036753] [ 1534] 0 1534 7525 119 20 0 -1000 sshd
> > [ 2577.036756] [ 1551] 0 1551 3703 39 11 0 0 agetty
> > [ 2577.036759] [ 1552] 0 1552 3703 38 11 0 0 agetty
> > [ 2577.036763] [ 1553] 0 1553 3703 38 10 0 0 agetty
> > [ 2577.036766] [ 1554] 0 1554 3703 39 11 0 0 agetty
> > [ 2577.036769] [ 1555] 0 1555 3703 39 11 0 0 agetty
> > [ 2577.036772] [ 1556] 0 1556 3703 40 11 0 0 agetty
> > [ 2577.036776] [ 1557] 0 1557 24818 273 39 0 0 sshd
> > [ 2577.036779] [ 1563] 0 1563 5320 847 15 0 0 bash
> > [ 2577.036802] [ 1333] 0 1333 3450 75 11 0 0 tmux
> > [ 2577.036805] [ 1335] 0 1335 7293 1842 18 0 0 tmux
> > [ 2577.036808] [ 1336] 0 1336 5337 858 15 0 0 bash
> > [ 2577.036812] [ 1389] 0 1389 5330 843 14 0 0 bash
> > [ 2577.036815] [22751] 0 22751 5372 856 15 0 0 bash
> > [ 2577.036821] [ 8286] 109 8286 369574 1204 56 0 0 sheep
> > [ 2577.036824] [ 8287] 109 8287 8617 47 12 0 0 sheep
> > [ 2577.036827] [ 8531] 109 8531 418726 2380 63 0 0 sheep
> > [ 2577.036830] [ 8532] 109 8532 8617 47 12 0 0 sheep
> > [ 2577.036834] [ 8799] 109 8799 353185 2295 54 0 0 sheep
> > [ 2577.036837] [ 8800] 109 8800 8617 46 12 0 0 sheep
> > [ 2577.036841] [ 5941] 0 5941 24095 14199 50 0 0 emerge
> > [ 2577.036845] [ 6134] 250 6134 1061 37 7 0 0 sandbox
> > [ 2577.036848] [ 6136] 250 6136 6081 496 15 0 0 ebuild.sh
> > [ 2577.036852] [ 6153] 250 6153 6087 518 14 0 0 ebuild.sh
> > [ 2577.036855] [ 6175] 250 6175 5846 255 15 0 0 configure
> > [ 2577.036859] [ 6432] 0 6430 372619 296187 601 0 0 dog
> > [ 2577.036863] [ 6456] 250 6456 5846 254 14 0 0 configure
> > [ 2577.036866] [ 6457] 250 6457 18555 70 9 0 0 conftest
> > [ 2577.036869] Out of memory: Kill process 6432 (dog) score 753 or sacrifice child
> > [ 2577.036872] Killed process 6432 (dog) total-vm:1490476kB, anon-rss:1184744kB, file-rss:4kB
>
>
> Hmm, I did another test. Even I don't hit ctrl+c while creating vdi dog
> clucter check is killed by OOM.
> Box has 1.5G RAM.
> Is it expected that dog cluster check needs so much memory?
> sheepdog at 77fa9112
Yes, "dog vdi check" needs amount of memory when it checks large
VDI. The problem is caused because dog doesn't manage worker threads
well. Could you add an issue to our tracker?
https://bugs.launchpad.net/sheepdog-project/+bugs
Solving this problem is not so difficult but I have other tasks,
sorry...
Thanks,
Hitoshi
More information about the sheepdog-users
mailing list