[sheepdog] [sheepdog/sheepdog] 088913: zk_control: fix "purge" threshold and default beha...

Wed Feb 8 06:53:54 CET 2017

  Branch: refs/heads/stable-1.0
  Home:   https://github.com/sheepdog/sheepdog
  Commit: 088913a6887b81c9c72c6f5b256b51564ce3a01e
      https://github.com/sheepdog/sheepdog/commit/088913a6887b81c9c72c6f5b256b51564ce3a01e
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-20 (Fri, 20 Jan 2017)

  Changed paths:
    M tools/zk_control.c

  Log Message:
  -----------
  zk_control: fix "purge" threshold and default behavior

Now zk_control purge ...
* can purge znodes created within last 24 hours
* purges znode created before more than 24 hours
  by default i.e. when threshold is not given explicitly
* returns error when threshold is negative value (bug fix)

Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
(cherry picked from commit 9a58c28a2af02d1a1a856bb770aed984bb536b70)

  Commit: f4aef345849b286ffe2567409ace16156cc07e1a
      https://github.com/sheepdog/sheepdog/commit/f4aef345849b286ffe2567409ace16156cc07e1a
  Author: Hiroshi Miura <miurahr at linux.com>
  Date:   2017-01-20 (Fri, 20 Jan 2017)

  Changed paths:
    M tools/zk_control.c

  Log Message:
  -----------
  Log exact number of deleted nodes when purge fails

Signed-off-by: Hiroshi Miura <miurahr at linux.com>
(cherry picked from commit 34d3ca833afa63e52eb2106f71f52ee78222d6e9)

  Commit: 133c7cc82b96a89155c01675c47c5d948333031a
      https://github.com/sheepdog/sheepdog/commit/133c7cc82b96a89155c01675c47c5d948333031a
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M dog/benchmark.c
    M include/work.h
    M lib/work.c
    M sheep/sheep.c
    M tests/unit/lib/test_work.c
    M tests/unit/sheep/test_recovery.c

  Log Message:
  -----------
  Remove WQ_UNLIMITED and change the limit of threads for WQ_DYNAMIC

This patch deletes WQ_UNLIMITED option because, with this option,
sheep may consume a very large amount of memory (in some cases,
to be killed by OOM-killer) by creating worker threads infinitely.

This patch also changes the upper limit for the number of threads
for WQ_DYNAMIC option from "#nodes*2" to "max(#nodes,#cores,16)*2".

Signed-off-by: Teruaki Ishizaki <ishizaki.teruaki at lab.ntt.co.jp>
Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
(cherry picked from commit cbe755af178566fa2aa8b0b18e2b8d50c1e74ec7)

  Commit: 0e3479fc90008b22a4d2489b57d94dfeea37045d
      https://github.com/sheepdog/sheepdog/commit/0e3479fc90008b22a4d2489b57d94dfeea37045d
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M include/work.h
    M lib/work.c
    M sheep/sheep.c

  Log Message:
  -----------
  sheep: add a new option for setting threads for dynamic workqueues

This commit adds a new option -x to sheep for configuring the maximum
number of threads for dynamic workqueues.

Example:

  # Set the max number of threads for dynamic workqueues to 500
  $ sheep -x 500

If this is not passed, sheep uses a default formula for configuring
the maximum number (that is, max(#nodes,#cores,16)*2).

This is for tuning performance, so please be careful to use. Sheep
works very slowly if you give a too small number, or may be shot by
OOM-killer under heavy load if you give a huge number.

Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
(cherry picked from commit dd924bdb67b0f5fa668afb24a1152be0c93b1796)

  Commit: cc3beb94df5866c845f03dee7a74a73e42d13f12
      https://github.com/sheepdog/sheepdog/commit/cc3beb94df5866c845f03dee7a74a73e42d13f12
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M lib/work.c

  Log Message:
  -----------
  sheep: fix logic for setting the max number of threads for dynamic WQ

This commit makes a minor change on "doubling growth". If the double
of the current number of threads for a dynamic queue exceeds the
maximum number, that workqueue grows into that maximum number.

Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
(cherry picked from commit c42b9d5f0cd5998911cc271f31dbb934380c4ddf)

  Commit: 8cc97015483068f5c157a7848fe27bc40830fc22
      https://github.com/sheepdog/sheepdog/commit/8cc97015483068f5c157a7848fe27bc40830fc22
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M tools/zk_control.c

  Log Message:
  -----------
  Merge branch 'for-v1.0.2-fix-zk_control' into for-v1.0.2

  Commit: cbf50bdb8b015634db8cc84acd5cee114a6efe22
      https://github.com/sheepdog/sheepdog/commit/cbf50bdb8b015634db8cc84acd5cee114a6efe22
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M dog/benchmark.c
    M include/work.h
    M lib/work.c
    M sheep/sheep.c
    M tests/unit/lib/test_work.c
    M tests/unit/sheep/test_recovery.c

  Log Message:
  -----------
  Merge branch 'for-v1.0.2-fix-max-threads' into for-v1.0.2

  Commit: bdb08f65bde89f3e1719e5a82ce1b4e026179e69
      https://github.com/sheepdog/sheepdog/commit/bdb08f65bde89f3e1719e5a82ce1b4e026179e69
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M sheep/ops.c

  Log Message:
  -----------
  sheep: remove object list cache of ledger when removed from store

Each object list cache should be removed when it is removed from
object store. Inode and data ones are done so by peer_remove_obj.
However, ledger objects are not so by peer_decref_object, and
remaining cache leads to unnecessary recovery trial and error log.

This commit lets sheep remove object list cache of ledger when it is
removed from store.

Fix #341

Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
(cherry picked from commit 254730a6992f74ad635db4417e493de4b45afe6b)

  Commit: 941879a64eaaf060c74ed4e6afb1a348ce205c7e
      https://github.com/sheepdog/sheepdog/commit/941879a64eaaf060c74ed4e6afb1a348ce205c7e
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-01-23 (Mon, 23 Jan 2017)

  Changed paths:
    M dog/benchmark.c
    M dog/common.c
    M dog/farm/farm.c
    M dog/farm/object_tree.c
    M dog/vdi.c
    M lib/sd_inode.c
    M sheep/gateway.c
    M sheep/group.c
    M sheep/http/kv.c
    M sheep/http/oalloc.c
    M sheep/nfs/fs.c
    M sheep/nfs/nfs.c
    M sheep/object_list_cache.c
    M sheep/ops.c
    M sheep/recovery.c
    M sheep/request.c
    M sheep/store/common.c
    M sheep/store/plain_store.c
    M sheep/store/tree_store.c
    M sheep/vdi.c
    M sheepfs/volume.c
    M tests/functional/028.out
    M tests/functional/042.out
    M tests/functional/080.out

  Log Message:
  -----------
  Print object ID in '"%016" PRIx64'

This commit lets Sheepdog print object ID in 16-digit zero-padded
hexadecimal, that is, '"%016" PRIx64'.

I think this is operator-friendly because s/he can tell whether
that object is a data object (00...), an inode object (80...), a
ledger object (08...) and so on, simply by looking at the printed.

Fix #340

Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
(cherry picked from commit e882d76dd632168f680dc31af81cfbdbc42225d9)

  Commit: a5a238c3c679a07a094cf531838107abd74a0319
      https://github.com/sheepdog/sheepdog/commit/a5a238c3c679a07a094cf531838107abd74a0319
  Author: Kazuhisa Hara <khara at sios.com>
  Date:   2017-02-08 (Wed, 08 Feb 2017)

  Changed paths:
    M sheepdog.spec.in

  Log Message:
  -----------
  rpm: support for systemd and sysvinit

In the current spec.in, rpmbuild fails in sysvinit distribution(ex: el6).
Therefore, set behavior in the case of systemd, and explicitly divide it
from sysvinit behavior.

Signed-off-by: Kazuhisa Hara <khara at sios.com>
(cherry picked from commit 454b84a0dc7f61bccff5c435f7a742c9bd14e94f)

  Commit: 9bb7a0c233ace5c162bff0c79bcbcd4db7da919a
      https://github.com/sheepdog/sheepdog/commit/9bb7a0c233ace5c162bff0c79bcbcd4db7da919a
  Author: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>
  Date:   2017-02-08 (Wed, 08 Feb 2017)

  Changed paths:
    M CHANGELOG.md
    M configure.ac

  Log Message:
  -----------
  sheepdog stable release v1.0.2_rc0

Signed-off-by: Takashi Menjo <menjo.takashi at lab.ntt.co.jp>

Compare: https://github.com/sheepdog/sheepdog/compare/7d00020ba45e...9bb7a0c233ac