From patchwork Wed Mar 31 03:12:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 12174243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D53CCC433C1 for ; Wed, 31 Mar 2021 03:12:51 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 98290619B1 for ; Wed, 31 Mar 2021 03:12:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98290619B1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 6211F100EB32A; Tue, 30 Mar 2021 20:12:51 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=134.134.136.20; helo=mga02.intel.com; envelope-from=vishal.l.verma@intel.com; receiver= Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5CCE5100EB85F for ; Tue, 30 Mar 2021 20:12:48 -0700 (PDT) IronPort-SDR: V70fN+7E2Kt9hDhhappehkFgNC5iQUrdQJQgy0x7g2Byth+BQlk1Q9HJ/jn9Rs5mf3CIMqfGoi NH5lXzAEam/A== X-IronPort-AV: E=McAfee;i="6000,8403,9939"; a="179035513" X-IronPort-AV: E=Sophos;i="5.81,291,1610438400"; d="scan'208";a="179035513" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2021 20:12:48 -0700 IronPort-SDR: 07P68APtm6vdbbMfnPkPEBezEfc2owFP7AQFI2vPCe9PP4p96PwbmoK5P1Ud4y6IIr/6cnm1MD y3HoOwRMUpcQ== X-IronPort-AV: E=Sophos;i="5.81,291,1610438400"; d="scan'208";a="438581731" Received: from choffma1-mobl.amr.corp.intel.com (HELO omniknight.intel.com) ([10.212.71.210]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2021 20:12:47 -0700 From: Vishal Verma To: Subject: [ndctl PATCH 1/3] daxctl: fail reconfigure-device based on kernel onlining policy Date: Tue, 30 Mar 2021 21:12:27 -0600 Message-Id: <20210331031229.384068-2-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210331031229.384068-1-vishal.l.verma@intel.com> References: <20210331031229.384068-1-vishal.l.verma@intel.com> MIME-Version: 1.0 Message-ID-Hash: 44COBQOGPLMPT3B66EIQ7M3Q7BG6EPGD X-Message-ID-Hash: 44COBQOGPLMPT3B66EIQ7M3Q7BG6EPGD X-MailFrom: vishal.l.verma@intel.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Dave Hansen , Chunye Xu , Dave Hansen X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: If the kernel has a policy set to auto-online any new memory blocks, we know that an attempt to reconfigure a device either in ZONE_MOVABLE, or with the --no-online is going to fail. While we detect this race after the fact, and print a warning, that is often insufficient as the user may be forced to reboot to get out of the situation, resulting in an unpleasant experience. Detect whether the kernel policy is set to auto-online. If so, fail device reconfigure operations that we know can't be satisfied. Allow for overriding this safety check via the -f (--force) option. Update the man page to talk about this, and the unit test to test for an expected failure by enabling auto-onlining. Cc: Dave Hansen Reported-by: Chunye Xu Reported-by: Dan Williams Signed-off-by: Vishal Verma --- .../daxctl/daxctl-reconfigure-device.txt | 12 ++++++- daxctl/lib/libdaxctl-private.h | 1 + daxctl/lib/libdaxctl.c | 21 +++++++++++ daxctl/libdaxctl.h | 1 + daxctl/device.c | 10 ++++++ daxctl/lib/libdaxctl.sym | 5 +++ test/daxctl-devices.sh | 36 +++++++++++++++++++ 7 files changed, 85 insertions(+), 1 deletion(-) diff --git a/Documentation/daxctl/daxctl-reconfigure-device.txt b/Documentation/daxctl/daxctl-reconfigure-device.txt index ad33eda..f112b3c 100644 --- a/Documentation/daxctl/daxctl-reconfigure-device.txt +++ b/Documentation/daxctl/daxctl-reconfigure-device.txt @@ -119,6 +119,10 @@ recommended to use the --no-online option described below. This will abridge the device reconfiguration operation to just hotplugging the memory, and refrain from then onlining it. +In case daxctl detects that there is a kernel policy to auto-online blocks +(via /sys/devices/system/memory/auto_online_blocks), then reconfiguring to +system-ram will result in a failure. This can be overridden with '--force'. + OPTIONS ------- include::region-option.txt[] @@ -162,12 +166,18 @@ include::movable-options.txt[] -f:: --force:: - When converting from "system-ram" mode to "devdax", it is expected + - When converting from "system-ram" mode to "devdax", it is expected that all the memory sections are first made offline. By default, daxctl won't touch online memory. However with this option, attempt to offline the memory on the NUMA node associated with the dax device before converting it back to "devdax" mode. + - Additionally, if a kernel policy to auto-online blocks is detected, + reconfiguration to system-ram fails. With this option, the failure can + be overridden to allow reconfiguration regardless of kernel policy. + Doing this may result in a successful reconfiguration, but it may + not be possible to subsequently offline the memory without a reboot. + include::human-option.txt[] diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index af257fd..ae45311 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -111,6 +111,7 @@ struct daxctl_memory { char *node_path; unsigned long block_size; enum memory_zones zone; + bool auto_online; }; diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 479e8f6..879f7e6 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -1644,3 +1644,24 @@ DAXCTL_EXPORT int daxctl_memory_is_movable(struct daxctl_memory *mem) return rc; return (mem->zone == MEM_ZONE_MOVABLE) ? 1 : 0; } + +DAXCTL_EXPORT int daxctl_dev_will_auto_online_memory(struct daxctl_dev *dev) +{ + const char *auto_path = "/sys/devices/system/memory/auto_online_blocks"; + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char buf[SYSFS_ATTR_SIZE]; + + /* + * If we can't read the policy for some reason, don't fail yet. Assume + * the auto-onlining policy is absent, and carry on. If onlining blocks + * does result in the memory being in an inconsistent state, we have a + * check and warning for it after the fact + */ + if (sysfs_read_attr(ctx, auto_path, buf) != 0) + err(ctx, "%s: Unable to determine auto-online policy: %s\n", + devname, strerror(errno)); + + /* match both "online" and "online_movable" */ + return !strncmp(buf, "online", 6); +} diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index e82b274..30ab51a 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -71,6 +71,7 @@ int daxctl_dev_disable(struct daxctl_dev *dev); int daxctl_dev_enable_devdax(struct daxctl_dev *dev); int daxctl_dev_enable_ram(struct daxctl_dev *dev); int daxctl_dev_get_target_node(struct daxctl_dev *dev); +int daxctl_dev_will_auto_online_memory(struct daxctl_dev *dev); struct daxctl_memory; struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev); diff --git a/daxctl/device.c b/daxctl/device.c index 0721a57..a427b7d 100644 --- a/daxctl/device.c +++ b/daxctl/device.c @@ -541,8 +541,18 @@ static int disable_devdax_device(struct daxctl_dev *dev) static int reconfig_mode_system_ram(struct daxctl_dev *dev) { + const char *devname = daxctl_dev_get_devname(dev); int rc, skip_enable = 0; + if (param.no_online || !param.no_movable) { + if (!param.force && daxctl_dev_will_auto_online_memory(dev)) { + fprintf(stderr, + "%s: error: kernel policy will auto-online memory, aborting\n", + devname); + return -EBUSY; + } + } + if (daxctl_dev_is_enabled(dev)) { rc = disable_devdax_device(dev); if (rc < 0) diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index a4e1684..892e393 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -91,3 +91,8 @@ global: daxctl_mapping_get_size; daxctl_dev_set_mapping; } LIBDAXCTL_7; + +LIBDAXCTL_9 { +global: + daxctl_dev_will_auto_online_memory; +} LIBDAXCTL_8; diff --git a/test/daxctl-devices.sh b/test/daxctl-devices.sh index 496e4f2..eed5906 100755 --- a/test/daxctl-devices.sh +++ b/test/daxctl-devices.sh @@ -64,6 +64,26 @@ daxctl_get_mode() "$DAXCTL" list -d "$1" | jq -er '.[].mode' } +set_online_policy() +{ + echo "online" > /sys/devices/system/memory/auto_online_blocks +} + +unset_online_policy() +{ + echo "offline" > /sys/devices/system/memory/auto_online_blocks +} + +save_online_policy() +{ + saved_policy="$(cat /sys/devices/system/memory/auto_online_blocks)" +} + +restore_online_policy() +{ + echo "$saved_policy" > /sys/devices/system/memory/auto_online_blocks +} + daxctl_test() { local daxdev @@ -71,6 +91,9 @@ daxctl_test() daxdev=$(daxctl_get_dev "$testdev") test -n "$daxdev" + # these tests need to run with kernel onlining policy turned off + save_online_policy + unset_online_policy "$DAXCTL" reconfigure-device -N -m system-ram "$daxdev" [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] "$DAXCTL" online-memory "$daxdev" @@ -81,6 +104,19 @@ daxctl_test() [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] "$DAXCTL" reconfigure-device -f -m devdax "$daxdev" [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] + + # this tests for reconfiguration failure if an online-policy is set + set_online_policy + : "This command is expected to fail:" + if ! "$DAXCTL" reconfigure-device -N -m system-ram "$daxdev"; then + echo "reconfigure failed as expected" + else + echo "reconfigure succeded, expected failure" + restore_online_policy + return 1 + fi + + restore_online_policy } find_testdev From patchwork Wed Mar 31 03:12:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 12174245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_50, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6BFEC433E1 for ; Wed, 31 Mar 2021 03:12:52 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C619619E8 for ; Wed, 31 Mar 2021 03:12:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C619619E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 84ABD100EB330; Tue, 30 Mar 2021 20:12:51 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=134.134.136.20; helo=mga02.intel.com; envelope-from=vishal.l.verma@intel.com; receiver= Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 50825100EB85F for ; Tue, 30 Mar 2021 20:12:49 -0700 (PDT) IronPort-SDR: EojhTVuWTz8efJyhoTeHKmBNTtVJ+Sd8Hb/5GoiokWJdd1v7L91SndmF7kHXzgg/eCuHAtEQ81 FtUpEXgZUSGQ== X-IronPort-AV: E=McAfee;i="6000,8403,9939"; a="179035523" X-IronPort-AV: E=Sophos;i="5.81,291,1610438400"; d="scan'208";a="179035523" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2021 20:12:49 -0700 IronPort-SDR: L3XRBIKZfjsJaeuT7BexLdTBR6JnBNxGgmxjbuRVuQvC9KTro3gUyna6nOgLnsumax8pFAZtU3 QiBEgCufQweQ== X-IronPort-AV: E=Sophos;i="5.81,291,1610438400"; d="scan'208";a="438581738" Received: from choffma1-mobl.amr.corp.intel.com (HELO omniknight.intel.com) ([10.212.71.210]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2021 20:12:48 -0700 From: Vishal Verma To: Subject: [ndctl PATCH 2/3] libdaxctl: add an API to check if a device is active Date: Tue, 30 Mar 2021 21:12:28 -0600 Message-Id: <20210331031229.384068-3-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210331031229.384068-1-vishal.l.verma@intel.com> References: <20210331031229.384068-1-vishal.l.verma@intel.com> MIME-Version: 1.0 Message-ID-Hash: 7CXJC2A33CSDHNDEQYOKDDSGH6WH66IJ X-Message-ID-Hash: 7CXJC2A33CSDHNDEQYOKDDSGH6WH66IJ X-MailFrom: vishal.l.verma@intel.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Dave Hansen , Chunye Xu X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Add an API to check whether a daxctl device is active in system-ram mode. This would be used from libndctl during ndctl_namespace_disable_safe(), so that we don't disable/destroy an underlying namespace while the memory is active and online. Reported-by: Chunye Xu Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 10 ++++++++++ daxctl/libdaxctl.h | 1 + daxctl/lib/libdaxctl.sym | 1 + 3 files changed, 12 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 879f7e6..860bd9c 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -1665,3 +1665,13 @@ DAXCTL_EXPORT int daxctl_dev_will_auto_online_memory(struct daxctl_dev *dev) /* match both "online" and "online_movable" */ return !strncmp(buf, "online", 6); } + +DAXCTL_EXPORT int daxctl_dev_has_online_memory(struct daxctl_dev *dev) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + + if (mem) + return daxctl_memory_is_online(mem); + else + return 0; +} diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index 30ab51a..683ae9c 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -72,6 +72,7 @@ int daxctl_dev_enable_devdax(struct daxctl_dev *dev); int daxctl_dev_enable_ram(struct daxctl_dev *dev); int daxctl_dev_get_target_node(struct daxctl_dev *dev); int daxctl_dev_will_auto_online_memory(struct daxctl_dev *dev); +int daxctl_dev_has_online_memory(struct daxctl_dev *dev); struct daxctl_memory; struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev); diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 892e393..a13e93d 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -95,4 +95,5 @@ global: LIBDAXCTL_9 { global: daxctl_dev_will_auto_online_memory; + daxctl_dev_has_online_memory; } LIBDAXCTL_8; From patchwork Wed Mar 31 03:12:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 12174247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C131C433DB for ; Wed, 31 Mar 2021 03:12:54 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1D70F619EF for ; Wed, 31 Mar 2021 03:12:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1D70F619EF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9B110100EB333; Tue, 30 Mar 2021 20:12:52 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=134.134.136.20; helo=mga02.intel.com; envelope-from=vishal.l.verma@intel.com; receiver= Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2CC2E100EB323 for ; Tue, 30 Mar 2021 20:12:50 -0700 (PDT) IronPort-SDR: PEQGl2zOqWkOdt3IPsyYyWD/goHMxxLL6I/kqWXcZ6YqmFM4qPFxbBACJIpxIa3EBrxgnw3evp K167W3oOoM2Q== X-IronPort-AV: E=McAfee;i="6000,8403,9939"; a="179035534" X-IronPort-AV: E=Sophos;i="5.81,291,1610438400"; d="scan'208";a="179035534" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2021 20:12:50 -0700 IronPort-SDR: ah/oYgQQVMTxNEhRuqrBu9AjlMr9xU+wiuaar9ZDwf/qH4OQhJVT3i7IsFGZXb/oRrYZbe2iDy 6teRvFN2mtlQ== X-IronPort-AV: E=Sophos;i="5.81,291,1610438400"; d="scan'208";a="438581744" Received: from choffma1-mobl.amr.corp.intel.com (HELO omniknight.intel.com) ([10.212.71.210]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2021 20:12:49 -0700 From: Vishal Verma To: Subject: [ndctl PATCH 3/3] libndctl: check for active system-ram before disabling daxctl devices Date: Tue, 30 Mar 2021 21:12:29 -0600 Message-Id: <20210331031229.384068-4-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210331031229.384068-1-vishal.l.verma@intel.com> References: <20210331031229.384068-1-vishal.l.verma@intel.com> MIME-Version: 1.0 Message-ID-Hash: 7IB5YTDWAB5QQIVLXSH6P37UQCPEZFRN X-Message-ID-Hash: 7IB5YTDWAB5QQIVLXSH6P37UQCPEZFRN X-MailFrom: vishal.l.verma@intel.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: Dave Hansen , Chunye Xu X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Teach ndctl_namespace_disable_safe() to look at the state of a daxctl_dev with respect to whether it is active in 'system-ram' mode before disabling it. This is similar to checking whether a filesystem is actively mounted on a namespace before disabling it. Without this, libndctl would happily disable a devdax namespace while the device was active in system-ram mode. If the namespace was subsequently also destroyed, this would leave the memory without any sort of a 'handle' to perform any subsequent operation on it, and the system would have to be rebooted to get out of this situation. Reported-by: Chunye Xu Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- ndctl/lib/libndctl.c | 25 ++++++++++++++++++++++++- test/daxctl-devices.sh | 16 ++++++++++++++++ 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/ndctl/lib/libndctl.c b/ndctl/lib/libndctl.c index 2f6d806..2eda56c 100644 --- a/ndctl/lib/libndctl.c +++ b/ndctl/lib/libndctl.c @@ -4593,21 +4593,40 @@ NDCTL_EXPORT int ndctl_namespace_disable_invalidate(struct ndctl_namespace *ndns return ndctl_namespace_disable(ndns); } +static int ndctl_dax_has_active_memory(struct ndctl_dax *dax) +{ + struct daxctl_region *dax_region; + struct daxctl_dev *dax_dev; + + dax_region = ndctl_dax_get_daxctl_region(dax); + if (!dax_region) + return 0; + + daxctl_dev_foreach(dax_region, dax_dev) + if (daxctl_dev_has_online_memory(dax_dev)) + return 1; + + return 0; +} + NDCTL_EXPORT int ndctl_namespace_disable_safe(struct ndctl_namespace *ndns) { const char *devname = ndctl_namespace_get_devname(ndns); struct ndctl_ctx *ctx = ndctl_namespace_get_ctx(ndns); struct ndctl_pfn *pfn = ndctl_namespace_get_pfn(ndns); struct ndctl_btt *btt = ndctl_namespace_get_btt(ndns); + struct ndctl_dax *dax = ndctl_namespace_get_dax(ndns); const char *bdev = NULL; + int fd, active = 0; char path[50]; - int fd; unsigned long long size = ndctl_namespace_get_size(ndns); if (pfn && ndctl_pfn_is_enabled(pfn)) bdev = ndctl_pfn_get_block_device(pfn); else if (btt && ndctl_btt_is_enabled(btt)) bdev = ndctl_btt_get_block_device(btt); + else if (dax && ndctl_dax_is_enabled(dax)) + active = ndctl_dax_has_active_memory(dax); else if (ndctl_namespace_is_enabled(ndns)) bdev = ndctl_namespace_get_block_device(ndns); @@ -4632,6 +4651,10 @@ NDCTL_EXPORT int ndctl_namespace_disable_safe(struct ndctl_namespace *ndns) devname, bdev, strerror(errno)); return -errno; } + } else if (active) { + dbg(ctx, "%s: active as system-ram, refusing to disable\n", + devname); + return -EBUSY; } else { if (size == 0) /* No disable necessary due to no capacity allocated */ diff --git a/test/daxctl-devices.sh b/test/daxctl-devices.sh index eed5906..56c9691 100755 --- a/test/daxctl-devices.sh +++ b/test/daxctl-devices.sh @@ -105,6 +105,22 @@ daxctl_test() "$DAXCTL" reconfigure-device -f -m devdax "$daxdev" [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] + # fail 'ndctl-disable-namespace' while the devdax namespace is active + # as system-ram. If this test fails, a reboot will be required to + # recover from the resulting state. + test -n "$testdev" + "$DAXCTL" reconfigure-device -m system-ram "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] + if ! "$NDCTL" disable-namespace "$testdev"; then + echo "disable-namespace failed as expected" + else + echo "disable-namespace succeded, expected failure" + echo "reboot required to recover from this state" + return 1 + fi + "$DAXCTL" reconfigure-device -f -m devdax "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] + # this tests for reconfiguration failure if an online-policy is set set_online_policy : "This command is expected to fail:"