From patchwork Thu Aug 17 07:30:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shinichiro Kawasaki X-Patchwork-Id: 13356065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52E30C2FC0E for ; Thu, 17 Aug 2023 07:31:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344134AbjHQHa3 (ORCPT ); Thu, 17 Aug 2023 03:30:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348466AbjHQHa2 (ORCPT ); Thu, 17 Aug 2023 03:30:28 -0400 Received: from esa1.hgst.iphmx.com (esa1.hgst.iphmx.com [68.232.141.245]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F07241FC8 for ; Thu, 17 Aug 2023 00:30:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1692257423; x=1723793423; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=kWnODAqt0Mw2BZKaBie2yj1ClUJdr2nuftoERPO8Oi4=; b=V0in4o9215LSn7AOCSvjgJAcnGloPNQD6gH5+HeXec8rry1AONHN38jn IS/5DyP05UDspRTNyInOC3RGm0eEDfaw8LkAN1Inj4Z//Fz/+VBwwLBey RwxTrKYG1u3euFz8ytjl4aptHQkOqUrtW2qNXKIPqjXF6+YRSgR4289Rh OcEki8bKfW1meeqixcBqi94ymxJTtUkxRuX4WmoBzOlVhwv1UmaLnmko8 TZJgPaiPJG55mXpixxNMdQ/WgLj65kRL0oKC4vrZCsnMX2Hi3JlvjPe5P zEGvxBDvk4hUVQ+wNaFihcF9ubK4EvTHdk9Bd/F4XpULKIDw8vo2meJaz Q==; X-IronPort-AV: E=Sophos;i="6.01,179,1684771200"; d="scan'208";a="353323160" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 17 Aug 2023 15:30:23 +0800 IronPort-SDR: CsRLVQE9ihaZRJfvWx/thg6c6QdW7ENDsVbm9n3pRUV3UfBgZbUOk4WnzLrW8KvYfwHxgxykZV 7vz/wr2Q1zlBRJev0CVewbge+HTSLW9oip3s0pIjn9cKI93rjbzeaT+4zOn/BTVMua7MtW1pU+ 7N2u+tzylOeBKRyjXENKz/109id+GBUSqDVLoCr7HDOyXEuAoxsraJSCqc2NsUb86gkFv9bx+3 SwrZHcEcDvHHDnCeCKPv0C50Eg/WshqP/DIThFOnhaNiVgtkW6YebN5TL4vRvpwlq5HA/LhJz1 5Bg= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 16 Aug 2023 23:37:59 -0700 IronPort-SDR: LAr0+QMRHuwnWLVYg6EaJ+VLplf4pxXvEatSUgI4U3v9CFhhYDtJ3Nn//+eb8hsbFgBimh1EzD 7DxhSAi9se9Q6P9OB3zB/Coc2TdNflkJAAz2moDjTScg1dId40qB92ekl/7OQkdtbXDswp6FXU WiEZR5FzrEfR/aL1IctiboqG+ApjcU4eDKaj3AvDlRQun2Os/4vxM8TthoiyMNNhCCL/f3nSgT JkACwd9+Edof+HCK1FEIyXSkgGIT2zbu4VrOdVjtqoxis6TSUPnNrLFaaJM3TJHRxOUMDDUalx 61E= WDCIronportException: Internal Received: from shindev.dhcp.fujisawa.hgst.com (HELO shindev.fujisawa.hgst.com) ([10.149.53.55]) by uls-op-cesaip01.wdc.com with ESMTP; 17 Aug 2023 00:30:22 -0700 From: Shin'ichiro Kawasaki To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Yi Zhang , Daniel Wagner , Sagi Grimberg , Bart Van Assche , Shin'ichiro Kawasaki Subject: [PATCH blktests v2 1/2] nvme/rc: fix nvme device readiness check after _nvme_connect_subsys Date: Thu, 17 Aug 2023 16:30:20 +0900 Message-Id: <20230817073021.3674602-1-shinichiro.kawasaki@wdc.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The helper function _nvme_connect_subsys() creates a nvme device. It may take some time after the function call until the device gets ready for I/O. So it is expected that the test cases call _find_nvme_dev() after _nvme_connect_subsys() before I/O. _find_nvme_dev() returns the path of the created device, and it also waits for uuid and wwid sysfs attributes of the created device get ready. This wait works as the wait for the device I/O readiness. However, this wait by _find_nvme_dev() has two problems. The first problem is missing call of _find_nvme_dev(). The test case nvme/047 calls _nvme_connect_subsys() twice, but _find_nvme_dev() is called only for the first _nvme_connect_subsys() call. This causes too early I/O to the device with tcp transport [1]. Fix this by moving the wait for the device readiness from _find_nvme_dev() to _nvme_connect_subsys(). Also add --wait-for option to _nvme_connect_subsys(). It allows to skip the wait in _nvmet_passthru_target_connect() which has its own wait for device readiness. The second problem is wrong paths for the sysfs attributes. The paths do not include namespace index, so the check for the attributes always fail. Still _find_nvme_dev() does 1 second wait and allows the device get ready for I/O in most cases, but this is not intended behavior. Fix the paths by adding the namespace index. On top of the checks for sysfs attributes, add 'udevadm settle' and a check for the created device file. These ensures that the create device is ready for I/O. [1] https://lore.kernel.org/linux-block/CAHj4cs9GNohGUjohNw93jrr8JGNcRYC-ienAZz+sa7az1RK77w@mail.gmail.com/ Fixes: c766fccf3aff ("Make the NVMe tests more reliable") Signed-off-by: Shin'ichiro Kawasaki Reviewed-by: Daniel Wagner --- Changes from v1: * Added --wait-for option to _nvme_connect_subsys() * Added 'udevadm settle' before readiness check tests/nvme/rc | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/tests/nvme/rc b/tests/nvme/rc index 0b964e9..797483e 100644 --- a/tests/nvme/rc +++ b/tests/nvme/rc @@ -428,6 +428,8 @@ _nvme_connect_subsys() { local keep_alive_tmo="" local reconnect_delay="" local ctrl_loss_tmo="" + local wait_for="ns" + local dev i while [[ $# -gt 0 ]]; do case $1 in @@ -483,6 +485,10 @@ _nvme_connect_subsys() { ctrl_loss_tmo="$2" shift 2 ;; + --wait-for) + wait_for="$2" + shift 2 + ;; *) positional_args+=("$1") shift @@ -532,6 +538,21 @@ _nvme_connect_subsys() { fi nvme connect "${ARGS[@]}" 2> /dev/null + + # Wait until device file and uuid/wwid sysfs attributes get ready for + # namespace 1. + if [[ ${wait_for} == ns ]]; then + udevadm settle + dev=$(_find_nvme_dev "$subsysnqn") + for ((i = 0; i < 10; i++)); do + if [[ -b /dev/${dev}n1 && + -e /sys/block/${dev}n1/uuid && + -e /sys/block/${dev}n1/wwid ]]; then + return + fi + sleep .1 + done + fi } _nvme_discover() { @@ -758,13 +779,6 @@ _find_nvme_dev() { subsysnqn="$(cat "/sys/class/nvme/${dev}/subsysnqn")" if [[ "$subsysnqn" == "$subsys" ]]; then echo "$dev" - for ((i = 0; i < 10; i++)); do - if [[ -e /sys/block/$dev/uuid && - -e /sys/block/$dev/wwid ]]; then - return - fi - sleep .1 - done fi done } @@ -794,7 +808,8 @@ _nvmet_passthru_target_connect() { local trtype=$1 local subsys_name=$2 - _nvme_connect_subsys "${trtype}" "${subsys_name}" || return + _nvme_connect_subsys "${trtype}" "${subsys_name}" --wait-for=none || + return nsdev=$(_find_nvme_passthru_loop_dev "${subsys_name}") # The following tests can race with the creation From patchwork Thu Aug 17 07:30:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shinichiro Kawasaki X-Patchwork-Id: 13356066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A3A6C41513 for ; Thu, 17 Aug 2023 07:31:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348424AbjHQHa3 (ORCPT ); Thu, 17 Aug 2023 03:30:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348467AbjHQHa2 (ORCPT ); Thu, 17 Aug 2023 03:30:28 -0400 Received: from esa1.hgst.iphmx.com (esa1.hgst.iphmx.com [68.232.141.245]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 015CF1FCE for ; Thu, 17 Aug 2023 00:30:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1692257424; x=1723793424; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UUWnka0lvWrfZ4NWtFcx8XJGzhEQIbMjxWq1Vz1NalM=; b=Kf/lXpYNgZONb4hQq2koNh68ZLgfBLx34X7XC60fXA6yeH6SVvTKVsgn aA/Srmgd+knf5Wsk0kNocrtrSnON7m9zCrcehAcLdG//eBkZV62ljo/bt L3ugqt/WpKnikD4FBcYARH13muSz1GzoL/cfvEWHQ7uPIZ4Gq/Q2wFNEB gXmknMdoZChbwHQrV3hY81R6+cnLVwIHiy7oCI4KECGPvPBLY9n/9Xwip OUPBeigP1m5VaiLiac1LVux6DNbZVHEJ381u5CGCgUcm3YKE6Evws98JO meOH+neN5/pKSsesvRW4/97Z8/0YnDq8pSQWTDXp8Z+dBE/gBCc6ycaDO g==; X-IronPort-AV: E=Sophos;i="6.01,179,1684771200"; d="scan'208";a="353323165" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 17 Aug 2023 15:30:24 +0800 IronPort-SDR: 7ErwrZ3dp+8GW5FJyj7828dmNR/dfSeKFmuL1DdZXG9+x5OYZKOUz4hzYIQ4z68MdyRhc/Wg5U lz1dw3N4OWpmoxPGacs/iWMyAiNhAD/Olr/XVF9CCtaSynC4snE2TjDzu0x6PSupqZbu0F8C/6 V/QL6FpInFjiUkCN7svwbZ3Q07B6XvsX79ZtqHjaAbmYTZsl+L2+gHAKrW0q/C39K7kUONu1/M 3g/jOZYRKG6XVMPs/O8UUPQpk46nheSQEfrgxgl49Si8lHlxkdyvA73Rlh7sS5OwRtjh6n2Wyx v/s= Received: from uls-op-cesaip01.wdc.com ([10.248.3.36]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 16 Aug 2023 23:38:01 -0700 IronPort-SDR: 8LjGFSSJVg+jApn3yw0OMSkd9bVshYGe6Bu5zGrlSFZELSAdq8SilxFMx6AK0BPuVRVcZtnoJb unTVjMuZTsoP9EJlvwi3a5xGvZE7GO3Sf17VnNLzZkOF7h8f0n69pzY1OOuvmkTMzJNaPlVDn4 +vPK5Fa19SC6KSDWrXYPsQwYjH/8Hp4OjnKKf3ub6ZTtByGPOjIxuLh3b6KLHxAkye+BEvRsVu PxPa4ax9BMlX+MMwwEAT4AKLfWKzam4vOB+uUR2M1hBR6C9hYgAzSFWnUe3KScvBgtEpFTQxSv gCo= WDCIronportException: Internal Received: from shindev.dhcp.fujisawa.hgst.com (HELO shindev.fujisawa.hgst.com) ([10.149.53.55]) by uls-op-cesaip01.wdc.com with ESMTP; 17 Aug 2023 00:30:23 -0700 From: Shin'ichiro Kawasaki To: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Yi Zhang , Daniel Wagner , Sagi Grimberg , Bart Van Assche , Shin'ichiro Kawasaki Subject: [PATCH blktests v2 2/2] nvme: remove "udevadm settle" after _nvme_connect_subsys Date: Thu, 17 Aug 2023 16:30:21 +0900 Message-Id: <20230817073021.3674602-2-shinichiro.kawasaki@wdc.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230817073021.3674602-1-shinichiro.kawasaki@wdc.com> References: <20230817073021.3674602-1-shinichiro.kawasaki@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The previous commit introduced "udevadm settle" command at the end of _nvme_connect_subsys. Then the command is no longer required after calling _nvme_connect_subsys in test cases. Signed-off-by: Shin'ichiro Kawasaki Reviewed-by: Daniel Wagner --- tests/nvme/005 | 2 -- tests/nvme/008 | 2 -- tests/nvme/009 | 2 -- tests/nvme/040 | 1 - tests/nvme/041 | 2 -- tests/nvme/042 | 3 --- tests/nvme/043 | 4 ---- tests/nvme/044 | 8 -------- tests/nvme/045 | 2 -- 9 files changed, 26 deletions(-) diff --git a/tests/nvme/005 b/tests/nvme/005 index 6646b82..4ca87ff 100755 --- a/tests/nvme/005 +++ b/tests/nvme/005 @@ -40,8 +40,6 @@ test() { nvmedev=$(_find_nvme_dev "${def_subsysnqn}") - udevadm settle - echo 1 > "/sys/class/nvme/${nvmedev}/reset_controller" _nvme_disconnect_ctrl "${nvmedev}" diff --git a/tests/nvme/008 b/tests/nvme/008 index 3921fc6..bd5e10f 100755 --- a/tests/nvme/008 +++ b/tests/nvme/008 @@ -40,8 +40,6 @@ test() { cat "/sys/block/${nvmedev}n1/uuid" cat "/sys/block/${nvmedev}n1/wwid" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" _remove_nvmet_subsystem_from_port "${port}" "${def_subsysnqn}" diff --git a/tests/nvme/009 b/tests/nvme/009 index aac3c1e..c9a4b57 100755 --- a/tests/nvme/009 +++ b/tests/nvme/009 @@ -36,8 +36,6 @@ test() { cat "/sys/block/${nvmedev}n1/uuid" cat "/sys/block/${nvmedev}n1/wwid" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" _remove_nvmet_subsystem_from_port "${port}" "${def_subsysnqn}" diff --git a/tests/nvme/040 b/tests/nvme/040 index 1a9be5c..ed6df3b 100755 --- a/tests/nvme/040 +++ b/tests/nvme/040 @@ -35,7 +35,6 @@ test() { _create_nvmet_host "${def_subsysnqn}" "${def_hostnqn}" _nvme_connect_subsys "${nvme_trtype}" "${def_subsysnqn}" - udevadm settle nvmedev=$(_find_nvme_dev "${def_subsysnqn}") # start fio job diff --git a/tests/nvme/041 b/tests/nvme/041 index cb27666..bc84412 100755 --- a/tests/nvme/041 +++ b/tests/nvme/041 @@ -57,8 +57,6 @@ test() { --hostid "${def_hostid}" \ --dhchap-secret "${hostkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" _remove_nvmet_subsystem_from_port "${port}" "${def_subsysnqn}" diff --git a/tests/nvme/042 b/tests/nvme/042 index 9180fce..47e1b95 100755 --- a/tests/nvme/042 +++ b/tests/nvme/042 @@ -50,7 +50,6 @@ test() { --hostnqn "${def_hostnqn}" \ --hostid "${def_hostid}" \ --dhchap-secret "${hostkey}" - udevadm settle _nvme_disconnect_subsys "${def_subsysnqn}" done @@ -69,8 +68,6 @@ test() { --hostid "${def_hostid}" \ --dhchap-secret "${hostkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" done diff --git a/tests/nvme/043 b/tests/nvme/043 index f084229..15676f8 100755 --- a/tests/nvme/043 +++ b/tests/nvme/043 @@ -55,8 +55,6 @@ test() { --hostid "${def_hostid}" \ --dhchap-secret "${hostkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" done @@ -71,8 +69,6 @@ test() { --hostid "${def_hostid}" \ --dhchap-secret "${hostkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" done diff --git a/tests/nvme/044 b/tests/nvme/044 index 5eb163d..9407ac6 100755 --- a/tests/nvme/044 +++ b/tests/nvme/044 @@ -59,8 +59,6 @@ test() { --hostid "${def_hostid}" \ --dhchap-secret "${hostkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" # Step 2: Connect with host authentication @@ -72,8 +70,6 @@ test() { --dhchap-secret "${hostkey}" \ --dhchap-ctrl-secret "${hostkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" # Step 3: Connect with host authentication @@ -85,8 +81,6 @@ test() { --dhchap-secret "${hostkey}" \ --dhchap-ctrl-secret "${ctrlkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" # Step 4: Connect with host authentication @@ -99,8 +93,6 @@ test() { --dhchap-secret "${hostkey}" \ --dhchap-ctrl-secret "${invkey}" - udevadm settle - _nvme_disconnect_subsys "${def_subsysnqn}" _remove_nvmet_subsystem_from_port "${port}" "${def_subsysnqn}" diff --git a/tests/nvme/045 b/tests/nvme/045 index 8364d5e..396bcde 100755 --- a/tests/nvme/045 +++ b/tests/nvme/045 @@ -61,8 +61,6 @@ test() { --dhchap-secret "${hostkey}" \ --dhchap-ctrl-secret "${ctrlkey}" - udevadm settle - echo "Re-authenticate with original host key" ctrldev=$(_find_nvme_dev "${def_subsysnqn}")