From patchwork Tue Oct 15 03:28:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835699 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 028481714B2 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; cv=none; b=qILj/DB4c/p/Red+UY5LAq1j7VKMrqVV/YAhMY+wcbjPP4tbcuuCSCbdfv9jL2k0zvIS+qxlL811izqTa/Yyyk0axEyg8iLpHGBfzars6Q6cyIV62j/pUmKDqY+02/eqVhDml22YudBcpQ8rCJc2vq0BSfyNxbfr+HwaFHjCess= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; c=relaxed/simple; bh=K1uD84RpdQMm62QBvQQrkNvq4iYFIL7AhA9ICg0G2PE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mmYKGZ4eGgxKPglDMMNgR8Ssm5T83rNlx+prr6Foj9TeqCxGtQFAaMDb8fHNn8vAYcM/0FjwFgaHmRLdr3XXaBvbPKcfxRb+8qcqN5XYExCpogujK/KGQimxpUkBeKl4IwRtf0opf4O0nsmQuI64ywyOfy1SvsQeSs+cXAllgUY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=crrW40ih; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="crrW40ih" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fDbPEXd1L46Bz924xjYAv09ESeJm5szPAAZ96dLVYtA=; b=crrW40ihEL6n/X5CvFmkePFiC+IMisxupvKAGBEU/EVSruk0Ax4tR7Sr9IxIOTAcN6GTso WfnaZUF9779XsrjWNDOBiIOoMgUkhUhn+tIi/rUCUdGxUSEnHT+5IXQF8MozYj5jk3a46s hUfoUR1lmRHjVLAiak0iD0SPNdYmEnQ= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-389-ZrSkgpFLM3-dkeiK0_hkiQ-1; Mon, 14 Oct 2024 23:28:38 -0400 X-MC-Unique: ZrSkgpFLM3-dkeiK0_hkiQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7B94119560A5; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 225A919560AE; Tue, 15 Oct 2024 03:28:36 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SZjT2693283 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:35 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SZE22693282; Mon, 14 Oct 2024 23:28:35 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 01/23] libmultipath: store checker_check() result in checker struct Date: Mon, 14 Oct 2024 23:28:13 -0400 Message-ID: <20241015032835.2693247-2-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com checker_check() is now a void function that stores the path state in the checker struct. It can be retrieved later using checker_get_state(). Right now, this is called immediately after checker_check(), but in the future, it can be deferred to another time. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/checkers.c | 26 +++++++++++++------------- libmultipath/checkers.h | 4 +++- libmultipath/discovery.c | 3 ++- libmultipath/propsel.c | 1 + 4 files changed, 19 insertions(+), 15 deletions(-) diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c index fdb91e17..c4918d28 100644 --- a/libmultipath/checkers.c +++ b/libmultipath/checkers.c @@ -299,28 +299,28 @@ void checker_put (struct checker * dst) free_checker_class(src); } -int checker_check (struct checker * c, int path_state) +int checker_get_state(struct checker *c) { - int r; + return c ? c->path_state : PATH_UNCHECKED; +} +void checker_check (struct checker * c, int path_state) +{ if (!c) - return PATH_WILD; + return; c->msgid = CHECKER_MSGID_NONE; if (c->disable) { c->msgid = CHECKER_MSGID_DISABLED; - return PATH_UNCHECKED; - } - if (!strncmp(c->cls->name, NONE, 4)) - return path_state; - - if (c->fd < 0) { + c->path_state = PATH_UNCHECKED; + } else if (!strncmp(c->cls->name, NONE, 4)) { + c->path_state = path_state; + } else if (c->fd < 0) { c->msgid = CHECKER_MSGID_NO_FD; - return PATH_WILD; + c->path_state = PATH_WILD; + } else { + c->path_state = c->cls->check(c); } - r = c->cls->check(c); - - return r; } const char *checker_name(const struct checker *c) diff --git a/libmultipath/checkers.h b/libmultipath/checkers.h index 102351f6..6e54d8f0 100644 --- a/libmultipath/checkers.h +++ b/libmultipath/checkers.h @@ -123,6 +123,7 @@ struct checker { int fd; unsigned int timeout; int disable; + int path_state; short msgid; /* checker-internal extra status */ void * context; /* store for persistent data */ void ** mpcontext; /* store for persistent data shared @@ -169,7 +170,8 @@ struct checker_context { }; int start_checker_thread (pthread_t *thread, const pthread_attr_t *attr, struct checker_context *ctx); -int checker_check (struct checker *, int); +int checker_get_state(struct checker *c); +void checker_check (struct checker *, int); int checker_is_sync(const struct checker *); const char *checker_name (const struct checker *); void reset_checker_classes(void); diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index e94705bf..5648be60 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -2007,7 +2007,8 @@ get_state (struct path * pp, struct config *conf, int daemon, int oldstate) checker_set_async(c); else checker_set_sync(c); - state = checker_check(c, oldstate); + checker_check(c, oldstate); + state = checker_get_state(c); condlog(3, "%s: %s state = %s", pp->dev, checker_name(c), checker_state_name(state)); if (state != PATH_UP && state != PATH_GHOST && diff --git a/libmultipath/propsel.c b/libmultipath/propsel.c index a3fce203..ad771d35 100644 --- a/libmultipath/propsel.c +++ b/libmultipath/propsel.c @@ -700,6 +700,7 @@ out: condlog(3, "%s: path_checker = %s %s", pp->dev, checker_name(c), origin); c->timeout = pp->checker_timeout; + c->path_state = PATH_UNCHECKED; return 0; } From patchwork Tue Oct 15 03:28:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835700 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 010A71714B0 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; cv=none; b=ZKVK8r6E/aaXbkvESazaR6M69qlXHC5RoAXR5OlN1U+7hweodAtZ8eir0V7ymDiFY5G04MpxhmFUAQ+sSiqNRfjFaU/yLJKMUc4n5YwP4c6+/NUJqcTIp/hgW/E4tO4x2/Ox68PK2UZ4SnLNt2dI86EMPMkPklpzHq6Yl/OVJVE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; c=relaxed/simple; bh=/gwPJEkUJRxZWnG/yNxxk0/KQxOhLA+xhqgoyL3Bg6c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KrDo2MEDqSlJB8kt3zG6fI+Ov4kcg+ilhx+gCaFCWLiIRM0OIvJz0w9ePitblzIh6+k0qkjgb/kKQ9bAljCo6HqJb5cBW6EVzRJcIvYhLemWyCrESypWvqutGRJcfh5wb6BUL7XI9d3Pkpe9G+M5LZG/sCwAqBvP8ejYQ4OLRoI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VKUpvLI5; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VKUpvLI5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SG/QEWVYn2IyaP1Pf7//bD8wY3Ah2v2mDwF8hGeVz+E=; b=VKUpvLI55vMGhBeDvT1YWhhu9p2f6S63ePhGm7/p2WupuT47KakK2KQIcnRzgOZw6csqwt y5za1jK0WBaMVpKWw2iwN89IGWateJWJFroq5VJMwh4x38jT6MTkqO1GD2mGdkB/e4KggQ AqRjvHPe4xQ9ScA4JMD7eOXkb7nobwg= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-389-kCxG-Bm-OD2EIJ-MRrbysw-1; Mon, 14 Oct 2024 23:28:38 -0400 X-MC-Unique: kCxG-Bm-OD2EIJ-MRrbysw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8450E19560B4; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 271741956054; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SZsY2693287 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SZm82693286; Mon, 14 Oct 2024 23:28:35 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 02/23] libmultipath: add missing checker function prototypes Date: Mon, 14 Oct 2024 23:28:14 -0400 Message-ID: <20241015032835.2693247-3-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/checkers.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libmultipath/checkers.h b/libmultipath/checkers.h index 6e54d8f0..fb1160af 100644 --- a/libmultipath/checkers.h +++ b/libmultipath/checkers.h @@ -188,6 +188,9 @@ int libcheck_check(struct checker *); int libcheck_init(struct checker *); void libcheck_free(struct checker *); void *libcheck_thread(struct checker_context *ctx); +void libcheck_reset(void); +int libcheck_mp_init(struct checker *); + /* * msgid => message map. From patchwork Tue Oct 15 03:28:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835711 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23E36172BCE for ; Tue, 15 Oct 2024 03:28:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; cv=none; b=o/+bMArQfM/oyGfRYYhhWy1SeP+N5qzNackC2A8CmksSvBDbjJzJeOjMBdTNpBV4b8E3eE5NcrSbFKxC1tcuxGmm6+M8HxN3QJG2LaWj/fsZM2Tl4n5vIC36CdcA84TRyI8J3iWblyI5lXXCJecU7Rl3w0wkblc0T1Gtyl7QVyk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; c=relaxed/simple; bh=XYrA09mkFsb2Fo7BUR6/8WmKh2VrTUllBGTwXh6awXk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qcIfxQG7ZD6TWxKkdDpHCYbYfMmALH2Dn14B8IYaO/Rt7Vox3Kqq5KbxK2Mgjx2oDsOM1iebSsApDpuks4caRBJjn2/qW6Ccq04xmpWbT51V2NMe+Jr9iqtlpxSDfn/iIyikp9EESHtbMR8EHkiAGGiE6szsSruNp8RoOcmfOsE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NjxV9MhU; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NjxV9MhU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7HWCQyiqKQb3VwL7z1bSCn9lfcUcYkeaxu5HGByVueA=; b=NjxV9MhUmzBb1hNIyYHwJws4FAvqnnASamFCb0wpXwoqVZIiAZysAVHMFrzoGEV2BJwcJv JyNas9EmvSLYRShRlKXRRihq9klMlV8D0Sl0JrjUgfeBedl0aeWvThZl1muBE8ZuVz1+rr 8c9lFqu/DTLpOha6c6yYtxIUduaTaTg= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-433-tfes7t1tNpagolYCGttZeQ-1; Mon, 14 Oct 2024 23:28:38 -0400 X-MC-Unique: tfes7t1tNpagolYCGttZeQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9CB72195608C; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 66B5E1955F43; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sa232693291 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SaxM2693290; Mon, 14 Oct 2024 23:28:36 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 03/23] libmultipath: split out the code to wait for pending checkers Date: Mon, 14 Oct 2024 23:28:15 -0400 Message-ID: <20241015032835.2693247-4-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com This patch adds a new optional symbol for the dynamic path checker libraries, libcheck_pending. This is currently unused, but can be called on pending checkers to check if they've completed and return their value. The "tur" and "directio" checkers are the only ones which can return PATH_PENDING. They now implement libcheck_pending() as a wrapper around the code that libcheck_check uses to wait for pending paths, which has been broken out into its own function. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/checkers.c | 4 +- libmultipath/checkers.h | 1 + libmultipath/checkers/directio.c | 86 ++++++++++++++++++++++---------- libmultipath/checkers/tur.c | 65 ++++++++++++++++-------- 4 files changed, 109 insertions(+), 47 deletions(-) diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c index c4918d28..298aec78 100644 --- a/libmultipath/checkers.c +++ b/libmultipath/checkers.c @@ -26,6 +26,7 @@ struct checker_class { void (*free)(struct checker *); /* to free the context */ void (*reset)(void); /* to reset the global variables */ void *(*thread)(void *); /* async thread entry point */ + int (*pending)(struct checker *); /* to recheck pending paths */ const char **msgtable; short msgtable_size; }; @@ -180,7 +181,8 @@ static struct checker_class *add_checker_class(const char *name) c->mp_init = (int (*)(struct checker *)) dlsym(c->handle, "libcheck_mp_init"); c->reset = (void (*)(void)) dlsym(c->handle, "libcheck_reset"); c->thread = (void *(*)(void*)) dlsym(c->handle, "libcheck_thread"); - /* These 3 functions can be NULL. call dlerror() to clear out any + c->pending = (int (*)(struct checker *)) dlsym(c->handle, "libcheck_pending"); + /* These 4 functions can be NULL. call dlerror() to clear out any * error string */ dlerror(); diff --git a/libmultipath/checkers.h b/libmultipath/checkers.h index fb1160af..b2342a1b 100644 --- a/libmultipath/checkers.h +++ b/libmultipath/checkers.h @@ -190,6 +190,7 @@ void libcheck_free(struct checker *); void *libcheck_thread(struct checker_context *ctx); void libcheck_reset(void); int libcheck_mp_init(struct checker *); +int libcheck_pending(struct checker *c); /* diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index 8e87878b..3f88b40d 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -288,14 +288,36 @@ get_events(struct aio_group *aio_grp, struct timespec *timeout) return got_events; } +static void +check_pending(struct directio_context *ct, struct timespec endtime) +{ + int r; + struct timespec currtime, timeout; + + while(1) { + get_monotonic_time(&currtime); + timespecsub(&endtime, &currtime, &timeout); + if (timeout.tv_sec < 0) + timeout.tv_sec = timeout.tv_nsec = 0; + + r = get_events(ct->aio_grp, &timeout); + + if (ct->req->state != PATH_PENDING) { + ct->running = 0; + return; + } else if (r == 0 || + (timeout.tv_sec == 0 && timeout.tv_nsec == 0)) + return; + } +} + static int check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) { struct timespec timeout = { .tv_nsec = 1000 }; struct stat sb; int rc; - long r; - struct timespec currtime, endtime; + struct timespec endtime; if (fstat(fd, &sb) == 0) { LOG(4, "called for %x", (unsigned) sb.st_rdev); @@ -330,21 +352,11 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) endtime.tv_sec += timeout.tv_sec; endtime.tv_nsec += timeout.tv_nsec; normalize_timespec(&endtime); - while(1) { - r = get_events(ct->aio_grp, &timeout); - if (ct->req->state != PATH_PENDING) { - ct->running = 0; - return ct->req->state; - } else if (r == 0 || - (timeout.tv_sec == 0 && timeout.tv_nsec == 0)) - break; + check_pending(ct, endtime); + if (ct->req->state != PATH_PENDING) + return ct->req->state; - get_monotonic_time(&currtime); - timespecsub(&endtime, &currtime, &timeout); - if (timeout.tv_sec < 0) - timeout.tv_sec = timeout.tv_nsec = 0; - } if (ct->running > timeout_secs || sync) { struct io_event event; @@ -360,17 +372,9 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) return rc; } -int libcheck_check (struct checker * c) +static void set_msgid(struct checker *c, int state) { - int ret; - struct directio_context * ct = (struct directio_context *)c->context; - - if (!ct) - return PATH_UNCHECKED; - - ret = check_state(c->fd, ct, checker_is_sync(c), c->timeout); - - switch (ret) + switch (state) { case PATH_UNCHECKED: c->msgid = MSG_DIRECTIO_UNKNOWN; @@ -387,5 +391,37 @@ int libcheck_check (struct checker * c) default: break; } +} + +int libcheck_pending(struct checker *c) +{ + struct timespec endtime; + struct directio_context *ct = (struct directio_context *)c->context; + + /* The if path checker isn't running, just return the exiting value. */ + if (!ct || !ct->running) + return c->path_state; + + if (ct->req->state == PATH_PENDING) { + get_monotonic_time(&endtime); + check_pending(ct, endtime); + } else + ct->running = 0; + set_msgid(c, ct->req->state); + + return ct->req->state; +} + +int libcheck_check (struct checker * c) +{ + int ret; + struct directio_context * ct = (struct directio_context *)c->context; + + if (!ct) + return PATH_UNCHECKED; + + ret = check_state(c->fd, ct, checker_is_sync(c), c->timeout); + set_msgid(c, ret); + return ret; } diff --git a/libmultipath/checkers/tur.c b/libmultipath/checkers/tur.c index a2905af5..95af5214 100644 --- a/libmultipath/checkers/tur.c +++ b/libmultipath/checkers/tur.c @@ -323,6 +323,49 @@ static int tur_check_async_timeout(struct checker *c) return (now.tv_sec > ct->time); } +int check_pending(struct checker *c, struct timespec endtime) +{ + struct tur_checker_context *ct = c->context; + int r, tur_status = PATH_PENDING; + + pthread_mutex_lock(&ct->lock); + + for (r = 0; + r == 0 && ct->state == PATH_PENDING && + ct->msgid == MSG_TUR_RUNNING; + r = pthread_cond_timedwait(&ct->active, &ct->lock, &endtime)); + + if (!r) { + tur_status = ct->state; + c->msgid = ct->msgid; + } + pthread_mutex_unlock(&ct->lock); + if (tur_status == PATH_PENDING && c->msgid == MSG_TUR_RUNNING) { + condlog(4, "%d:%d : tur checker still running", + major(ct->devt), minor(ct->devt)); + } else { + int running = uatomic_xchg(&ct->running, 0); + if (running) + pthread_cancel(ct->thread); + ct->thread = 0; + } + + return tur_status; +} + +int libcheck_pending(struct checker *c) +{ + struct timespec endtime; + struct tur_checker_context *ct = c->context; + + /* The if path checker isn't running, just return the exiting value. */ + if (!ct || !ct->thread) + return c->path_state; + + get_monotonic_time(&endtime); + return check_pending(c, endtime); +} + int libcheck_check(struct checker * c) { struct tur_checker_context *ct = c->context; @@ -437,27 +480,7 @@ int libcheck_check(struct checker * c) return tur_check(c->fd, c->timeout, &c->msgid); } tur_timeout(&tsp); - pthread_mutex_lock(&ct->lock); - - for (r = 0; - r == 0 && ct->state == PATH_PENDING && - ct->msgid == MSG_TUR_RUNNING; - r = pthread_cond_timedwait(&ct->active, &ct->lock, &tsp)); - - if (!r) { - tur_status = ct->state; - c->msgid = ct->msgid; - } - pthread_mutex_unlock(&ct->lock); - if (tur_status == PATH_PENDING && c->msgid == MSG_TUR_RUNNING) { - condlog(4, "%d:%d : tur checker still running", - major(ct->devt), minor(ct->devt)); - } else { - int running = uatomic_xchg(&ct->running, 0); - if (running) - pthread_cancel(ct->thread); - ct->thread = 0; - } + tur_status = check_pending(c, tsp); } return tur_status; From patchwork Tue Oct 15 03:28:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835698 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A13E0170A3A for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; cv=none; b=YEcKkSqt+1+iEUBxNh7W9mbDTCWHU2QNK7APVElXWK1kgwP2mfVuq+LNi2vi26t2kcq2pSvSkwngzTGgSzkbJnmePRDENt0dioaAWAo5VsBrAbzx8zqI42auiFTcocpRfSJs+Ll9EinGcbGE95pwgvJaxyx3K+9dQtNeyU3t4NQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; c=relaxed/simple; bh=9gf0xS5uvMboH734GTPPdJpFwAl6aDwk3bX/llWFMFI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rMt5mcok9uIh355STgvH00vWwojs6mNKQgGBVcjqLpaNMnjbCdKX0wJA8yzdjI2NiT5pHDOrjtid04f5UNwn5crTthzEo4LOhPA7CIJkyYZgbihyaIHlGSyW5XRVpQhbEY31j3jNlnznpH7uhDpoppygZN9WoVlcH+UTwTl4HSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=P4dbc3Dl; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="P4dbc3Dl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6pAoNay9S+X0hEUCnIb37okRQ6By4AndDEubmxEliNU=; b=P4dbc3Dlg4wp/C+TBh9phDOCydQ9jmUa+vpM7pTQ930HQyjsa9EbWEpgKnj4rGNyUelykP /JtlQSillRq64QZ4FamRFgmMGF5f3GfpxAfc2xCPSd4WGQZk/G+Fj0+PUYIrEOM1zNUUmR kFhDEdkYEWE1fiExi32roxkuoBrwBkY= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-580-LFGa6-jDOsOisbkZVYvMzg-1; Mon, 14 Oct 2024 23:28:38 -0400 X-MC-Unique: LFGa6-jDOsOisbkZVYvMzg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C22651955F41; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 66B181956089; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sas22693295 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SaTX2693294; Mon, 14 Oct 2024 23:28:36 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 04/23] libmultipath: remove pending wait code from libcheck_check calls Date: Mon, 14 Oct 2024 23:28:16 -0400 Message-ID: <20241015032835.2693247-5-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com When the tur and directio checkers start an asynchronous checker, they now immediately return with the path in PATH_PENDING, instead of waiting in checker_check(). Instead the wait now happens in checker_get_state(). Additionally, the directio checker now waits for 1 ms, like the tur checker does. Also like the tur checker it now only waits once. If it is still pending after the first call to checker_get_state(). It will not wait at all on future calls, and will just process the already completed IOs. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/checkers.c | 7 +++- libmultipath/checkers/directio.c | 57 +++++++++++++++++--------------- libmultipath/checkers/tur.c | 13 +++----- 3 files changed, 41 insertions(+), 36 deletions(-) diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c index 298aec78..ce3e48bd 100644 --- a/libmultipath/checkers.c +++ b/libmultipath/checkers.c @@ -303,7 +303,12 @@ void checker_put (struct checker * dst) int checker_get_state(struct checker *c) { - return c ? c->path_state : PATH_UNCHECKED; + if (!c || !c->cls) + return PATH_UNCHECKED; + if (c->path_state != PATH_PENDING || !c->cls->pending) + return c->path_state; + c->path_state = c->cls->pending(c); + return c->path_state; } void checker_check (struct checker * c, int path_state) diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index 3f88b40d..904e3071 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -60,10 +60,11 @@ const char *libcheck_msgtable[] = { #define LOG(prio, fmt, args...) condlog(prio, "directio: " fmt, ##args) struct directio_context { - int running; + unsigned int running; int reset_flags; struct aio_group *aio_grp; struct async_req *req; + struct timespec endtime; }; static struct aio_group * @@ -314,19 +315,16 @@ check_pending(struct directio_context *ct, struct timespec endtime) static int check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) { - struct timespec timeout = { .tv_nsec = 1000 }; struct stat sb; int rc; + struct io_event event; struct timespec endtime; if (fstat(fd, &sb) == 0) { LOG(4, "called for %x", (unsigned) sb.st_rdev); } - if (sync > 0) { + if (sync > 0) LOG(4, "called in synchronous mode"); - timeout.tv_sec = timeout_secs; - timeout.tv_nsec = 0; - } if (ct->running) { if (ct->req->state != PATH_PENDING) { @@ -345,31 +343,26 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) LOG(3, "io_submit error %i", -rc); return PATH_UNCHECKED; } + get_monotonic_time(&ct->endtime); + ct->endtime.tv_nsec += 1000 * 1000; + normalize_timespec(&ct->endtime); } ct->running++; + if (!sync) + return PATH_PENDING; get_monotonic_time(&endtime); - endtime.tv_sec += timeout.tv_sec; - endtime.tv_nsec += timeout.tv_nsec; + endtime.tv_sec += timeout_secs; normalize_timespec(&endtime); check_pending(ct, endtime); if (ct->req->state != PATH_PENDING) return ct->req->state; - if (ct->running > timeout_secs || sync) { - struct io_event event; - - LOG(3, "abort check on timeout"); - - io_cancel(ct->aio_grp->ioctx, &ct->req->io, &event); - rc = PATH_DOWN; - } else { - LOG(4, "async io pending"); - rc = PATH_PENDING; - } + LOG(3, "abort check on timeout"); - return rc; + io_cancel(ct->aio_grp->ioctx, &ct->req->io, &event); + return PATH_DOWN; } static void set_msgid(struct checker *c, int state) @@ -395,21 +388,31 @@ static void set_msgid(struct checker *c, int state) int libcheck_pending(struct checker *c) { - struct timespec endtime; + int rc; + struct io_event event; struct directio_context *ct = (struct directio_context *)c->context; /* The if path checker isn't running, just return the exiting value. */ if (!ct || !ct->running) return c->path_state; - if (ct->req->state == PATH_PENDING) { - get_monotonic_time(&endtime); - check_pending(ct, endtime); - } else + if (ct->req->state == PATH_PENDING) + check_pending(ct, ct->endtime); + else ct->running = 0; - set_msgid(c, ct->req->state); + rc = ct->req->state; + if (rc == PATH_PENDING) { + if (ct->running > c->timeout) { + LOG(3, "abort check on timeout"); + io_cancel(ct->aio_grp->ioctx, &ct->req->io, &event); + rc = PATH_DOWN; + } + else + LOG(4, "async io pending"); + } + set_msgid(c, rc); - return ct->req->state; + return rc; } int libcheck_check (struct checker * c) diff --git a/libmultipath/checkers/tur.c b/libmultipath/checkers/tur.c index 95af5214..81db565b 100644 --- a/libmultipath/checkers/tur.c +++ b/libmultipath/checkers/tur.c @@ -58,6 +58,7 @@ struct tur_checker_context { int msgid; struct checker_context ctx; unsigned int nr_timeouts; + struct timespec endtime; }; int libcheck_init (struct checker * c) @@ -323,7 +324,7 @@ static int tur_check_async_timeout(struct checker *c) return (now.tv_sec > ct->time); } -int check_pending(struct checker *c, struct timespec endtime) +int check_pending(struct checker *c) { struct tur_checker_context *ct = c->context; int r, tur_status = PATH_PENDING; @@ -333,7 +334,7 @@ int check_pending(struct checker *c, struct timespec endtime) for (r = 0; r == 0 && ct->state == PATH_PENDING && ct->msgid == MSG_TUR_RUNNING; - r = pthread_cond_timedwait(&ct->active, &ct->lock, &endtime)); + r = pthread_cond_timedwait(&ct->active, &ct->lock, &ct->endtime)); if (!r) { tur_status = ct->state; @@ -355,21 +356,18 @@ int check_pending(struct checker *c, struct timespec endtime) int libcheck_pending(struct checker *c) { - struct timespec endtime; struct tur_checker_context *ct = c->context; /* The if path checker isn't running, just return the exiting value. */ if (!ct || !ct->thread) return c->path_state; - get_monotonic_time(&endtime); - return check_pending(c, endtime); + return check_pending(c); } int libcheck_check(struct checker * c) { struct tur_checker_context *ct = c->context; - struct timespec tsp; pthread_attr_t attr; int tur_status, r; @@ -479,8 +477,7 @@ int libcheck_check(struct checker * c) " sync mode", major(ct->devt), minor(ct->devt)); return tur_check(c->fd, c->timeout, &c->msgid); } - tur_timeout(&tsp); - tur_status = check_pending(c, tsp); + tur_timeout(&ct->endtime); } return tur_status; From patchwork Tue Oct 15 03:28:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835716 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28E5C1714B5 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; cv=none; b=eEEXY4STW8w7TpAdNlbCDhLZEisUB5RUkBaaIKLGuK293BSH2R/+Zzn0GY+rwVl0I8BgHd4zH5zZ5ETF7ZT/L7yyDvCjL8AsCS+22erk4gvpMohddEwfrjiDLRwy/XxwKg2yaipBCxBX/C731QKZHlIgcm+6h3NtnQvEmWNyo20= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; c=relaxed/simple; bh=ag4LoPZGTQKSadSxgqREFDqvcz5GzuEwdtaJacB+dKk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sRbp9dsR8xDO0lKjMesJUH3UKIFFK3rs0QsjAWDp8niFB4+fhz2wPmTbElkDoFf3Uuxdl7dA3XGig7ujiPxngDZ/uUTiNSUUNHtvPC7qTDb46R1ogT3cRmquGL10cUm0Igzpyy6aHBBXlYvBmV6j8q7PEbNRvbp7UwyZecmL+BE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Qb/mp8GW; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Qb/mp8GW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b3MeuiXHpoDkGrIha11WWIwSL5siHJTfu9lyMXGZ7iA=; b=Qb/mp8GW3TFHw7LUTfv8VqIfUdnIP22+nMu6YtNRrSUhx0euJpelalULOaWvsbplzexZR8 Ktto7eJEMrgBkuktILqFBoG/TXdAP5IVuKDzgTNfmsdBjxt+pB5thvsVK9GF/PEAIKflRL eLAXnrT/j/iASHcXoDLKL4EcH38Uc4s= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-441--bIZUFyTMtm9175i4ZpSrw-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: -bIZUFyTMtm9175i4ZpSrw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EFBC21956089; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A27E919560AE; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SaDU2693299 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SaoL2693298; Mon, 14 Oct 2024 23:28:36 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 05/23] multipath-tools tests: fix up directio tests Date: Mon, 14 Oct 2024 23:28:17 -0400 Message-ID: <20241015032835.2693247-6-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Make the directio tests work with libcheck_pending() being separate from libcheck_check Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- tests/directio.c | 133 +++++++++++++++++++++++++++++------------------ 1 file changed, 82 insertions(+), 51 deletions(-) diff --git a/tests/directio.c b/tests/directio.c index 763929e5..2e22f831 100644 --- a/tests/directio.c +++ b/tests/directio.c @@ -219,13 +219,22 @@ static void return_io_getevents_nr(struct timespec *ts, int nr, ev_off += i; } -void do_check_state(struct checker *c, int sync, int timeout, int chk_state) +void do_check_state(struct checker *c, int sync, int chk_state) { struct directio_context * ct = (struct directio_context *)c->context; if (!ct->running) will_return(__wrap_io_submit, 1); - assert_int_equal(check_state(test_fd, ct, sync, timeout), chk_state); + assert_int_equal(check_state(test_fd, ct, sync, c->timeout), chk_state); + if (sync) { + assert_int_equal(ev_off, 0); + memset(mock_events, 0, sizeof(mock_events)); + } +} + +void do_libcheck_pending(struct checker *c, int chk_state) +{ + assert_int_equal(libcheck_pending(c), chk_state); assert_int_equal(ev_off, 0); memset(mock_events, 0, sizeof(mock_events)); } @@ -245,12 +254,13 @@ void do_libcheck_reset(int nr_aio_grps) assert_int_equal(ioctx_count, 0); } -static void do_libcheck_init(struct checker *c, int blocksize, +static void do_libcheck_init(struct checker *c, int blocksize, int timeout, struct async_req **req) { struct directio_context * ct; c->fd = test_fd; + c->timeout = timeout; wrap_will_return(WRAP_IOCTL, blocksize); assert_int_equal(libcheck_init(c), 0); ct = (struct directio_context *)c->context; @@ -305,7 +315,7 @@ static void test_init_reset_init(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c, 4096, NULL); + do_libcheck_init(&c, 4096, 0, NULL); aio_grp = get_aio_grp(&c); check_aio_grp(aio_grp, 1, 0); list_for_each_entry(tmp_grp, &aio_grp_list, node) @@ -314,7 +324,7 @@ static void test_init_reset_init(void **state) check_aio_grp(aio_grp, 0, 0); do_libcheck_reset(1); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c, 4096, NULL); + do_libcheck_init(&c, 4096, 0, NULL); aio_grp = get_aio_grp(&c); check_aio_grp(aio_grp, 1, 0); list_for_each_entry(tmp_grp, &aio_grp_list, node) @@ -340,11 +350,11 @@ static void test_init_free(void **state) struct directio_context * ct; if (i % 3 == 0) - do_libcheck_init(&c[i], 512, NULL); + do_libcheck_init(&c[i], 512, 0, NULL); else if (i % 3 == 1) - do_libcheck_init(&c[i], 1024, NULL); + do_libcheck_init(&c[i], 1024, 0, NULL); else - do_libcheck_init(&c[i], 4096, NULL); + do_libcheck_init(&c[i], 4096, 0, NULL); ct = (struct directio_context *)c[i].context; assert_non_null(ct->aio_grp); if ((i & 1023) == 0) @@ -385,7 +395,7 @@ static void test_multi_init_free(void **state) for (count = 0, i = 0; i < 4096; count++) { /* usually init, but occasionally free checkers */ if (count == 0 || (count % 5 != 0 && count % 7 != 0)) { - do_libcheck_init(&c[i], 4096, NULL); + do_libcheck_init(&c[i], 4096, 0, NULL); i++; } else { i--; @@ -404,7 +414,7 @@ static void test_multi_init_free(void **state) i--; libcheck_free(&c[i]); } else { - do_libcheck_init(&c[i], 4096, NULL); + do_libcheck_init(&c[i], 4096, 0, NULL); i++; } } @@ -420,9 +430,9 @@ static void test_check_state_simple(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c, 4096, &req); + do_libcheck_init(&c, 4096, 30, &req); return_io_getevents_nr(NULL, 1, &req, &res); - do_check_state(&c, 1, 30, PATH_UP); + do_check_state(&c, 1, PATH_UP); libcheck_free(&c); do_libcheck_reset(1); } @@ -435,10 +445,10 @@ static void test_check_state_timeout(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c, 4096, NULL); + do_libcheck_init(&c, 4096, 30, NULL); aio_grp = get_aio_grp(&c); return_io_getevents_none(); - do_check_state(&c, 1, 30, PATH_DOWN); + do_check_state(&c, 1, PATH_DOWN); check_aio_grp(aio_grp, 1, 0); libcheck_free(&c); do_libcheck_reset(1); @@ -452,16 +462,20 @@ static void test_check_state_async_timeout(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c, 4096, NULL); + do_libcheck_init(&c, 4096, 3, NULL); aio_grp = get_aio_grp(&c); + do_check_state(&c, 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c, 0, 3, PATH_PENDING); + do_libcheck_pending(&c, PATH_PENDING); + do_check_state(&c, 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c, 0, 3, PATH_PENDING); + do_libcheck_pending(&c, PATH_PENDING); + do_check_state(&c, 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c, 0, 3, PATH_PENDING); + do_libcheck_pending(&c, PATH_PENDING); + do_check_state(&c, 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c, 0, 3, PATH_DOWN); + do_libcheck_pending(&c, PATH_DOWN); check_aio_grp(aio_grp, 1, 0); libcheck_free(&c); do_libcheck_reset(1); @@ -477,14 +491,16 @@ static void test_free_with_pending(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c[0], 4096, &req); - do_libcheck_init(&c[1], 4096, NULL); + do_libcheck_init(&c[0], 4096, 30, &req); + do_libcheck_init(&c[1], 4096, 30, NULL); aio_grp = get_aio_grp(c); + do_check_state(&c[0], 0, PATH_PENDING); + do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[0], 0, 30, PATH_PENDING); + do_libcheck_pending(&c[0], PATH_PENDING); return_io_getevents_nr(NULL, 1, &req, &res); return_io_getevents_none(); - do_check_state(&c[1], 0, 30, PATH_PENDING); + do_libcheck_pending(&c[1], PATH_PENDING); assert_true(is_checker_running(&c[0])); assert_true(is_checker_running(&c[1])); check_aio_grp(aio_grp, 2, 0); @@ -505,9 +521,10 @@ static void test_orphaned_aio_group(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); for (i = 0; i < AIO_GROUP_SIZE; i++) { - do_libcheck_init(&c[i], 4096, NULL); + do_libcheck_init(&c[i], 4096, 30, NULL); + do_check_state(&c[i], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[i], 0, 30, PATH_PENDING); + do_libcheck_pending(&c[i], PATH_PENDING); } aio_grp = get_aio_grp(c); check_aio_grp(aio_grp, AIO_GROUP_SIZE, 0); @@ -539,19 +556,19 @@ static void test_timeout_cancel_failed(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); for (i = 0; i < 2; i++) - do_libcheck_init(&c[i], 4096, &reqs[i]); + do_libcheck_init(&c[i], 4096, 30, &reqs[i]); aio_grp = get_aio_grp(c); return_io_getevents_none(); - do_check_state(&c[0], 1, 30, PATH_DOWN); + do_check_state(&c[0], 1, PATH_DOWN); assert_true(is_checker_running(&c[0])); check_aio_grp(aio_grp, 2, 0); return_io_getevents_none(); - do_check_state(&c[0], 1, 30, PATH_DOWN); + do_check_state(&c[0], 1, PATH_DOWN); assert_true(is_checker_running(&c[0])); return_io_getevents_nr(NULL, 1, &reqs[0], &res[0]); return_io_getevents_nr(NULL, 1, &reqs[1], &res[1]); - do_check_state(&c[1], 1, 30, PATH_UP); - do_check_state(&c[0], 1, 30, PATH_UP); + do_check_state(&c[1], 1, PATH_UP); + do_check_state(&c[0], 1, PATH_UP); for (i = 0; i < 2; i++) { assert_false(is_checker_running(&c[i])); libcheck_free(&c[i]); @@ -571,28 +588,37 @@ static void test_async_timeout_cancel_failed(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); for (i = 0; i < 2; i++) - do_libcheck_init(&c[i], 4096, &reqs[i]); + do_libcheck_init(&c[i], 4096, 2, &reqs[i]); + do_check_state(&c[0], 0, PATH_PENDING); + do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[0], 0, 2, PATH_PENDING); + do_libcheck_pending(&c[0], PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[1], 0, 2, PATH_PENDING); + do_libcheck_pending(&c[1], PATH_PENDING); + do_check_state(&c[0], 0, PATH_PENDING); + do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[0], 0, 2, PATH_PENDING); + do_libcheck_pending(&c[0], PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[1], 0, 2, PATH_PENDING); + do_libcheck_pending(&c[1], PATH_PENDING); + do_check_state(&c[0], 0, PATH_PENDING); + do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[0], 0, 2, PATH_DOWN); + do_libcheck_pending(&c[0], PATH_DOWN); if (!test_dev) { /* can't pick which even gets returned on real devices */ return_io_getevents_nr(NULL, 1, &reqs[1], &res[1]); - do_check_state(&c[1], 0, 2, PATH_UP); + do_libcheck_pending(&c[1], PATH_UP); } + do_check_state(&c[0], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[0], 0, 2, PATH_DOWN); + do_libcheck_pending(&c[0], PATH_DOWN); assert_true(is_checker_running(&c[0])); + do_check_state(&c[1], 0, PATH_PENDING); + do_check_state(&c[0], 0, PATH_PENDING); return_io_getevents_nr(NULL, 2, reqs, res); - do_check_state(&c[1], 0, 2, PATH_UP); - do_check_state(&c[0], 0, 2, PATH_UP); + do_libcheck_pending(&c[1], PATH_UP); + do_libcheck_pending(&c[0], PATH_UP); for (i = 0; i < 2; i++) { assert_false(is_checker_running(&c[i])); libcheck_free(&c[i]); @@ -612,15 +638,17 @@ static void test_orphan_checker_cleanup(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); for (i = 0; i < 2; i++) - do_libcheck_init(&c[i], 4096, &reqs[i]); + do_libcheck_init(&c[i], 4096, 30, &reqs[i]); aio_grp = get_aio_grp(c); + do_check_state(&c[0], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[0], 0, 30, PATH_PENDING); + do_libcheck_pending(&c[0], PATH_PENDING); check_aio_grp(aio_grp, 2, 0); libcheck_free(&c[0]); check_aio_grp(aio_grp, 2, 1); + do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_nr(NULL, 2, reqs, res); - do_check_state(&c[1], 0, 2, PATH_UP); + do_libcheck_pending(&c[1], PATH_UP); check_aio_grp(aio_grp, 1, 0); libcheck_free(&c[1]); check_aio_grp(aio_grp, 0, 0); @@ -636,10 +664,11 @@ static void test_orphan_reset_cleanup(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); - do_libcheck_init(&c, 4096, NULL); + do_libcheck_init(&c, 4096, 30, NULL); orphan_aio_grp = get_aio_grp(&c); + do_check_state(&c, 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c, 0, 30, PATH_PENDING); + do_libcheck_pending(&c, PATH_PENDING); check_aio_grp(orphan_aio_grp, 1, 0); libcheck_free(&c); check_aio_grp(orphan_aio_grp, 1, 1); @@ -671,10 +700,10 @@ static void test_check_state_blksize(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); for (i = 0; i < 3; i++) - do_libcheck_init(&c[i], blksize[i], &reqs[i]); + do_libcheck_init(&c[i], blksize[i], 30, &reqs[i]); for (i = 0; i < 3; i++) { return_io_getevents_nr(NULL, 1, &reqs[i], &res[i]); - do_check_state(&c[i], 1, 30, chk_state[i]); + do_check_state(&c[i], 1, chk_state[i]); } for (i = 0; i < 3; i++) { assert_false(is_checker_running(&c[i])); @@ -695,19 +724,21 @@ static void test_check_state_async(void **state) assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); for (i = 0; i < 257; i++) - do_libcheck_init(&c[i], 4096, &reqs[i]); + do_libcheck_init(&c[i], 4096, 30, &reqs[i]); for (i = 0; i < 256; i++) { + do_check_state(&c[i], 0, PATH_PENDING); return_io_getevents_none(); - do_check_state(&c[i], 0, 30, PATH_PENDING); + do_libcheck_pending(&c[i], PATH_PENDING); assert_true(is_checker_running(&c[i])); } + do_check_state(&c[256], 0, PATH_PENDING); return_io_getevents_nr(&full_timeout, 256, reqs, res); return_io_getevents_nr(NULL, 1, &reqs[256], &res[256]); - do_check_state(&c[256], 0, 30, PATH_UP); + do_libcheck_pending(&c[256], PATH_UP); assert_false(is_checker_running(&c[256])); libcheck_free(&c[256]); for (i = 0; i < 256; i++) { - do_check_state(&c[i], 0, 30, PATH_UP); + do_check_state(&c[i], 0, PATH_UP); assert_false(is_checker_running(&c[i])); libcheck_free(&c[i]); } From patchwork Tue Oct 15 03:28:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835710 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6B551714C6 for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; cv=none; b=PGjcLZWUvvM8qzamKFmUbEDOg/MwsMc9ABiWSB5nI+DDSDnqBQvItxrrWvlIMDhbsGymRvDQjO+Iq1TdzMha3Hk/pRLE/tkjhol/olvlF6CXqm6iCaMBg8OmP7uKbKnHUpJlpC2IAzr0tpU5i6cGdnkxrsQgo8MQNEQl1yAF5YE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; c=relaxed/simple; bh=DZTb0BWSmeyBfjiIkjFphYb8nQXGB1t8tf042fhVx2c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ARzSC6lCFFFBHGe4Ga/Tn03zTg3V6WJ4W2h5DKVEpq+O5zVERcKjV85Zwl9oxAPfA/2hvrUh/mn/85+feZ4mfwkCmZEherHhDB6r7SfW37I1Z3AiUhXxRjFm81TskyOM0WpgeBGJXDpMHiUqnHDts43DD5MmwogpnlftJkXkvdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=L1zIlkSg; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="L1zIlkSg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CxVWR9mGuSCvNQeAMZWWeiixsel1OjZZaSBgvQsHKWs=; b=L1zIlkSgsyb3WuiLPVobRvboF3f4dqj/uaQBO5MhZlL2FR5syeXNHlp7t9kwL+N3LHJkIR 0mEobYcxHsl441P0AnIBYHqnOLPbf7ETxJXS9yNlzFe6cllq+Zg3+aQHqhon5EYg21AG8b 2TIGRDBpG42mBAz+7NMbwOq9MgLX8x4= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-590-MQqgUpL2OK2aH5ABbllDng-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: MQqgUpL2OK2aH5ABbllDng-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 030391956080; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C214019560A2; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SatN2693303 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3Sa1S2693302; Mon, 14 Oct 2024 23:28:36 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 06/23] libmultipath: split get_state into two functions Date: Mon, 14 Oct 2024 23:28:18 -0400 Message-ID: <20241015032835.2693247-7-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com get_state() is now split into start_checker(), which runs the checker but doesn't wait for async checkers, and get_state(), which returns the state from the checker, after waiting for async checkers if necessary. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/discovery.c | 22 ++++++++++++++++------ libmultipath/discovery.h | 4 +++- libmultipath/libmultipath.version | 1 + multipathd/main.c | 4 +++- 4 files changed, 23 insertions(+), 8 deletions(-) diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index 5648be60..e0f46ff2 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -1974,30 +1974,29 @@ cciss_ioctl_pathinfo(struct path *pp) } int -get_state (struct path * pp, struct config *conf, int daemon, int oldstate) +start_checker (struct path * pp, struct config *conf, int daemon, int oldstate) { struct checker * c = &pp->checker; - int state; if (!checker_selected(c)) { if (daemon) { if (pathinfo(pp, conf, DI_SYSFS) != PATHINFO_OK) { condlog(3, "%s: couldn't get sysfs pathinfo", pp->dev); - return PATH_UNCHECKED; + return -1; } } select_detect_checker(conf, pp); select_checker(conf, pp); if (!checker_selected(c)) { condlog(3, "%s: No checker selected", pp->dev); - return PATH_UNCHECKED; + return -1; } checker_set_fd(c, pp->fd); if (checker_init(c, pp->mpp?&pp->mpp->mpcontext:NULL)) { checker_clear(c); condlog(3, "%s: checker init failed", pp->dev); - return PATH_UNCHECKED; + return -1; } } if (pp->mpp && !c->mpcontext) @@ -2008,6 +2007,15 @@ get_state (struct path * pp, struct config *conf, int daemon, int oldstate) else checker_set_sync(c); checker_check(c, oldstate); + return 0; +} + +int +get_state (struct path * pp) +{ + struct checker * c = &pp->checker; + int state; + state = checker_get_state(c); condlog(3, "%s: %s state = %s", pp->dev, checker_name(c), checker_state_name(state)); @@ -2455,7 +2463,9 @@ int pathinfo(struct path *pp, struct config *conf, int mask) if (mask & DI_CHECKER) { if (path_state == PATH_UP) { - int newstate = get_state(pp, conf, 0, path_state); + int newstate = PATH_UNCHECKED; + if (start_checker(pp, conf, 0, path_state) == 0) + newstate = get_state(pp); if (newstate != PATH_PENDING || pp->state == PATH_UNCHECKED || pp->state == PATH_WILD) diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h index c93abf1c..f3e0c618 100644 --- a/libmultipath/discovery.h +++ b/libmultipath/discovery.h @@ -34,7 +34,9 @@ int path_discovery (vector pathvec, int flag); int path_get_tpgs(struct path *pp); /* This function never returns TPGS_UNDEF */ int do_tur (char *); int path_offline (struct path *); -int get_state (struct path * pp, struct config * conf, int daemon, int state); +int start_checker(struct path * pp, struct config * conf, int daemon, + int state); +int get_state(struct path * pp); int get_vpd_sgio (int fd, int pg, int vend_id, char * str, int maxlen); int pathinfo (struct path * pp, struct config * conf, int mask); int alloc_path_with_pathinfo (struct config *conf, struct udev_device *udevice, diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index 21d48da6..bb34d9e8 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -160,6 +160,7 @@ global: remove_wwid; replace_wwids; reset_checker_classes; + start_checker; select_all_tg_pt; select_action; select_find_multipaths_timeout; diff --git a/multipathd/main.c b/multipathd/main.c index 1b7fd04f..4f752adb 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2324,7 +2324,9 @@ check_path_state(struct path *pp) if (newstate == PATH_UP) { conf = get_multipath_config(); pthread_cleanup_push(put_multipath_config, conf); - newstate = get_state(pp, conf, 1, newstate); + newstate = PATH_UNCHECKED; + if (start_checker(pp, conf, 1, newstate) == 0) + newstate = get_state(pp); pthread_cleanup_pop(1); } else { checker_clear_message(&pp->checker); From patchwork Tue Oct 15 03:28:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835713 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B133A17554A for ; Tue, 15 Oct 2024 03:28:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; cv=none; b=E17j/UjGUiYzRMo/wAJG5yIQbKbixfgSOqRlMqoKu+XjyiYCBpmxMBAIgDnjuhZvIAkmuETh0+fP5DvqpW3oLYM4Kl/CjE6M/WD0S6ZQONiBQUFVVijkKJCnN5KkLZqVnL6wBUdiCgdxHznZoMhJTw1AIs8ubFIy0wB10DGxY4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; c=relaxed/simple; bh=ev9hbR9J8QxR3vyZrwK3UV9bUOZUj0BJOpliD5gDO5s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nCSKm5FSRHmcmphntRWnkXNLxfEHDqzoOYoqYZVLe/jiQtLX74wxtKOWVINSTBBW7F35Etv1pf34x9e/BRlQSdH+reSnXW1ntDzwXWG59HHGhnjXaLHrlyrb3FnAch2ZW+GPhBpr64jx7SMzhWKIjrtPjqT/krre2pRJByqu+UQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gDMJPyew; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gDMJPyew" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mWIDSC0L+QVlymRHw3Vd+wWnbhNrMXo3bWFIYFiW1CA=; b=gDMJPyewY1L2PeQKYRnLBLNVx+Q7z7/GtyW/uMAntsIWey6C1w05GMOazlvil853zBve8R 5Jg2riE+OUmPQyDbbcJHD02co+SIw7wmz4pSVtp9XbLq5bfLAjVOQeDn5R6wVNCkywkHQ9 7HSzprm+kNmdzpGCSHdVv+4Mf8g4Wy4= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-295-7cZ90DEFMU22njhyfYDrkQ-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: 7cZ90DEFMU22njhyfYDrkQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3E77F19560AA; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 074761956054; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sa652693307 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SaYC2693306; Mon, 14 Oct 2024 23:28:36 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 07/23] libmultipath: change path_offline to path_sysfs_state Date: Mon, 14 Oct 2024 23:28:19 -0400 Message-ID: <20241015032835.2693247-8-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Instead of pp->offline being a binary value, change it to show the actual result of looking at the sysfs state, and rename it to pp->sysfs_state. Also change the function name from path_offline() to path_sysfs_state(). Adapt the tests for pp->offline. This should not change how multipath currently works. pp->sysfs_state will be used in future patches. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/discovery.c | 65 ++++++++++++++++--------------- libmultipath/discovery.h | 2 +- libmultipath/libmultipath.version | 2 +- libmultipath/print.c | 2 +- libmultipath/structs.h | 2 +- multipathd/main.c | 4 +- 6 files changed, 40 insertions(+), 37 deletions(-) diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index e0f46ff2..6ccdfa0b 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -1076,18 +1076,16 @@ detect_alua(struct path * pp) return; } - if (pp->fd == -1 || pp->offline) + if (pp->fd == -1 || pp->sysfs_state == PATH_DOWN) return; ret = get_target_port_group(pp); if (ret < 0 || get_asymmetric_access_state(pp, ret) < 0) { - int state; - if (ret == -RTPG_INQUIRY_FAILED) return; - state = path_offline(pp); - if (state != PATH_UP) + path_sysfs_state(pp); + if (pp->sysfs_state != PATH_UP) return; pp->tpgs = TPGS_NONE; @@ -1800,7 +1798,7 @@ common_sysfs_pathinfo (struct path * pp) } int -path_offline (struct path * pp) +path_sysfs_state(struct path * pp) { struct udev_device * parent; char buff[SCSI_STATE_SIZE]; @@ -1814,7 +1812,8 @@ path_offline (struct path * pp) subsys_type = "nvme"; } else { - return PATH_UP; + pp->sysfs_state = PATH_UP; + goto out; } parent = pp->udev; @@ -1827,16 +1826,18 @@ path_offline (struct path * pp) if (!parent) { condlog(1, "%s: failed to get sysfs information", pp->dev); - return PATH_REMOVED; + pp->sysfs_state = PATH_REMOVED; + goto out; } memset(buff, 0x0, SCSI_STATE_SIZE); err = sysfs_attr_get_value(parent, "state", buff, sizeof(buff)); if (!sysfs_attr_value_ok(err, sizeof(buff))) { if (err == -ENXIO) - return PATH_REMOVED; + pp->sysfs_state = PATH_REMOVED; else - return PATH_DOWN; + pp->sysfs_state = PATH_DOWN; + goto out; } @@ -1844,31 +1845,34 @@ path_offline (struct path * pp) if (pp->bus == SYSFS_BUS_SCSI) { if (!strncmp(buff, "offline", 7)) { - pp->offline = 1; - return PATH_DOWN; + pp->sysfs_state = PATH_DOWN; + goto out; + } else if (!strncmp(buff, "blocked", 7) || + !strncmp(buff, "quiesce", 7)) { + pp->sysfs_state = PATH_PENDING; + goto out; + } else if (!strncmp(buff, "running", 7)) { + pp->sysfs_state = PATH_UP; + goto out; } - pp->offline = 0; - if (!strncmp(buff, "blocked", 7) || - !strncmp(buff, "quiesce", 7)) - return PATH_PENDING; - else if (!strncmp(buff, "running", 7)) - return PATH_UP; } else if (pp->bus == SYSFS_BUS_NVME) { if (!strncmp(buff, "dead", 4)) { - pp->offline = 1; - return PATH_DOWN; + pp->sysfs_state = PATH_DOWN; + goto out; + } else if (!strncmp(buff, "new", 3) || + !strncmp(buff, "deleting", 8)) { + pp->sysfs_state = PATH_PENDING; + goto out; + } else if (!strncmp(buff, "live", 4)) { + pp->sysfs_state = PATH_UP; + goto out; } - pp->offline = 0; - if (!strncmp(buff, "new", 3) || - !strncmp(buff, "deleting", 8)) - return PATH_PENDING; - else if (!strncmp(buff, "live", 4)) - return PATH_UP; } - - return PATH_DOWN; + pp->sysfs_state = PATH_DOWN; +out: + return pp->sysfs_state; } static int @@ -2052,8 +2056,7 @@ get_prio (struct path * pp) old_prio = pp->priority; pp->priority = prio_getprio(p, pp); if (pp->priority < 0) { - /* this changes pp->offline, but why not */ - int state = path_offline(pp); + int state = path_sysfs_state(pp); if (state == PATH_DOWN || state == PATH_PENDING) { pp->priority = old_prio; @@ -2424,7 +2427,7 @@ int pathinfo(struct path *pp, struct config *conf, int mask) return PATHINFO_SKIPPED; } - path_state = path_offline(pp); + path_state = path_sysfs_state(pp); if (path_state == PATH_REMOVED) goto blank; else if (mask & DI_NOIO) { diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h index f3e0c618..7d42eae5 100644 --- a/libmultipath/discovery.h +++ b/libmultipath/discovery.h @@ -33,7 +33,7 @@ struct config; int path_discovery (vector pathvec, int flag); int path_get_tpgs(struct path *pp); /* This function never returns TPGS_UNDEF */ int do_tur (char *); -int path_offline (struct path *); +int path_sysfs_state(struct path *); int start_checker(struct path * pp, struct config * conf, int daemon, int state); int get_state(struct path * pp); diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index bb34d9e8..6439d3a7 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -146,7 +146,7 @@ global: path_discovery; path_get_tpgs; pathinfo; - path_offline; + path_sysfs_state; print_all_paths; print_foreign_topology; print_multipath_topology__; diff --git a/libmultipath/print.c b/libmultipath/print.c index e536c5c0..00c03ace 100644 --- a/libmultipath/print.c +++ b/libmultipath/print.c @@ -514,7 +514,7 @@ snprint_offline (struct strbuf *buff, const struct path * pp) { if (!pp || !pp->mpp) return append_strbuf_str(buff, "unknown"); - else if (pp->offline) + else if (pp->sysfs_state == PATH_DOWN) return append_strbuf_str(buff, "offline"); else return append_strbuf_str(buff, "running"); diff --git a/libmultipath/structs.h b/libmultipath/structs.h index 074faca6..d8231e95 100644 --- a/libmultipath/structs.h +++ b/libmultipath/structs.h @@ -362,7 +362,7 @@ struct path { unsigned int tick; unsigned int pending_ticks; int bus; - int offline; + int sysfs_state; int state; int dmstate; int chkrstate; diff --git a/multipathd/main.c b/multipathd/main.c index 4f752adb..33a57041 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -96,7 +96,7 @@ void * mpath_pr_event_handler_fn (void * ); do { \ if (pp->mpp && checker_selected(&pp->checker) && \ lvl <= libmp_verbosity) { \ - if (pp->offline) \ + if (pp->sysfs_state == PATH_DOWN) \ condlog(lvl, "%s: %s - path offline", \ pp->mpp->alias, pp->dev); \ else { \ @@ -2320,7 +2320,7 @@ check_path_state(struct path *pp) int newstate; struct config *conf; - newstate = path_offline(pp); + newstate = path_sysfs_state(pp); if (newstate == PATH_UP) { conf = get_multipath_config(); pthread_cleanup_push(put_multipath_config, conf); From patchwork Tue Oct 15 03:28:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835706 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04C2E170A27 for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; cv=none; b=Q2IhU8UQ2FUCqwn8BSa2DpDFp+vF3isOW8zUxfWsT7krkPaCoNd9yRBdtcvIKM72P50oyCToSoDsUSSaiyGe/NvHfYO8woHFDZSr/s5OrbMl3cMZvIkqSpbwXj0jsqpb5BDQywCY8ohgqzUi6qqmOYBYK3/syn3ywk6ciNiMIwc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; c=relaxed/simple; bh=M9eNzj/l2YhC0Ow+8Y85N5SSEpFfJWxEJcScAAC0SPQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ONd0Qp57/iDIdIH+q+xq85NTEr8yAHIGhP3FBHHQxxb/SQ/rV7pmIvwnKmWaw3HDssYIWETZGL0Q92sfRa9hUTaPJJ6wRkkpPNMdtD1hWxsZLZCVzkpSbay4u9HYA7SaIyF5R4YkZo9TfO09IhzS55v5LmO6JgvhyapZF8RDI3A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ODdaI6o7; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ODdaI6o7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/xSNWjnq4MvF1/6rUIiggT/NFv8MV1T6dyoFZEajvUg=; b=ODdaI6o7/G5Q/4cLPQe6lt6JbuoERhetDYL4j2blO5OWdophfRQgGm8AI1u1Nijg82Lm7T aTbsy6/5gR4HQWFu5fXPYbZ6mj9EkEYrIm2tPmiFHh0DgDYbAcHcnMrwoCQt+eEdb8K7R1 bi1KyTt+rlsy/yqIvF35ttKbZ8ASoM0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-530-AxgDOgmYOjSlWDPoDbIlGg-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: AxgDOgmYOjSlWDPoDbIlGg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3F2EB1956096; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F3E4B19560AE; Tue, 15 Oct 2024 03:28:37 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SawE2693311 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:36 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SaHp2693310; Mon, 14 Oct 2024 23:28:36 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 08/23] multipathd: split check_path_state into two functions Date: Mon, 14 Oct 2024 23:28:20 -0400 Message-ID: <20241015032835.2693247-9-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com check_path_state() is now split into start_path_check(), which calls path_sysfs_state() and if the path is up also calls start_checker(), and get_new_state() which gets the new state from either pp->sysfs_state or get_state(). Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- multipathd/main.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 33a57041..16c0531e 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2314,25 +2314,31 @@ should_skip_path(struct path *pp){ return 0; } -static int -check_path_state(struct path *pp) +static void +start_path_check(struct path *pp) { - int newstate; struct config *conf; - newstate = path_sysfs_state(pp); - if (newstate == PATH_UP) { + if (path_sysfs_state(pp) == PATH_UP) { conf = get_multipath_config(); pthread_cleanup_push(put_multipath_config, conf); - newstate = PATH_UNCHECKED; - if (start_checker(pp, conf, 1, newstate) == 0) - newstate = get_state(pp); + start_checker(pp, conf, 1, PATH_UNCHECKED); pthread_cleanup_pop(1); } else { checker_clear_message(&pp->checker); condlog(3, "%s: state %s, checker not called", - pp->dev, checker_state_name(newstate)); + pp->dev, checker_state_name(pp->sysfs_state)); } +} + +static int +get_new_state(struct path *pp) +{ + int newstate = pp->sysfs_state; + struct config *conf; + + if (newstate == PATH_UP) + newstate = get_state(pp); /* * Wait for uevent for removed paths; * some LLDDs like zfcp keep paths unavailable @@ -2413,7 +2419,8 @@ do_check_path (struct vectors * vecs, struct path * pp) pp->checkint = checkint; }; - newstate = check_path_state(pp); + start_path_check(pp); + newstate = get_new_state(pp); if (newstate == PATH_WILD || newstate == PATH_UNCHECKED) return CHECK_PATH_SKIPPED; /* @@ -2752,7 +2759,8 @@ handle_uninitialized_path(struct vectors * vecs, struct path * pp, } } - newstate = check_path_state(pp); + start_path_check(pp); + newstate = get_new_state(pp); if (!strlen(pp->wwid) && (pp->initialized == INIT_FAILED || pp->initialized == INIT_NEW) && From patchwork Tue Oct 15 03:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835703 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6F6F1714BD for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; cv=none; b=re2HFmOH2BfoMZ29eSeXAgJMuRf9prCBrl0z1HVgjV5qg2N/T1vpiIplsYbUfXHGG+wdRN80AGa3mscJ23bl0VKninGp+/v+DLUf3VNQ3DyeTKgXVFRe0RnUDbPTvWqzTXhnowVbTL+miVw400i94XBx5jX6c8pGpRKzhEm0Wmo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; c=relaxed/simple; bh=9T8Hn2v/DAA05EWxuLl/0vPGncBmmOjV0Qau/nh5324=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pJViEDa1+SVLmx8AYlYVSays+usR/TQ7TBXdl28vZ2aFbal5Cg81+sP/RoMkSHmbhwpeKR3FEjbpD6+cP4xZLzeaxH9czASrgSei+F9H8sym7F9sDjIV/u+yTJHJvvolfvyTl/e4RuNAry5/jbiv9rqT73etT8/HIX9RSiu8Uvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Iv688+PW; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Iv688+PW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aSJMRVFe0C+Fen574tMyVMu/sjlFYHEciXXA4GnGkuw=; b=Iv688+PWrEXxYpdSwB04dm0NLut5rktsAUNy89NtLzsJ4FiXyV7C8ra3kAu+G6sWxp72lx dtz85gBZX6XCU7iJpkpFsuYUqdfyHyNcszhPGRlpk4bW0shXOCexY5ybISFtnwdKINmZUC Y+q7pBjF9cNaGfySKCRIs2UsmKxPqh4= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-435-qGi7ZMJ8PuSjJERQYuPsBg-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: qGi7ZMJ8PuSjJERQYuPsBg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7EFC219560B1; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3F56F1956089; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SbfG2693315 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3Sbbg2693314; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 09/23] multipathd: rename do_check_path to update_path_state Date: Mon, 14 Oct 2024 23:28:21 -0400 Message-ID: <20241015032835.2693247-10-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Move the code that starts the path checker from do_check_path() into check_path(), rename the remainder of do_check_path() to update_path_state() and call that from check_path(). Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- multipathd/main.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 16c0531e..488aa225 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2396,7 +2396,7 @@ enum check_path_return { }; static int -do_check_path (struct vectors * vecs, struct path * pp) +update_path_state (struct vectors * vecs, struct path * pp) { int newstate; int new_path_up = 0; @@ -2414,12 +2414,6 @@ do_check_path (struct vectors * vecs, struct path * pp) marginal_pathgroups = conf->marginal_pathgroups; put_multipath_config(conf); - if (pp->checkint == CHECKINT_UNDEF) { - condlog(0, "%s: BUG: checkint is not set", pp->dev); - pp->checkint = checkint; - }; - - start_path_check(pp); newstate = get_new_state(pp); if (newstate == PATH_WILD || newstate == PATH_UNCHECKED) return CHECK_PATH_SKIPPED; @@ -2639,7 +2633,7 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks, time_t start_secs) { int r; - unsigned int adjust_int, max_checkint; + unsigned int adjust_int, checkint, max_checkint; struct config *conf; time_t next_idx, goal_idx; @@ -2652,14 +2646,21 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks, return CHECK_PATH_SKIPPED; conf = get_multipath_config(); + checkint = conf->checkint; max_checkint = conf->max_checkint; adjust_int = conf->adjust_int; put_multipath_config(conf); - r = do_check_path(vecs, pp); + if (pp->checkint == CHECKINT_UNDEF) { + condlog(0, "%s: BUG: checkint is not set", pp->dev); + pp->checkint = checkint; + } + + start_path_check(pp); + r = update_path_state(vecs, pp); /* - * do_check_path() removed or orphaned the path. + * update_path_state() removed or orphaned the path. */ if (r == CHECK_PATH_REMOVED || !pp->mpp) return r; From patchwork Tue Oct 15 03:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835694 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 144C3170A29 for ; Tue, 15 Oct 2024 03:28:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962923; cv=none; b=EWkY7mEc5WydCwk/jJec6hhIAXUvZ/HHWm4MtGtajpEckb1xc4PfR1XFIVD5+yXOudLXeOdldfAmACOeUG7thKPTbQIyT+gNEHSv/tM9zO5ZI+Bd9xe0mCGkFhoyo9auGqkeRl4WfB80DOXsXmTpRzWPXVpOXl9Ko2qs8yt9fgA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962923; c=relaxed/simple; bh=jBpQ6zE+7D2Me1PRYsOFpxR6pHrr0D5aXd5/aBz1ivk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eCJTIfBnw0DFDfsj0F2HA2vQvng99KdlmUReXD+Jv31+tgkdrkAzTLkJ7k4vlsNn5T8dPz2kj5ztcrHJumUixo4q5ssDREo3k9UZxaspqjpbHiQd9MWvm6KGIGxj00SCS8eNjlJVRJK/XZNW3/QTnuW21h9kqXC5T8guKp+60fw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SyxBz+zp; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SyxBz+zp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3mXkKGvLzCDO7253OcHK4EFZvaEdEDLktaTSNH4bi+w=; b=SyxBz+zp44pT6m4Pvg+Wi/sdn17kNwMg5o5nZ3I7BqFx9+cE89MCzxZCMn0A6yQm3vn3af H94bbjPmhDdBCeFSN3VKbA9/8HWQT8ofgbOiqARfOVsjJlJ4lYbRy74d4Q6gaYbksxDYbC o96onEXWI6Q13ip53HhI0aFy+zvwqY8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-532-fkiCE6kzPOSOStDddo635Q-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: fkiCE6kzPOSOStDddo635Q-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9A01519560BA; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5C21B300019D; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sbld2693319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3Sbxs2693318; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 10/23] multipathd: split check_path into two functions Date: Mon, 14 Oct 2024 23:28:22 -0400 Message-ID: <20241015032835.2693247-11-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Split out the code that updates a path's state and sets up the next check time into its own function, update_path(). Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- multipathd/main.c | 42 ++++++++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 488aa225..ac54ace4 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2390,6 +2390,7 @@ sync_mpp(struct vectors * vecs, struct multipath *mpp, unsigned int ticks) } enum check_path_return { + CHECK_PATH_STARTED, CHECK_PATH_CHECKED, CHECK_PATH_SKIPPED, CHECK_PATH_REMOVED, @@ -2629,14 +2630,8 @@ update_path_state (struct vectors * vecs, struct path * pp) } static int -check_path (struct vectors * vecs, struct path * pp, unsigned int ticks, - time_t start_secs) +check_path (struct path * pp, unsigned int ticks) { - int r; - unsigned int adjust_int, checkint, max_checkint; - struct config *conf; - time_t next_idx, goal_idx; - if (pp->initialized == INIT_REMOVED) return CHECK_PATH_SKIPPED; @@ -2645,18 +2640,27 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks, if (pp->tick) return CHECK_PATH_SKIPPED; - conf = get_multipath_config(); - checkint = conf->checkint; - max_checkint = conf->max_checkint; - adjust_int = conf->adjust_int; - put_multipath_config(conf); - if (pp->checkint == CHECKINT_UNDEF) { + struct config *conf; + condlog(0, "%s: BUG: checkint is not set", pp->dev); - pp->checkint = checkint; + conf = get_multipath_config(); + pp->checkint = conf->checkint; + put_multipath_config(conf); } start_path_check(pp); + return CHECK_PATH_STARTED; +} + +static int +update_path(struct vectors * vecs, struct path * pp, time_t start_secs) +{ + int r; + unsigned int adjust_int, max_checkint; + struct config *conf; + time_t next_idx, goal_idx; + r = update_path_state(vecs, pp); /* @@ -2685,6 +2689,10 @@ check_path (struct vectors * vecs, struct path * pp, unsigned int ticks, if (pp->tick == 1) return r; + conf = get_multipath_config(); + max_checkint = conf->max_checkint; + adjust_int = conf->adjust_int; + put_multipath_config(conf); /* * every mpp has a goal_idx in the range of * 0 <= goal_idx < conf->max_checkint @@ -2818,8 +2826,10 @@ check_paths(struct vectors *vecs, unsigned int ticks, int *num_paths_p) if (!pp->mpp || pp->is_checked) continue; pp->is_checked = true; - rc = check_path(vecs, pp, ticks, - start_time.tv_sec); + rc = check_path(pp, ticks); + if (rc == CHECK_PATH_STARTED) + rc = update_path(vecs, pp, + start_time.tv_sec); if (rc == CHECK_PATH_CHECKED) (*num_paths_p)++; if (++paths_checked % 128 == 0) From patchwork Tue Oct 15 03:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835693 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C69D154456 for ; Tue, 15 Oct 2024 03:28:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962923; cv=none; b=mb8BO9OKobnWoYLtEsgiZtrtj4NRXP8EuK6KYw/7aF1xDv4bBALNSe9D+MybZJTazD7iySTYNGZwq6J3RKSjpI1+HpGJUlJqlH85RVzO54OQJ7IBTVOqzx0zlr9w/p6xfmRDh1sVX++cCfG+50aiwHa6w+BD8iFuVC4PyKks5sA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962923; c=relaxed/simple; bh=99K6VTJJFJxLyjokNBy4ckNi1r7YQLB+jPluDJU/KQA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Gv161CGp/rkuXjq9MAV8xmEriUdhFYlpYnrXXurU1M1oGPZNI6NDTktI1tDB1k1hgLrnCviONuJI+wtw9QE1oVq6twEyJyUeNYmVMSTPgxw8XC4VemTFmagCms/X/yArzElNnhXrbB/ju93Fe3dBzSm482U1t1sokjzfi9yQ8p0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=A4u45iTQ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A4u45iTQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ppC/N7vQ2ltmCpoFIF5mEowm4dAipA83IZBEj/LI+qM=; b=A4u45iTQtlMShF2s39vRZ1YLttsZSyvUwdaPnWT9yLvWceJ8ePIoGpgvyArsWOTHkevRg1 lWd+Sb7Se43MPQEYVuiVcdkj3/aIG658XBvxJGsqK81RMTCEpEdWzpX0Dxll4Y8q2i6Aes eSlOPjeuTYG/TvOjveHca9HaXl6YSM0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-443-z6vC0DhBMG6GCU0cG8_d7w-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: z6vC0DhBMG6GCU0cG8_d7w-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C20EC19560AF; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 83B6A1956054; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SbA32693323 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3Sb8Y2693322; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 11/23] multipathd: split handle_uninitialized_path into two functions Date: Mon, 14 Oct 2024 23:28:23 -0400 Message-ID: <20241015032835.2693247-12-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Split handle_uninitialized_path() into check_uninitialized_path, which handles udev retriggers for INIT_MISSING_UDEV paths and starts the path checker for INIT_FAILED and INIT_NEW paths, and update_uninitialized_path() which gets the path checker result and reruns pathinfo if the path is up. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- multipathd/main.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index ac54ace4..89f02a3a 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2715,13 +2715,10 @@ update_path(struct vectors * vecs, struct path * pp, time_t start_secs) } static int -handle_uninitialized_path(struct vectors * vecs, struct path * pp, - unsigned int ticks) +check_uninitialized_path(struct path * pp, unsigned int ticks) { - int newstate; int retrigger_tries; struct config *conf; - int ret; if (pp->initialized != INIT_NEW && pp->initialized != INIT_FAILED && pp->initialized != INIT_MISSING_UDEV) @@ -2735,6 +2732,7 @@ handle_uninitialized_path(struct vectors * vecs, struct path * pp, conf = get_multipath_config(); retrigger_tries = conf->retrigger_tries; pp->tick = conf->max_checkint; + pp->checkint = conf->checkint; put_multipath_config(conf); if (pp->initialized == INIT_MISSING_UDEV) { @@ -2769,6 +2767,15 @@ handle_uninitialized_path(struct vectors * vecs, struct path * pp, } start_path_check(pp); + return CHECK_PATH_STARTED; +} + +static int +update_uninitialized_path(struct vectors * vecs, struct path * pp) +{ + int newstate, ret; + struct config *conf; + newstate = get_new_state(pp); if (!strlen(pp->wwid) && @@ -2859,7 +2866,9 @@ next_mpp: continue; pp->is_checked = true; - rc = handle_uninitialized_path(vecs, pp, ticks); + rc = check_uninitialized_path(pp, ticks); + if (rc == CHECK_PATH_STARTED) + rc = update_uninitialized_path(vecs, pp); if (rc == CHECK_PATH_REMOVED) i--; else if (rc == CHECK_PATH_CHECKED) From patchwork Tue Oct 15 03:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835695 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 577B8170A30 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; cv=none; b=lR+0sDLN/nJM/0SbsRNkb/i9qh/chVIMFkca/U06TG2Zv7PvjfBSC2+1KINSKZvzhBroKr5CTRFMZjvL9IDvlen9fj5cAWsycszkWutcAaUG6KyPr6gRSo47pgK4aWnB+0tXjgjVfmWoaHa8xToliGeaRD99rqlQ+TgMxIvgH/8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; c=relaxed/simple; bh=wWPnR+o3byB0vhX6nlRIAyV5Xl/EVbcg+NTourlorjE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=X/Z7Dl9sPZpbkDqbs8498nLyFINoxZZKNRwncA3ZQDawUHDmDOrFZiUfLdrUFSw9WgBrMElRAQQO0GBds8hb3FZoB/FHW6EXXuYEyJ3JOE1hFXjPKl1DkwNumbPcilPfsMNF4m2pt5PPxVGSM0/48pCAGoC6wA4wEq+rwxvQo+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aUON/tCp; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aUON/tCp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3ocU7+I2+d1xMXa/ZMQZXn9qyzm2+esou0DE/soHhxk=; b=aUON/tCpxmYiLBGHL65OTjefdQQwj435LzQYWC340fpiKziaxvS4ul6oru1D2hdM81cYCf DPdPRgKNeCRADsSHi44snM76gHxv1BCXnwvLqqTHjDRl1SyEtDrRIPIPGxkshqdIFS8gpR BKKwY9lNDFPFoV38i2MPG5IhdX9TUT8= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-437-UuDIdsfKNvuoIMdwBK-NCQ-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: UuDIdsfKNvuoIMdwBK-NCQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AC2BF195609D; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 773AA19560AE; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sbxr2693327 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SbdP2693326; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 12/23] multipathd: split check_paths into two functions Date: Mon, 14 Oct 2024 23:28:24 -0400 Message-ID: <20241015032835.2693247-13-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Instead of checking and updating each path, the checkerloop now starts the checkers on all the paths in check_paths(), and then goes back and updates all the paths in update_paths(). Since the async checkers use an absolute time to wait for before returning PATH_PENDING, only one checker actually needs to be waited for in update_paths(). The rest will already have reached their timeout when update_path() is called for them. The check_paths() and update_paths() loop over the pathvec instead of looping through the multipath device paths to avoid having to restart checking of the device paths when the multipath device needs to be resynced while updating the paths. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/structs.h | 10 +++- multipathd/main.c | 104 +++++++++++++++++++---------------------- 2 files changed, 56 insertions(+), 58 deletions(-) diff --git a/libmultipath/structs.h b/libmultipath/structs.h index d8231e95..af8e31e9 100644 --- a/libmultipath/structs.h +++ b/libmultipath/structs.h @@ -314,6 +314,14 @@ enum recheck_wwid_states { RECHECK_WWID_ON = YNU_YES, }; +enum check_path_states { + CHECK_PATH_UNCHECKED, + CHECK_PATH_STARTED, + CHECK_PATH_CHECKED, + CHECK_PATH_SKIPPED, + CHECK_PATH_REMOVED, +}; + struct vpd_vendor_page { int pg; const char *name; @@ -395,7 +403,7 @@ struct path { int fast_io_fail; unsigned int dev_loss; int eh_deadline; - bool is_checked; + enum check_path_states is_checked; bool can_use_env_uid; unsigned int checker_timeout; /* configlet pointers */ diff --git a/multipathd/main.c b/multipathd/main.c index 89f02a3a..ca84331d 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2389,13 +2389,6 @@ sync_mpp(struct vectors * vecs, struct multipath *mpp, unsigned int ticks) do_sync_mpp(vecs, mpp); } -enum check_path_return { - CHECK_PATH_STARTED, - CHECK_PATH_CHECKED, - CHECK_PATH_SKIPPED, - CHECK_PATH_REMOVED, -}; - static int update_path_state (struct vectors * vecs, struct path * pp) { @@ -2776,6 +2769,10 @@ update_uninitialized_path(struct vectors * vecs, struct path * pp) int newstate, ret; struct config *conf; + if (pp->initialized != INIT_NEW && pp->initialized != INIT_FAILED && + pp->initialized != INIT_MISSING_UDEV) + return CHECK_PATH_SKIPPED; + newstate = get_new_state(pp); if (!strlen(pp->wwid) && @@ -2805,80 +2802,69 @@ update_uninitialized_path(struct vectors * vecs, struct path * pp) enum checker_state { CHECKER_STARTING, - CHECKER_RUNNING, + CHECKER_CHECKING_PATHS, + CHECKER_UPDATING_PATHS, CHECKER_FINISHED, }; static enum checker_state -check_paths(struct vectors *vecs, unsigned int ticks, int *num_paths_p) +check_paths(struct vectors *vecs, unsigned int ticks) { unsigned int paths_checked = 0; struct timespec diff_time, start_time, end_time; - struct multipath *mpp; struct path *pp; - int i, rc; + int i; get_monotonic_time(&start_time); - vector_foreach_slot(vecs->mpvec, mpp, i) { - struct pathgroup *pgp; - struct path *pp; - int j, k; - bool check_for_waiters = false; - /* maps can be rechecked, so this is not always 0 */ - int synced_count = mpp->synced_count; - - vector_foreach_slot (mpp->pg, pgp, j) { - vector_foreach_slot (pgp->paths, pp, k) { - if (!pp->mpp || pp->is_checked) - continue; - pp->is_checked = true; - rc = check_path(pp, ticks); - if (rc == CHECK_PATH_STARTED) - rc = update_path(vecs, pp, - start_time.tv_sec); - if (rc == CHECK_PATH_CHECKED) - (*num_paths_p)++; - if (++paths_checked % 128 == 0) - check_for_waiters = true; - /* - * mpp has been removed or resynced. Path may - * have been removed. - */ - if (VECTOR_SLOT(vecs->mpvec, i) != mpp || - synced_count != mpp->synced_count) { - i--; - goto next_mpp; - } - } - } -next_mpp: - if (check_for_waiters && + vector_foreach_slot(vecs->pathvec, pp, i) { + if (pp->is_checked != CHECK_PATH_UNCHECKED) + continue; + if (pp->mpp) + pp->is_checked = check_path(pp, ticks); + else + pp->is_checked = check_uninitialized_path(pp, ticks); + if (++paths_checked % 128 == 0 && (lock_has_waiters(&vecs->lock) || waiting_clients())) { get_monotonic_time(&end_time); timespecsub(&end_time, &start_time, &diff_time); if (diff_time.tv_sec > 0) - return CHECKER_RUNNING; + return CHECKER_CHECKING_PATHS; } } + return CHECKER_UPDATING_PATHS; +} + +static enum checker_state +update_paths(struct vectors *vecs, int *num_paths_p, time_t start_secs) +{ + unsigned int paths_checked = 0; + struct timespec diff_time, start_time, end_time; + struct path *pp; + int i, rc; + + get_monotonic_time(&start_time); + vector_foreach_slot(vecs->pathvec, pp, i) { - if (pp->mpp || pp->is_checked) + if (pp->is_checked != CHECK_PATH_STARTED) continue; - pp->is_checked = true; - - rc = check_uninitialized_path(pp, ticks); - if (rc == CHECK_PATH_STARTED) + if (pp->mpp) + rc = update_path(vecs, pp, start_secs); + else rc = update_uninitialized_path(vecs, pp); if (rc == CHECK_PATH_REMOVED) i--; - else if (rc == CHECK_PATH_CHECKED) - (*num_paths_p)++; + else { + pp->is_checked = rc; + if (rc == CHECK_PATH_CHECKED) + (*num_paths_p)++; + } if (++paths_checked % 128 == 0 && (lock_has_waiters(&vecs->lock) || waiting_clients())) { get_monotonic_time(&end_time); timespecsub(&end_time, &start_time, &diff_time); if (diff_time.tv_sec > 0) - return CHECKER_RUNNING; + return CHECKER_UPDATING_PATHS; } } return CHECKER_FINISHED; @@ -2946,10 +2932,14 @@ checkerloop (void *ap) vector_foreach_slot(vecs->mpvec, mpp, i) sync_mpp(vecs, mpp, ticks); vector_foreach_slot(vecs->pathvec, pp, i) - pp->is_checked = false; - checker_state = CHECKER_RUNNING; + pp->is_checked = CHECK_PATH_UNCHECKED; + checker_state = CHECKER_CHECKING_PATHS; } - checker_state = check_paths(vecs, ticks, &num_paths); + if (checker_state == CHECKER_CHECKING_PATHS) + checker_state = check_paths(vecs, ticks); + if (checker_state == CHECKER_UPDATING_PATHS) + checker_state = update_paths(vecs, &num_paths, + start_time.tv_sec); lock_cleanup_pop(vecs->lock); if (checker_state != CHECKER_FINISHED) { /* Yield to waiters */ From patchwork Tue Oct 15 03:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835697 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B619D1714A8 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; cv=none; b=kJAs6Osc12+rHmP1cAz/g+B5qwGE0jA9WQKiyN5BMCtWMuWnyrvbbXHDT/cHIzOz5NqdERb76WFxvHhgYdqpAoOXZjLfc7JpRVc+xAVk9kzC3VRpUOaq087ZmJLFSEoZWavBMgi5jOottwJTQtk1Jdj8hYeD9IIFccV824oMNHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; c=relaxed/simple; bh=QwWLCaoqkEO7MdxjSSX1AlBvBOqamwc2C9mCQH8UZjg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uzWs+R1ZBNoFrcdkzr08maQjGfFrb6m7TcC3PgWyFD0LoLcXc1UrTXIv/kNLkDZW+pxv+DNM117wXidDs2WjeW0wDT4tDZrqAuTyk7m7wHzQhgWFsrSWX/dzEmIkFvRFkrwK6VdfQoveYS0fIMEwkJtnuQdYPd7Wvv4vFNTiQzk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SBTKhyj/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SBTKhyj/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uMHUFgDHX/rlgtd8bQh97l/dzcQRe0tvCTscS6K1NIQ=; b=SBTKhyj/8qvaAw0HGDHMB/zGaXLJVsQUsD6RNKYOzXs5viVRbyeVakrg5Wqp96Rn2svuk2 ULtvI5Fg5ZYb5YHU9H2WlJRQHcL0OfoPZbB4zIQmIGzSuRzKCUt/RqqCHAuKEiUH+kcYae 0ep7GyOXW3ENLkqh36ZbnDsRJTYwN0E= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-596-s_ZhUqiYP1KHJCvCNlKyPw-1; Mon, 14 Oct 2024 23:28:40 -0400 X-MC-Unique: s_ZhUqiYP1KHJCvCNlKyPw-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F41F919560B5; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B53181956056; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3SbsS2693331 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SbE32693330; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 13/23] multipathd: fix "fail path" and "reinstate path" commands Date: Mon, 14 Oct 2024 23:28:25 -0400 Message-ID: <20241015032835.2693247-14-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Now that multipathd can drop the vecs lock and sleep in-between when the checker runs and when the path gets updated, it is possible for the user to either fail or reinstate the path in this window. If a path gets manually failed in the window between when the checker starts and the path is updated, the checker will have already been started, on the path, and if multipathd read the results like normal, it could simply reinstate the path. To avoid this, when a path checker is disabled, the checker path_state and message are updated, just like they are when checker_check() is run on a disabled path, so that when checker_get_state() is called, it will always return the same results for a disabled checker, regardless of when it was disabled. Reinstating the path doesn't cause many problems, but still can be improved. Since the checker was disabled when it would have been started, it didn't run during this cycle, only the kernel state will get updated. The rest of the path update changes won't happen until the next time the checker runs. This is the case regardless of whether or not the path was reinstated in the window between when the checker starts and the path is updated. To make reinstated paths get updated sooner, pp->tick is now set to 1 when the path is reinstated. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/checkers.c | 2 ++ multipathd/cli_handlers.c | 1 + 2 files changed, 3 insertions(+) diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c index ce3e48bd..f3e98352 100644 --- a/libmultipath/checkers.c +++ b/libmultipath/checkers.c @@ -251,6 +251,8 @@ void checker_disable (struct checker * c) if (!c) return; c->disable = 1; + c->msgid = CHECKER_MSGID_DISABLED; + c->path_state = PATH_UNCHECKED; } int checker_init (struct checker * c, void ** mpctxt_addr) diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c index 42603544..184c3f91 100644 --- a/multipathd/cli_handlers.c +++ b/multipathd/cli_handlers.c @@ -1108,6 +1108,7 @@ cli_reinstate(void * v, struct strbuf *reply, void * data) pp->mpp->alias, pp->dev_t); checker_enable(&pp->checker); + pp->tick = 1; return dm_reinstate_path(pp->mpp->alias, pp->dev_t); } From patchwork Tue Oct 15 03:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835701 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 278051714B4 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; cv=none; b=gpYdmYiL0f95QqpmWQFotE6Dx1DxBPxqPV0Hw3XO/NdH4zcUXc8QUnCGo5SvyDEFia0+zNr4/Sr5yB9Ahm2g1TVGRwC0TOxgIOd26ZDcW1MuBcsEBaXHxnD9DbWubfAATw0QpAEIt9aHejksDxlifPpsxS4TnlY0jWwVSkUGJQg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; c=relaxed/simple; bh=e6OzqhsAonj4bA/TeGJUiRtgmpbsZ7PzyPONeAorJIE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BYA8EAzSeh+D6howiG3hthXsqB3R3CFZO1j6tpnlkCNY8rgNBNdq4rCIdM8YnKuWAFLk0WDbBVh7MAvvjLTLtBtdFGrSc4r8CNbeZtl2fxO54r/EiVDAoUJhFGH/19+FsVB+J4G7I8p7MwZXu6qND841GubQnzbm9qV0uDYk0fY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=f4k4lt0U; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="f4k4lt0U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q0amrwaMhpzwHVsclVNYT1+bKgK5IvtHmiQkFNtvjBE=; b=f4k4lt0U2McP6dmmhtQFHezTvVwDA/FL4rIkNe3YZlPMj38vWKdq/8Dzkj1jr407tmGK98 /cGEDhG9mae4fsaL/bHM4mQrPPdF6FZSx7V2t7MdJ9AG8g5P6Q2I/ilZVL7ZeGHHejwiEQ 1GPK38SkEV5oaO2ENTPlVJ7jvi1FKOo= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-297-IOYvprN6M1ewdmcsPR7ndQ-1; Mon, 14 Oct 2024 23:28:39 -0400 X-MC-Unique: IOYvprN6M1ewdmcsPR7ndQ-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 095EB195608B; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C875D1955E8F; Tue, 15 Oct 2024 03:28:38 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sbq02693335 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SbDP2693334; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 14/23] multipathd: update priority once after updating all paths Date: Mon, 14 Oct 2024 23:28:26 -0400 Message-ID: <20241015032835.2693247-15-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Instead of updating the path priorities and possibly reloading the multipath device when each path is updated, wait till all paths have been updated, and then go through the multipath devices updating the priorities once, reloading if necessary. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/structs.h | 9 +++ multipathd/main.c | 174 +++++++++++++++++++++++++++-------------- 2 files changed, 123 insertions(+), 60 deletions(-) diff --git a/libmultipath/structs.h b/libmultipath/structs.h index af8e31e9..1f531d30 100644 --- a/libmultipath/structs.h +++ b/libmultipath/structs.h @@ -318,6 +318,7 @@ enum check_path_states { CHECK_PATH_UNCHECKED, CHECK_PATH_STARTED, CHECK_PATH_CHECKED, + CHECK_PATH_NEW_UP, CHECK_PATH_SKIPPED, CHECK_PATH_REMOVED, }; @@ -421,6 +422,13 @@ enum prflag_value { PRFLAG_SET, }; +enum prio_update_type { + PRIO_UPDATE_NONE, + PRIO_UPDATE_NORMAL, + PRIO_UPDATE_NEW_PATH, + PRIO_UPDATE_MARGINAL, +}; + struct multipath { char wwid[WWID_SIZE]; char alias_old[WWID_SIZE]; @@ -464,6 +472,7 @@ struct multipath { int queue_mode; unsigned int sync_tick; int synced_count; + enum prio_update_type prio_update; uid_t uid; gid_t gid; mode_t mode; diff --git a/multipathd/main.c b/multipathd/main.c index ca84331d..f48dbe25 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -1996,15 +1996,13 @@ mpvec_garbage_collector (struct vectors * vecs) * best pathgroup, and this is the first path in the pathgroup to come back * up, then switch to this pathgroup */ static int -followover_should_failback(struct path * pp) +do_followover_should_failback(struct path * pp) { struct pathgroup * pgp; struct path *pp1; int i; - if (pp->mpp->pgfailback != -FAILBACK_FOLLOWOVER || - !pp->mpp->pg || !pp->pgindex || - pp->pgindex != pp->mpp->bestpg) + if (!pp->pgindex || pp->pgindex != pp->mpp->bestpg) return 0; pgp = VECTOR_SLOT(pp->mpp->pg, pp->pgindex - 1); @@ -2017,6 +2015,26 @@ followover_should_failback(struct path * pp) return 1; } +static int +followover_should_failback(struct multipath *mpp) +{ + struct path *pp; + struct pathgroup * pgp; + int i, j; + + if (mpp->pgfailback != -FAILBACK_FOLLOWOVER || !mpp->pg) + return 0; + + vector_foreach_slot (mpp->pg, pgp, i) { + vector_foreach_slot (pgp->paths, pp, j) { + if (pp->is_checked == CHECK_PATH_NEW_UP && + do_followover_should_failback(pp)) + return 1; + } + } + return 0; +} + static void missing_uev_wait_tick(struct vectors *vecs) { @@ -2132,41 +2150,58 @@ partial_retrigger_tick(vector pathvec) } } -static int update_prio(struct path *pp, int force_refresh_all) +static bool update_prio(struct multipath *mpp, bool refresh_all) { int oldpriority; - struct path *pp1; + struct path *pp; struct pathgroup * pgp; - int i, j, changed = 0; + int i, j; + bool changed = false; + bool skipped_path = false; struct config *conf; - oldpriority = pp->priority; - if (pp->state != PATH_DOWN) { - conf = get_multipath_config(); - pthread_cleanup_push(put_multipath_config, conf); - pathinfo(pp, conf, DI_PRIO); - pthread_cleanup_pop(1); + vector_foreach_slot (mpp->pg, pgp, i) { + vector_foreach_slot (pgp->paths, pp, j) { + if (pp->state == PATH_DOWN) + continue; + /* + * refresh_all will be set if the mpp has any path + * for whom pp->marginal switched values or for whom + * pp->is_checked == CHECK_PATH_NEW_UP + */ + if (!refresh_all && + pp->is_checked != CHECK_PATH_CHECKED) { + skipped_path = true; + continue; + } + oldpriority = pp->priority; + conf = get_multipath_config(); + pthread_cleanup_push(put_multipath_config, conf); + pathinfo(pp, conf, DI_PRIO); + pthread_cleanup_pop(1); + if (pp->priority != oldpriority) + changed = true; + } } - - if (pp->priority != oldpriority) - changed = 1; - else if (!force_refresh_all) - return 0; - - vector_foreach_slot (pp->mpp->pg, pgp, i) { - vector_foreach_slot (pgp->paths, pp1, j) { - if (pp1 == pp || pp1->state == PATH_DOWN) + if (!changed || !skipped_path) + return changed; + /* + * If a path changed priorities, refresh the priorities of any + * paths we skipped + */ + vector_foreach_slot (mpp->pg, pgp, i) { + vector_foreach_slot (pgp->paths, pp, j) { + if (pp->state == PATH_DOWN) + continue; + if (pp->is_checked == CHECK_PATH_CHECKED) continue; - oldpriority = pp1->priority; conf = get_multipath_config(); pthread_cleanup_push(put_multipath_config, conf); - pathinfo(pp1, conf, DI_PRIO); + pathinfo(pp, conf, DI_PRIO); pthread_cleanup_pop(1); - if (pp1->priority != oldpriority) - changed = 1; } } - return changed; + return true; } static int reload_map(struct vectors *vecs, struct multipath *mpp, @@ -2393,14 +2428,12 @@ static int update_path_state (struct vectors * vecs, struct path * pp) { int newstate; - int new_path_up = 0; int chkr_new_path_up = 0; int disable_reinstate = 0; int oldchkrstate = pp->chkrstate; unsigned int checkint, max_checkint; struct config *conf; - int marginal_pathgroups, marginal_changed = 0; - bool need_reload; + int marginal_pathgroups; conf = get_multipath_config(); checkint = conf->checkint; @@ -2462,7 +2495,7 @@ update_path_state (struct vectors * vecs, struct path * pp) } if (!pp->marginal) { pp->marginal = 1; - marginal_changed = 1; + pp->mpp->prio_update = PRIO_UPDATE_MARGINAL; } } else { if (pp->marginal || pp->state == PATH_DELAYED) @@ -2470,7 +2503,7 @@ update_path_state (struct vectors * vecs, struct path * pp) pp->dev); if (marginal_pathgroups && pp->marginal) { pp->marginal = 0; - marginal_changed = 1; + pp->mpp->prio_update = PRIO_UPDATE_MARGINAL; } } } @@ -2537,7 +2570,8 @@ update_path_state (struct vectors * vecs, struct path * pp) */ if (!disable_reinstate) reinstate_path(pp); - new_path_up = 1; + if (pp->mpp->prio_update != PRIO_UPDATE_MARGINAL) + pp->mpp->prio_update = PRIO_UPDATE_NEW_PATH; if (oldchkrstate != PATH_UP && oldchkrstate != PATH_GHOST) chkr_new_path_up = 1; @@ -2588,38 +2622,48 @@ update_path_state (struct vectors * vecs, struct path * pp) LOG_MSG(2, pp); } } - + if (pp->mpp->prio_update == PRIO_UPDATE_NONE && + (newstate == PATH_UP || newstate == PATH_GHOST)) + pp->mpp->prio_update = PRIO_UPDATE_NORMAL; pp->state = newstate; + return chkr_new_path_up ? CHECK_PATH_NEW_UP : CHECK_PATH_CHECKED; +} - if (pp->mpp->wait_for_udev) - return CHECK_PATH_CHECKED; - /* - * path prio refreshing - */ - condlog(4, "path prio refresh"); - - if (marginal_changed) { - update_prio(pp, 1); - reload_and_sync_map(pp->mpp, vecs); - } else if (update_prio(pp, new_path_up) && - pp->mpp->pgpolicyfn == (pgpolicyfn *)group_by_prio && - pp->mpp->pgfailback == -FAILBACK_IMMEDIATE) { +static int +update_mpp_prio(struct vectors *vecs, struct multipath *mpp) +{ + bool need_reload, changed; + enum prio_update_type prio_update = mpp->prio_update; + mpp->prio_update = PRIO_UPDATE_NONE; + + if (mpp->wait_for_udev || prio_update == PRIO_UPDATE_NONE) + return 0; + condlog(4, "prio refresh"); + + changed = update_prio(mpp, prio_update != PRIO_UPDATE_NORMAL); + if (prio_update == PRIO_UPDATE_MARGINAL) + return reload_and_sync_map(mpp, vecs); + if (changed && mpp->pgpolicyfn == (pgpolicyfn *)group_by_prio && + mpp->pgfailback == -FAILBACK_IMMEDIATE) { condlog(2, "%s: path priorities changed. reloading", - pp->mpp->alias); - reload_and_sync_map(pp->mpp, vecs); - } else if (need_switch_pathgroup(pp->mpp, &need_reload)) { - if (pp->mpp->pgfailback > 0 && - (new_path_up || pp->mpp->failback_tick <= 0)) - pp->mpp->failback_tick = pp->mpp->pgfailback + 1; - else if (pp->mpp->pgfailback == -FAILBACK_IMMEDIATE || - (chkr_new_path_up && followover_should_failback(pp))) { + mpp->alias); + return reload_and_sync_map(mpp, vecs); + } + if (need_switch_pathgroup(mpp, &need_reload)) { + if (mpp->pgfailback > 0 && + (prio_update == PRIO_UPDATE_NEW_PATH || + mpp->failback_tick <= 0)) + mpp->failback_tick = mpp->pgfailback + 1; + else if (mpp->pgfailback == -FAILBACK_IMMEDIATE || + (prio_update == PRIO_UPDATE_NEW_PATH && + followover_should_failback(mpp))) { if (need_reload) - reload_and_sync_map(pp->mpp, vecs); + return reload_and_sync_map(mpp, vecs); else - switch_pathgroup(pp->mpp); + switch_pathgroup(mpp); } } - return CHECK_PATH_CHECKED; + return 0; } static int @@ -2856,7 +2900,7 @@ update_paths(struct vectors *vecs, int *num_paths_p, time_t start_secs) i--; else { pp->is_checked = rc; - if (rc == CHECK_PATH_CHECKED) + if (rc == CHECK_PATH_CHECKED || rc == CHECK_PATH_NEW_UP) (*num_paths_p)++; } if (++paths_checked % 128 == 0 && @@ -2929,8 +2973,10 @@ checkerloop (void *ap) vector_foreach_slot(vecs->mpvec, mpp, i) mpp->synced_count = 0; if (checker_state == CHECKER_STARTING) { - vector_foreach_slot(vecs->mpvec, mpp, i) + vector_foreach_slot(vecs->mpvec, mpp, i) { sync_mpp(vecs, mpp, ticks); + mpp->prio_update = PRIO_UPDATE_NONE; + } vector_foreach_slot(vecs->pathvec, pp, i) pp->is_checked = CHECK_PATH_UNCHECKED; checker_state = CHECKER_CHECKING_PATHS; @@ -2940,6 +2986,14 @@ checkerloop (void *ap) if (checker_state == CHECKER_UPDATING_PATHS) checker_state = update_paths(vecs, &num_paths, start_time.tv_sec); + if (checker_state == CHECKER_FINISHED) { + vector_foreach_slot(vecs->mpvec, mpp, i) { + if (update_mpp_prio(vecs, mpp) == 2) { + /* multipath device deleted */ + i--; + } + } + } lock_cleanup_pop(vecs->lock); if (checker_state != CHECKER_FINISHED) { /* Yield to waiters */ From patchwork Tue Oct 15 03:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835708 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9C7C170A29 for ; Tue, 15 Oct 2024 03:28:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; cv=none; b=aKwiGioBe7DvprUu/dGJsh6MVkrgMEZ1HqT0IggG6pS0w9u8860ZT2nmmM7O03XfD8OKzqk6TEBbTrBhVW4/6OT4oez2KhoPw1FHjIyTmNxdQlC75MnIcfeWiUuYeY3N3WV6G4KOmJJjJY/o2gMHxzaMwleP0QcCCKG2fdR5JCI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; c=relaxed/simple; bh=TVueBc3knMc5YbTSIAYV3r+I9/3bai/9AUTG9nnnTEo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BTRgJMIuyvy30Q+jD+Fa2NhBAtWisseAK+g6BlJeGbWtRFP1CZRi9JzyRdh4sHUrXG4MD94xdfRhpNJH4BYtjKA1+tR7LxnX+LHfb4bJHR1JikKKd2aXvIpN5zHqMmfbYiPo68KAdAdgK0P1gCFsp/t+3rHdVOnd1wJ7Dps12cM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hbudSlWx; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hbudSlWx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GRzj54dB3uT+CxggsXo/PP5NtZgRtoGn5Aag07Cnl9A=; b=hbudSlWxPu/bqE2O/mEtxnzqOvH0JXmJMxTfrfK+KDXWB4eh8Dq+UO3HkLQJNl+G3wR98G 1n/FsRaPt6gjtfyEatLX8ThY9HDXSSldVgQ/I+chVx7gfrHAkxgksyh1Bs35IO0qfi98+5 Z+qT5WhWUXXL+FHT2N4d3cf5e6nQwrU= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-339-uGhWESlDO6uHrR5S4Pb73w-1; Mon, 14 Oct 2024 23:28:40 -0400 X-MC-Unique: uGhWESlDO6uHrR5S4Pb73w-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5619B195608C; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2110719560A2; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sbwc2693339 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:37 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SbjX2693338; Mon, 14 Oct 2024 23:28:37 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 15/23] multipathd: simplify checking for followover_should_failback Date: Mon, 14 Oct 2024 23:28:27 -0400 Message-ID: <20241015032835.2693247-16-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com The code needs to check that pp->pgindex equals pp->mpp->bestpg and that they aren't both 0 (which is an invalid pathgroup id). Since we're already checking that they are equal, we only need to check one of them to see if it's non-zero. Instead of checking if pp->pgindex is non-zero for every path, just check mpp->bestpg once for the multipath device and return immediately if it's zero. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- multipathd/main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index f48dbe25..ea4adac8 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2002,7 +2002,7 @@ do_followover_should_failback(struct path * pp) struct path *pp1; int i; - if (!pp->pgindex || pp->pgindex != pp->mpp->bestpg) + if (pp->pgindex != pp->mpp->bestpg) return 0; pgp = VECTOR_SLOT(pp->mpp->pg, pp->pgindex - 1); @@ -2022,7 +2022,7 @@ followover_should_failback(struct multipath *mpp) struct pathgroup * pgp; int i, j; - if (mpp->pgfailback != -FAILBACK_FOLLOWOVER || !mpp->pg) + if (mpp->pgfailback != -FAILBACK_FOLLOWOVER || !mpp->pg || !mpp->bestpg) return 0; vector_foreach_slot (mpp->pg, pgp, i) { From patchwork Tue Oct 15 03:28:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835705 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABC2017107F for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; cv=none; b=RUJHlRF75Z7vjiPjSSOayRaXtKCdafdatJI/NCtq6al4YYliQD7dnjbOlSgRNy3ombbBQ9cKvhYkoZcW/r1cPwGl9wSQB/7HvO7h6DIMr/hzl6QEz7A0jH9E+eA4DfeF5QtJbmBaOZNOaihLI4Ex26ygb1ceLYwvREtoIWcCFOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; c=relaxed/simple; bh=Ama3sgic4Q55+LMItVyZvetCA/GkdYvfV2UOI8OnrTo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GYMKrQcqQasLCahmg5tq91PNgdhLf66QSirgKsRCvXb4H6vZEFFIB1mo0SPwAWt9guJMRwgHe/kqykDvzP36ngrOIPnrr6RFMOfFpTkIJUKylSOjikee0A2yqbmxLrbKRvFGKHcW2/ClffHkvhaWXufWy5JtVopKw78CuA9xcbQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UM9NGpNN; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UM9NGpNN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Md4emmpgtWKs6xlxe1zNgmqs5la/lqParoon+C+2nh0=; b=UM9NGpNNzYrQVnNO0hQ821gcE6VaW747m7JFKmwufV59h6l6L7+Lz5eA6mzAXBWYS8Zfo1 IOXhPqY9wTrsLyowuxC0CTII8iLjoBQme7xxI+2QCqbunApYHy4InLvOqTNK12WhbUSK5p lGgfhyhac1lkPIio7k0IggfaE+6HqCM= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-495-2xmCHBjyNGGTkRFZ7wq4Zg-1; Mon, 14 Oct 2024 23:28:40 -0400 X-MC-Unique: 2xmCHBjyNGGTkRFZ7wq4Zg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9F7B5195608F; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4879C19560AE; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3ScNj2693343 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:38 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3Scpg2693342; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 16/23] multipathd: only refresh prios on PATH_UP and PATH_GHOST Date: Mon, 14 Oct 2024 23:28:28 -0400 Message-ID: <20241015032835.2693247-17-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com The way that multipathd was handling priority refreshes had some issues. Some of the multipath prioritizers can hang when run on a failed path. Multipathd was only skipping paths in PATH_DOWN, but there are other states where the prioritizer is also likely to hang, such as PATH_TIMEOUT, PATH_SHAKY, and to a lesser extent, PATH_DELAYED. Also, before the recent patch splitting the priority updating from the path checking, multipathd wasn't consistent with which states would cause paths to get their priorities updated. If a path changed its state to anything other than PATH_UP or PATH_GHOST, it wouldn't get its priority updated. But if a path kept the same state its priority would get updated as long at the state wasn't PATH_DOWN. For safety's sake, a path's priority should only get refreshed when its in the PATH_UP or PATH_GHOST state. This shouldn't cause problems. Only paths that are in the PATH_UP or PATH_GHOST state are usable by the kenel and contibute to the pathgroup's priority. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- multipathd/main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index ea4adac8..d479d0c0 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2162,7 +2162,7 @@ static bool update_prio(struct multipath *mpp, bool refresh_all) vector_foreach_slot (mpp->pg, pgp, i) { vector_foreach_slot (pgp->paths, pp, j) { - if (pp->state == PATH_DOWN) + if (pp->state != PATH_UP && pp->state != PATH_GHOST) continue; /* * refresh_all will be set if the mpp has any path @@ -2191,7 +2191,7 @@ static bool update_prio(struct multipath *mpp, bool refresh_all) */ vector_foreach_slot (mpp->pg, pgp, i) { vector_foreach_slot (pgp->paths, pp, j) { - if (pp->state == PATH_DOWN) + if (pp->state != PATH_UP && pp->state != PATH_GHOST) continue; if (pp->is_checked == CHECK_PATH_CHECKED) continue; From patchwork Tue Oct 15 03:28:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835696 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABC70171092 for ; Tue, 15 Oct 2024 03:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; cv=none; b=g8uJW3svMFeRWJRjE6+FyKFHNUxakvMU/8Z9j0IdahNiF2IhVHUXZWNnTo5gZrueuIUDsctUBKkQAO6Yp8WXbXQxs4pINUweL4O91sBd09gjgK+QYMSTbGPvF3nh28fZ/IQ6g5PpNXu18xWec07tS+gdMOlOaLFnpodBulOSDVk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962924; c=relaxed/simple; bh=fogHgKkf06/XAZau6gZhQRwgh5+C538jFtU/BkSksRQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=vC+mEwnmSfLVrQbtMNMFk3qxFf2OPfc0nUsqRwMx2NS7jAzynoI9gpyPLIVv9I2sZcF91CEiDkQfczb1i21Wf+uAMAfUfli4qZV8n+62eEWkmm7x3LIrgmzy2Fpsr8M+cIRYZyDLpcnDSkTONqliquEQ7YPkA+r/w2LgrAhzS58= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=F/KzRx84; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="F/KzRx84" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0u3XKLQL68D5VjpSMku/sQH35STdRSYAt+YMguSAzLM=; b=F/KzRx84DsR42Hch25bw/vJUUFSwpeSGGvDAoPkHeHbt7IxmWRPlEF/1KHjM1bHxtjBZlO /+YyUmz1bwYI9328BlZWj2OnvJqUknTtRImsF9dekTbsfw9++Fsgp3XoL0aTdfrJx/Htyp KiE0jilHM34PFlsKrMUwNJ51cCxH7vE= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-495-eWd8tcHaODySHtEkXJC4RA-1; Mon, 14 Oct 2024 23:28:40 -0400 X-MC-Unique: eWd8tcHaODySHtEkXJC4RA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B120919560B4; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 71F571956089; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3ScOa2693347 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:38 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3ScOZ2693346; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 17/23] multipathd: remove pointless check Date: Mon, 14 Oct 2024 23:28:29 -0400 Message-ID: <20241015032835.2693247-18-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Since we just returned if newstate wasn't PATH_UP or PATH_GHOST, it must obviously be PATH_UP or PATH_GHOST at this point in the code. We don't even wrap all the code for dealing with a path that just came up in this if-block, It's only the persistent resrvation code. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- multipathd/main.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index d479d0c0..ec172021 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2550,19 +2550,19 @@ update_path_state (struct vectors * vecs, struct path * pp) return CHECK_PATH_CHECKED; } - if (newstate == PATH_UP || newstate == PATH_GHOST) { - if (pp->mpp->prflag != PRFLAG_UNSET) { - int prflag = pp->mpp->prflag; - /* - * Check Persistent Reservation. - */ - condlog(2, "%s: checking persistent " - "reservation registration", pp->dev); - mpath_pr_event_handle(pp); - if (pp->mpp->prflag == PRFLAG_SET && - prflag != PRFLAG_SET) - pr_register_active_paths(pp->mpp); - } + /* newstate == PATH_UP || newstate == PATH_GHOST */ + + if (pp->mpp->prflag != PRFLAG_UNSET) { + int prflag = pp->mpp->prflag; + /* + * Check Persistent Reservation. + */ + condlog(2, "%s: checking persistent " + "reservation registration", pp->dev); + mpath_pr_event_handle(pp); + if (pp->mpp->prflag == PRFLAG_SET && + prflag != PRFLAG_SET) + pr_register_active_paths(pp->mpp); } /* From patchwork Tue Oct 15 03:28:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835709 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C057D172767 for ; Tue, 15 Oct 2024 03:28:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; cv=none; b=DjucgqiI2pV6lOtinDimSMilOBROn78IuzFs+1z6t/PQy+xACDIRBqNv2LCg/YsRjFK5D4l5YCYqeq7SxkLiSD8+nZVOmwJ6SniRjJ9+WeBo0Qk1Eof6GWaP1cKRMYww4g0r9xh3pNFANR6et6YkgWCobba55+LiIbXk7Gz6WRA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; c=relaxed/simple; bh=zaNexoUneOEQPf+9ysIfEbZaWN6PLoQIcIse4OA0nXs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=i6qAwTRsZzD75Ld0W7tjS3wBt2l9wtuTV8MAYxMRuu0hodNfbpEZH4+XaV7Qk/PCzlCJECPIUQPMuDTuVxxNsvrzHVbiXIK2jbI8TDs9U1d19pgLFKIrZjaMYixMF8qKsU6bXwlG6kyvt7jOx+r7KjZPodSDpqxWtrj8K55MjNM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fwfTe7r8; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fwfTe7r8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kR0kv7vtx28nx1HwYUaVB0jOK/P9d+OGolWX1vzDtWg=; b=fwfTe7r8Jywo/wxq7XLRZ/5uwAAHinXgG19QUaw9HbRkpvp07KG4DusijIpdzAxB079rzf +p1cq3fgp8R+2z6UHqmr2TnM/n6Jevh2JeNp6YxtAEQKOTyDNT3GzgT/fI1U2/Ww19T1WJ 758zh9tC3kjipOGGre+gBIFH7ta4JiA= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-516-3QE-Q-feOfK9F1Zn28uFcw-1; Mon, 14 Oct 2024 23:28:40 -0400 X-MC-Unique: 3QE-Q-feOfK9F1Zn28uFcw-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AFA8A1955EE6; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 78F4D3000198; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3ScT82693351 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:38 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3ScIP2693350; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 18/23] multipathd: fix deferred_failback_tick for reload removes Date: Mon, 14 Oct 2024 23:28:30 -0400 Message-ID: <20241015032835.2693247-19-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com If reload_and_sync_map() removes the multipath device, deferred_failback_tick() needs to decrement the counter so that it doesn't skip the following device. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- multipathd/main.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index ec172021..846be5a9 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2098,9 +2098,12 @@ deferred_failback_tick (struct vectors *vecs) if (!mpp->failback_tick && need_switch_pathgroup(mpp, &need_reload)) { - if (need_reload) - reload_and_sync_map(mpp, vecs); - else + if (need_reload) { + if (reload_and_sync_map(mpp, vecs) == 2) { + /* multipath device removed */ + i--; + } + } else switch_pathgroup(mpp); } } From patchwork Tue Oct 15 03:28:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835704 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE6D51714C4 for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; cv=none; b=GZ0Rj/BRSRWhzwNkMXrPHWSBWft452H/+dvbmvbP4zU16fDGV2Wimf2vJ9hAs9G3oDCOH99tUmylNNRmupgSECbssWOvcbeSLAktFVkna9oM1V3sFkNLiyfYI5lFwldCDdu6d7c8mQ+N15z3YVSD/ctC5aYvnx98aIrZIdFLz/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; c=relaxed/simple; bh=carlqwuHvzs4hWAk5yJVCrw1UpAvroQeY7+FaTa3NqA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=XAD0QTC+Jr7lxk5ViN1HMQOHNkR3XoMV6oXLZH9QaaDkcgJ1toi9Iqtb5H/BILF4YhvhiWZLwUGZeHdsFbC8ERTTtBYTC0BWBMw7DRKAlIyPYmjU1Z4z+Nsj2Zmg1nEG0FF5pFI3xE7yppAHJtSo8X4WVfQwuy8CYkvBdBjOUaM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=B6SHz7rC; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="B6SHz7rC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k6iSHZPtUFiEmqZCod3IM3UIOhtJux4wqtbcI3I8ouc=; b=B6SHz7rC+38uXDqjCxKtTJhK1T4ivDufAer+fl9GnJ4cdnNTEvFBAbCSqx68bDDJs5To7B QNZj+hJI3DGyVBM+oACY34pBhSJOg5VqM/cTdUbxx+QZN50yrD2UpHSZM2u6XIuTZHbbeU sDVatMEeIbHfuHvr5Nmf0l2XteuOKKY= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-369-sPubHlDDMRWV6oIqGCNUng-1; Mon, 14 Oct 2024 23:28:40 -0400 X-MC-Unique: sPubHlDDMRWV6oIqGCNUng-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BA7A01955EE8; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8400D19560A2; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sct02693355 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:38 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3ScGU2693354; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 19/23] libmultipath: add libcheck_need_wait checker function Date: Mon, 14 Oct 2024 23:28:31 -0400 Message-ID: <20241015032835.2693247-20-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Add a new optional checker class function, libcheck_need_wait() and a new function to call it, checker_need_wait(). This can be used to see if a path_checker is currently running. This will be used to determine if there are pending checkers that need to be waited for. Signed-off-by: Benjamin Marzinski Reviewed-by: Martin Wilck --- libmultipath/checkers.c | 12 +++++++++++- libmultipath/checkers.h | 4 +++- libmultipath/checkers/directio.c | 11 +++++++++++ libmultipath/checkers/tur.c | 11 +++++++++++ 4 files changed, 36 insertions(+), 2 deletions(-) diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c index f3e98352..e2eda58d 100644 --- a/libmultipath/checkers.c +++ b/libmultipath/checkers.c @@ -27,6 +27,7 @@ struct checker_class { void (*reset)(void); /* to reset the global variables */ void *(*thread)(void *); /* async thread entry point */ int (*pending)(struct checker *); /* to recheck pending paths */ + bool (*need_wait)(struct checker *); /* checker needs waiting for */ const char **msgtable; short msgtable_size; }; @@ -182,7 +183,8 @@ static struct checker_class *add_checker_class(const char *name) c->reset = (void (*)(void)) dlsym(c->handle, "libcheck_reset"); c->thread = (void *(*)(void*)) dlsym(c->handle, "libcheck_thread"); c->pending = (int (*)(struct checker *)) dlsym(c->handle, "libcheck_pending"); - /* These 4 functions can be NULL. call dlerror() to clear out any + c->need_wait = (bool (*)(struct checker *)) dlsym(c->handle, "libcheck_need_wait"); + /* These 5 functions can be NULL. call dlerror() to clear out any * error string */ dlerror(); @@ -313,6 +315,14 @@ int checker_get_state(struct checker *c) return c->path_state; } +bool checker_need_wait(struct checker *c) +{ + if (!c || !c->cls || c->path_state != PATH_PENDING || + !c->cls->need_wait) + return false; + return c->cls->need_wait(c); +} + void checker_check (struct checker * c, int path_state) { if (!c) diff --git a/libmultipath/checkers.h b/libmultipath/checkers.h index b2342a1b..da91f499 100644 --- a/libmultipath/checkers.h +++ b/libmultipath/checkers.h @@ -2,6 +2,7 @@ #define CHECKERS_H_INCLUDED #include +#include #include "list.h" #include "defaults.h" @@ -171,6 +172,7 @@ struct checker_context { int start_checker_thread (pthread_t *thread, const pthread_attr_t *attr, struct checker_context *ctx); int checker_get_state(struct checker *c); +bool checker_need_wait(struct checker *c); void checker_check (struct checker *, int); int checker_is_sync(const struct checker *); const char *checker_name (const struct checker *); @@ -191,7 +193,7 @@ void *libcheck_thread(struct checker_context *ctx); void libcheck_reset(void); int libcheck_mp_init(struct checker *); int libcheck_pending(struct checker *c); - +bool libcheck_need_wait(struct checker *c); /* * msgid => message map. diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index 904e3071..54c12f7b 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -65,6 +65,7 @@ struct directio_context { struct aio_group *aio_grp; struct async_req *req; struct timespec endtime; + bool checked_state; }; static struct aio_group * @@ -295,6 +296,7 @@ check_pending(struct directio_context *ct, struct timespec endtime) int r; struct timespec currtime, timeout; + ct->checked_state = true; while(1) { get_monotonic_time(&currtime); timespecsub(&endtime, &currtime, &timeout); @@ -327,6 +329,7 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) LOG(4, "called in synchronous mode"); if (ct->running) { + ct->checked_state = true; if (ct->req->state != PATH_PENDING) { ct->running = 0; return ct->req->state; @@ -346,6 +349,7 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) get_monotonic_time(&ct->endtime); ct->endtime.tv_nsec += 1000 * 1000; normalize_timespec(&ct->endtime); + ct->checked_state = false; } ct->running++; if (!sync) @@ -386,6 +390,13 @@ static void set_msgid(struct checker *c, int state) } } +bool libcheck_need_wait(struct checker *c) +{ + struct directio_context *ct = (struct directio_context *)c->context; + return (ct && ct->running && ct->req->state == PATH_PENDING && + !ct->checked_state); +} + int libcheck_pending(struct checker *c) { int rc; diff --git a/libmultipath/checkers/tur.c b/libmultipath/checkers/tur.c index 81db565b..f7e6a483 100644 --- a/libmultipath/checkers/tur.c +++ b/libmultipath/checkers/tur.c @@ -59,6 +59,7 @@ struct tur_checker_context { struct checker_context ctx; unsigned int nr_timeouts; struct timespec endtime; + bool checked_state; }; int libcheck_init (struct checker * c) @@ -351,9 +352,17 @@ int check_pending(struct checker *c) ct->thread = 0; } + ct->checked_state = true; return tur_status; } +bool libcheck_need_wait(struct checker *c) +{ + struct tur_checker_context *ct = c->context; + return (ct && ct->thread && uatomic_read(&ct->running) != 0 && + !ct->checked_state); +} + int libcheck_pending(struct checker *c) { struct tur_checker_context *ct = c->context; @@ -381,6 +390,7 @@ int libcheck_check(struct checker * c) * Async mode */ if (ct->thread) { + ct->checked_state = true; if (tur_check_async_timeout(c)) { int running = uatomic_xchg(&ct->running, 0); if (running) { @@ -463,6 +473,7 @@ int libcheck_check(struct checker * c) pthread_mutex_unlock(&ct->lock); ct->fd = c->fd; ct->timeout = c->timeout; + ct->checked_state = false; uatomic_add(&ct->holders, 1); uatomic_set(&ct->running, 1); tur_set_async_timeout(c); From patchwork Tue Oct 15 03:28:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835702 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A7E81714BE for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; cv=none; b=DR2Jc5RViIO0yNh1IoSrOAwE/CUWq6SFsBj5UxkoT8x4dqBJWAECIWZpekMfixeESssBAujIum06PKfJ2uqIJFTKRcaISYyql5OFxjjYPvcUOxyNyE6LBrphYXv64CaJy0Ic8BvNL/GrUuw/2ucfvCePaze8gNey8Idv3Gz6uQ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962925; c=relaxed/simple; bh=mA9bug/wjM6NVN3hGIgpfQ5/c+bEOey9J9VlY6D2yqY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lAtslNJJfbhVg809FSoJGWCTuK4BWjSh8mDVrB6SUBKlmr7nvnNV0tMXY5MXQE3llaZkKF1+vYIa/YW/MLawJX1NlpFlQAflyxnskAU66PMCm6BYo66rc712te6Z+k5+bfaBhBbsXUcpsIQ0vqDQfoP32GBAReD/8+poIII9sIA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HoDXJ0CM; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HoDXJ0CM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X+5TCCZC21OsLunQEarU+AbAt+6HTEvgDbEmt+d8pas=; b=HoDXJ0CMD3JQLpCuJqokCxNGJOkqDPnlXRydWkx5jrQH3NHQE2FOB6pZc1XA20CsDfGbO+ kbzTcaanfcYfw8KkRRNfZiyuWwMMsmLWjNxsHr6L/mIDPrTmwKHTlYv6d8Lpqf28NVFH34 KMoM1FcscjdMwpMk6x+gR+W2mo0kXZs= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-499--MDOB4QrNN2ZZTHzNfB6bQ-1; Mon, 14 Oct 2024 23:28:41 -0400 X-MC-Unique: -MDOB4QrNN2ZZTHzNfB6bQ-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0717819560BE; Tue, 15 Oct 2024 03:28:40 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C48B519560A2; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3ScZS2693359 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:38 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3Sc3L2693358; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 20/23] libmultipath: don't wait in libcheck_pending Date: Mon, 14 Oct 2024 23:28:32 -0400 Message-ID: <20241015032835.2693247-21-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Disable waiting in the libcheck_pending() checker class functions. Instead, they now just check if the checker has already completed. A future patch will re-add waiting outside of libcheck_pending(). Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/checkers/directio.c | 41 ++++++++++++++++---------------- libmultipath/checkers/tur.c | 20 +++------------- 2 files changed, 23 insertions(+), 38 deletions(-) diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index 54c12f7b..27227894 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -64,7 +64,6 @@ struct directio_context { int reset_flags; struct aio_group *aio_grp; struct async_req *req; - struct timespec endtime; bool checked_state; }; @@ -291,18 +290,17 @@ get_events(struct aio_group *aio_grp, struct timespec *timeout) } static void -check_pending(struct directio_context *ct, struct timespec endtime) +check_pending(struct directio_context *ct, struct timespec timeout) { int r; - struct timespec currtime, timeout; + struct timespec endtime, currtime; ct->checked_state = true; + get_monotonic_time(&endtime); + endtime.tv_sec += timeout.tv_sec; + endtime.tv_nsec += timeout.tv_nsec; + normalize_timespec(&endtime); while(1) { - get_monotonic_time(&currtime); - timespecsub(&endtime, &currtime, &timeout); - if (timeout.tv_sec < 0) - timeout.tv_sec = timeout.tv_nsec = 0; - r = get_events(ct->aio_grp, &timeout); if (ct->req->state != PATH_PENDING) { @@ -311,6 +309,10 @@ check_pending(struct directio_context *ct, struct timespec endtime) } else if (r == 0 || (timeout.tv_sec == 0 && timeout.tv_nsec == 0)) return; + get_monotonic_time(&currtime); + timespecsub(&endtime, &currtime, &timeout); + if (timeout.tv_sec < 0) + timeout.tv_sec = timeout.tv_nsec = 0; } } @@ -320,7 +322,7 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) struct stat sb; int rc; struct io_event event; - struct timespec endtime; + struct timespec timeout = { .tv_sec = timeout_secs }; if (fstat(fd, &sb) == 0) { LOG(4, "called for %x", (unsigned) sb.st_rdev); @@ -346,20 +348,13 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) LOG(3, "io_submit error %i", -rc); return PATH_UNCHECKED; } - get_monotonic_time(&ct->endtime); - ct->endtime.tv_nsec += 1000 * 1000; - normalize_timespec(&ct->endtime); ct->checked_state = false; } ct->running++; if (!sync) return PATH_PENDING; - get_monotonic_time(&endtime); - endtime.tv_sec += timeout_secs; - normalize_timespec(&endtime); - - check_pending(ct, endtime); + check_pending(ct, timeout); if (ct->req->state != PATH_PENDING) return ct->req->state; @@ -402,13 +397,16 @@ int libcheck_pending(struct checker *c) int rc; struct io_event event; struct directio_context *ct = (struct directio_context *)c->context; + struct timespec no_wait = { .tv_sec = 0 }; /* The if path checker isn't running, just return the exiting value. */ - if (!ct || !ct->running) - return c->path_state; + if (!ct || !ct->running) { + rc = c->path_state; + goto out; + } if (ct->req->state == PATH_PENDING) - check_pending(ct, ct->endtime); + check_pending(ct, no_wait); else ct->running = 0; rc = ct->req->state; @@ -421,8 +419,9 @@ int libcheck_pending(struct checker *c) else LOG(4, "async io pending"); } - set_msgid(c, rc); +out: + set_msgid(c, rc); return rc; } diff --git a/libmultipath/checkers/tur.c b/libmultipath/checkers/tur.c index f7e6a483..e70a2e11 100644 --- a/libmultipath/checkers/tur.c +++ b/libmultipath/checkers/tur.c @@ -58,7 +58,6 @@ struct tur_checker_context { int msgid; struct checker_context ctx; unsigned int nr_timeouts; - struct timespec endtime; bool checked_state; }; @@ -299,14 +298,6 @@ void *libcheck_thread(struct checker_context *ctx) return ((void *)0); } - -static void tur_timeout(struct timespec *tsp) -{ - get_monotonic_time(tsp); - tsp->tv_nsec += 1000 * 1000; /* 1 millisecond */ - normalize_timespec(tsp); -} - static void tur_set_async_timeout(struct checker *c) { struct tur_checker_context *ct = c->context; @@ -328,16 +319,12 @@ static int tur_check_async_timeout(struct checker *c) int check_pending(struct checker *c) { struct tur_checker_context *ct = c->context; - int r, tur_status = PATH_PENDING; + int tur_status = PATH_PENDING; pthread_mutex_lock(&ct->lock); - for (r = 0; - r == 0 && ct->state == PATH_PENDING && - ct->msgid == MSG_TUR_RUNNING; - r = pthread_cond_timedwait(&ct->active, &ct->lock, &ct->endtime)); - - if (!r) { + if (ct->state != PATH_PENDING || ct->msgid != MSG_TUR_RUNNING) + { tur_status = ct->state; c->msgid = ct->msgid; } @@ -488,7 +475,6 @@ int libcheck_check(struct checker * c) " sync mode", major(ct->devt), minor(ct->devt)); return tur_check(c->fd, c->timeout, &c->msgid); } - tur_timeout(&ct->endtime); } return tur_status; From patchwork Tue Oct 15 03:28:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835714 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C362171E65 for ; Tue, 15 Oct 2024 03:28:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; cv=none; b=QXN3Dwku//W2aMmQ2F1KdzwMdssHVSretrpS+XoaGZ0p5nya85/xNZ4gu/9TzlmdHzQA0zhli3A3M77Q1fxiMW6EpDrqW3DnCwOVgzMlH3w1stO9bBwu3bpzTjZH5+C8EtpzQdpLyMQayLLZ6O23a0pCYlq8WleRFYMb6AnUKBA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; c=relaxed/simple; bh=wQ1nePph6dUXU7bvFvowIYwNCYq8PFKyHFWr8UlTcHI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GTFi61+OgS7un6kAxTQkGbexOGAHKvqI6OPa61DpWyXEdAa5d79qlgaLTE/DAnVmaB7NhHUngNZglQ+rcI47t1Uupn/DxV4WyqF43u5vpKkrhNzAHSLAt/sx7efyVO8j5QW5sbgqDJzh4u4m4R4Ev2spNsUaDTSeXWiHdxGxsv8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=g3I+eXCq; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g3I+eXCq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i/IF6FkL0st3WJlofb0F9sBmzyu28qPBTHH4HvimkHs=; b=g3I+eXCqxv80jWNurxX3Gr3oEYZOtmmQS0qQJPc/M9OTjXIo0ft+QWHnNHaAhXtp0F1Dfd KcGOssQp/gnKdctgeDcxCTI7jliMf94qoBJaolPjTArIwqNTmsYumbiN+mkZ0D5+oTdjak P5HCIuxjP8Y2fS4YvrfQBLaeKj15xS4= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-640-593OGlR_MZSdXvv0qEQmlw-1; Mon, 14 Oct 2024 23:28:41 -0400 X-MC-Unique: 593OGlR_MZSdXvv0qEQmlw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 45D0119560A5; Tue, 15 Oct 2024 03:28:40 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 102581955F43; Tue, 15 Oct 2024 03:28:39 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Scdw2693363 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:38 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3ScBj2693362; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 21/23] multipathd: wait for checkers to complete Date: Mon, 14 Oct 2024 23:28:33 -0400 Message-ID: <20241015032835.2693247-22-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Multipath again waits for running checkers to complete. In pathinfo(), mutipath will just wait 1ms if the checker is running. In checkerloop(), if there are running checkers, multipathd will drop the vecs lock and wait for 5ms. The difference it wait times is because multipathd cannot drop the vecs lock in pathinfo. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/discovery.c | 9 ++++++++- libmultipath/libmultipath.version | 1 + multipathd/main.c | 12 +++++++++++- 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c index 6ccdfa0b..1d48c30a 100644 --- a/libmultipath/discovery.c +++ b/libmultipath/discovery.c @@ -2467,8 +2467,15 @@ int pathinfo(struct path *pp, struct config *conf, int mask) if (mask & DI_CHECKER) { if (path_state == PATH_UP) { int newstate = PATH_UNCHECKED; - if (start_checker(pp, conf, 0, path_state) == 0) + if (start_checker(pp, conf, 0, path_state) == 0) { + if (checker_need_wait(&pp->checker)) { + struct timespec wait = { + .tv_nsec = 1000 * 1000, + }; + nanosleep(&wait, NULL); + } newstate = get_state(pp); + } if (newstate != PATH_PENDING || pp->state == PATH_UNCHECKED || pp->state == PATH_WILD) diff --git a/libmultipath/libmultipath.version b/libmultipath/libmultipath.version index 6439d3a7..c2e5f552 100644 --- a/libmultipath/libmultipath.version +++ b/libmultipath/libmultipath.version @@ -62,6 +62,7 @@ global: checker_enable; checker_message; checker_name; + checker_need_wait; checker_state_name; check_foreign; cleanup_bindings; diff --git a/multipathd/main.c b/multipathd/main.c index 846be5a9..e92f3ebd 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2850,6 +2850,7 @@ update_uninitialized_path(struct vectors * vecs, struct path * pp) enum checker_state { CHECKER_STARTING, CHECKER_CHECKING_PATHS, + CHECKER_WAITING_FOR_PATHS, CHECKER_UPDATING_PATHS, CHECKER_FINISHED, }; @@ -2861,6 +2862,7 @@ check_paths(struct vectors *vecs, unsigned int ticks) struct timespec diff_time, start_time, end_time; struct path *pp; int i; + bool need_wait = false; get_monotonic_time(&start_time); @@ -2871,6 +2873,9 @@ check_paths(struct vectors *vecs, unsigned int ticks) pp->is_checked = check_path(pp, ticks); else pp->is_checked = check_uninitialized_path(pp, ticks); + if (pp->is_checked == CHECK_PATH_STARTED && + checker_need_wait(&pp->checker)) + need_wait = true; if (++paths_checked % 128 == 0 && (lock_has_waiters(&vecs->lock) || waiting_clients())) { get_monotonic_time(&end_time); @@ -2879,7 +2884,7 @@ check_paths(struct vectors *vecs, unsigned int ticks) return CHECKER_CHECKING_PATHS; } } - return CHECKER_UPDATING_PATHS; + return need_wait ? CHECKER_WAITING_FOR_PATHS : CHECKER_UPDATING_PATHS; } static enum checker_state @@ -3001,6 +3006,11 @@ checkerloop (void *ap) if (checker_state != CHECKER_FINISHED) { /* Yield to waiters */ struct timespec wait = { .tv_nsec = 10000, }; + if (checker_state == CHECKER_WAITING_FOR_PATHS) { + /* wait 5ms */ + wait.tv_nsec = 5 * 1000 * 1000; + checker_state = CHECKER_UPDATING_PATHS; + } nanosleep(&wait, NULL); } } From patchwork Tue Oct 15 03:28:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835707 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CD17171650 for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; cv=none; b=Zkqo0UEtG8nTM8PxLhdKKOyTbyaday4whmkXNFwnDci0YeSeUnwLoKXHQenBhjhY7U/LDQvvv8ny+Ii4zOiCpizx5j+WXHWNLBxMjL6hYGjrhPuY78K/vJCBHq8aSKXqSl5aosBcwzhdkI0epj6biIQsiPc4IDetT1O/E7fYKJ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962926; c=relaxed/simple; bh=BzJngYBcYcGRH3T9avRonYEJrI9SngwMl5W5X84CrKU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=vALKdQfzlX7E9YCghebGJk6bt+69Ze0itCTYTf0ouqxP+bW6cinqbhGB2l+u9VU03Hl3uMm69JsEsnRu6KQG51Fj5jF2J0Cz73qlCkvRv7+8LDqbfzjrG+HqPPpTXimHvygaQyg2o2y/5tA+BgEeileOrPnChBJEta8sa7sFhH4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=iyA76Wbu; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iyA76Wbu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rQDEPPXMo3/Bb2SHhlH4+sI+igtzxVWyAt9dqVb547s=; b=iyA76Wbu+wvLTeyHpxIMH84H1bs1tpaqncHVIVBV9GMuNbknwNI5arbJqPPY7MYhuxQ+JR Id7Rw3eB6d4FPNXDPph3WyfUNfmzNkiRFzQ5VZge2vt4qp1IzLqKz732hrLpAmfrEKUEFz 4ehwj/kJoFM+45Twf53XJ6lq+jB/CxM= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-371-vmsAUfXKNRyVnwGbwIXg6A-1; Mon, 14 Oct 2024 23:28:41 -0400 X-MC-Unique: vmsAUfXKNRyVnwGbwIXg6A-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4C65D1956096; Tue, 15 Oct 2024 03:28:40 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1765B19560AE; Tue, 15 Oct 2024 03:28:40 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Scg72693367 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:39 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3ScZX2693366; Mon, 14 Oct 2024 23:28:38 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck , Martin Wilck Subject: [PATCH v5 22/23] multipath-tools tests: fix up directio tests Date: Mon, 14 Oct 2024 23:28:34 -0400 Message-ID: <20241015032835.2693247-23-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Make the directio tests work with no waiting in the libcheck_pending() and the new libcheck_need_wait() call. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- tests/directio.c | 49 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 46 insertions(+), 3 deletions(-) diff --git a/tests/directio.c b/tests/directio.c index 2e22f831..5ee482a8 100644 --- a/tests/directio.c +++ b/tests/directio.c @@ -313,23 +313,28 @@ static void test_init_reset_init(void **state) struct checker c = {.cls = NULL}; struct aio_group *aio_grp, *tmp_grp; + assert_int_equal(libcheck_need_wait(&c), false); assert_true(list_empty(&aio_grp_list)); will_return(__wrap_io_setup, 0); do_libcheck_init(&c, 4096, 0, NULL); + assert_int_equal(libcheck_need_wait(&c), false); aio_grp = get_aio_grp(&c); check_aio_grp(aio_grp, 1, 0); list_for_each_entry(tmp_grp, &aio_grp_list, node) assert_ptr_equal(aio_grp, tmp_grp); libcheck_free(&c); + assert_int_equal(libcheck_need_wait(&c), false); check_aio_grp(aio_grp, 0, 0); do_libcheck_reset(1); will_return(__wrap_io_setup, 0); do_libcheck_init(&c, 4096, 0, NULL); + assert_int_equal(libcheck_need_wait(&c), false); aio_grp = get_aio_grp(&c); check_aio_grp(aio_grp, 1, 0); list_for_each_entry(tmp_grp, &aio_grp_list, node) assert_ptr_equal(aio_grp, tmp_grp); libcheck_free(&c); + assert_int_equal(libcheck_need_wait(&c), false); check_aio_grp(aio_grp, 0, 0); do_libcheck_reset(1); } @@ -433,6 +438,7 @@ static void test_check_state_simple(void **state) do_libcheck_init(&c, 4096, 30, &req); return_io_getevents_nr(NULL, 1, &req, &res); do_check_state(&c, 1, PATH_UP); + assert_int_equal(libcheck_need_wait(&c), false); libcheck_free(&c); do_libcheck_reset(1); } @@ -465,17 +471,25 @@ static void test_check_state_async_timeout(void **state) do_libcheck_init(&c, 4096, 3, NULL); aio_grp = get_aio_grp(&c); do_check_state(&c, 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), true); return_io_getevents_none(); do_libcheck_pending(&c, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); do_check_state(&c, 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); return_io_getevents_none(); do_libcheck_pending(&c, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); do_check_state(&c, 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); return_io_getevents_none(); do_libcheck_pending(&c, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); do_check_state(&c, 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); return_io_getevents_none(); do_libcheck_pending(&c, PATH_DOWN); + assert_int_equal(libcheck_need_wait(&c), false); check_aio_grp(aio_grp, 1, 0); libcheck_free(&c); do_libcheck_reset(1); @@ -496,11 +510,16 @@ static void test_free_with_pending(void **state) aio_grp = get_aio_grp(c); do_check_state(&c[0], 0, PATH_PENDING); do_check_state(&c[1], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), true); + assert_int_equal(libcheck_need_wait(&c[1]), true); return_io_getevents_none(); do_libcheck_pending(&c[0], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), false); + assert_int_equal(libcheck_need_wait(&c[1]), true); return_io_getevents_nr(NULL, 1, &req, &res); - return_io_getevents_none(); do_libcheck_pending(&c[1], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), false); + assert_int_equal(libcheck_need_wait(&c[1]), false); assert_true(is_checker_running(&c[0])); assert_true(is_checker_running(&c[1])); check_aio_grp(aio_grp, 2, 0); @@ -523,8 +542,10 @@ static void test_orphaned_aio_group(void **state) for (i = 0; i < AIO_GROUP_SIZE; i++) { do_libcheck_init(&c[i], 4096, 30, NULL); do_check_state(&c[i], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[i]), true); return_io_getevents_none(); do_libcheck_pending(&c[i], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[i]), false); } aio_grp = get_aio_grp(c); check_aio_grp(aio_grp, AIO_GROUP_SIZE, 0); @@ -560,15 +581,19 @@ static void test_timeout_cancel_failed(void **state) aio_grp = get_aio_grp(c); return_io_getevents_none(); do_check_state(&c[0], 1, PATH_DOWN); + assert_int_equal(libcheck_need_wait(&c[0]), false); assert_true(is_checker_running(&c[0])); check_aio_grp(aio_grp, 2, 0); return_io_getevents_none(); do_check_state(&c[0], 1, PATH_DOWN); + assert_int_equal(libcheck_need_wait(&c[0]), false); assert_true(is_checker_running(&c[0])); return_io_getevents_nr(NULL, 1, &reqs[0], &res[0]); return_io_getevents_nr(NULL, 1, &reqs[1], &res[1]); do_check_state(&c[1], 1, PATH_UP); + assert_int_equal(libcheck_need_wait(&c[1]), false); do_check_state(&c[0], 1, PATH_UP); + assert_int_equal(libcheck_need_wait(&c[0]), false); for (i = 0; i < 2; i++) { assert_false(is_checker_running(&c[i])); libcheck_free(&c[i]); @@ -590,11 +615,15 @@ static void test_async_timeout_cancel_failed(void **state) for (i = 0; i < 2; i++) do_libcheck_init(&c[i], 4096, 2, &reqs[i]); do_check_state(&c[0], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), true); do_check_state(&c[1], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[1]), true); return_io_getevents_none(); do_libcheck_pending(&c[0], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), false); return_io_getevents_none(); do_libcheck_pending(&c[1], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[1]), false); do_check_state(&c[0], 0, PATH_PENDING); do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_none(); @@ -605,10 +634,12 @@ static void test_async_timeout_cancel_failed(void **state) do_check_state(&c[1], 0, PATH_PENDING); return_io_getevents_none(); do_libcheck_pending(&c[0], PATH_DOWN); + assert_int_equal(libcheck_need_wait(&c[0]), false); if (!test_dev) { /* can't pick which even gets returned on real devices */ return_io_getevents_nr(NULL, 1, &reqs[1], &res[1]); do_libcheck_pending(&c[1], PATH_UP); + assert_int_equal(libcheck_need_wait(&c[1]), false); } do_check_state(&c[0], 0, PATH_PENDING); return_io_getevents_none(); @@ -641,14 +672,19 @@ static void test_orphan_checker_cleanup(void **state) do_libcheck_init(&c[i], 4096, 30, &reqs[i]); aio_grp = get_aio_grp(c); do_check_state(&c[0], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), true); return_io_getevents_none(); do_libcheck_pending(&c[0], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[0]), false); check_aio_grp(aio_grp, 2, 0); libcheck_free(&c[0]); + assert_int_equal(libcheck_need_wait(&c[0]), false); check_aio_grp(aio_grp, 2, 1); do_check_state(&c[1], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[1]), true); return_io_getevents_nr(NULL, 2, reqs, res); do_libcheck_pending(&c[1], PATH_UP); + assert_int_equal(libcheck_need_wait(&c[1]), false); check_aio_grp(aio_grp, 1, 0); libcheck_free(&c[1]); check_aio_grp(aio_grp, 0, 0); @@ -667,8 +703,10 @@ static void test_orphan_reset_cleanup(void **state) do_libcheck_init(&c, 4096, 30, NULL); orphan_aio_grp = get_aio_grp(&c); do_check_state(&c, 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), true); return_io_getevents_none(); do_libcheck_pending(&c, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c), false); check_aio_grp(orphan_aio_grp, 1, 0); libcheck_free(&c); check_aio_grp(orphan_aio_grp, 1, 1); @@ -704,6 +742,7 @@ static void test_check_state_blksize(void **state) for (i = 0; i < 3; i++) { return_io_getevents_nr(NULL, 1, &reqs[i], &res[i]); do_check_state(&c[i], 1, chk_state[i]); + assert_int_equal(libcheck_need_wait(&c[i]), false); } for (i = 0; i < 3; i++) { assert_false(is_checker_running(&c[i])); @@ -727,18 +766,22 @@ static void test_check_state_async(void **state) do_libcheck_init(&c[i], 4096, 30, &reqs[i]); for (i = 0; i < 256; i++) { do_check_state(&c[i], 0, PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[i]), true); return_io_getevents_none(); do_libcheck_pending(&c[i], PATH_PENDING); + assert_int_equal(libcheck_need_wait(&c[i]), false); assert_true(is_checker_running(&c[i])); } do_check_state(&c[256], 0, PATH_PENDING); - return_io_getevents_nr(&full_timeout, 256, reqs, res); - return_io_getevents_nr(NULL, 1, &reqs[256], &res[256]); + assert_int_equal(libcheck_need_wait(&c[256]), true); + return_io_getevents_nr(&full_timeout, 257, reqs, res); do_libcheck_pending(&c[256], PATH_UP); + assert_int_equal(libcheck_need_wait(&c[256]), false); assert_false(is_checker_running(&c[256])); libcheck_free(&c[256]); for (i = 0; i < 256; i++) { do_check_state(&c[i], 0, PATH_UP); + assert_int_equal(libcheck_need_wait(&c[i]), false); assert_false(is_checker_running(&c[i])); libcheck_free(&c[i]); } From patchwork Tue Oct 15 03:28:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13835715 X-Patchwork-Delegate: bmarzins@redhat.com Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E74841714CC for ; Tue, 15 Oct 2024 03:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; cv=none; b=GhX5w1CmpVzp8vZPopsuEA8pwKnK328C/3hNIkS0YZlABKTI6Br4CXyeSKrCuijWAoQVsdlVTHmfjgRDVY3VhwslVestZxIy8E51AkO+WbA0oyw/j4yJJRIE1NAQqnwcUeqj/OVJKvE+T6vZf0ibyNdbwZtI01lITwwjO9yE3Sk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728962927; c=relaxed/simple; bh=VDbE0x5886dg1Q4JogrsSN1jO4LASeBe6qliyHPK8AA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gdvs8G4dwDGLqAlg7nDI1ZGyK4InVrr8eEznNqR705zDKJDbb+1yCY7TspSxXbEWpMosOJXguBPkY6lMFeLfqvYkDPrIE8ySLvKM6CJ2WjKhkX1yQycUFPRwMEwXVMS4qxkUGpTf6etycLmxQJmucAnHxeAol5CYhh+43EBKmEM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aMcAwWr2; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aMcAwWr2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728962922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Trk8JV6w/ZdQPRK33bjSsbgKTL9TRmdgDAwoQzFLOAY=; b=aMcAwWr2A4NqUmUbEvTcgxAZOex1lzfLkAx9XHsvjaIsOjacwVwTTN4pOX6+o02b1qLX1P 1UrYQZlxFKrtN8nlbhFW6ifI/2dc/FVSfav1CmaUMS1GPdDf4v8w3n550NBIVI+B+XTxYK XNRNquXHAC5q5kGveGCvWjUNbSoG41s= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-499-4w8xBhQmPJaSRpwIyh8rUg-1; Mon, 14 Oct 2024 23:28:41 -0400 X-MC-Unique: 4w8xBhQmPJaSRpwIyh8rUg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7230119560AE; Tue, 15 Oct 2024 03:28:40 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3CB3F1956054; Tue, 15 Oct 2024 03:28:40 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 49F3Sdvs2693371 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 23:28:39 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 49F3SdLF2693370; Mon, 14 Oct 2024 23:28:39 -0400 From: Benjamin Marzinski To: Christophe Varoqui Cc: device-mapper development , Martin Wilck Subject: [PATCH v5 23/23] multipathd: use timestamps to tell when the directio checker timed out Date: Mon, 14 Oct 2024 23:28:35 -0400 Message-ID: <20241015032835.2693247-24-bmarzins@redhat.com> In-Reply-To: <20241015032835.2693247-1-bmarzins@redhat.com> References: <20241015032835.2693247-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Instead of counting the number of times the path checker has been called and treating that as the number of seconds that have passed, calculate the actual timestamp when the checker will time out, and check that instead. Signed-off-by: Benjamin Marzinski --- libmultipath/checkers/directio.c | 44 ++++++++++++++++++++++---------- 1 file changed, 31 insertions(+), 13 deletions(-) diff --git a/libmultipath/checkers/directio.c b/libmultipath/checkers/directio.c index 27227894..d35a6bac 100644 --- a/libmultipath/checkers/directio.c +++ b/libmultipath/checkers/directio.c @@ -60,13 +60,28 @@ const char *libcheck_msgtable[] = { #define LOG(prio, fmt, args...) condlog(prio, "directio: " fmt, ##args) struct directio_context { - unsigned int running; - int reset_flags; + time_t timeout; + int reset_flags; struct aio_group *aio_grp; struct async_req *req; bool checked_state; }; +bool is_running(struct directio_context *ct) { + return (ct->timeout != 0); +} + +void start_running(struct directio_context *ct, int timeout_secs) { + struct timespec now; + + get_monotonic_time(&now); + ct->timeout = now.tv_sec + timeout_secs; +} + +void stop_running(struct directio_context *ct) { + ct->timeout = 0; +} + static struct aio_group * add_aio_group(void) { @@ -234,9 +249,9 @@ void libcheck_free (struct checker * c) } } - if (ct->running && ct->req->state != PATH_PENDING) - ct->running = 0; - if (!ct->running) { + if (is_running(ct) && ct->req->state != PATH_PENDING) + stop_running(ct); + if (!is_running(ct)) { free(ct->req->buf); free(ct->req); ct->aio_grp->holders--; @@ -304,7 +319,7 @@ check_pending(struct directio_context *ct, struct timespec timeout) r = get_events(ct->aio_grp, &timeout); if (ct->req->state != PATH_PENDING) { - ct->running = 0; + stop_running(ct); return; } else if (r == 0 || (timeout.tv_sec == 0 && timeout.tv_nsec == 0)) @@ -330,10 +345,10 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) if (sync > 0) LOG(4, "called in synchronous mode"); - if (ct->running) { + if (is_running(ct)) { ct->checked_state = true; if (ct->req->state != PATH_PENDING) { - ct->running = 0; + stop_running(ct); return ct->req->state; } } else { @@ -348,9 +363,9 @@ check_state(int fd, struct directio_context *ct, int sync, int timeout_secs) LOG(3, "io_submit error %i", -rc); return PATH_UNCHECKED; } + start_running(ct, timeout_secs); ct->checked_state = false; } - ct->running++; if (!sync) return PATH_PENDING; @@ -388,7 +403,7 @@ static void set_msgid(struct checker *c, int state) bool libcheck_need_wait(struct checker *c) { struct directio_context *ct = (struct directio_context *)c->context; - return (ct && ct->running && ct->req->state == PATH_PENDING && + return (ct && is_running(ct) && ct->req->state == PATH_PENDING && !ct->checked_state); } @@ -400,7 +415,7 @@ int libcheck_pending(struct checker *c) struct timespec no_wait = { .tv_sec = 0 }; /* The if path checker isn't running, just return the exiting value. */ - if (!ct || !ct->running) { + if (!ct || !is_running(ct)) { rc = c->path_state; goto out; } @@ -408,10 +423,13 @@ int libcheck_pending(struct checker *c) if (ct->req->state == PATH_PENDING) check_pending(ct, no_wait); else - ct->running = 0; + stop_running(ct); rc = ct->req->state; if (rc == PATH_PENDING) { - if (ct->running > c->timeout) { + struct timespec now; + + get_monotonic_time(&now); + if (now.tv_sec > ct->timeout) { LOG(3, "abort check on timeout"); io_cancel(ct->aio_grp->ioctx, &ct->req->io, &event); rc = PATH_DOWN;