From patchwork Mon Oct 23 01:58:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13432254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 394D2C0032E for ; Mon, 23 Oct 2023 02:11:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233046AbjJWCLp (ORCPT ); Sun, 22 Oct 2023 22:11:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229470AbjJWCLo (ORCPT ); Sun, 22 Oct 2023 22:11:44 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6032490 for ; Sun, 22 Oct 2023 19:11:42 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0DD7E1FE03; Mon, 23 Oct 2023 02:11:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698027101; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wp0WlWrHsij9v1ELwAOMtktOa0wyT2s2v9GtDArTVP8=; b=tRzQYpmkqRky2iLdpRQRafeL0vw6T7w4MZ0BrNA41vsptrowCs2PwrE0/lXSARc8p4mtax 86hI09OpWhtre2UwDq3tl2Q4a0o1NVJbbCBdvVinFkaPDG45ncEVnooo/eiw58hZzyY0R8 93APs1kjT0OUemI0P4+GuGqYymnBLe4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698027101; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wp0WlWrHsij9v1ELwAOMtktOa0wyT2s2v9GtDArTVP8=; b=yyiBD8YPpZBz6SUWp1FV1Y3THe+YmuMrENDjgA9BUk5d56HMo/5OKs1RDPMM9h8SX4/4w6 IoqCBfHAvLXk9zBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D5638132FD; Mon, 23 Oct 2023 02:11:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 5UARI1vWNWVlbwAAMHmgww (envelope-from ); Mon, 23 Oct 2023 02:11:39 +0000 From: NeilBrown To: Steve Dickson Cc: linux-nfs@vger.kernel.org, Trond Myklebust Subject: [PATCH 1/6] export: fix handling of error from match_fsid() Date: Mon, 23 Oct 2023 12:58:31 +1100 Message-ID: <20231023021052.5258-2-neilb@suse.de> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231023021052.5258-1-neilb@suse.de> References: <20231023021052.5258-1-neilb@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spamd-Result: default: False [0.79 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_MISSING_CHARSET(2.50)[]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.11)[66.16%] Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org If match_fsid() returns -1 we shouldn't assume that the path definitely doesn't match the fsid, though it might not. This is a similar situation to where an export is expected to be a mount point, but is found not to be one. So it can be handled the same way, by setting 'dev_missing'. This will only have an effect if no other path matched the fsid, which is what we want. The current code results in nothing being exported if any export point, or any mount point beneath a crossmnt export point, fails a 'stat' request, which is too harsh. Signed-off-by: NeilBrown --- support/export/cache.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/support/export/cache.c b/support/export/cache.c index 19bbba556060..e4595020f43f 100644 --- a/support/export/cache.c +++ b/support/export/cache.c @@ -858,7 +858,8 @@ static void nfsd_fh(int f) case 0: continue; case -1: - goto out; + dev_missing ++; + continue; } if (is_ipaddr_client(dom) && !ipaddr_client_matches(exp, ai)) From patchwork Mon Oct 23 01:58:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13432255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68C04CDB474 for ; Mon, 23 Oct 2023 02:11:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233060AbjJWCLu (ORCPT ); Sun, 22 Oct 2023 22:11:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229489AbjJWCLs (ORCPT ); Sun, 22 Oct 2023 22:11:48 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2029790 for ; Sun, 22 Oct 2023 19:11:47 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A72F121847; Mon, 23 Oct 2023 02:11:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698027105; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LL7tWXUl7A3RdvdqOHiqksj0Bp6yAXgIslcHvnRez98=; b=zbTPTsMvqZ64y/mjngFdKmMibTMsmvfu0oNgNTwRAChoeRR/XOSAGt96wnvV7tn0I6TClc yBSo4K7cXDqP7cImBx1Q7l3h1DPD4Py+UpqOViWNgZYpYo8cbTXADwosZGWzSx5MR9OgKH sa6aG/0irgKivwC4FsNMh7tqF0cTn80= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698027105; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LL7tWXUl7A3RdvdqOHiqksj0Bp6yAXgIslcHvnRez98=; b=vGZcSFLUG0rJgPxUYvMipkma+9wlRa+YlA04HPGbi7y9kKof+6P7NaBk0IiMQ12Ji6+ebE 2IRIyV66rVIcjQAQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 75BB4132FD; Mon, 23 Oct 2023 02:11:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id KXZGC2DWNWVpbwAAMHmgww (envelope-from ); Mon, 23 Oct 2023 02:11:44 +0000 From: NeilBrown To: Steve Dickson Cc: linux-nfs@vger.kernel.org, Trond Myklebust Subject: [PATCH 2/6] export: add EACCES to the list of known path_lookup_error() errors. Date: Mon, 23 Oct 2023 12:58:32 +1100 Message-ID: <20231023021052.5258-3-neilb@suse.de> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231023021052.5258-1-neilb@suse.de> References: <20231023021052.5258-1-neilb@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spamd-Result: default: False [0.52 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_MISSING_CHARSET(2.50)[]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.38)[77.21%] Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org If a 'stat' results in EACCES (for root), then it is likely a permanent problem. One possible cause is a 'fuser' filesystem which only gives any access to the user which mounted it. So it is reasonable for EACCES to be a "path lookup error" Signed-off-by: NeilBrown --- support/export/cache.c | 1 + 1 file changed, 1 insertion(+) diff --git a/support/export/cache.c b/support/export/cache.c index e4595020f43f..5307f6c8d872 100644 --- a/support/export/cache.c +++ b/support/export/cache.c @@ -77,6 +77,7 @@ static bool path_lookup_error(int err) case ENAMETOOLONG: case ENOENT: case ENOTDIR: + case EACCES: return 1; } return 0; From patchwork Mon Oct 23 01:58:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13432256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6667DC0032E for ; Mon, 23 Oct 2023 02:11:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233007AbjJWCL4 (ORCPT ); Sun, 22 Oct 2023 22:11:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229489AbjJWCLz (ORCPT ); Sun, 22 Oct 2023 22:11:55 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9ABAA8E for ; Sun, 22 Oct 2023 19:11:52 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 004A31FD8E; Mon, 23 Oct 2023 02:11:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698027111; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=29Y+6+lN91+ttI9zymDHVFSaeYXd6uip7j1l91qeT0M=; b=gdTAY0Wi0V3pNBOQ/TPizjQn7BVUayD3J/dg21GDwoPQQ3bvwP2rehiIzmXrvP6cGZ2a3F Oznv9GJ843tLHV1giq3ooLMNaMwKCw7zhwCQTTfMeFKBrOZrDeaXxKKMl8FpE+Gwlp+6iW FWT55DVuJvGh7yhb2noLaY2aumxyPaI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698027111; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=29Y+6+lN91+ttI9zymDHVFSaeYXd6uip7j1l91qeT0M=; b=sZ5Sl/JxZvJoBbqPfyyLhUvoL3asCXv4jc2dkpBTDX6FQ1Q1pY8KhWBKrQpUwlKMmrwdTV 5Wl1bkt/5WjgmFCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C75DB132FD; Mon, 23 Oct 2023 02:11:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0iljL2TWNWVwbwAAMHmgww (envelope-from ); Mon, 23 Oct 2023 02:11:48 +0000 From: NeilBrown To: Steve Dickson Cc: linux-nfs@vger.kernel.org, Trond Myklebust Subject: [PATCH 3/6] export: move cache_open() before workers are forked. Date: Mon, 23 Oct 2023 12:58:33 +1100 Message-ID: <20231023021052.5258-4-neilb@suse.de> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231023021052.5258-1-neilb@suse.de> References: <20231023021052.5258-1-neilb@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spamd-Result: default: False [0.90 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_MISSING_CHARSET(2.50)[]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.00)[39.91%] Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org If each worker has a separate open on a cache channel, then each worker will potentially receive every upcall request resulting in duplicated work. A worker will only not see a request that another worker sees if that other worker answers the request before this worker gets a chance to read it. To avoid duplicate effort between threads and so get maximum benefit from multiple threads, open the cache channels before forking. Note that the kernel provides locking so that only one thread can be reading to writing to any channel at any given moment. Fixes: 5fc3bac9e0c3 ("mountd: Ensure we don't share cache file descriptors among processes.") Signed-off-by: NeilBrown --- utils/exportd/exportd.c | 8 ++++++-- utils/mountd/mountd.c | 8 ++++++-- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c index 2dd12cb6015b..6f866445efc2 100644 --- a/utils/exportd/exportd.c +++ b/utils/exportd/exportd.c @@ -289,12 +289,16 @@ main(int argc, char **argv) else if (num_threads > MAX_THREADS) num_threads = MAX_THREADS; + /* Open cache channel files BEFORE forking so each upcall is + * only handled by one thread. Kernel provides locking for both + * read and write. + */ + cache_open(); + if (num_threads > 1) fork_workers(); - /* Open files now to avoid sharing descriptors among forked processes */ - cache_open(); v4clients_init(); /* Process incoming upcalls */ diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c index bcf749fabbb3..f9c62cded66c 100644 --- a/utils/mountd/mountd.c +++ b/utils/mountd/mountd.c @@ -916,12 +916,16 @@ main(int argc, char **argv) else if (num_threads > MAX_THREADS) num_threads = MAX_THREADS; + /* Open cache channel files BEFORE forking so each upcall is + * only handled by one thread. Kernel provides locking for both + * read and write. + */ + cache_open(); + if (num_threads > 1) fork_workers(); nfsd_path_init(); - /* Open files now to avoid sharing descriptors among forked processes */ - cache_open(); v4clients_init(); xlog(L_NOTICE, "Version " VERSION " starting"); From patchwork Mon Oct 23 01:58:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13432257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B271DC0032E for ; Mon, 23 Oct 2023 02:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229470AbjJWCMB (ORCPT ); Sun, 22 Oct 2023 22:12:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233057AbjJWCMA (ORCPT ); Sun, 22 Oct 2023 22:12:00 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B9EF8E for ; Sun, 22 Oct 2023 19:11:57 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DF85E1FE05; Mon, 23 Oct 2023 02:11:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698027115; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k+tJrpWVDJJ7KETSa91IT4Yko+pzXy4M/FqN16u/imA=; b=s5LsyowArzqA8jhivPkChCZo47Ziif6yhlsjuYRwbsSaSV1PQx9dworgojmvuu1oHSrsK/ rDKhoDv4kHEcmaCGsY7lzNxDF05YzdUY6HPgHaV5zn8efU3psM5+SqAwAZNZmCUmCqSgyC U+BwDJR8bsIvaWrfgZmmiJa329kMjOc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698027115; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k+tJrpWVDJJ7KETSa91IT4Yko+pzXy4M/FqN16u/imA=; b=9LY8056gLdOPZckx/ThPj7qOa7rcXyTMj9p6Iv9QcEvVDKisdvBCchyE162cxWlTab9cSX 57mZlbYlpeHL7lBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 6C925132FD; Mon, 23 Oct 2023 02:11:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QpwwCWrWNWV2bwAAMHmgww (envelope-from ); Mon, 23 Oct 2023 02:11:54 +0000 From: NeilBrown To: Steve Dickson Cc: linux-nfs@vger.kernel.org, Trond Myklebust Subject: [PATCH 4/6] Move fork_workers() and wait_for_workers() in cache.c Date: Mon, 23 Oct 2023 12:58:34 +1100 Message-ID: <20231023021052.5258-5-neilb@suse.de> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231023021052.5258-1-neilb@suse.de> References: <20231023021052.5258-1-neilb@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out2.suse.de; none X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_MISSING_CHARSET(2.50)[]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Both mountd and exported have fork_workers() and wait_for_workers() which are nearly identical. Move this code into cache.c (adding a cache_ prefix to the function names) and leave the minor differences in the two callers. Also remove duplicate declarations from mountd.h. Signed-off-by: NeilBrown --- support/export/cache.c | 75 ++++++++++++++++++++++++++++++++- support/export/export.h | 2 + utils/exportd/exportd.c | 90 ++++++---------------------------------- utils/mountd/mountd.c | 91 ++++++----------------------------------- utils/mountd/mountd.h | 9 ---- 5 files changed, 99 insertions(+), 168 deletions(-) diff --git a/support/export/cache.c b/support/export/cache.c index 5307f6c8d872..1874156af5e5 100644 --- a/support/export/cache.c +++ b/support/export/cache.c @@ -1,10 +1,9 @@ - /* * Handle communication with knfsd internal cache * * We open /proc/net/rpc/{auth.unix.ip,nfsd.export,nfsd.fh}/channel * and listen for requests (using my_svc_run) - * + * */ #ifdef HAVE_CONFIG_H @@ -16,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -1775,3 +1775,74 @@ cache_get_filehandle(nfs_export *exp, int len, char *p) fh.fh_size = qword_get(&bp, (char *)fh.fh_handle, NFS3_FHSIZE); return &fh; } + +/* Wait for all worker child processes to exit and reap them */ +void +cache_wait_for_workers(char *prog) +{ + int status; + pid_t pid; + + for (;;) { + + pid = waitpid(0, &status, 0); + + if (pid < 0) { + if (errno == ECHILD) + return; /* no more children */ + xlog(L_FATAL, "%s: can't wait: %s\n", prog, + strerror(errno)); + } + + /* Note: because we SIG_IGN'd SIGCHLD earlier, this + * does not happen on 2.6 kernels, and waitpid() blocks + * until all the children are dead then returns with + * -ECHILD. But, we don't need to do anything on the + * death of individual workers, so we don't care. */ + xlog(L_NOTICE, "%s: reaped child %d, status %d\n", + prog, (int)pid, status); + } +} + +/* Fork num_threads worker children and wait for them */ +int +cache_fork_workers(char *prog, int num_threads) +{ + int i; + pid_t pid; + + if (num_threads <= 1) + return 1; + + xlog(L_NOTICE, "%s: starting %d threads\n", prog, num_threads); + + for (i = 0 ; i < num_threads ; i++) { + pid = fork(); + if (pid < 0) { + xlog(L_FATAL, "%s: cannot fork: %s\n", prog, + strerror(errno)); + } + if (pid == 0) { + /* worker child */ + + /* Re-enable the default action on SIGTERM et al + * so that workers die naturally when sent them. + * Only the parent unregisters with pmap and + * hence needs to do special SIGTERM handling. */ + struct sigaction sa; + sa.sa_handler = SIG_DFL; + sa.sa_flags = 0; + sigemptyset(&sa.sa_mask); + sigaction(SIGHUP, &sa, NULL); + sigaction(SIGINT, &sa, NULL); + sigaction(SIGTERM, &sa, NULL); + + /* fall into my_svc_run in caller */ + return 1; + } + } + + /* in parent */ + cache_wait_for_workers(prog); + return 0; +} diff --git a/support/export/export.h b/support/export/export.h index 8d5a0d3004ef..ce561f9fbd3e 100644 --- a/support/export/export.h +++ b/support/export/export.h @@ -29,6 +29,8 @@ int v4clients_process(fd_set *fdset); struct nfs_fh_len * cache_get_filehandle(nfs_export *exp, int len, char *p); int cache_export(nfs_export *exp, char *path); +int cache_fork_workers(char *prog, int num_threads); +void cache_wait_for_workers(char *prog); bool ipaddr_client_matches(nfs_export *exp, struct addrinfo *ai); bool namelist_client_matches(nfs_export *exp, char *dom); diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c index 6f866445efc2..d07a885c6763 100644 --- a/utils/exportd/exportd.c +++ b/utils/exportd/exportd.c @@ -16,7 +16,6 @@ #include #include #include -#include #include "nfslib.h" #include "conffile.h" @@ -54,90 +53,19 @@ static char shortopts[] = "d:fghs:t:liT:"; */ inline static void set_signals(void); -/* Wait for all worker child processes to exit and reap them */ -static void -wait_for_workers (void) -{ - int status; - pid_t pid; - - for (;;) { - - pid = waitpid(0, &status, 0); - - if (pid < 0) { - if (errno == ECHILD) - return; /* no more children */ - xlog(L_FATAL, "mountd: can't wait: %s\n", - strerror(errno)); - } - - /* Note: because we SIG_IGN'd SIGCHLD earlier, this - * does not happen on 2.6 kernels, and waitpid() blocks - * until all the children are dead then returns with - * -ECHILD. But, we don't need to do anything on the - * death of individual workers, so we don't care. */ - xlog(L_NOTICE, "mountd: reaped child %d, status %d\n", - (int)pid, status); - } -} - inline void cleanup_lockfiles (void) { unlink(etab.lockfn); } -/* Fork num_threads worker children and wait for them */ static void -fork_workers(void) -{ - int i; - pid_t pid; - - xlog(L_NOTICE, "mountd: starting %d threads\n", num_threads); - - for (i = 0 ; i < num_threads ; i++) { - pid = fork(); - if (pid < 0) { - xlog(L_FATAL, "mountd: cannot fork: %s\n", - strerror(errno)); - } - if (pid == 0) { - /* worker child */ - - /* Re-enable the default action on SIGTERM et al - * so that workers die naturally when sent them. - * Only the parent unregisters with pmap and - * hence needs to do special SIGTERM handling. */ - struct sigaction sa; - sa.sa_handler = SIG_DFL; - sa.sa_flags = 0; - sigemptyset(&sa.sa_mask); - sigaction(SIGHUP, &sa, NULL); - sigaction(SIGINT, &sa, NULL); - sigaction(SIGTERM, &sa, NULL); - - /* fall into my_svc_run in caller */ - return; - } - } - - /* in parent */ - wait_for_workers(); - cleanup_lockfiles(); - free_state_path_names(&etab); - xlog(L_NOTICE, "exportd: no more workers, exiting\n"); - exit(0); -} - -static void killer (int sig) { if (num_threads > 1) { /* play Kronos and eat our children */ kill(0, SIGTERM); - wait_for_workers(); + cache_wait_for_workers("exportd"); } cleanup_lockfiles(); free_state_path_names(&etab); @@ -145,6 +73,7 @@ killer (int sig) exit(0); } + static void sig_hup (int UNUSED(sig)) { @@ -152,8 +81,9 @@ sig_hup (int UNUSED(sig)) xlog (L_NOTICE, "Received SIGHUP... Ignoring.\n"); return; } -inline static void -set_signals(void) + +inline static void +set_signals(void) { struct sigaction sa; @@ -295,9 +225,13 @@ main(int argc, char **argv) */ cache_open(); - if (num_threads > 1) - fork_workers(); - + if (cache_fork_workers(progname, num_threads) == 0) { + /* We forked, waited, and now need to clean up */ + cleanup_lockfiles(); + free_state_path_names(&etab); + xlog(L_NOTICE, "%s: no more workers, exiting\n", progname); + exit(0); + } v4clients_init(); diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c index f9c62cded66c..dbd5546df61c 100644 --- a/utils/mountd/mountd.c +++ b/utils/mountd/mountd.c @@ -21,7 +21,6 @@ #include #include #include -#include #include "conffile.h" #include "xmalloc.h" @@ -119,90 +118,17 @@ cleanup_lockfiles (void) unlink(rmtab.lockfn); } -/* Wait for all worker child processes to exit and reap them */ -static void -wait_for_workers (void) -{ - int status; - pid_t pid; - - for (;;) { - - pid = waitpid(0, &status, 0); - - if (pid < 0) { - if (errno == ECHILD) - return; /* no more children */ - xlog(L_FATAL, "mountd: can't wait: %s\n", - strerror(errno)); - } - - /* Note: because we SIG_IGN'd SIGCHLD earlier, this - * does not happen on 2.6 kernels, and waitpid() blocks - * until all the children are dead then returns with - * -ECHILD. But, we don't need to do anything on the - * death of individual workers, so we don't care. */ - xlog(L_NOTICE, "mountd: reaped child %d, status %d\n", - (int)pid, status); - } -} - -/* Fork num_threads worker children and wait for them */ -static void -fork_workers(void) -{ - int i; - pid_t pid; - - xlog(L_NOTICE, "mountd: starting %d threads\n", num_threads); - - for (i = 0 ; i < num_threads ; i++) { - pid = fork(); - if (pid < 0) { - xlog(L_FATAL, "mountd: cannot fork: %s\n", - strerror(errno)); - } - if (pid == 0) { - /* worker child */ - - /* Re-enable the default action on SIGTERM et al - * so that workers die naturally when sent them. - * Only the parent unregisters with pmap and - * hence needs to do special SIGTERM handling. */ - struct sigaction sa; - sa.sa_handler = SIG_DFL; - sa.sa_flags = 0; - sigemptyset(&sa.sa_mask); - sigaction(SIGHUP, &sa, NULL); - sigaction(SIGINT, &sa, NULL); - sigaction(SIGTERM, &sa, NULL); - - /* fall into my_svc_run in caller */ - return; - } - } - - /* in parent */ - wait_for_workers(); - unregister_services(); - cleanup_lockfiles(); - free_state_path_names(&etab); - free_state_path_names(&rmtab); - xlog(L_NOTICE, "mountd: no more workers, exiting\n"); - exit(0); -} - /* * Signal handler. */ -static void +static void killer (int sig) { unregister_services(); if (num_threads > 1) { /* play Kronos and eat our children */ kill(0, SIGTERM); - wait_for_workers(); + cache_wait_for_workers("mountd"); } cleanup_lockfiles(); free_state_path_names(&etab); @@ -220,7 +146,7 @@ sig_hup (int UNUSED(sig)) } bool_t -mount_null_1_svc(struct svc_req *rqstp, void *UNUSED(argp), +mount_null_1_svc(struct svc_req *rqstp, void *UNUSED(argp), void *UNUSED(resp)) { struct sockaddr *sap = nfs_getrpccaller(rqstp->rq_xprt); @@ -922,8 +848,15 @@ main(int argc, char **argv) */ cache_open(); - if (num_threads > 1) - fork_workers(); + if (cache_fork_workers("mountd", num_threads) == 0) { + /* We forked, waited, and now need to clean up */ + unregister_services(); + cleanup_lockfiles(); + free_state_path_names(&etab); + free_state_path_names(&rmtab); + xlog(L_NOTICE, "mountd: no more workers, exiting\n"); + exit(0); + } nfsd_path_init(); v4clients_init(); diff --git a/utils/mountd/mountd.h b/utils/mountd/mountd.h index d30775313f66..bd5c9576d8ad 100644 --- a/utils/mountd/mountd.h +++ b/utils/mountd/mountd.h @@ -51,13 +51,4 @@ void mountlist_del(char *host, const char *path); void mountlist_del_all(const struct sockaddr *sap); mountlist mountlist_list(void); -void cache_open(void); -struct nfs_fh_len * - cache_get_filehandle(nfs_export *exp, int len, char *p); -int cache_export(nfs_export *exp, char *path); - -bool ipaddr_client_matches(nfs_export *exp, struct addrinfo *ai); -bool namelist_client_matches(nfs_export *exp, char *dom); -bool client_matches(nfs_export *exp, char *dom, struct addrinfo *ai); - #endif /* MOUNTD_H */ From patchwork Mon Oct 23 01:58:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13432258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5D22CDB474 for ; Mon, 23 Oct 2023 02:12:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229489AbjJWCMG (ORCPT ); Sun, 22 Oct 2023 22:12:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233097AbjJWCMF (ORCPT ); Sun, 22 Oct 2023 22:12:05 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89654135 for ; Sun, 22 Oct 2023 19:12:02 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 26F1E21A7F; Mon, 23 Oct 2023 02:12:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698027121; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dzwlzp/M6AhoIJULxUZ0rApD1agxj3RFV36T21E6jWo=; b=js/bYquh0SKd8RYixV17Wr23vmY62P3NO+N3BOki2JQTeWq7i+ZdiwtRGDMy4LfpELBrdL ato60a+9NV48ZOM2VDPE7C6A8InyB3a6BTdrI9KRwZZrecQSfdKtUWx2XaP/f8NlwQEsy7 iHhpIGcXcHVk/fj17vvXnhMMwCnBit4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698027121; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dzwlzp/M6AhoIJULxUZ0rApD1agxj3RFV36T21E6jWo=; b=wYphfpHlfRJBHSzxyeYijKioz1HOcxpZm3vwqFyuwun4RoBGclMmZ34hG+yEu+S900l0CL 6a++kpmreG08IaBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EEABF132FD; Mon, 23 Oct 2023 02:11:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id jfIMKW/WNWWHbwAAMHmgww (envelope-from ); Mon, 23 Oct 2023 02:11:59 +0000 From: NeilBrown To: Steve Dickson Cc: linux-nfs@vger.kernel.org, Trond Myklebust Subject: [PATCH 5/6] Share process_loop code between mountd and exportd. Date: Mon, 23 Oct 2023 12:58:35 +1100 Message-ID: <20231023021052.5258-6-neilb@suse.de> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231023021052.5258-1-neilb@suse.de> References: <20231023021052.5258-1-neilb@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spamd-Result: default: False [0.61 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_MISSING_CHARSET(2.50)[]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.29)[74.73%] Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org There is substantial commonality between cache_process_loop() used by exportd and my_svc_run() used by mountd. Remove the looping from cache_process_loop() renaming it to cache_process() and call it in a loop from exportd. my_svc_run() now calls cache_process() for all the common functionality and adds code specific to being an RPC server. Signed-off-by: NeilBrown --- support/export/cache.c | 49 +++++++++++++++++++++-------------------- support/export/export.h | 1 + utils/exportd/exportd.c | 5 +++-- utils/mountd/svc_run.c | 23 ++++--------------- 4 files changed, 33 insertions(+), 45 deletions(-) diff --git a/support/export/cache.c b/support/export/cache.c index 1874156af5e5..a01eba4f6619 100644 --- a/support/export/cache.c +++ b/support/export/cache.c @@ -1602,40 +1602,41 @@ int cache_process_req(fd_set *readfds) } /** - * cache_process_loop - process incoming upcalls + * cache_process - process incoming upcalls + * Returns -ve on error, or number of fds in svc_fds + * that might need processing. */ -void cache_process_loop(void) +int cache_process(fd_set *readfds) { - fd_set readfds; + fd_set fdset; int selret; - FD_ZERO(&readfds); - - for (;;) { - - cache_set_fds(&readfds); - v4clients_set_fds(&readfds); - - selret = select(FD_SETSIZE, &readfds, - (void *) 0, (void *) 0, (struct timeval *) 0); + if (!readfds) { + FD_ZERO(&fdset); + readfds = &fdset; + } + cache_set_fds(readfds); + v4clients_set_fds(readfds); + selret = select(FD_SETSIZE, readfds, + (void *) 0, (void *) 0, (struct timeval *) 0); - switch (selret) { - case -1: - if (errno == EINTR || errno == ECONNREFUSED - || errno == ENETUNREACH || errno == EHOSTUNREACH) - continue; - xlog(L_ERROR, "my_svc_run() - select: %m"); - return; + switch (selret) { + case -1: + if (errno == EINTR || errno == ECONNREFUSED + || errno == ENETUNREACH || errno == EHOSTUNREACH) + return 0; + return -1; - default: - cache_process_req(&readfds); - v4clients_process(&readfds); - } + default: + selret -= cache_process_req(readfds); + selret -= v4clients_process(readfds); + if (selret < 0) + selret = 0; } + return selret; } - /* * Give IP->domain and domain+path->options to kernel * % echo nfsd $IP $[now+DEFAULT_TTL] $domain > /proc/net/rpc/auth.unix.ip/channel diff --git a/support/export/export.h b/support/export/export.h index ce561f9fbd3e..e2009ccdc443 100644 --- a/support/export/export.h +++ b/support/export/export.h @@ -31,6 +31,7 @@ struct nfs_fh_len * int cache_export(nfs_export *exp, char *path); int cache_fork_workers(char *prog, int num_threads); void cache_wait_for_workers(char *prog); +int cache_process(fd_set *readfds); bool ipaddr_client_matches(nfs_export *exp, struct addrinfo *ai); bool namelist_client_matches(nfs_export *exp, char *dom); diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c index d07a885c6763..a2e370ac506f 100644 --- a/utils/exportd/exportd.c +++ b/utils/exportd/exportd.c @@ -236,9 +236,10 @@ main(int argc, char **argv) v4clients_init(); /* Process incoming upcalls */ - cache_process_loop(); + while (cache_process(NULL) >= 0) + ; - xlog(L_ERROR, "%s: process loop terminated unexpectedly. Exiting...\n", + xlog(L_ERROR, "%s: process loop terminated unexpectedly(%m). Exiting...\n", progname); free_state_path_names(&etab); diff --git a/utils/mountd/svc_run.c b/utils/mountd/svc_run.c index 167b9757bde2..2aaf3756bbb1 100644 --- a/utils/mountd/svc_run.c +++ b/utils/mountd/svc_run.c @@ -97,28 +97,13 @@ my_svc_run(void) int selret; for (;;) { - readfds = svc_fdset; - cache_set_fds(&readfds); - v4clients_set_fds(&readfds); - - selret = select(FD_SETSIZE, &readfds, - (void *) 0, (void *) 0, (struct timeval *) 0); - - - switch (selret) { - case -1: - if (errno == EINTR || errno == ECONNREFUSED - || errno == ENETUNREACH || errno == EHOSTUNREACH) - continue; + selret = cache_process(&readfds); + if (selret < 0) { xlog(L_ERROR, "my_svc_run() - select: %m"); return; - - default: - selret -= cache_process_req(&readfds); - selret -= v4clients_process(&readfds); - if (selret) - svc_getreqset(&readfds); } + if (selret) + svc_getreqset(&readfds); } } From patchwork Mon Oct 23 01:58:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13432269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EC76C0032E for ; Mon, 23 Oct 2023 02:12:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233057AbjJWCMN (ORCPT ); Sun, 22 Oct 2023 22:12:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233094AbjJWCMM (ORCPT ); Sun, 22 Oct 2023 22:12:12 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB137E6 for ; Sun, 22 Oct 2023 19:12:07 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4F69821A80; Mon, 23 Oct 2023 02:12:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698027126; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GDCSGpqzzQCPsWJnFwARKruyZ2iXRyLzv259sJdXiwE=; b=afeTQ7mXkyYdbgtEKuSYzWrtRlNWK/cTAVgg3QXY+LYjaNo4xCXFPkYSK2V4Rwfst/3EWS F8k88gQQEi3oqvogw940iZR/Iaw5AuZ/gSxXe1Y0r9JXa2yDT6n+g34S4uXDhdT0kT1UE3 odzgtxr4/mMlD09hds9cRYZR+GQfh7k= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698027126; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GDCSGpqzzQCPsWJnFwARKruyZ2iXRyLzv259sJdXiwE=; b=twdbu0D/g6tyU74HoZ17i88eIkkID54jA378UPB7mgSkhwcE2cK8QML9HJ4tCyL4SF0YSS Rv8GEnVuMpcfVxBg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 23A19132FD; Mon, 23 Oct 2023 02:12:04 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id VccwM3TWNWWTbwAAMHmgww (envelope-from ); Mon, 23 Oct 2023 02:12:04 +0000 From: NeilBrown To: Steve Dickson Cc: linux-nfs@vger.kernel.org, Trond Myklebust Subject: [PATCH 6/6] cache: periodically retry requests that couldn't be answered. Date: Mon, 23 Oct 2023 12:58:36 +1100 Message-ID: <20231023021052.5258-7-neilb@suse.de> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231023021052.5258-1-neilb@suse.de> References: <20231023021052.5258-1-neilb@suse.de> MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spamd-Result: default: False [-2.10 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_MISSING_CHARSET(2.50)[]; BROKEN_CONTENT_TYPE(1.50)[]; NEURAL_HAM_LONG(-3.00)[-1.000]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Requests from the kernel to map the fsid from a filehandle to a path name sometimes cannot be answered because the filesystems isn't available now but might be available later. This happens if an export is marked "mountpoint" but the mountpoint isn't currently mounted. In this case it might get mounted in the future. It also happens in an NFS filesystem is being re-exported and the server is unresponsive. In that case (if it was mounted "softerr") we get ETIMEDOUT from a stat() attempt and so cannot give either a positive or negative response. These cases are currently handled poorly. No answer is returned to the kernel so it will continue waiting for an answer - and never get one even if the NFS server comes back or the mountpoint is mounted. We cannot report a soft error to the kernel so much retry ourselves. With this patch we record the request when the lookup fails with dev_missing or similar and retry every 2 minutes. Signed-off-by: NeilBrown --- support/export/cache.c | 121 +++++++++++++++++++++++++++++++++++------ 1 file changed, 103 insertions(+), 18 deletions(-) diff --git a/support/export/cache.c b/support/export/cache.c index a01eba4f6619..6c0a44a3a209 100644 --- a/support/export/cache.c +++ b/support/export/cache.c @@ -759,7 +759,15 @@ static struct addrinfo *lookup_client_addr(char *dom) return ret; } -static void nfsd_fh(int f) +#define RETRY_SEC 120 +struct delayed { + char *message; + time_t last_attempt; + int f; + struct delayed *next; +} *delayed; + +static int nfsd_handle_fh(int f, char *bp, int blen) { /* request are: * domain fsidtype fsid @@ -777,21 +785,13 @@ static void nfsd_fh(int f) nfs_export *exp; int i; int dev_missing = 0; - char buf[RPC_CHAN_BUF_SIZE], *bp; - int blen; + char buf[RPC_CHAN_BUF_SIZE]; int did_uncover = 0; - - blen = cache_read(f, buf, sizeof(buf)); - if (blen <= 0 || buf[blen-1] != '\n') return; - buf[blen-1] = 0; - - xlog(D_CALL, "nfsd_fh: inbuf '%s'", buf); - - bp = buf; + int ret = 0; dom = malloc(blen); if (dom == NULL) - return; + return ret; if (qword_get(&bp, dom, blen) <= 0) goto out; if (qword_get_int(&bp, &fsidtype) != 0) @@ -893,8 +893,10 @@ static void nfsd_fh(int f) /* The missing dev could be what we want, so just be * quiet rather than returning stale yet */ - if (dev_missing) + if (dev_missing) { + ret = 1; goto out; + } } else if (found->e_mountpoint && !is_mountpoint(found->e_mountpoint[0]? found->e_mountpoint: @@ -904,7 +906,7 @@ static void nfsd_fh(int f) xlog(L_WARNING, "%s not exported as %d not a mountpoint", found->e_path, found->e_mountpoint); */ - /* FIXME we need to make sure we re-visit this later */ + ret = 1; goto out; } @@ -933,7 +935,68 @@ out: free(found_path); nfs_freeaddrinfo(ai); free(dom); - xlog(D_CALL, "nfsd_fh: found %p path %s", found, found ? found->e_path : NULL); + if (!ret) + xlog(D_CALL, "nfsd_fh: found %p path %s", + found, found ? found->e_path : NULL); + return ret; +} + +static void nfsd_fh(int f) +{ + struct delayed *d, **dp; + char inbuf[RPC_CHAN_BUF_SIZE]; + int blen; + + blen = cache_read(f, inbuf, sizeof(inbuf)); + if (blen <= 0 || inbuf[blen-1] != '\n') return; + inbuf[blen-1] = 0; + + xlog(D_CALL, "nfsd_fh: inbuf '%s'", inbuf); + + if (nfsd_handle_fh(f, inbuf, blen) == 0) + return; + /* We don't have a definitive answer to give the kernel. + * This is because an export marked "mountpoint" isn't a + * mountpoint, or because a stat of a mountpoint fails with + * a strange error like ETIMEDOUT as is possible with an + * NFS mount marked "softerr" which is being re-exported. + * + * We cannot tell the kernel to retry, so we have to + * retry ourselves. + */ + d = malloc(sizeof(*d)); + + if (!d) + return; + d->message = strndup(inbuf, blen); + if (!d->message) { + free(d); + return; + } + d->f = f; + d->last_attempt = time(NULL); + d->next = NULL; + dp = &delayed; + while (*dp) + dp = &(*dp)->next; + *dp = d; +} + +static void nfsd_retry_fh(struct delayed *d) +{ + struct delayed **dp; + + if (nfsd_handle_fh(d->f, d->message, strlen(d->message)+1) == 0) { + free(d->message); + free(d); + return; + } + d->last_attempt = time(NULL); + d->next = NULL; + dp = &delayed; + while (*dp) + dp = &(*dp)->next; + *dp = d; } #ifdef HAVE_JUNCTION_SUPPORT @@ -1512,7 +1575,7 @@ static void nfsd_export(int f) * This will cause it not to appear in the V4 Pseudo-root * and so a "mount" of this path will fail, just like with * V3. - * And filehandle for this mountpoint from an earlier + * Any filehandle for this mountpoint from an earlier * mount will block in nfsd.fh lookup. */ xlog(L_WARNING, @@ -1610,6 +1673,7 @@ int cache_process(fd_set *readfds) { fd_set fdset; int selret; + struct timeval tv = { 24*3600, 0 }; if (!readfds) { FD_ZERO(&fdset); @@ -1618,8 +1682,29 @@ int cache_process(fd_set *readfds) cache_set_fds(readfds); v4clients_set_fds(readfds); - selret = select(FD_SETSIZE, readfds, - (void *) 0, (void *) 0, (struct timeval *) 0); + if (delayed) { + time_t now = time(NULL); + time_t delay; + if (delayed->last_attempt > now) + /* Clock updated - retry immediately */ + delayed->last_attempt = now - RETRY_SEC; + delay = delayed->last_attempt + RETRY_SEC - now; + if (delay < 0) + delay = 0; + tv.tv_sec = delay; + } + selret = select(FD_SETSIZE, readfds, NULL, NULL, &tv); + + if (delayed) { + time_t now = time(NULL); + struct delayed *d = delayed; + + if (d->last_attempt + RETRY_SEC <= now) { + delayed = d->next; + d->next = NULL; + nfsd_retry_fh(d); + } + } switch (selret) { case -1: