From patchwork Tue Feb 9 21:23:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Dickson X-Patchwork-Id: 12079335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01030C433E6 for ; Tue, 9 Feb 2021 22:26:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B428664DA8 for ; Tue, 9 Feb 2021 22:26:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233785AbhBIWZb (ORCPT ); Tue, 9 Feb 2021 17:25:31 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:54428 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233840AbhBIWJB (ORCPT ); Tue, 9 Feb 2021 17:09:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1612908406; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wE+fMXUcVEsuXdUx1P7iy+Wcubauj8G6lDryuKeSjH4=; b=gmHtol3b5bMwjjyAMEq9YZMxL3Mk36gyFo2rw1mDGVLjPw1ExyGTMgWPqxnIyavgXQrwaN +Y9zmdH0x6Yh5QlsIKzm0B43GgdmWTjjYJkg6Nd+3PDAO80mFeUY76IDPOYW9ND65cOH1f XtPXTinh1hFfJ/s0VlNHRTNBRk8cmk4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-265-mIXxMFsSNtC27Uiv13Sliw-1; Tue, 09 Feb 2021 16:22:17 -0500 X-MC-Unique: mIXxMFsSNtC27Uiv13Sliw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 83653CC620 for ; Tue, 9 Feb 2021 21:22:16 +0000 (UTC) Received: from madhat.home.dicksonnet.net (ovpn-113-50.phx2.redhat.com [10.3.113.50]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4396A3828 for ; Tue, 9 Feb 2021 21:22:16 +0000 (UTC) From: Steve Dickson To: Linux NFS Mailing list Subject: [PATCH 3/5] exportd: multiple threads Date: Tue, 9 Feb 2021 16:23:40 -0500 Message-Id: <20210209212342.233111-4-steved@redhat.com> In-Reply-To: <20210209212342.233111-1-steved@redhat.com> References: <20210209212342.233111-1-steved@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Ported the multiple thread code from mountd (commit 11d34d11) Signed-off-by: Steve Dickson --- nfs.conf | 3 + systemd/nfs.conf.man | 9 +++ utils/exportd/exportd.c | 118 ++++++++++++++++++++++++++++++++++++-- utils/exportd/exportd.man | 7 +++ 4 files changed, 133 insertions(+), 4 deletions(-) diff --git a/nfs.conf b/nfs.conf index 9fcf1bf..4b344fa 100644 --- a/nfs.conf +++ b/nfs.conf @@ -29,6 +29,9 @@ # port=0 # udp-port=0 # +[exportd] +# debug="all|auth|call|general|parse" +# threads=1 [mountd] # debug="all|auth|call|general|parse" # manage-gids=n diff --git a/systemd/nfs.conf.man b/systemd/nfs.conf.man index 16e0ec4..a4379fd 100644 --- a/systemd/nfs.conf.man +++ b/systemd/nfs.conf.man @@ -128,6 +128,15 @@ then the client will be able to mount the path as but on the server, this will resolve to the path .BR /my/root/filesystem . +.TP +.B exportd +Recognized values: +.B threads + +See +.BR exportd (8) +for details. + .TP .B nfsdcltrack Recognized values: diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c index 150938c..bf5f431 100644 --- a/utils/exportd/exportd.c +++ b/utils/exportd/exportd.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #include "nfslib.h" #include "conffile.h" @@ -26,6 +28,13 @@ extern void my_svc_run(void); struct state_paths etab; struct state_paths rmtab; +/* Number of mountd threads to start. Default is 1 and + * that's probably enough unless you need hundreds of + * clients to be able to mount at once. */ +static int num_threads = 1; +/* Arbitrary limit on number of threads */ +#define MAX_THREADS 64 + int manage_gids; int use_ipaddr = -1; @@ -34,6 +43,7 @@ static struct option longopts[] = { "foreground", 0, 0, 'F' }, { "debug", 1, 0, 'd' }, { "help", 0, 0, 'h' }, + { "num-threads", 1, 0, 't' }, { NULL, 0, 0, 0 } }; @@ -42,10 +52,85 @@ static struct option longopts[] = */ inline static void set_signals(void); +/* Wait for all worker child processes to exit and reap them */ +static void +wait_for_workers (void) +{ + int status; + pid_t pid; + + for (;;) { + + pid = waitpid(0, &status, 0); + + if (pid < 0) { + if (errno == ECHILD) + return; /* no more children */ + xlog(L_FATAL, "mountd: can't wait: %s\n", + strerror(errno)); + } + + /* Note: because we SIG_IGN'd SIGCHLD earlier, this + * does not happen on 2.6 kernels, and waitpid() blocks + * until all the children are dead then returns with + * -ECHILD. But, we don't need to do anything on the + * death of individual workers, so we don't care. */ + xlog(L_NOTICE, "mountd: reaped child %d, status %d\n", + (int)pid, status); + } +} + +/* Fork num_threads worker children and wait for them */ +static void +fork_workers(void) +{ + int i; + pid_t pid; + + xlog(L_NOTICE, "mountd: starting %d threads\n", num_threads); + + for (i = 0 ; i < num_threads ; i++) { + pid = fork(); + if (pid < 0) { + xlog(L_FATAL, "mountd: cannot fork: %s\n", + strerror(errno)); + } + if (pid == 0) { + /* worker child */ + + /* Re-enable the default action on SIGTERM et al + * so that workers die naturally when sent them. + * Only the parent unregisters with pmap and + * hence needs to do special SIGTERM handling. */ + struct sigaction sa; + sa.sa_handler = SIG_DFL; + sa.sa_flags = 0; + sigemptyset(&sa.sa_mask); + sigaction(SIGHUP, &sa, NULL); + sigaction(SIGINT, &sa, NULL); + sigaction(SIGTERM, &sa, NULL); + + /* fall into my_svc_run in caller */ + return; + } + } + + /* in parent */ + wait_for_workers(); + xlog(L_NOTICE, "exportd: no more workers, exiting\n"); + exit(0); +} + static void killer (int sig) { + if (num_threads > 1) { + /* play Kronos and eat our children */ + kill(0, SIGTERM); + wait_for_workers(); + } xlog (L_NOTICE, "Caught signal %d, exiting.", sig); + exit(0); } static void @@ -78,10 +163,20 @@ static void usage(const char *prog, int n) { fprintf(stderr, - "Usage: %s [-f|--foreground] [-h|--help] [-d kind|--debug kind]\n", prog); + "Usage: %s [-f|--foreground] [-h|--help] [-d kind|--debug kind]\n" +" [-t num|--num-threads=num]\n", prog); exit(n); } +inline static void +read_exportd_conf(char *progname) +{ + conf_init_file(NFS_CONFFILE); + + xlog_from_conffile(progname); + + num_threads = conf_get_num("exportd", "threads", num_threads); +} int main(int argc, char **argv) { @@ -98,10 +193,10 @@ main(int argc, char **argv) /* Initialize logging. */ xlog_open(progname); - conf_init_file(NFS_CONFFILE); - xlog_from_conffile(progname); + /* Read in config setting */ + read_exportd_conf(progname); - while ((c = getopt_long(argc, argv, "d:fh", longopts, NULL)) != EOF) { + while ((c = getopt_long(argc, argv, "d:fht:", longopts, NULL)) != EOF) { switch (c) { case 'd': xlog_sconfig(optarg, 1); @@ -112,6 +207,9 @@ main(int argc, char **argv) case 'h': usage(progname, 0); break; + case 't': + num_threads = atoi (optarg); + break; case '?': default: usage(progname, 1); @@ -132,6 +230,18 @@ main(int argc, char **argv) set_signals(); daemon_ready(); + /* silently bounds check num_threads */ + if (foreground) + num_threads = 1; + else if (num_threads < 1) + num_threads = 1; + else if (num_threads > MAX_THREADS) + num_threads = MAX_THREADS; + + if (num_threads > 1) + fork_workers(); + + /* Open files now to avoid sharing descriptors among forked processes */ cache_open(); diff --git a/utils/exportd/exportd.man b/utils/exportd/exportd.man index 96e133c..fae47d0 100644 --- a/utils/exportd/exportd.man +++ b/utils/exportd/exportd.man @@ -44,6 +44,13 @@ Run in foreground (do not daemonize) .TP .B \-h " or " \-\-help Display usage message. +.TP +.BR "\-t N" " or " "\-\-num\-threads=N " or " \-\-num\-threads N " +This option specifies the number of worker threads that rpc.mountd +spawns. The default is 1 thread, which is probably enough. More +threads are usually only needed for NFS servers which need to handle +mount storms of hundreds of NFS mounts in a few seconds, or when +your DNS server is slow or unreliable. .SH CONFIGURATION FILE Many of the options that can be set on the command line can also be controlled through values set in the