From patchwork Tue Apr 18 09:33:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Weinberger X-Patchwork-Id: 13215320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6E61C77B76 for ; Tue, 18 Apr 2023 09:34:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230423AbjDRJe1 (ORCPT ); Tue, 18 Apr 2023 05:34:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231363AbjDRJeV (ORCPT ); Tue, 18 Apr 2023 05:34:21 -0400 Received: from lithops.sigma-star.at (lithops.sigma-star.at [195.201.40.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC10259CC for ; Tue, 18 Apr 2023 02:34:15 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by lithops.sigma-star.at (Postfix) with ESMTP id 046BC6431C4A; Tue, 18 Apr 2023 11:34:14 +0200 (CEST) Received: from lithops.sigma-star.at ([127.0.0.1]) by localhost (lithops.sigma-star.at [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 9PY6-9HQh7KW; Tue, 18 Apr 2023 11:34:13 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by lithops.sigma-star.at (Postfix) with ESMTP id F17B36431C58; Tue, 18 Apr 2023 11:34:12 +0200 (CEST) Received: from lithops.sigma-star.at ([127.0.0.1]) by localhost (lithops.sigma-star.at [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id BBppqH7DqBGX; Tue, 18 Apr 2023 11:34:12 +0200 (CEST) Received: from blindfold.corp.sigma-star.at (unknown [82.150.214.1]) by lithops.sigma-star.at (Postfix) with ESMTPSA id 952AE63CC168; Tue, 18 Apr 2023 11:34:12 +0200 (CEST) From: Richard Weinberger To: linux-nfs@vger.kernel.org Cc: david@sigma-star.at, david.oberhollenzer@sigma-star.at, luis.turcitu@appsbroker.com, david.young@appsbroker.com, trond.myklebust@hammerspace.com, anna.schumaker@netapp.com, steved@redhat.com, chris.chilvers@appsbroker.com, Richard Weinberger Subject: [PATCH 2/8] Implement reexport= export option Date: Tue, 18 Apr 2023 11:33:44 +0200 Message-Id: <20230418093350.4550-3-richard@nod.at> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20230418093350.4550-1-richard@nod.at> References: <20230418093350.4550-1-richard@nod.at> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org When re-exporting a NFS volume it is mandatory to specify either a UUID or numerical fsid= option because nfsd is unable to derive an identifier on its own. For NFS cross mounts this becomes a problem because nfsd also needs an identifier for every crossed mount. A common workaround is stating every single subvolume in the exports list too. But this defeats the purpose of the crossmnt option and is tedious. This is where the reexport= tries to help. It offers various strategies to automatically derive a identifier for NFS volumes and sub volumes. Currently two strategies are implemented: 1. auto-fsidnum In this mode mountd/exportd will create a new numerical fsid for a NFS volume and subvolume. The numbers are stored in a database, via fsidd, such that the server will always use the same fsid. The entry in the exports file allowed to skip the fsid= option but stating a UUID is allowed, if needed. This mode has the obvious downside that load balancing is by default not possible since multiple re-exporting NFS servers would generate different ids. It is possible if all load balancers use the same database. This can be achieved by using nfs-utils' fsidd and placing it's sqlit database on a network share which supports file locks or by implementing your own fsidd which is able to provide consistent fsids across multiple re-exporting nfs servers. 2. predefined-fsidnum This mode works just like auto-fsidnum but does not generate ids for you. It helps in the load balancing case. A system administrator has to manually maintain the database and install it on all re-exporting NFS servers. If you have a massive amount of subvolumes this mode will help because you don't have to bloat the exports list. Signed-off-by: Richard Weinberger --- support/export/export.c | 20 ++++++++++++ support/nfs/Makefile.am | 1 + support/nfs/exports.c | 62 ++++++++++++++++++++++++++++++++++++++ systemd/Makefile.am | 2 ++ utils/exportd/Makefile.am | 4 ++- utils/exportfs/Makefile.am | 3 ++ utils/exportfs/exportfs.c | 11 +++++++ utils/mount/Makefile.am | 3 +- utils/mountd/Makefile.am | 2 ++ 9 files changed, 106 insertions(+), 2 deletions(-) diff --git a/support/export/export.c b/support/export/export.c index 03390dfc..3e48c42d 100644 --- a/support/export/export.c +++ b/support/export/export.c @@ -25,6 +25,7 @@ #include "exportfs.h" #include "nfsd_path.h" #include "xlog.h" +#include "reexport.h" exp_hash_table exportlist[MCL_MAXTYPES] = {{NULL, {{NULL,NULL}, }}, }; static int export_hash(char *); @@ -115,6 +116,7 @@ export_read(char *fname, int ignore_hosts) nfs_export *exp; int volumes = 0; + int reexport_found = 0; setexportent(fname, "r"); while ((eep = getexportent(0,1)) != NULL) { @@ -126,7 +128,25 @@ export_read(char *fname, int ignore_hosts) } else warn_duplicated_exports(exp, eep); + + if (eep->e_reexport) + reexport_found = 1; } + + if (reexport_found) { + for (int i = 0; i < MCL_MAXTYPES; i++) { + for (exp = exportlist[i].p_head; exp; exp = exp->m_next) { + if (exp->m_export.e_reexport) + continue; + + if (exp->m_export.e_flags & NFSEXP_FSID) { + xlog(L_ERROR, "When a reexport= option is present no manully assigned numerical fsid= options are allowed"); + return -1; + } + } + } + } + endexportent(); return volumes; diff --git a/support/nfs/Makefile.am b/support/nfs/Makefile.am index 67e3a8e1..2e1577cc 100644 --- a/support/nfs/Makefile.am +++ b/support/nfs/Makefile.am @@ -9,6 +9,7 @@ libnfs_la_SOURCES = exports.c rmtab.c xio.c rpcmisc.c rpcdispatch.c \ svc_socket.c cacheio.c closeall.c nfs_mntent.c \ svc_create.c atomicio.c strlcat.c strlcpy.c libnfs_la_LIBADD = libnfsconf.la +libnfs_la_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport libnfsconf_la_SOURCES = conffile.c xlog.c diff --git a/support/nfs/exports.c b/support/nfs/exports.c index da8ace3a..72e632f4 100644 --- a/support/nfs/exports.c +++ b/support/nfs/exports.c @@ -31,6 +31,7 @@ #include "xlog.h" #include "xio.h" #include "pseudoflavors.h" +#include "reexport.h" #define EXPORT_DEFAULT_FLAGS \ (NFSEXP_READONLY|NFSEXP_ROOTSQUASH|NFSEXP_GATHERED_WRITES|NFSEXP_NOSUBTREECHECK) @@ -104,6 +105,7 @@ static void init_exportent (struct exportent *ee, int fromkernel) ee->e_nsqgids = 0; ee->e_uuid = NULL; ee->e_ttl = default_ttl; + ee->e_reexport = REEXP_NONE; } struct exportent * @@ -313,6 +315,23 @@ putexportent(struct exportent *ep) } if (ep->e_uuid) fprintf(fp, "fsid=%s,", ep->e_uuid); + + if (ep->e_reexport) { + fprintf(fp, "reexport="); + switch (ep->e_reexport) { + case REEXP_AUTO_FSIDNUM: + fprintf(fp, "auto-fsidnum"); + break; + case REEXP_PREDEFINED_FSIDNUM: + fprintf(fp, "predefined-fsidnum"); + break; + default: + xlog(L_ERROR, "unknown reexport method %i", ep->e_reexport); + fprintf(fp, "none"); + } + fprintf(fp, ","); + } + if (ep->e_mountpoint) fprintf(fp, "mountpoint%s%s,", ep->e_mountpoint[0]?"=":"", ep->e_mountpoint); @@ -619,6 +638,7 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr) char *flname = efname?efname:"command line"; int flline = efp?efp->x_line:0; unsigned int active = 0; + int saw_reexport = 0; squids = ep->e_squids; nsquids = ep->e_nsquids; sqgids = ep->e_sqgids; nsqgids = ep->e_nsqgids; @@ -725,6 +745,13 @@ bad_option: } } else if (strncmp(opt, "fsid=", 5) == 0) { char *oe; + + if (saw_reexport) { + xlog(L_ERROR, "%s:%d: 'fsid=' has to be before 'reexport=' %s\n", + flname, flline, opt); + goto bad_option; + } + if (strcmp(opt+5, "root") == 0) { ep->e_fsid = 0; setflags(NFSEXP_FSID, active, ep); @@ -772,6 +799,41 @@ bad_option: } else if (strncmp(opt, "xprtsec=", 8) == 0) { if (!parse_xprtsec(opt + 8, ep)) goto bad_option; + } else if (strncmp(opt, "reexport=", 9) == 0) { + char *strategy = strchr(opt, '='); + + if (!strategy) { + xlog(L_ERROR, "%s:%d: bad option %s\n", + flname, flline, opt); + goto bad_option; + } + strategy++; + + if (saw_reexport) { + xlog(L_ERROR, "%s:%d: only one 'reexport=' is allowed%s\n", + flname, flline, opt); + goto bad_option; + } + + if (strcmp(strategy, "auto-fsidnum") == 0) { + ep->e_reexport = REEXP_AUTO_FSIDNUM; + } else if (strcmp(strategy, "predefined-fsidnum") == 0) { + ep->e_reexport = REEXP_PREDEFINED_FSIDNUM; + } else if (strcmp(strategy, "none") == 0) { + ep->e_reexport = REEXP_NONE; + } else { + xlog(L_ERROR, "%s:%d: bad option %s\n", + flname, flline, strategy); + goto bad_option; + } + + if (reexpdb_apply_reexport_settings(ep, flname, flline) != 0) + goto bad_option; + + if (ep->e_fsid) + setflags(NFSEXP_FSID, active, ep); + + saw_reexport = 1; } else { xlog(L_ERROR, "%s:%d: unknown keyword \"%s\"\n", flname, flline, opt); diff --git a/systemd/Makefile.am b/systemd/Makefile.am index 577c6a22..2e250dca 100644 --- a/systemd/Makefile.am +++ b/systemd/Makefile.am @@ -70,8 +70,10 @@ rpc_pipefs_generator_SOURCES = $(COMMON_SRCS) rpc-pipefs-generator.c nfs_server_generator_LDADD = ../support/export/libexport.a \ ../support/nfs/libnfs.la \ ../support/misc/libmisc.a \ + ../support/reexport/libreexport.a \ $(LIBPTHREAD) + rpc_pipefs_generator_LDADD = ../support/nfs/libnfs.la if INSTALL_SYSTEMD diff --git a/utils/exportd/Makefile.am b/utils/exportd/Makefile.am index c95bdee7..83024958 100644 --- a/utils/exportd/Makefile.am +++ b/utils/exportd/Makefile.am @@ -16,7 +16,9 @@ exportd_SOURCES = exportd.c exportd_LDADD = ../../support/export/libexport.a \ ../../support/nfs/libnfs.la \ ../../support/misc/libmisc.a \ - $(OPTLIBS) $(LIBBLKID) $(LIBPTHREAD) -luuid + ../support/reexport/libreexport.a \ + $(OPTLIBS) $(LIBBLKID) $(LIBPTHREAD) \ + -luuid exportd_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) \ -I$(top_srcdir)/support/export diff --git a/utils/exportfs/Makefile.am b/utils/exportfs/Makefile.am index 96524c72..7f8ce9fa 100644 --- a/utils/exportfs/Makefile.am +++ b/utils/exportfs/Makefile.am @@ -10,6 +10,9 @@ exportfs_SOURCES = exportfs.c exportfs_LDADD = ../../support/export/libexport.a \ ../../support/nfs/libnfs.la \ ../../support/misc/libmisc.a \ + ../../support/reexport/libreexport.a \ $(LIBWRAP) $(LIBNSL) $(LIBPTHREAD) +exportfs_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) -I$(top_srcdir)/support/reexport + MAINTAINERCLEANFILES = Makefile.in diff --git a/utils/exportfs/exportfs.c b/utils/exportfs/exportfs.c index 37b9e4b3..b03a047b 100644 --- a/utils/exportfs/exportfs.c +++ b/utils/exportfs/exportfs.c @@ -38,6 +38,7 @@ #include "exportfs.h" #include "xlog.h" #include "conffile.h" +#include "reexport.h" static void export_all(int verbose); static void exportfs(char *arg, char *options, int verbose); @@ -719,6 +720,16 @@ dump(int verbose, int export_format) c = dumpopt(c, "fsid=%d", ep->e_fsid); if (ep->e_uuid) c = dumpopt(c, "fsid=%s", ep->e_uuid); + if (ep->e_reexport) { + switch (ep->e_reexport) { + case REEXP_AUTO_FSIDNUM: + c = dumpopt(c, "reexport=%s", "auto-fsidnum"); + break; + case REEXP_PREDEFINED_FSIDNUM: + c = dumpopt(c, "reexport=%s", "predefined-fsidnum"); + break; + } + } if (ep->e_mountpoint) c = dumpopt(c, "mountpoint%s%s", ep->e_mountpoint[0]?"=":"", diff --git a/utils/mount/Makefile.am b/utils/mount/Makefile.am index 3101f7ab..5ff1148c 100644 --- a/utils/mount/Makefile.am +++ b/utils/mount/Makefile.am @@ -29,8 +29,9 @@ endif mount_nfs_LDADD = ../../support/nfs/libnfs.la \ ../../support/export/libexport.a \ + ../../support/reexport/libreexport.a \ ../../support/misc/libmisc.a \ - $(LIBTIRPC) + $(LIBTIRPC) $(LIBPTHREAD) mount_nfs_SOURCES = $(mount_common) diff --git a/utils/mountd/Makefile.am b/utils/mountd/Makefile.am index 13b25c90..197ef29b 100644 --- a/utils/mountd/Makefile.am +++ b/utils/mountd/Makefile.am @@ -17,9 +17,11 @@ mountd_SOURCES = mountd.c mount_dispatch.c rmtab.c \ mountd_LDADD = ../../support/export/libexport.a \ ../../support/nfs/libnfs.la \ ../../support/misc/libmisc.a \ + ../../support/reexport/libreexport.a \ $(OPTLIBS) \ $(LIBBSD) $(LIBWRAP) $(LIBNSL) $(LIBBLKID) -luuid $(LIBTIRPC) \ $(LIBPTHREAD) + mountd_CPPFLAGS = $(AM_CPPFLAGS) $(CPPFLAGS) \ -I$(top_builddir)/support/include \ -I$(top_srcdir)/support/export