From patchwork Sat Jun 13 16:27:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11602911 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37C91138C for ; Sat, 13 Jun 2020 16:28:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0ED282078A for ; Sat, 13 Jun 2020 16:28:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0ED282078A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id F222721F211; Sat, 13 Jun 2020 09:27:54 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 78DF521EBAC for ; Sat, 13 Jun 2020 09:27:28 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id B29116DF; Sat, 13 Jun 2020 12:27:19 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id AF8C7488; Sat, 13 Jun 2020 12:27:19 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Sat, 13 Jun 2020 12:27:15 -0400 Message-Id: <1592065636-28333-20-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1592065636-28333-1-git-send-email-jsimmons@infradead.org> References: <1592065636-28333-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 19/20] lustre: mdc: allow setting max_mod_rpcs_in_flight larger X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Andreas Dilger Allow setting mdc.*.max_mod_rpcs_in_flight > mdc.*.max_rpcs_in_flight by increasing the latter value, rather than returning an error and telling the user to do that. This matches the similar behavior if mdc.*.max_rpcs_in_flight is reduced lower than max_mod_rpcs_in_flight. If there are multiple MDTs, the "mdc.*.max_mod_rpcs_in_flight" param may be set from e.g. the MDT0000 config log before MDT0001 is fully configured, catching MDT0001 with ocd_maxmodrpcs = 0 before the OCD from the MDT has been filled in, and incorrectly trigger an error. If seen during setup, allow ocd_maxmodrpcs = (max_rpcs_in_flight - 1), since this will be fixed up later if mdc.*.max_rpcs_in_flight is set smaller in the config log (if set larger it doesn't matter). WC-bug-id: https://jira.whamcloud.com/browse/LU-13503 Lustre-commit: 6d314902e6d19 ("LU-13503 mdc: allow setting max_mod_rpcs_in_flight larger") Signed-off-by: Andreas Dilger Signed-off-by: Jian Yu Reviewed-on: https://review.whamcloud.com/38455 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/obdclass/genops.c | 50 +++++++++++++++++++++++++++++++++------------ 1 file changed, 37 insertions(+), 13 deletions(-) diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 607f0d6..1647fe9 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -1324,7 +1324,6 @@ u32 obd_get_max_rpcs_in_flight(struct client_obd *cli) int obd_set_max_rpcs_in_flight(struct client_obd *cli, u32 max) { struct obd_request_slot_waiter *orsw; - const char *type_name; u32 old; int diff; int rc; @@ -1333,14 +1332,18 @@ int obd_set_max_rpcs_in_flight(struct client_obd *cli, u32 max) if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; - type_name = cli->cl_import->imp_obd->obd_type->typ_name; - if (!strcmp(type_name, LUSTRE_MDC_NAME)) { + CDEBUG(D_INFO, "%s: max = %hu max_mod = %u rif = %u\n", + cli->cl_import->imp_obd->obd_name, max, + cli->cl_max_mod_rpcs_in_flight, cli->cl_max_rpcs_in_flight); + + if (strcmp(cli->cl_import->imp_obd->obd_type->typ_name, + LUSTRE_MDC_NAME) == 0) { /* * adjust max_mod_rpcs_in_flight to ensure it is always * strictly lower that max_rpcs_in_flight */ if (max < 2) { - CERROR("%s: cannot set max_rpcs_in_flight to 1 because it must be higher than max_mod_rpcs_in_flight value\n", + CERROR("%s: cannot set mdc.*.max_rpcs_in_flight=1\n", cli->cl_import->imp_obd->obd_name); return -ERANGE; } @@ -1385,22 +1388,43 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, u16 max) if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; - /* cannot exceed or equal max_rpcs_in_flight */ + CDEBUG(D_INFO, "%s: max = %hu flags = %llx, max_mod = %u rif = %u\n", + cli->cl_import->imp_obd->obd_name, max, ocd->ocd_connect_flags, + ocd->ocd_maxmodrpcs, cli->cl_max_rpcs_in_flight); + + if (max == OBD_MAX_RIF_MAX) + max = OBD_MAX_RIF_MAX - 1; + + /* Cannot exceed or equal max_rpcs_in_flight. If we are asked to + * increase this value, also bump up max_rpcs_in_flight to match. + */ if (max >= cli->cl_max_rpcs_in_flight) { - CERROR("%s: can't set max_mod_rpcs_in_flight to a value (%hu) higher or equal to max_rpcs_in_flight value (%u)\n", - cli->cl_import->imp_obd->obd_name, - max, cli->cl_max_rpcs_in_flight); + CDEBUG(D_INFO, + "%s: increasing max_rpcs_in_flight=%hu to allow larger max_mod_rpcs_in_flight=%u\n", + cli->cl_import->imp_obd->obd_name, max + 1, max); return -ERANGE; } - /* cannot exceed max modify RPCs in flight supported by the server */ - ocd = &cli->cl_import->imp_connect_data; - if (ocd->ocd_connect_flags & OBD_CONNECT_MULTIMODRPCS) + /* cannot exceed max modify RPCs in flight supported by the server, + * but verify ocd_connect_flags is at least initialized first. If + * not, allow it and fix value later in ptlrpc_connect_set_flags(). + */ + if (!ocd->ocd_connect_flags) { + maxmodrpcs = cli->cl_max_rpcs_in_flight - 1; + } else if (ocd->ocd_connect_flags & OBD_CONNECT_MULTIMODRPCS) { maxmodrpcs = ocd->ocd_maxmodrpcs; - else + if (maxmodrpcs == 0) { /* connection not finished yet */ + maxmodrpcs = cli->cl_max_rpcs_in_flight - 1; + CDEBUG(D_INFO, + "%s: partial connect, assume maxmodrpcs=%hu\n", + cli->cl_import->imp_obd->obd_name, maxmodrpcs); + } + } else { maxmodrpcs = 1; + } + if (max > maxmodrpcs) { - CERROR("%s: can't set max_mod_rpcs_in_flight to a value (%hu) higher than max_mod_rpcs_per_client value (%hu) returned by the server at connection\n", + CERROR("%s: can't set max_mod_rpcs_in_flight=%hu higher than ocd_maxmodrpcs=%hu returned by the server at connection\n", cli->cl_import->imp_obd->obd_name, max, maxmodrpcs); return -ERANGE;