From patchwork Thu Feb 27 21:14:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11410665 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5157517E0 for ; Thu, 27 Feb 2020 21:43:51 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A6F0246A1 for ; Thu, 27 Feb 2020 21:43:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A6F0246A1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 06FF63492EF; Thu, 27 Feb 2020 13:35:12 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4993021FC2C for ; Thu, 27 Feb 2020 13:20:15 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id B68BC8AAF; Thu, 27 Feb 2020 16:18:17 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id B55A647C; Thu, 27 Feb 2020 16:18:17 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Thu, 27 Feb 2020 16:14:04 -0500 Message-Id: <1582838290-17243-377-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> References: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 376/622] lustre: obdclass: protect imp_sec using rwlock_t X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Dongyang , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Li Dongyang We've seen spinlock contention on imp_lock in sptlrpc_import_sec_ref(), introduce a new rwlock imp_sec_lock to protect imp_sec instead of using imp_lock. This patch also removes imp_sec_mutex from obd_import, which is not needed, to avoid confusion between imp_sec_lock/mutex. WC-bug-id: https://jira.whamcloud.com/browse/LU-11775 Lustre-commit: 8ed361345154 ("LU-11775 obdclass: protect imp_sec using rwlock_t") Signed-off-by: Li Dongyang Reviewed-on: https://review.whamcloud.com/33861 Reviewed-by: Alexey Lyashkov Reviewed-by: Alexandr Boyko Reviewed-by: Sebastien Buisson Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_import.h | 2 +- fs/lustre/obdclass/genops.c | 2 +- fs/lustre/ptlrpc/sec.c | 15 ++++++--------- fs/lustre/ptlrpc/sec_config.c | 4 ++-- 4 files changed, 10 insertions(+), 13 deletions(-) diff --git a/fs/lustre/include/lustre_import.h b/fs/lustre/include/lustre_import.h index f16d621..ff171d1 100644 --- a/fs/lustre/include/lustre_import.h +++ b/fs/lustre/include/lustre_import.h @@ -206,7 +206,7 @@ struct obd_import { * @{ */ struct ptlrpc_sec *imp_sec; - struct mutex imp_sec_mutex; + rwlock_t imp_sec_lock; time64_t imp_sec_expire; /** @} */ diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index fd9dd96..2b1175f 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -997,7 +997,7 @@ struct obd_import *class_new_import(struct obd_device *obd) imp->imp_last_success_conn = 0; imp->imp_state = LUSTRE_IMP_NEW; imp->imp_obd = class_incref(obd, "import", imp); - mutex_init(&imp->imp_sec_mutex); + rwlock_init(&imp->imp_sec_lock); init_waitqueue_head(&imp->imp_recovery_waitq); INIT_WORK(&imp->imp_zombie_work, obd_zombie_imp_cull); diff --git a/fs/lustre/ptlrpc/sec.c b/fs/lustre/ptlrpc/sec.c index 789b5cb..d82809f 100644 --- a/fs/lustre/ptlrpc/sec.c +++ b/fs/lustre/ptlrpc/sec.c @@ -303,13 +303,13 @@ static int import_sec_check_expire(struct obd_import *imp) { int adapt = 0; - spin_lock(&imp->imp_lock); + write_lock(&imp->imp_sec_lock); if (imp->imp_sec_expire && imp->imp_sec_expire < ktime_get_real_seconds()) { adapt = 1; imp->imp_sec_expire = 0; } - spin_unlock(&imp->imp_lock); + write_unlock(&imp->imp_sec_lock); if (!adapt) return 0; @@ -1317,9 +1317,9 @@ struct ptlrpc_sec *sptlrpc_import_sec_ref(struct obd_import *imp) { struct ptlrpc_sec *sec; - spin_lock(&imp->imp_lock); + read_lock(&imp->imp_sec_lock); sec = sptlrpc_sec_get(imp->imp_sec); - spin_unlock(&imp->imp_lock); + read_unlock(&imp->imp_sec_lock); return sec; } @@ -1332,10 +1332,10 @@ static void sptlrpc_import_sec_install(struct obd_import *imp, LASSERT_ATOMIC_POS(&sec->ps_refcount); - spin_lock(&imp->imp_lock); + write_lock(&imp->imp_sec_lock); old_sec = imp->imp_sec; imp->imp_sec = sec; - spin_unlock(&imp->imp_lock); + write_unlock(&imp->imp_sec_lock); if (old_sec) { sptlrpc_sec_kill(old_sec); @@ -1455,8 +1455,6 @@ int sptlrpc_import_sec_adapt(struct obd_import *imp, sptlrpc_flavor2name(&sf, str, sizeof(str))); } - mutex_lock(&imp->imp_sec_mutex); - newsec = sptlrpc_sec_create(imp, svc_ctx, &sf, sp); if (newsec) { sptlrpc_import_sec_install(imp, newsec); @@ -1467,7 +1465,6 @@ int sptlrpc_import_sec_adapt(struct obd_import *imp, rc = -EPERM; } - mutex_unlock(&imp->imp_sec_mutex); out: sptlrpc_sec_put(sec); return rc; diff --git a/fs/lustre/ptlrpc/sec_config.c b/fs/lustre/ptlrpc/sec_config.c index e4b1a075..9ced6c7 100644 --- a/fs/lustre/ptlrpc/sec_config.c +++ b/fs/lustre/ptlrpc/sec_config.c @@ -846,11 +846,11 @@ void sptlrpc_conf_client_adapt(struct obd_device *obd) imp = obd->u.cli.cl_import; if (imp) { - spin_lock(&imp->imp_lock); + write_lock(&imp->imp_sec_lock); if (imp->imp_sec) imp->imp_sec_expire = ktime_get_real_seconds() + SEC_ADAPT_DELAY; - spin_unlock(&imp->imp_lock); + write_unlock(&imp->imp_sec_lock); } up_read(&obd->u.cli.cl_sem);