From patchwork Thu Feb 27 21:12:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11410881 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1148924 for ; Thu, 27 Feb 2020 21:49:42 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89A4F24690 for ; Thu, 27 Feb 2020 21:49:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89A4F24690 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id DA40F34A77B; Thu, 27 Feb 2020 13:41:09 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6F80521C9CA for ; Thu, 27 Feb 2020 13:19:51 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id DEC508A43; Thu, 27 Feb 2020 16:18:16 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id DC65746D; Thu, 27 Feb 2020 16:18:16 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Thu, 27 Feb 2020 16:12:51 -0500 Message-Id: <1582838290-17243-304-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> References: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 303/622] lustre: llite: Revalidate dentries in ll_intent_file_open X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin We might get a lookup lock in response to our open request and we definitely want to ensure that our dentry is valid, so it could actually be matched by dcache code in future operations. Benchmark results: This patch can significantly improve open-create + stat on the same client. This patch in combination with two others: https://review.whamcloud.com/#/c/33584 https://review.whamcloud.com/#/c/33585 Improves the 'stat' side of open-create + stat by >10x. Without patches (master branch commit 26a7abe): mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d /cache1/out/ -F -C -T -v -w 32k Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 3838.205 3838.204 3838.204 0.000 File stat : 33459.289 33459.249 33459.271 0.011 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 3146.841 3146.841 3146.841 0.000 Tree removal : 0.000 0.000 0.000 0.000 With the three patches: mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d /cache1/out/ -F -C -T -v -w 32k SUMMARY rate: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 3822.440 3822.439 3822.440 0.000 File stat : 350620.140 350615.980 350617.193 1.051 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 2076.727 2076.727 2076.727 0.000 Tree removal : 0.000 0.000 0.000 0.000 Note 33K stats/second vs 350K stats/second. ls -l time of the mdtest directory is also reduced from 23.5 seconds to 5.8 seconds. WC-bug-id: https://jira.whamcloud.com/browse/LU-10948 Lustre-commit: 14ca3157b21d ("LU-10948 llite: Revalidate dentries in ll_intent_file_open") Signed-off-by: Oleg Drokin Reviewed-on: https://review.whamcloud.com/32157 Reviewed-by: Mike Pershin Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index e9d0ff9..191b0f9 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -419,25 +419,12 @@ void ll_dom_finish_open(struct inode *inode, struct ptlrpc_request *req, struct page *vmpage; struct niobuf_remote *rnb; char *data; - struct lustre_handle lockh; - struct ldlm_lock *lock; unsigned long index, start; struct niobuf_local lnb; - bool dom_lock = false; if (!obj) return; - if (it->it_lock_mode != 0) { - lockh.cookie = it->it_lock_handle; - lock = ldlm_handle2lock(&lockh); - if (lock) - dom_lock = ldlm_has_dom(lock); - LDLM_LOCK_PUT(lock); - } - if (!dom_lock) - return; - if (!req_capsule_has_field(&req->rq_pill, &RMF_NIOBUF_INLINE, RCL_SERVER)) return; @@ -576,8 +563,27 @@ static int ll_intent_file_open(struct dentry *de, void *lmm, int lmmsize, rc = ll_prep_inode(&inode, req, NULL, itp); if (!rc && itp->it_lock_mode) { - ll_dom_finish_open(d_inode(de), req, itp); + struct lustre_handle handle = {.cookie = itp->it_lock_handle}; + struct ldlm_lock *lock; + bool has_dom_bit = false; + + /* If we got a lock back and it has a LOOKUP bit set, + * make sure the dentry is marked as valid so we can find it. + * We don't need to care about actual hashing since other bits + * of kernel will deal with that later. + */ + lock = ldlm_handle2lock(&handle); + if (lock) { + has_dom_bit = ldlm_has_dom(lock); + if (lock->l_policy_data.l_inodebits.bits & + MDS_INODELOCK_LOOKUP) + d_lustre_revalidate(de); + + LDLM_LOCK_PUT(lock); + } ll_set_lock_data(sbi->ll_md_exp, inode, itp, NULL); + if (has_dom_bit) + ll_dom_finish_open(inode, req, itp); } out: