From patchwork Wed Oct 19 05:19:42 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Ren X-Patchwork-Id: 9383603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 76B00600CA for ; Wed, 19 Oct 2016 05:21:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69A0D28FF7 for ; Wed, 19 Oct 2016 05:21:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5E3A1296E6; Wed, 19 Oct 2016 05:21:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E590A28FF7 for ; Wed, 19 Oct 2016 05:21:30 +0000 (UTC) Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u9J5LEad003223 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 19 Oct 2016 05:21:16 GMT Received: from oss.oracle.com (oss-old-reserved.oracle.com [137.254.22.2]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u9J5LDQK003976 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 19 Oct 2016 05:21:14 GMT Received: from localhost ([127.0.0.1] helo=lb-oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1bwjJR-000327-Uj; Tue, 18 Oct 2016 22:21:13 -0700 Received: from userv0021.oracle.com ([156.151.31.71]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1bwjId-00030c-Mg for ocfs2-devel@oss.oracle.com; Tue, 18 Oct 2016 22:20:23 -0700 Received: from aserp1030.oracle.com (aserp1030.oracle.com [141.146.126.68]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id u9J5KMoe025450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 19 Oct 2016 05:20:22 GMT Received: from userp2040.oracle.com (userp2040.oracle.com [156.151.31.90]) by aserp1030.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u9J5KLli031577 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 19 Oct 2016 05:20:22 GMT Received: from pps.filterd (userp2040.oracle.com [127.0.0.1]) by userp2040.oracle.com (8.16.0.17/8.16.0.17) with SMTP id u9J5HcNF044822 for ; Wed, 19 Oct 2016 05:20:21 GMT Authentication-Results: oracle.com; spf=pass smtp.mailfrom=zren@suse.com Received: from prv3-mh.provo.novell.com (victor.provo.novell.com [137.65.250.26]) by userp2040.oracle.com with ESMTP id 2661k80rqe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Oct 2016 05:20:21 +0000 Received: from laptop.apac.novell.com (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by prv3-mh.provo.novell.com with ESMTP (TLS encrypted); Tue, 18 Oct 2016 23:20:09 -0600 From: Eric Ren To: ocfs2-devel@oss.oracle.com Date: Wed, 19 Oct 2016 13:19:42 +0800 Message-Id: <1476854382-28101-3-git-send-email-zren@suse.com> X-Mailer: git-send-email 2.6.6 In-Reply-To: <1476854382-28101-1-git-send-email-zren@suse.com> References: <1476854382-28101-1-git-send-email-zren@suse.com> X-ServerName: victor.provo.novell.com X-Proofpoint-SPF-Result: pass X-Proofpoint-SPF-Record: v=spf1 include:microfocus.com ~all X-Proofpoint-Virus-Version: vendor=nai engine=5800 definitions=8322 signatures=670725 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1610190093 Subject: [Ocfs2-devel] [DRAFT 2/2] ocfs2: fix deadlock caused by recursive cluster locking X-BeenThere: ocfs2-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ocfs2-devel-bounces@oss.oracle.com Errors-To: ocfs2-devel-bounces@oss.oracle.com X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Virus-Scanned: ClamAV using ClamSMTP The deadlock issue happens when running discontiguous block group testing on multiple nodes. The easier way to reproduce is to do "chmod -R 777 /mnt/ocfs2" things like this on multiple nodes at the same time by pssh. This is indeed another deadlock caused by: commit 743b5f1434f5 ("ocfs2: take inode lock in ocfs2_iop_set/get_acl()"). The reason had been explained well by Tariq Saeed in this thread: https://oss.oracle.com/pipermail/ocfs2-devel/2015-September/011085.html For this case, the ocfs2_inode_lock() is misused recursively as below: do_sys_open do_filp_open path_openat may_open inode_permission __inode_permission ocfs2_permission <====== ocfs2_inode_lock() generic_permission get_acl ocfs2_iop_get_acl <====== ocfs2_inode_lock() ocfs2_inode_lock_full_nested <===== deadlock if a remote EX request comes between two ocfs2_inode_lock() Fix by checking if the cluster lock has been acquired aready in the call-chain path. Fixes: commit 743b5f1434f5 ("ocfs2: take inode lock in ocfs2_iop_set/get_acl()") Signed-off-by: Eric Ren --- fs/ocfs2/acl.c | 39 +++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-) diff --git a/fs/ocfs2/acl.c b/fs/ocfs2/acl.c index bed1fcb..7e3544e 100644 --- a/fs/ocfs2/acl.c +++ b/fs/ocfs2/acl.c @@ -283,16 +283,24 @@ int ocfs2_set_acl(handle_t *handle, int ocfs2_iop_set_acl(struct inode *inode, struct posix_acl *acl, int type) { struct buffer_head *bh = NULL; + struct ocfs2_holder *oh; + struct ocfs2_lock_res *lockres = &OCFS2_I(inode)->ip_inode_lockres; int status = 0; - status = ocfs2_inode_lock(inode, &bh, 1); - if (status < 0) { - if (status != -ENOENT) - mlog_errno(status); - return status; + oh = ocfs2_is_locked_by_me(lockres); + if (!oh) { + status = ocfs2_inode_lock(inode, &bh, 1); + if (status < 0) { + if (status != -ENOENT) + mlog_errno(status); + return status; + } } + status = ocfs2_set_acl(NULL, inode, bh, type, acl, NULL, NULL); - ocfs2_inode_unlock(inode, 1); + + if (!oh) + ocfs2_inode_unlock(inode, 1); brelse(bh); return status; } @@ -302,21 +310,28 @@ struct posix_acl *ocfs2_iop_get_acl(struct inode *inode, int type) struct ocfs2_super *osb; struct buffer_head *di_bh = NULL; struct posix_acl *acl; + struct ocfs2_holder *oh; + struct ocfs2_lock_res *lockres = &OCFS2_I(inode)->ip_inode_lockres; int ret; osb = OCFS2_SB(inode->i_sb); if (!(osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL)) return NULL; - ret = ocfs2_inode_lock(inode, &di_bh, 0); - if (ret < 0) { - if (ret != -ENOENT) - mlog_errno(ret); - return ERR_PTR(ret); + + oh = ocfs2_is_locked_by_me(lockres); + if (!oh) { + ret = ocfs2_inode_lock(inode, &di_bh, 0); + if (ret < 0) { + if (ret != -ENOENT) + mlog_errno(ret); + return ERR_PTR(ret); + } } acl = ocfs2_get_acl_nolock(inode, type, di_bh); - ocfs2_inode_unlock(inode, 0); + if (!oh) + ocfs2_inode_unlock(inode, 0); brelse(di_bh); return acl; }