From patchwork Wed Apr 16 04:03:36 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 3997311 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id B3ED4BFF02 for ; Wed, 16 Apr 2014 04:19:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C450920200 for ; Wed, 16 Apr 2014 04:19:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D819B20219 for ; Wed, 16 Apr 2014 04:19:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751203AbaDPESh (ORCPT ); Wed, 16 Apr 2014 00:18:37 -0400 Received: from cantor2.suse.de ([195.135.220.15]:38800 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751181AbaDPESd (ORCPT ); Wed, 16 Apr 2014 00:18:33 -0400 Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 715C0AB0E; Wed, 16 Apr 2014 04:18:32 +0000 (UTC) From: NeilBrown To: linux-mm@kvack.org, linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org Date: Wed, 16 Apr 2014 14:03:36 +1000 Subject: [PATCH 06/19] nfsd: set PF_FSTRANS for nfsd threads. Cc: xfs@oss.sgi.com cc: Peter Zijlstra , Ingo Molnar Message-ID: <20140416040336.10604.60493.stgit@notabene.brown> In-Reply-To: <20140416033623.10604.69237.stgit@notabene.brown> References: <20140416033623.10604.69237.stgit@notabene.brown> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a localhost mount is present, then it is easy to deadlock NFS by nfsd entering direct reclaim and calling nfs_release_page() which requires nfsd to perform an fsync() (which it cannot do because it is reclaiming memory). By setting PF_FSTRANS we stop the memory allocator from ever attempting any FS operation would could deadlock. We need this flag set for any thread which is handling a request from the local host, but we also need to always have it for at least 1 or 2 threads so that we don't end up with all threads blocked in allocation. When we set PF_FSTRANS we also tell lockdep that we are handling reclaim so that it can detect deadlocks for us. Signed-off-by: NeilBrown --- fs/nfsd/nfssvc.c | 18 ++++++++++++++++++ include/linux/sunrpc/svc.h | 1 + net/sunrpc/svc.c | 6 ++++++ 3 files changed, 25 insertions(+) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index 9a4a5f9e7468..6af8bc2daf7d 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -565,6 +565,8 @@ nfsd(void *vrqstp) struct svc_xprt *perm_sock = list_entry(rqstp->rq_server->sv_permsocks.next, typeof(struct svc_xprt), xpt_list); struct net *net = perm_sock->xpt_net; int err; + unsigned int pflags = 0; + gfp_t reclaim_state = 0; /* Lock module and set up kernel thread */ mutex_lock(&nfsd_mutex); @@ -611,14 +613,30 @@ nfsd(void *vrqstp) ; if (err == -EINTR) break; + if (rqstp->rq_local && !current_test_flags(PF_FSTRANS)) { + current_set_flags_nested(&pflags, PF_FSTRANS); + atomic_inc(&rqstp->rq_pool->sp_nr_fstrans); + reclaim_state = lockdep_set_current_reclaim_state(GFP_KERNEL); + } validate_process_creds(); svc_process(rqstp); validate_process_creds(); + if (current_test_flags(PF_FSTRANS) && + atomic_dec_if_positive(&rqstp->rq_pool->sp_nr_fstrans) >= 0) { + current_restore_flags_nested(&pflags, PF_FSTRANS); + lockdep_restore_current_reclaim_state(reclaim_state); + } } /* Clear signals before calling svc_exit_thread() */ flush_signals(current); + if (current_test_flags(PF_FSTRANS)) { + current_restore_flags_nested(&pflags, PF_FSTRANS); + lockdep_restore_current_reclaim_state(reclaim_state); + atomic_dec(&rqstp->rq_pool->sp_nr_fstrans); + } + mutex_lock(&nfsd_mutex); nfsdstats.th_cnt --; diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index a0dbbd1e00e9..4b274aba51dd 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -48,6 +48,7 @@ struct svc_pool { struct list_head sp_threads; /* idle server threads */ struct list_head sp_sockets; /* pending sockets */ unsigned int sp_nrthreads; /* # of threads in pool */ + atomic_t sp_nr_fstrans; /* # threads with PF_FSTRANS */ struct list_head sp_all_threads; /* all server threads */ struct svc_pool_stats sp_stats; /* statistics on pool operation */ int sp_task_pending;/* has pending task */ diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 5de6801cd924..8b13f35b6cbb 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -477,6 +477,12 @@ __svc_create(struct svc_program *prog, unsigned int bufsize, int npools, INIT_LIST_HEAD(&pool->sp_threads); INIT_LIST_HEAD(&pool->sp_sockets); INIT_LIST_HEAD(&pool->sp_all_threads); + /* The number of threads with PF_FSTRANS set + * should never be reduced below 2, except when + * threads exit. So we use atomic_dec_if_positive() + * on this value. + */ + atomic_set(&pool->sp_nr_fstrans, -2); spin_lock_init(&pool->sp_lock); }