From patchwork Wed May 22 13:54:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Haakon Bugge X-Patchwork-Id: 13670862 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FDC61411F8; Wed, 22 May 2024 13:55:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386112; cv=none; b=un1mAZN+q5netJ4+ONH0EPSQR22pI6le51r7SRmm01E5T0+y114QdDLt8z8IUaoMHJl1/sjule4tDzvS7YLZQX24SP88A5siSFrNOgGzbbw4JbQWEziKZ/K0qXHWzWKaCpLqt9nreTgCg3Uh3GFNjRktgOqqFtMKcrPVkpqW/TA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386112; c=relaxed/simple; bh=Fv4RgdofNF0g1JF1RNb7kP1z5vab40UP16jtAY2qMKY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=s+fYvpcU2F56TPWHTuL4cqPtDgWMQQ/eo6RhYCgYuC3rlcCwB4CyBhgln1mpdiZ3k4YB7VJiPD1yVOKRKTZDxn+DYie/Pd6lG1c2Xq2eqKa6CTkkWDSqCCQMUrU/fTzavHomBjWz5cctx6scE3FUzgicshmPNsHTgUukuHSInek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Wa+rZaRe; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Wa+rZaRe" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCqd0G013793; Wed, 22 May 2024 13:54:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2023-11-20; bh=cjEST1jMAgisAXPjhkC6jVf+PoUCWzwnDuoQDxOX6/o=; b=Wa+rZaReQlz/5ZMOaVX0p4JEkqiySf9/aqo3lRo3aY+Ei/FC255TnJ/HWlonXpNIDUSW pSR16H2Y+37pgifh3wjlaMzQff7xiL+koB7m/llJZUY9PAvXW/elZHByQTz9J5frk38m 3Rnd+jLiL9s1Cx45yS0SWtJI6VNq3Fk9e1QbSwj4Ze6/9WxofOh1mAjoVmXVlqi7YEVS gbF3NN1ZRRF7YjsRU5+lN8meYdlgVvt7Rh81x8PeKSt7WdwS1fvCxVJir6E/LL0vUqJB LiBKk+IfWUFdMNlMnRzgJX7KtHD1W4HYiBE11L45L2oUo6CzLJT970SZg6dd94xTE5aI Cw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y6mcdyt5e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:54:56 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44MDEjnA019603; Wed, 22 May 2024 13:54:56 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3y6js98t0h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:54:56 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44MDsm2q016070; Wed, 22 May 2024 13:54:55 GMT Received: from lab61.no.oracle.com (lab61.no.oracle.com [10.172.144.82]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3y6js98su1-3; Wed, 22 May 2024 13:54:55 +0000 From: =?utf-8?q?H=C3=A5kon_Bugge?= To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com Cc: Jason Gunthorpe , Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , =?utf-8?q?H=C3=A5kon_Bugge?= , Chuck Lever , Shiraz Saleem , Yang Li Subject: [PATCH v3 1/6] workqueue: Inherit per-process allocation flags Date: Wed, 22 May 2024 15:54:34 +0200 Message-Id: <20240522135444.1685642-3-haakon.bugge@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240522135444.1685642-1-haakon.bugge@oracle.com> References: <20240522135444.1685642-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16 definitions=2024-05-22_07,2024-05-22_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405220093 X-Proofpoint-ORIG-GUID: QnE95IRACn-0412yPTR0rINuSd72tkji X-Proofpoint-GUID: QnE95IRACn-0412yPTR0rINuSd72tkji For drivers/modules running inside a memalloc_flags_{save,restore} region, if a work-queue is created, we make sure work executed on the work-queue inherits the same flag(s). This in order to conditionally enable drivers to work aligned with block I/O devices. This commit makes sure that any work queued later on work-queues created during module initialization, when current's flags has any of the PF_MEMALLOC* set, will inherit the same flags. We do this in order to enable drivers to be used as a network block I/O device. This in order to support XFS or other file-systems on top of a raw block device which uses said drivers as the network transport layer. Under intense memory pressure, we get memory reclaims. Assume the file-system reclaims memory, goes to the raw block device, which calls into said drivers. Now, if regular GFP_KERNEL allocations in the drivers require reclaims to be fulfilled, we end up in a circular dependency. We break this circular dependency by: 1. Force all allocations in the drivers to use GFP_NOIO, by means of a parenthetic use of memalloc_flags_{save,restore} on all relevant entry points, setting/clearing the PF_MEMALLOC_NOIO bit. 2. Make sure work-queues inherits current->flags wrt. PF_MEMALLOC_NOIO, such that work executed on the work-queue inherits the same flag(s). That is what this commit contributes with. Signed-off-by: Håkon Bugge --- v2 -> v3: * Add support for all PF_MEMALLOC* flags * Re-worded commit message v1 -> v2: * Added missing hunk in alloc_workqueue() --- include/linux/workqueue.h | 9 ++++++ kernel/workqueue.c | 60 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index fb39938945365..f8c87f824272b 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -406,9 +406,18 @@ enum wq_flags { __WQ_DRAINING = 1 << 16, /* internal: workqueue is draining */ __WQ_ORDERED = 1 << 17, /* internal: workqueue is ordered */ __WQ_LEGACY = 1 << 18, /* internal: create*_workqueue() */ + __WQ_MEMALLOC = 1 << 19, /* internal: execute work with MEMALLOC */ + __WQ_MEMALLOC_NOFS = 1 << 20, /* internal: execute work with MEMALLOC_NOFS */ + __WQ_MEMALLOC_NOIO = 1 << 21, /* internal: execute work with MEMALLOC_NOIO */ + __WQ_MEMALLOC_NORECLAIM = 1 << 22, /* internal: execute work with MEMALLOC_NORECLAIM */ + __WQ_MEMALLOC_NOWARN = 1 << 23, /* internal: execute work with MEMALLOC_NOWARN */ + __WQ_MEMALLOC_PIN = 1 << 24, /* internal: execute work with MEMALLOC_PIN */ /* BH wq only allows the following flags */ __WQ_BH_ALLOWS = WQ_BH | WQ_HIGHPRI, + + __WQ_PF_MEMALLOC_MASK = PF_MEMALLOC | PF_MEMALLOC_NOFS | PF_MEMALLOC_NOIO | + PF_MEMALLOC_NORECLAIM | PF_MEMALLOC_NOWARN | PF_MEMALLOC_PIN, }; enum wq_consts { diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 003474c9a77d0..28ed6b9556e91 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -51,6 +51,7 @@ #include #include #include +#include #include #include #include @@ -3113,6 +3114,28 @@ static bool manage_workers(struct worker *worker) return true; } +static unsigned int wq_build_memalloc_flags(struct pool_workqueue *pwq) +{ + unsigned int pf_flags = 0; + +#define BUILD_PF_FLAGS_FROM_WQ(name) \ + do { \ + if (pwq->wq->flags & __WQ_ ## name) \ + pf_flags |= PF_ ## name; \ + } while (0) + + BUILD_PF_FLAGS_FROM_WQ(MEMALLOC); + BUILD_PF_FLAGS_FROM_WQ(MEMALLOC_NOFS); + BUILD_PF_FLAGS_FROM_WQ(MEMALLOC_NOIO); + BUILD_PF_FLAGS_FROM_WQ(MEMALLOC_NORECLAIM); + BUILD_PF_FLAGS_FROM_WQ(MEMALLOC_NOWARN); + BUILD_PF_FLAGS_FROM_WQ(MEMALLOC_PIN); + +#undef BUILD_PF_FLAGS_FROM_WQ + + return pf_flags; +} + /** * process_one_work - process single work * @worker: self @@ -3136,6 +3159,8 @@ __acquires(&pool->lock) unsigned long work_data; int lockdep_start_depth, rcu_start_depth; bool bh_draining = pool->flags & POOL_BH_DRAINING; + unsigned int memalloc_flags = wq_build_memalloc_flags(pwq); + unsigned int memalloc_flags_old; #ifdef CONFIG_LOCKDEP /* * It is permissible to free the struct work_struct from @@ -3148,6 +3173,10 @@ __acquires(&pool->lock) lockdep_copy_map(&lockdep_map, &work->lockdep_map); #endif + /* Set inherited alloc flags */ + if (memalloc_flags) + memalloc_flags_old = memalloc_flags_save(memalloc_flags); + /* ensure we're on the correct CPU */ WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); @@ -3284,6 +3313,10 @@ __acquires(&pool->lock) /* must be the last step, see the function comment */ pwq_dec_nr_in_flight(pwq, work_data); + + /* Restore alloc flags */ + if (memalloc_flags) + memalloc_flags_restore(memalloc_flags_old); } /** @@ -5637,6 +5670,30 @@ static void wq_adjust_max_active(struct workqueue_struct *wq) } while (activated); } +/** + * wq_set_memalloc_flags - Test current->flags for PF_MEMALLOC_FOO_BAR + * flag bits and set the corresponding __WQ_MEMALLOC_FOO_BAR in the + * WQ's flags variable. + * @flags_ptr: Pointer to wq->flags + */ +static void wq_set_memalloc_flags(unsigned int *flags_ptr) +{ +#define TEST_PF_SET_WQ(name) \ + do { \ + if (current->flags & PF_ ## name) \ + *flags_ptr |= __WQ_ ## name; \ + } while (0) + + TEST_PF_SET_WQ(MEMALLOC); + TEST_PF_SET_WQ(MEMALLOC_NOFS); + TEST_PF_SET_WQ(MEMALLOC_NOIO); + TEST_PF_SET_WQ(MEMALLOC_NORECLAIM); + TEST_PF_SET_WQ(MEMALLOC_NOWARN); + TEST_PF_SET_WQ(MEMALLOC_PIN); + +#undef TEST_PF_SET_WQ +} + __printf(1, 4) struct workqueue_struct *alloc_workqueue(const char *fmt, unsigned int flags, @@ -5695,6 +5752,9 @@ struct workqueue_struct *alloc_workqueue(const char *fmt, /* init wq */ wq->flags = flags; + if (current->flags & __WQ_PF_MEMALLOC_MASK) + wq_set_memalloc_flags(&wq->flags); + wq->max_active = max_active; wq->min_active = min(max_active, WQ_DFL_MIN_ACTIVE); wq->saved_max_active = wq->max_active; From patchwork Wed May 22 13:54:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Haakon Bugge X-Patchwork-Id: 13670864 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA5DD1411C5; Wed, 22 May 2024 13:55:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386126; cv=none; b=iutEE0BKTP3U1cpEcnbMTb9LOoLT8fwSzcEQIhv02nbbRXvn/91vq5OkCqcM6GrhpoCI9eq7kSpQ1SVYI9fLt30EGFnXFIeQT4rv0xBZ1CGnZX+3jPtDgAKs/MshJ/PJKdellbtJuCqY2pj7emawGUT0LJhbHHeDJK9yCu3cDRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386126; c=relaxed/simple; bh=wn0kyr49Fw4TbkQOrXhATkFp4mPn3l9JzX1iWbLKDoY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=TzpJip73RibpG5imhnkO7etjcCxPsv+5Lpz9Ig+x9u35ZjN0m3vFRf6FO0iFzR6KQ4cq7wBqjtOqcZm9i/U4o6MVBCOOv98DcvEfvYYLA+gBNpgLdFlXa1C38HmwWXEeQ++W6g39cLLy6neFik4RZiQOPgB6W/xL9WFPFqS7IJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=aYxxDDNv; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="aYxxDDNv" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCqiA2013837; Wed, 22 May 2024 13:55:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2023-11-20; bh=utuWZV+u4jRdeZAZ5ktMHl3a6nwr7vDtTRv3hhU5VeI=; b=aYxxDDNvxO8VJhrKT6Jxob+u2ZEd9KNcE49/TzlVYEK0CHCxarLuFIYxuKgtRPKs9JhC n/bjs4gGqjKbcZH3PDY16LFKbo4JqJ0q7VRAVM2cGAk9R6HU9bpC4rwRluCMpVPnibRn uFa1AD2BL0gYtmBK44D6muqH0q5YRx9uQWCibF2cobwIhNguP+pwE4LFqglyJwun8IlL ERqyY/88n8pig2X0t61uZwRlGXp1IzU0GmapN/0dGE3mWwm+PrDmeyQcZ4Y88uZPfcFr r9b0XMqaZmOigEvJ/SEDCRviTdgV6OwEJEuC9apdAI6UQoK6B51O8nHO5I8KCoD+NG8k +A== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y6mcdyt5p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:03 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCPx2o019534; Wed, 22 May 2024 13:55:02 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3y6js98t49-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:02 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44MDsm2w016070; Wed, 22 May 2024 13:55:01 GMT Received: from lab61.no.oracle.com (lab61.no.oracle.com [10.172.144.82]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3y6js98su1-5; Wed, 22 May 2024 13:55:01 +0000 From: =?utf-8?q?H=C3=A5kon_Bugge?= To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com Cc: Jason Gunthorpe , Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , =?utf-8?q?H=C3=A5kon_Bugge?= , Chuck Lever , Shiraz Saleem , Yang Li Subject: [PATCH v3 2/6] rds: Brute force GFP_NOIO Date: Wed, 22 May 2024 15:54:36 +0200 Message-Id: <20240522135444.1685642-5-haakon.bugge@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240522135444.1685642-1-haakon.bugge@oracle.com> References: <20240522135444.1685642-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16 definitions=2024-05-22_07,2024-05-22_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405220093 X-Proofpoint-ORIG-GUID: 6Z6NDXFq4Nx7vrjIH4IxqguVRljDzK0P X-Proofpoint-GUID: 6Z6NDXFq4Nx7vrjIH4IxqguVRljDzK0P For most entry points to RDS, we call memalloc_noio_{save,restore} in a parenthetic fashion when enabled by the module parameter force_noio. We skip the calls to memalloc_noio_{save,restore} in rds_ioctl(), as no memory allocations are executed in this function or its callees. The reason we execute memalloc_noio_{save,restore} in rds_poll(), is due to the following call chain: rds_poll() poll_wait() __pollwait() poll_get_entry() __get_free_page(GFP_KERNEL) The function rds_setsockopt() allocates memory in its callee's rds_get_mr() and rds_get_mr_for_dest(). Hence, we need memalloc_noio_{save,restore} in rds_setsockopt(). In rds_getsockopt(), we have rds_info_getsockopt() that allocates memory. Hence, we need memalloc_noio_{save,restore} in rds_getsockopt(). All the above, in order to conditionally enable RDS to become a block I/O device. Signed-off-by: Håkon Bugge --- v1 -> v2: * s/EXPORT_SYMBOL/static/ for the rds_force_noio variable as pin-pointed by Simon * Straightened the reverse xmas tree two places * Fixed C/P error in rds_cancel_sent_to() where I had two _save()s and no _restore() as reported by Simon --- net/rds/af_rds.c | 59 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c index 8435a20968ef5..846ad20b3783a 100644 --- a/net/rds/af_rds.c +++ b/net/rds/af_rds.c @@ -37,10 +37,15 @@ #include #include #include +#include #include #include "rds.h" +static bool rds_force_noio; +module_param_named(force_noio, rds_force_noio, bool, 0444); +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); + /* this is just used for stats gathering :/ */ static DEFINE_SPINLOCK(rds_sock_lock); static unsigned long rds_sock_count; @@ -59,8 +64,12 @@ DECLARE_WAIT_QUEUE_HEAD(rds_poll_waitq); static int rds_release(struct socket *sock) { struct sock *sk = sock->sk; + unsigned int noio_flags; struct rds_sock *rs; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + if (!sk) goto out; @@ -90,6 +99,8 @@ static int rds_release(struct socket *sock) sock->sk = NULL; sock_put(sk); out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return 0; } @@ -214,9 +225,13 @@ static __poll_t rds_poll(struct file *file, struct socket *sock, { struct sock *sk = sock->sk; struct rds_sock *rs = rds_sk_to_rs(sk); + unsigned int noio_flags; __poll_t mask = 0; unsigned long flags; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + poll_wait(file, sk_sleep(sk), wait); if (rs->rs_seen_congestion) @@ -249,6 +264,8 @@ static __poll_t rds_poll(struct file *file, struct socket *sock, if (mask) rs->rs_seen_congestion = 0; + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return mask; } @@ -293,9 +310,13 @@ static int rds_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg) static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len) { struct sockaddr_in6 sin6; + unsigned int noio_flags; struct sockaddr_in sin; int ret = 0; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + /* racing with another thread binding seems ok here */ if (ipv6_addr_any(&rs->rs_bound_addr)) { ret = -ENOTCONN; /* XXX not a great errno */ @@ -324,6 +345,8 @@ static int rds_cancel_sent_to(struct rds_sock *rs, sockptr_t optval, int len) rds_send_drop_to(rs, &sin6); out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } @@ -485,8 +508,12 @@ static int rds_getsockopt(struct socket *sock, int level, int optname, { struct rds_sock *rs = rds_sk_to_rs(sock->sk); int ret = -ENOPROTOOPT, len; + unsigned int noio_flags; int trans; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + if (level != SOL_RDS) goto out; @@ -529,6 +556,8 @@ static int rds_getsockopt(struct socket *sock, int level, int optname, } out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } @@ -538,12 +567,16 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr, { struct sock *sk = sock->sk; struct sockaddr_in *sin; + unsigned int noio_flags; struct rds_sock *rs = rds_sk_to_rs(sk); int ret = 0; if (addr_len < offsetofend(struct sockaddr, sa_family)) return -EINVAL; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + lock_sock(sk); switch (uaddr->sa_family) { @@ -626,6 +659,8 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr, } release_sock(sk); + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } @@ -697,16 +732,28 @@ static int __rds_create(struct socket *sock, struct sock *sk, int protocol) static int rds_create(struct net *net, struct socket *sock, int protocol, int kern) { + unsigned int noio_flags; struct sock *sk; + int ret; if (sock->type != SOCK_SEQPACKET || protocol) return -ESOCKTNOSUPPORT; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + sk = sk_alloc(net, AF_RDS, GFP_KERNEL, &rds_proto, kern); - if (!sk) - return -ENOMEM; + if (!sk) { + ret = -ENOMEM; + goto out; + } - return __rds_create(sock, sk, protocol); + ret = __rds_create(sock, sk, protocol); +out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); + + return ret; } void rds_sock_addref(struct rds_sock *rs) @@ -895,8 +942,12 @@ u32 rds_gen_num; static int __init rds_init(void) { + unsigned int noio_flags; int ret; + if (rds_force_noio) + noio_flags = memalloc_noio_save(); + net_get_random_once(&rds_gen_num, sizeof(rds_gen_num)); ret = rds_bind_lock_init(); @@ -947,6 +998,8 @@ static int __init rds_init(void) out_bind: rds_bind_lock_destroy(); out: + if (rds_force_noio) + memalloc_noio_restore(noio_flags); return ret; } module_init(rds_init); From patchwork Wed May 22 13:54:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Haakon Bugge X-Patchwork-Id: 13670863 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA5AC1411C1; Wed, 22 May 2024 13:55:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386126; cv=none; b=qvtpCsGwzVqx86/mWnhBToqMaTqdnMD/bNzIubBwc5zh6gA7CUlQWu3+Z5sVdtOJfrEyvzBqTRfTjDBopMYv5FLENwoHTSMKq8/LGRRGizhocGYNS8YEPuSFk2EA5YbjKU21o1/gcS6qhy18wMzrD4o0C1xfAfJrPA7DS1pVwck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386126; c=relaxed/simple; bh=x+YRy/ThdkD01EFNyKjA9UeYpsvcSONGSu4lbSQxQow=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=cn6Q2DSxl0HSC4CUjoeVAzYezcuJg/CawCd5IYe/pyY0XcF6u9JbY/Cs/5s96q1pB73MpUUzc/l+vMMfvfMEvIJPqOJKEKtvDJlQy1Gjj1NGd23hPK+bsqU3UP9VV4Guli808Ogg9AEAwAJQ9i/vhDf32FjmeksvSSy6GmqE2Bc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Tr/pdOM2; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Tr/pdOM2" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCqdw8013783; Wed, 22 May 2024 13:55:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2023-11-20; bh=R3mG6Szp+rz+nLnvrLG6Q59rBqnQhSrkmf7l+8vA1YU=; b=Tr/pdOM2gDcwkeBgq+dC+/8/4gQUA7awtrNFNnm/26FGbz2TXfe3QlpxfC7p0S+95A6n HksE2kHO9YVSMUHcj/XRoOjI+ksENUVZFwvCAnKizixhjH7BBx9iCdPrfKMVDK0t5Dvm 7TzYXneyQu8dmUvyeGCT4lZhQ1p8rN2grzzh7+6PdsaSJksu49OYatvzdTohNgfhYZIx zBAfSYLP8crcZ1A8GKGQgimz6RIQsRVgWHhY+mht3Dj5BVEooEuTrGjhQM9By1aP0myk pFNQM6kf+tA4kJZAGmp2Iwhm7uDHg7s0UNTu0bVJASyqiAoX8cBxqeOBSFFLnE768GUd KQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y6mcdyt60-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:09 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCuWYm019512; Wed, 22 May 2024 13:55:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3y6js98t81-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:09 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44MDsm34016070; Wed, 22 May 2024 13:55:08 GMT Received: from lab61.no.oracle.com (lab61.no.oracle.com [10.172.144.82]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3y6js98su1-7; Wed, 22 May 2024 13:55:08 +0000 From: =?utf-8?q?H=C3=A5kon_Bugge?= To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com Cc: Jason Gunthorpe , Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , =?utf-8?q?H=C3=A5kon_Bugge?= , Chuck Lever , Shiraz Saleem , Yang Li Subject: [PATCH v3 3/6] RDMA/cma: Brute force GFP_NOIO Date: Wed, 22 May 2024 15:54:38 +0200 Message-Id: <20240522135444.1685642-7-haakon.bugge@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240522135444.1685642-1-haakon.bugge@oracle.com> References: <20240522135444.1685642-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16 definitions=2024-05-22_07,2024-05-22_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405220093 X-Proofpoint-ORIG-GUID: 5UH2sQqlIU1_ZWsukQVNPZex9ll28i-I X-Proofpoint-GUID: 5UH2sQqlIU1_ZWsukQVNPZex9ll28i-I In cma_init(), we call memalloc_noio_{save,restore} in a parenthetic fashion when enabled by the module parameter force_noio. This in order to conditionally enable rdma_cm to work aligned with block I/O devices. Any work queued later on work-queues created during module initialization will inherit the PF_MEMALLOC_{NOIO,NOFS} flag(s), due to commit ("workqueue: Inherit NOIO and NOFS alloc flags"). We do this in order to enable ULPs using the RDMA stack to be used as a network block I/O device. This to support a filesystem on top of a raw block device which uses said ULP(s) and the RDMA stack as the network transport layer. Under intense memory pressure, we get memory reclaims. Assume the filesystem reclaims memory, goes to the raw block device, which calls into the ULP in question, which calls the RDMA stack. Now, if regular GFP_KERNEL allocations in the ULP or the RDMA stack require reclaims to be fulfilled, we end up in a circular dependency. We break this circular dependency by: 1. Force all allocations in the ULP and the relevant RDMA stack to use GFP_NOIO, by means of a parenthetic use of memalloc_noio_{save,restore} on all relevant entry points. 2. Make sure work-queues inherits current->flags wrt. PF_MEMALLOC_{NOIO,NOFS}, such that work executed on the work-queue inherits the same flag(s). Signed-off-by: Håkon Bugge --- drivers/infiniband/core/cma.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 64ace0b968f07..6e82c18aeebb8 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -50,6 +50,10 @@ MODULE_LICENSE("Dual BSD/GPL"); #define CMA_IBOE_PACKET_LIFETIME 16 #define CMA_PREFERRED_ROCE_GID_TYPE IB_GID_TYPE_ROCE_UDP_ENCAP +static bool cma_force_noio; +module_param_named(force_noio, cma_force_noio, bool, 0444); +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); + static const char * const cma_events[] = { [RDMA_CM_EVENT_ADDR_RESOLVED] = "address resolved", [RDMA_CM_EVENT_ADDR_ERROR] = "address error", @@ -5426,6 +5430,10 @@ static struct pernet_operations cma_pernet_operations = { static int __init cma_init(void) { int ret; + unsigned int noio_flags; + + if (cma_force_noio) + noio_flags = memalloc_noio_save(); /* * There is a rare lock ordering dependency in cma_netdev_callback() @@ -5441,8 +5449,10 @@ static int __init cma_init(void) } cma_wq = alloc_ordered_workqueue("rdma_cm", WQ_MEM_RECLAIM); - if (!cma_wq) - return -ENOMEM; + if (!cma_wq) { + ret = -ENOMEM; + goto out; + } ret = register_pernet_subsys(&cma_pernet_operations); if (ret) @@ -5460,7 +5470,8 @@ static int __init cma_init(void) if (ret) goto err_ib; - return 0; + ret = 0; + goto out; err_ib: ib_unregister_client(&cma_client); @@ -5471,6 +5482,9 @@ static int __init cma_init(void) unregister_pernet_subsys(&cma_pernet_operations); err_wq: destroy_workqueue(cma_wq); +out: + if (cma_force_noio) + memalloc_noio_restore(noio_flags); return ret; } From patchwork Wed May 22 13:54:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Haakon Bugge X-Patchwork-Id: 13670867 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B6D7145B25; Wed, 22 May 2024 13:55:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386132; cv=none; b=YKnpnEMaxf69o9/5y7oQY6WXu99fsWvfm86680g1KYgK7jColTTrZnjgOL3tQGWpcRL0/jpWjRHBQ2jNRKlGkGDlKTuSvABSu0Tq+TgcB/GL2lK/ArDGpEvPPkVlK3LbN7H+pbx00tLA2JeTdiGrkD9dfFMa8xDDT+6dw0Ro6rg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386132; c=relaxed/simple; bh=x6BtFVfPBT7eGGlyetNktXEK7Tg7+wXGKEk5z5Om/YY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=Mue0YMaqp27EWf8dwbE4w29bp887YoZM+I7XnyaAY0FS9mfUrsxA5t5IB3F5ermRrYQ2jbQY2vpTIe8FzLuUYUCzEGWz7bfP7RN3ZwO7UdqGrGRyheSWX6DsuElCh4UcTXBj3Zmhd3P27f7j10VbMHzM0jGjOGSz1PG7InSLIXs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=VoW+2s9P; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="VoW+2s9P" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCqUrC022591; Wed, 22 May 2024 13:55:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2023-11-20; bh=2K+xgQ5YWBHKfRmK0QOxsZIYadN+1nqsW7eWDLqroyU=; b=VoW+2s9PuirWOHz44JeFPEtOB34W3CjzSANTZNDVSIq8ohyAZ1zo6U0Zc+7EUNuRCDTT HhzU4ljWxCs1ZiR93EiJmqLdZa7JlGRvlLQwl1gH3TOQduWcrG5Ha/sy1Dp31H/Mh78s lpIREU6N0jaDnRYmt5f1OajpV02sWDaswPZYswSISwANiFzk74NGHD6zw7UJ9MePREIp 2JLNcI099Yj4tmzEVzVpwkXTFo/8GAi5NOilDMskbj+g9sXG0ngr4zsktzE5yYPbhgs9 cVTBeTSKVfWEMUAMm7qhJP7vRWwshTE04/wxWt09YB5UpI/5Jkcuy6xnZ56d/5QydDwa Mw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y6kxv7sx2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:17 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44MDKcE9019530; Wed, 22 May 2024 13:55:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3y6js98tcc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:16 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44MDsm3A016070; Wed, 22 May 2024 13:55:15 GMT Received: from lab61.no.oracle.com (lab61.no.oracle.com [10.172.144.82]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3y6js98su1-9; Wed, 22 May 2024 13:55:15 +0000 From: =?utf-8?q?H=C3=A5kon_Bugge?= To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com Cc: Jason Gunthorpe , Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , =?utf-8?q?H=C3=A5kon_Bugge?= , Chuck Lever , Shiraz Saleem , Yang Li Subject: [PATCH v3 4/6] RDMA/cm: Brute force GFP_NOIO Date: Wed, 22 May 2024 15:54:40 +0200 Message-Id: <20240522135444.1685642-9-haakon.bugge@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240522135444.1685642-1-haakon.bugge@oracle.com> References: <20240522135444.1685642-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16 definitions=2024-05-22_07,2024-05-22_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405220093 X-Proofpoint-GUID: ch9XShqC5ZFNMlpM96vUEkr-4UNutiS6 X-Proofpoint-ORIG-GUID: ch9XShqC5ZFNMlpM96vUEkr-4UNutiS6 In ib_cm_init(), we call memalloc_noio_{save,restore} in a parenthetic fashion when enabled by the module parameter force_noio. This in order to conditionally enable ib_cm to work aligned with block I/O devices. Any work queued later on work-queues created during module initialization will inherit the PF_MEMALLOC_{NOIO,NOFS} flag(s), due to commit ("workqueue: Inherit NOIO and NOFS alloc flags"). We do this in order to enable ULPs using the RDMA stack to be used as a network block I/O device. This to support a filesystem on top of a raw block device which uses said ULP(s) and the RDMA stack as the network transport layer. Under intense memory pressure, we get memory reclaims. Assume the filesystem reclaims memory, goes to the raw block device, which calls into the ULP in question, which calls the RDMA stack. Now, if regular GFP_KERNEL allocations in ULP or the RDMA stack require reclaims to be fulfilled, we end up in a circular dependency. We break this circular dependency by: 1. Force all allocations in the ULP and the relevant RDMA stack to use GFP_NOIO, by means of a parenthetic use of memalloc_noio_{save,restore} on all relevant entry points. 2. Make sure work-queues inherits current->flags wrt. PF_MEMALLOC_{NOIO,NOFS}, such that work executed on the work-queue inherits the same flag(s). Signed-off-by: Håkon Bugge --- drivers/infiniband/core/cm.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index 07fb8d3c037f0..767eec38eb57d 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include @@ -35,6 +36,11 @@ MODULE_DESCRIPTION("InfiniBand CM"); MODULE_LICENSE("Dual BSD/GPL"); #define CM_DESTROY_ID_WAIT_TIMEOUT 10000 /* msecs */ + +static bool cm_force_noio; +module_param_named(force_noio, cm_force_noio, bool, 0444); +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); + static const char * const ibcm_rej_reason_strs[] = { [IB_CM_REJ_NO_QP] = "no QP", [IB_CM_REJ_NO_EEC] = "no EEC", @@ -4504,6 +4510,10 @@ static void cm_remove_one(struct ib_device *ib_device, void *client_data) static int __init ib_cm_init(void) { int ret; + unsigned int noio_flags; + + if (cm_force_noio) + noio_flags = memalloc_noio_save(); INIT_LIST_HEAD(&cm.device_list); rwlock_init(&cm.device_lock); @@ -4527,10 +4537,13 @@ static int __init ib_cm_init(void) if (ret) goto error3; - return 0; + goto error2; error3: destroy_workqueue(cm.wq); error2: + if (cm_force_noio) + memalloc_noio_restore(noio_flags); + return ret; } From patchwork Wed May 22 13:54:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haakon Bugge X-Patchwork-Id: 13670870 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 802721474B7; Wed, 22 May 2024 13:55:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386139; cv=none; b=cF1E4Bsv/U//zyFhHQK1ffSfOsMgyYyZaOV0MMoagOZ1wYrpPWYs0S4acvubJpMNkQLebsnVWKgybXyhoT2Idi02g5IrH8kf/3rAgf2TiemqZapeV8Kr/SXvyGRzF6tSCXdMLmqnFtFt+7Lo4JOAuSrQdo2wf8z2v6qVKGB93Rc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386139; c=relaxed/simple; bh=kK+OhbDo2JEyj7pc/ZTdnFaD3OwZTrp2X84sYD1+N90=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jQazxVspcQK3BW4HoaJr2r4ITXZKg5mZVpq47Q1OpJmamOakADl1zLAsJy+Tlpw2gaZXEAXEOCIZPrtF5KkyYR+5KEdP6rwViB9D0rMLKyaOVXUd6bTQlXIdDXGqYL2QEo4ZIW6kmgHTq7aL9PBRTpeWfxD4WKEcfdoLT5EeorQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=HP9ybbBS; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="HP9ybbBS" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCrKJS006713; Wed, 22 May 2024 13:55:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=IObyseWmhRp79zg0ks96haG4gisXWog/WVAkWgKvqZ4=; b=HP9ybbBSqzogk8n+cAqj9qnJ7c7dnxdVgWvLfOz42l2gwb9fnM7w0aHV70y62UmDy+5n hA2pLsxynaCyvGT6Vaz/crUkHOwgIoNs6wxbx9nkV3swxEgzh/3Blfpu2Tlp844gDwYO mi5Uflwifu+BOkYIgTxDt2krCMYd88VJ7uP8q5Z0zM53KED710Yb5DWd1J/tv8HoyQUK qtok4jQpsDQsjFw8IBnHjFknD3G/MDInd16H/MbekNg/2eDX0QcgyL8Ta0XS9qVnvcwp oQGD5Oew+PscnyLlEYmzaodhFaXLVocFEnEhrTngoOuRgfAmz/iBGhl9Gy5ViO+ecW9e hQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y6k8d7qru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:21 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44MD47EJ019531; Wed, 22 May 2024 13:55:19 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3y6js98ter-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:19 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44MDsm3D016070; Wed, 22 May 2024 13:55:19 GMT Received: from lab61.no.oracle.com (lab61.no.oracle.com [10.172.144.82]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3y6js98su1-10; Wed, 22 May 2024 13:55:19 +0000 From: =?utf-8?q?H=C3=A5kon_Bugge?= To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com Cc: Jason Gunthorpe , Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , =?utf-8?q?H=C3=A5kon_Bugge?= , Chuck Lever , Shiraz Saleem , Yang Li Subject: [PATCH v3 5/6] ml: Add ol_signing_keys.pem in certs dir Date: Wed, 22 May 2024 15:54:41 +0200 Message-Id: <20240522135444.1685642-10-haakon.bugge@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240522135444.1685642-1-haakon.bugge@oracle.com> References: <20240522135444.1685642-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16 definitions=2024-05-22_07,2024-05-22_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405220093 X-Proofpoint-GUID: 0x85Bf96kQPixahaWTzFnCc5zFp9Zd4E X-Proofpoint-ORIG-GUID: 0x85Bf96kQPixahaWTzFnCc5zFp9Zd4E From: Luci Bot ol_signing_keys.pem is copied in certs directory Signed-off-by: Luci Bot Reviewed-by: Mark Nicholson --- ml-rpm/ol8/kernel-mainline.spec | 11 +++++++++++ ml-rpm/ol9/kernel-mainline.spec | 11 +++++++++++ 2 files changed, 22 insertions(+) diff --git a/ml-rpm/ol8/kernel-mainline.spec b/ml-rpm/ol8/kernel-mainline.spec index ed47eb0706a87..8487d522984a3 100644 --- a/ml-rpm/ol8/kernel-mainline.spec +++ b/ml-rpm/ol8/kernel-mainline.spec @@ -483,6 +483,7 @@ BuildConflicts: rpm < 4.13.0.1-19 Source0: linux-%{kversion}.tar.bz2 %if %{signkernel}%{signmodules} +Source9: ol_signing_keys.pem Source10: x509.genkey %endif @@ -1110,6 +1111,11 @@ BuildContainerKernel() { perl -p -i -e "s/^EXTRAVERSION.*/EXTRAVERSION = ${ExtraVer}/" Makefile make %{?make_opts} mrproper + +%if %{signkernel}%{signmodules} + cp %{SOURCE10} %{SOURCE9} certs/. +%endif + cp configs/config-container .config make %{?make_opts} ARCH=$Arch %{?_kernel_cc} olddefconfig > /dev/null @@ -1159,6 +1165,11 @@ BuildKernel() { make %{?make_opts} mrproper +%if %{signkernel}%{signmodules} + cp %{SOURCE10} %{SOURCE9} certs/. +%endif + + %if %{signkernel}%{signmodules} cp %{SOURCE10} certs/. %endif diff --git a/ml-rpm/ol9/kernel-mainline.spec b/ml-rpm/ol9/kernel-mainline.spec index 0eb7d20043ed0..6a84976f01632 100644 --- a/ml-rpm/ol9/kernel-mainline.spec +++ b/ml-rpm/ol9/kernel-mainline.spec @@ -458,6 +458,7 @@ BuildConflicts: rpm < 4.13.0.1-19 Source0: linux-%{kversion}.tar.bz2 %if %{signkernel}%{signmodules} +Source9: ol_signing_keys.pem Source10: x509.genkey %endif @@ -1072,6 +1073,11 @@ BuildContainerKernel() { perl -p -i -e "s/^EXTRAVERSION.*/EXTRAVERSION = ${ExtraVer}/" Makefile make %{?make_opts} mrproper + +%if %{signkernel}%{signmodules} + cp %{SOURCE10} %{SOURCE9} certs/. +%endif + cp configs/config-container .config make %{?make_opts} ARCH=$Arch %{?_kernel_cc} olddefconfig > /dev/null @@ -1121,6 +1127,11 @@ BuildKernel() { make %{?make_opts} mrproper +%if %{signkernel}%{signmodules} + cp %{SOURCE10} %{SOURCE9} certs/. +%endif + + %if %{signkernel}%{signmodules} cp %{SOURCE10} certs/. %endif From patchwork Wed May 22 13:54:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Haakon Bugge X-Patchwork-Id: 13670872 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85DAB1487E2; Wed, 22 May 2024 13:55:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386145; cv=none; b=LakSVrBxtII9epj37cs7soEh/D8+RRCPmR4tY328Ef6EXql9w+yCGPIIDL/57gMn9YXs0totACzheZ7+K1mWAkuajIJvxynieu/rGFpVdr+IzsAR+0q9R98120oGp4DnG0WQnOca+dplF4RAnQfumLqn0nNHnJzgV86OvlD56qA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716386145; c=relaxed/simple; bh=Yo6kvNyM9fmaeDHPwmooDhBiayjHyrtV1nFwJzUbbDk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=JBIehGsMWzIXAYBiOYYIxxZpg2syin2bcjjZZbbaMLJRAp7YInUyr3JZXgRUUHOna7XS6jUfczf8SFUi1JxZ4pC3t3ubNzqdzJ7s0Tu3FHQtyB2DT+S9YcA1FN7c4ECaOOXxpkKV3XhqaTnyUukJNELkdRycaM7J80T6gSlmWFA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=HRbGvPrR; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="HRbGvPrR" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCqtA9006213; Wed, 22 May 2024 13:55:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2023-11-20; bh=aFDMQAl4uwXrHDgcWlkgT8qjneMf1b5JzDHTBMbZmhU=; b=HRbGvPrRFSOqHJiYW6w5AViVCDQZt3RM5JzJ9hN/MHx4GnMscLqzLWZmDFxOxF9zd+Rc lkkiqt48L4aC2wQjDhUYBc3Xw3ofnVDj5F5MkAcKa36phMslhcTmN2G4vszlV6Q7EahX BGHEMXey+kA9xbtp2aLRVCcsTlMHRAheCz6JkHzTE6JPXdR2j6L2Tj9d+yCo7O2Sr8ap gZYayg7JhfUOkO0DNKKF26X2MW0QsZMSCkPgY6JGrs1WXGmpeAzfI52fcBuxPd8a3hHi mdrf7RaUA3clXkbw7pjxj0ZHFXL7NQHYB5uH8XL1U30ZLsb6gxYQrIt8Dy8RCn58sBPU aQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y6k8d7qs4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:27 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44MCI27L019513; Wed, 22 May 2024 13:55:26 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3y6js98tn9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 May 2024 13:55:26 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44MDsm3J016070; Wed, 22 May 2024 13:55:25 GMT Received: from lab61.no.oracle.com (lab61.no.oracle.com [10.172.144.82]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3y6js98su1-12; Wed, 22 May 2024 13:55:25 +0000 From: =?utf-8?q?H=C3=A5kon_Bugge?= To: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com Cc: Jason Gunthorpe , Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , =?utf-8?q?H=C3=A5kon_Bugge?= , Chuck Lever , Shiraz Saleem , Yang Li Subject: [PATCH v3 6/6] net/mlx5: Brute force GFP_NOIO Date: Wed, 22 May 2024 15:54:43 +0200 Message-Id: <20240522135444.1685642-12-haakon.bugge@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240522135444.1685642-1-haakon.bugge@oracle.com> References: <20240522135444.1685642-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.12.28.16 definitions=2024-05-22_07,2024-05-22_01,2024-05-17_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 spamscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405220093 X-Proofpoint-GUID: TMB2Qk9oP0rCDeO2DErtPcBDv1hY1qzK X-Proofpoint-ORIG-GUID: TMB2Qk9oP0rCDeO2DErtPcBDv1hY1qzK In mlx5_core_init(), we call memalloc_noio_{save,restore} in a parenthetic fashion when enabled by the module parameter force_noio. This in order to conditionally enable mlx5_core to work aligned with I/O devices. Any work queued later on work-queues created during module initialization will inherit the PF_MEMALLOC_{NOIO,NOFS} flag(s), due to commit ("workqueue: Inherit NOIO and NOFS alloc flags"). We do this in order to enable ULPs using the RDMA stack and the mlx5_core driver to be used as a network block I/O device. This to support a filesystem on top of a raw block device which uses said ULP(s) and the RDMA stack as the network transport layer. Under intense memory pressure, we get memory reclaims. Assume the filesystem reclaims memory, goes to the raw block device, which calls into the ULP in question, which calls the RDMA stack. Now, if regular GFP_KERNEL allocations in ULP or the RDMA stack require reclaims to be fulfilled, we end up in a circular dependency. We break this circular dependency by: 1. Force all allocations in the ULP and the relevant RDMA stack to use GFP_NOIO, by means of a parenthetic use of memalloc_noio_{save,restore} on all relevant entry points. 2. Make sure work-queues inherits current->flags wrt. PF_MEMALLOC_{NOIO,NOFS}, such that work executed on the work-queue inherits the same flag(s). Signed-off-by: Håkon Bugge --- drivers/net/ethernet/mellanox/mlx5/core/main.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 6574c145dc1e2..66cef64a82c61 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -48,6 +48,7 @@ #include #include #include +#include #include "mlx5_core.h" #include "lib/eq.h" #include "fs_core.h" @@ -87,6 +88,10 @@ static unsigned int prof_sel = MLX5_DEFAULT_PROF; module_param_named(prof_sel, prof_sel, uint, 0444); MODULE_PARM_DESC(prof_sel, "profile selector. Valid range 0 - 2"); +static bool mlx5_core_force_noio; +module_param_named(force_noio, mlx5_core_force_noio, bool, 0444); +MODULE_PARM_DESC(force_noio, "Force the use of GFP_NOIO (Y/N)"); + static u32 sw_owner_id[4]; #define MAX_SW_VHCA_ID (BIT(__mlx5_bit_sz(cmd_hca_cap_2, sw_vhca_id)) - 1) static DEFINE_IDA(sw_vhca_ida); @@ -2308,8 +2313,12 @@ static void mlx5_core_verify_params(void) static int __init mlx5_init(void) { + unsigned int noio_flags; int err; + if (mlx5_core_force_noio) + noio_flags = memalloc_noio_save(); + WARN_ONCE(strcmp(MLX5_ADEV_NAME, KBUILD_MODNAME), "mlx5_core name not in sync with kernel module name"); @@ -2330,7 +2339,7 @@ static int __init mlx5_init(void) if (err) goto err_pci; - return 0; + goto out; err_pci: mlx5_sf_driver_unregister(); @@ -2338,6 +2347,9 @@ static int __init mlx5_init(void) mlx5e_cleanup(); err_debug: mlx5_unregister_debugfs(); +out: + if (mlx5_core_force_noio) + memalloc_noio_restore(noio_flags); return err; }