From patchwork Sun Mar 9 14:46:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 14008579 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC2D11EB5C4; Sun, 9 Mar 2025 14:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741531637; cv=none; b=VDDPKA5jxp0xc4E7IfMddojnaAbaWuSaXqEKL5JTCNN6W+0dAMnoUM/5clu80fTRLTWAThWcHwyOfoKDjunNVK5v6p8NJ0L6unfUsdvbBg30Q99vo8zr8F9mPvtQXRkQE8IdQuP96dsSxug3Xxv02OL7g1Vec1Q+2MrDOdFgU80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741531637; c=relaxed/simple; bh=KqoKUZEweth4eUnuFS1Nm52k3bfMfq298SudnyUOZxg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y9ic4Nd0HraR7Qgmh0lhQU0PoLQd4E5I28oqIvfvRk1VOvgXQjzyTfjDFWeeumAdbrLRUPke2/DVBnOUVKly/SCz9gieqY77/ref41dFjucMLDgdNj5pkuD16IhUxk6FR3oW8OunK2c4GdnYeT4dCKFgWMtYNGW8JfH7ZkewIJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=NKeRG1C6; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=FxQ5VLEa; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="NKeRG1C6"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="FxQ5VLEa" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1741531632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dgbrz2OEZ6aEoJpoB0C+hoamsafL5vBdsLc143vGtMY=; b=NKeRG1C6VaKiSwGdYsB5nzBCZxDLcaIyoabVuvOgX6ppaGrmV6/c7vJyx+f5ZBsVkbdDTv pTDhBJLKk1aIg0AWIx6lNditWebWlNEkTJXkufmBYC3K9ERHNx3Zrx4UwsppzvQpifT4af jxOa7GBQWr2f2J6MCKJVooa9Wru3KZNWJRd4y7jMX9Em8QXli3LS7yGAzAWNW3e9Pi/edV wp+b7GZREpY6Xdl++aKEVQ9CHtvd0hyKEaHntsWjn+83p575Yig1zf5HNUHbFVZsH+TN07 iDHNzdyOz0pa8X4jt0NO6KQrX3CyNJaJw7iM6sPsmy4fM2dfFDJXksqiQpaqbA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1741531632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dgbrz2OEZ6aEoJpoB0C+hoamsafL5vBdsLc143vGtMY=; b=FxQ5VLEaXR+FXN5bDRPqXF90SWUM1FxiatgPy6CNtcS+PZ/ixp3NCM2LlH1T+YxpqbSzOs EOEEE1Oh4oWM1kBQ== To: netdev@vger.kernel.org, linux-rt-devel@lists.linux.dev Cc: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Thomas Gleixner , Sebastian Andrzej Siewior , Allison Henderson , linux-rdma@vger.kernel.org Subject: [PATCH net-next 16/18] rds: Disable only bottom halves in rds_page_remainder_alloc(). Date: Sun, 9 Mar 2025 15:46:51 +0100 Message-ID: <20250309144653.825351-17-bigeasy@linutronix.de> In-Reply-To: <20250309144653.825351-1-bigeasy@linutronix.de> References: <20250309144653.825351-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 rds_page_remainder_alloc() is invoked from a preemptible context or a tasklet. There is no need to disable interrupts for locking. Use local_bh_disable() instead of local_irq_save() for locking. Cc: Allison Henderson Cc: linux-rdma@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior --- net/rds/page.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/net/rds/page.c b/net/rds/page.c index 7cc57e098ddb9..e0dd4f62ea47a 100644 --- a/net/rds/page.c +++ b/net/rds/page.c @@ -69,7 +69,6 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, gfp_t gfp) { struct rds_page_remainder *rem; - unsigned long flags; struct page *page; int ret; @@ -88,7 +87,7 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, } rem = &per_cpu(rds_page_remainders, get_cpu()); - local_irq_save(flags); + local_bh_disable(); while (1) { /* avoid a tiny region getting stuck by tossing it */ @@ -116,13 +115,13 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, } /* alloc if there is nothing for us to use */ - local_irq_restore(flags); + local_bh_enable(); put_cpu(); page = alloc_page(gfp); rem = &per_cpu(rds_page_remainders, get_cpu()); - local_irq_save(flags); + local_bh_disable(); if (!page) { ret = -ENOMEM; @@ -140,7 +139,7 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, rem->r_offset = 0; } - local_irq_restore(flags); + local_bh_enable(); put_cpu(); out: rdsdebug("bytes %lu ret %d %p %u %u\n", bytes, ret, From patchwork Sun Mar 9 14:46:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 14008580 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 729CD1E521D; Sun, 9 Mar 2025 14:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741531638; cv=none; b=efwWDVqoySPw24NoqXJxXUXF1ifeVuQupXWwTMSCZGr77CAurkucz4NzFAdPhFfwOK4sVPAFTuzZWZV5qnfeDoBN5StwQlnLghLhf4oQCl/ZY4ioWACiay6LezvzveVuLTBZNS0xdEW3RAGMDSAD7tMlpiBvumXhp0JlOqsxM0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741531638; c=relaxed/simple; bh=alvzr3rmRIJs8SYMbjK6X1RfWDp0ICwIjcxZCSinOLU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jQ2BPaTYEfgHAzCTVAyiUCxFdFRBAD2n7tC2dgglSIsBNd1qzX8i1Fx9T3rixPlSj6CafkYVuZN60rKZUYYHHHZ+Yk8f7Tx+f4LcIrtasuctaYz09EDQmYH5LvfC/ilgFyf6ohSs/dZ3vyZVHjut6HuVdqRQN/vo9A94ypU28D0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=b4k0GYge; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=qZ4Amo/H; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="b4k0GYge"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="qZ4Amo/H" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1741531632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QvIeJeOhBJ2MlknP0xO1PybUxd0Jemlt2VLER5LrqDY=; b=b4k0GYgefgxyxAJjh4MU39+73tYHC9owGD7B5c6QytvgXG6giRVS/Z/Fz7KWxk6ZxoAfLj 7f5ZtqXMTCR2IK88afKn2KpAwcgC9YVLnOV28o+1vg61bMoYUdW4SDB++JelXdhIopFODk 1NcB1d4gym92K5RqoautjBGKW/CLR49uPsMXLTuCOEzx1nctWBUekIT9bzQohAd7Ja2vft +hi7xbgm26H/0eypIFG6vPlNYavAJLMi3JCURbAX5N3ETx6fGCUKFmd30UnCYf4V8OQKzh Q/J+HNwB8fDZTvBpIBjLBxwxTVqLSXAdiQAYrJB0AfKgHoIb9nRIBj0Lxl/mIw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1741531632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QvIeJeOhBJ2MlknP0xO1PybUxd0Jemlt2VLER5LrqDY=; b=qZ4Amo/Hf+Eg2+oaw+s1iSO+6i/Qt5b/dWs8SxfvTyDeLmhT/P17iKwUiB3tC27H1kY4hZ XHSQdjCFeFQTI+AQ== To: netdev@vger.kernel.org, linux-rt-devel@lists.linux.dev Cc: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Thomas Gleixner , Sebastian Andrzej Siewior , Allison Henderson , linux-rdma@vger.kernel.org Subject: [PATCH net-next 17/18] rds: Acquire per-CPU pointer within BH disabled section. Date: Sun, 9 Mar 2025 15:46:52 +0100 Message-ID: <20250309144653.825351-18-bigeasy@linutronix.de> In-Reply-To: <20250309144653.825351-1-bigeasy@linutronix.de> References: <20250309144653.825351-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 rds_page_remainder_alloc() obtains the current CPU with get_cpu() while disabling preemption. Then the CPU number is used to access the per-CPU data structure via per_cpu(). This can be optimized by relying on local_bh_disable() to provide a stable CPU number/ avoid migration and then using this_cpu_ptr() to retrieve the data structure. Cc: Allison Henderson Cc: linux-rdma@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior --- net/rds/page.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/net/rds/page.c b/net/rds/page.c index e0dd4f62ea47a..58a8548a915a9 100644 --- a/net/rds/page.c +++ b/net/rds/page.c @@ -86,8 +86,8 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, goto out; } - rem = &per_cpu(rds_page_remainders, get_cpu()); local_bh_disable(); + rem = this_cpu_ptr(&rds_page_remainders); while (1) { /* avoid a tiny region getting stuck by tossing it */ @@ -116,12 +116,11 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, /* alloc if there is nothing for us to use */ local_bh_enable(); - put_cpu(); page = alloc_page(gfp); - rem = &per_cpu(rds_page_remainders, get_cpu()); local_bh_disable(); + rem = this_cpu_ptr(&rds_page_remainders); if (!page) { ret = -ENOMEM; @@ -140,7 +139,6 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, } local_bh_enable(); - put_cpu(); out: rdsdebug("bytes %lu ret %d %p %u %u\n", bytes, ret, ret ? NULL : sg_page(scat), ret ? 0 : scat->offset, From patchwork Sun Mar 9 14:46:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 14008581 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC0DA1E98F8; Sun, 9 Mar 2025 14:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741531638; cv=none; b=FH69F8/E7KTHTPA954BuvzCc2dI/svc8rEnshOrVsWSaruYclU207towOKKdptgmz/SNJ63bFGiCNnTJTtYZCPu3MYSzfxkQEkruWiGjDwi3xzwwcYW/YR3iw2ejVaGC4HxPzmFlG64iE6ZiaomRqnrVFeGauZmK8Qd1Mln2lvE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741531638; c=relaxed/simple; bh=EM/uznQ2YZTtHd2PJqyOgtHEa7X0gpcWflzaAAZAoes=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n1Cl7A+HpEhSzisCR40nwovk3YxWIPMQHWHznhhDCW7aIVEwZ+TxB3TSqWhVmIZjnkBQH7CUVxtBoahIhFxu47FBdQxjMwUtUvKZOkSkoUaoDwjLW0nwq4tIqde1nWqB17M4P0XeskMN8SuC61+Bv6X3+Ze4x+1sUBTIvcP2CGM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=OACsY6l9; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Ix7/O3Cy; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="OACsY6l9"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Ix7/O3Cy" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1741531633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ILRU884p1OAoaunx4VDmaSDgJgpHGl18bni0vDofXnE=; b=OACsY6l9XCBNmnz8p2IxMspoNEZ+22tc4fkJIUoq8KCksZe+eu5ShFI6kD2CPfOjZ8z6Z7 zi0gr988bG9nd1r8vjnAjEfSpkPsFdWH2k8aDtQUocAZrlky7tdsY1Y0Kbur2GF63PPbay lyXXgBa4tsQ2tQ9kz4XJhT5Uoximfli4JMRLi7fVt4iyoe/9WRiveVwhECk9Qlj01btE24 gXxBhcMeWZ0fMOnbhKnsGFyrym1qP/5xqFMiZHNgFMwffLNZ/kR4+Wd63z6m5Wa693J9se B6DDduKN+lZX/ejgIyHrgnrGTCvFM9esUPWzMQceX34PKp/efy66b3KFHhUYpQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1741531633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ILRU884p1OAoaunx4VDmaSDgJgpHGl18bni0vDofXnE=; b=Ix7/O3CyoYpZV7DwZk0ANGxA/wE9TXlnVQP3wfC3y5a0xK+i8ISaBE5PoyXK/0tIVlumix e9lpnZtl2Yx2ZXBQ== To: netdev@vger.kernel.org, linux-rt-devel@lists.linux.dev Cc: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Thomas Gleixner , Sebastian Andrzej Siewior , Allison Henderson , linux-rdma@vger.kernel.org Subject: [PATCH net-next 18/18] rds: Use nested-BH locking for rds_page_remainder. Date: Sun, 9 Mar 2025 15:46:53 +0100 Message-ID: <20250309144653.825351-19-bigeasy@linutronix.de> In-Reply-To: <20250309144653.825351-1-bigeasy@linutronix.de> References: <20250309144653.825351-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 rds_page_remainder is a per-CPU variable and relies on disabled BH for its locking. Without per-CPU locking in local_bh_disable() on PREEMPT_RT this data structure requires explicit locking. Add a local_lock_t to the data structure and use local_lock_nested_bh() for locking. This change adds only lockdep coverage and does not alter the functional behaviour for !PREEMPT_RT. Cc: Allison Henderson Cc: linux-rdma@vger.kernel.org Signed-off-by: Sebastian Andrzej Siewior --- net/rds/page.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/net/rds/page.c b/net/rds/page.c index 58a8548a915a9..afb151eac271c 100644 --- a/net/rds/page.c +++ b/net/rds/page.c @@ -40,10 +40,12 @@ struct rds_page_remainder { struct page *r_page; unsigned long r_offset; + local_lock_t bh_lock; }; -static -DEFINE_PER_CPU_SHARED_ALIGNED(struct rds_page_remainder, rds_page_remainders); +static DEFINE_PER_CPU_SHARED_ALIGNED(struct rds_page_remainder, rds_page_remainders) = { + .bh_lock = INIT_LOCAL_LOCK(bh_lock), +}; /** * rds_page_remainder_alloc - build up regions of a message. @@ -87,6 +89,7 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, } local_bh_disable(); + local_lock_nested_bh(&rds_page_remainders.bh_lock); rem = this_cpu_ptr(&rds_page_remainders); while (1) { @@ -115,11 +118,13 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, } /* alloc if there is nothing for us to use */ + local_unlock_nested_bh(&rds_page_remainders.bh_lock); local_bh_enable(); page = alloc_page(gfp); local_bh_disable(); + local_lock_nested_bh(&rds_page_remainders.bh_lock); rem = this_cpu_ptr(&rds_page_remainders); if (!page) { @@ -138,6 +143,7 @@ int rds_page_remainder_alloc(struct scatterlist *scat, unsigned long bytes, rem->r_offset = 0; } + local_unlock_nested_bh(&rds_page_remainders.bh_lock); local_bh_enable(); out: rdsdebug("bytes %lu ret %d %p %u %u\n", bytes, ret,