From patchwork Tue Apr 5 13:57:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A9A6C433F5 for ; Tue, 5 Apr 2022 14:00:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WyxdG4uEZMORkYmMUw3qg4OeV2CCce++5W+1yBOBPJ8=; b=Xg6vYXWKeiWyQA SPVHhFwuZhA5vCZajMg/14LVpF9Xf3K45xWjru+WZvdY734sKp89mcPfbW5Wr6pJl8z6CuJpqNw0P 9+FE6skmop9moZvwO/P1TJl/iLHqlLfFH0u8HxYYYWBhyMBcmf1UoMreajIL1e+cHQlr3JTMyldSx OV9cM3FARXDwMiHmKKnl5KbZFC0TcRMm6TalzQ22Bz3tTV0mDIEwKtbOAXoAXD3/FeXPBOL2+gAfY iZwEvwTzlU5LsDHYX54CTb518/ykdzqIm+Fi/bJ1Kj9ayPAoiTTcoWdAEUXi70yDAUDjdYSx5LB4U Mo5tHV5OjhG9vCxAk9yw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjiC-001GNs-DO; Tue, 05 Apr 2022 13:59:12 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjh9-001Fov-Ch for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:08 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DE855618AB; Tue, 5 Apr 2022 13:58:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 686A6C385AA; Tue, 5 Apr 2022 13:58:04 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 01/10] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Date: Tue, 5 Apr 2022 14:57:49 +0100 Message-Id: <20220405135758.774016-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065807_539021_5733EAEC X-CRM114-Status: GOOD ( 14.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for supporting a dynamic kmalloc() minimum alignment, allow architectures to define ARCH_KMALLOC_MINALIGN independently of ARCH_DMA_MINALIGN. In addition, always define ARCH_DMA_MINALIGN even if an architecture does not override it. After this patch, ARCH_DMA_MINALIGN is expected to be used in static alignment annotations and defined by an architecture to be the maximum alignment for all supported configurations/SoCs in a single Image. ARCH_KMALLOC_MINALIGN, if different, is the minimum alignment guaranteed by kmalloc(). Signed-off-by: Catalin Marinas Cc: Andrew Morton Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- include/linux/slab.h | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 373b3ef99f4e..d58211bdeceb 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -187,17 +187,30 @@ bool kmem_valid_obj(void *object); void kmem_dump_obj(void *object); #endif +/* + * slob does not support independent control of ARCH_KMALLOC_MINALIGN and + * ARCH_DMA_MINALIGN. + */ +#ifdef CONFIG_SLOB +#undef ARCH_KMALLOC_MINALIGN +#endif + /* * Some archs want to perform DMA into kmalloc caches and need a guaranteed * alignment larger than the alignment of a 64-bit integer. - * Setting ARCH_KMALLOC_MINALIGN in arch headers allows that. + * Setting ARCH_DMA_MINALIGN in arch headers allows that. */ -#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8 +#ifndef ARCH_DMA_MINALIGN +#define ARCH_DMA_MINALIGN __alignof__(unsigned long long) +#elif ARCH_DMA_MINALIGN > 8 && !defined(ARCH_KMALLOC_MINALIGN) #define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN -#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN -#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN) -#else +#endif + +#ifndef ARCH_KMALLOC_MINALIGN #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) +#else +#define KMALLOC_MIN_SIZE ARCH_KMALLOC_MINALIGN +#define KMALLOC_SHIFT_LOW ilog2(KMALLOC_MIN_SIZE) #endif /* From patchwork Tue Apr 5 13:57:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C762FC433EF for ; Tue, 5 Apr 2022 14:00:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Z0XvXHHWmzhsuRwfqeouXl5518BTBV1i+ODc+xzGp9g=; b=VA/k+v/Y03gwtX cveShFj/MRkD8LX4OPoN+eJ7xOux95nBgLbOshXRXBZRjIYtyry1+R2UTMcBCiPwnDPS4UMrOwZ27 +HoNeBscOFg7IRQy9gSXl8Br/dupkDQpSmxB4ePFReaiG1UZe6JXW1OUiqOmv5WZyXLfVM5WQzpxl dnF63OwRZnqvHxtQqFwA1EhA1dE077Ia6j2r80E55fC57+QClhp2Y/vffRl4Liy17krzZyuBJbFdO 4CTjt4TBuHqKjkA+9aNojhh8D6q2HMOqFfH3tw0O8PoZEDsREZz/JMerTMHvxI+MXoJIFz86xYoqN l1aeRUmDQYzD/el5yiDw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjie-001GaX-0D; Tue, 05 Apr 2022 13:59:40 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhB-001FqL-Qv for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:11 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 63FBE618AB; Tue, 5 Apr 2022 13:58:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7481C385A6; Tue, 5 Apr 2022 13:58:06 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, "Rafael J. Wysocki" Subject: [PATCH 02/10] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Tue, 5 Apr 2022 14:57:50 +0100 Message-Id: <20220405135758.774016-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065809_951848_83E73B84 X-CRM114-Status: GOOD ( 14.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" --- drivers/base/devres.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index 864d0b3f566e..0ed39464ad08 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -29,10 +29,10 @@ struct devres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same + * Thus we use ARCH_DMA_MINALIGN here and get at least the same * buffer alignment as if it was allocated by plain kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; struct devres_group { From patchwork Tue Apr 5 13:57:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 06D45C433F5 for ; Tue, 5 Apr 2022 14:01:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=v0D6BUZxxGyG/GOszz8CG+B2e9r4xBzdcNVB4LJxNeE=; b=jgbFJNfYx0cFYO Vmii4LFDpfuIC9QDvwy7qd3AoUwlcB4oJBG+zzL2YTmrNWBlkro1E4B4ueblaRTMREcEya41yhXAy 1YQvDaSIE9l/f8HvMcJwXwHPCUPb3drTN2AER4bx1kC3YihkIgEwP32TZ2b6b0yCkavi29wbTNmZ0 PUNG+SBr0KJLvyTllHWGVRjIzqXabiwySQ3sUpDMu8oArJnXyJSAuqkj+C+It/CgjUIAWPY83z/xj sx0ZJSozYrkiHRqSItjdccpdBE1KFQpNh3dhiHZoWtrJRNd2+bfGwBlXMWUfeXiBVLaZTF9wlQkKt ohIVXO5Gsp0egv8DRx2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjj8-001GpD-Ms; Tue, 05 Apr 2022 14:00:11 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhE-001Frn-Cj for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:13 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D8B37618AB; Tue, 5 Apr 2022 13:58:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BB97C385A9; Tue, 5 Apr 2022 13:58:09 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Daniel Vetter Subject: [PATCH 03/10] drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Tue, 5 Apr 2022 14:57:51 +0100 Message-Id: <20220405135758.774016-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065812_524790_62B5CD79 X-CRM114-Status: GOOD ( 14.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Daniel Vetter --- drivers/gpu/drm/drm_managed.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/drm_managed.c b/drivers/gpu/drm/drm_managed.c index 37d7db6223be..2d4c51f9d74e 100644 --- a/drivers/gpu/drm/drm_managed.c +++ b/drivers/gpu/drm/drm_managed.c @@ -48,10 +48,10 @@ struct drmres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same + * Thus we use ARCH_DMA_MINALIGN here and get at least the same * buffer alignment as if it was allocated by plain kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; static void free_dr(struct drmres *dr) From patchwork Tue Apr 5 13:57:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5450C433F5 for ; Tue, 5 Apr 2022 14:02:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=u2vxiXnFHmXJixSfPTDtGS4mFNTNOBGlRYkA6ydm58I=; b=X3vvY2ISiLaeJ5 qoigDJAFL9IFll8D2vr2nKo7yLxS3YM7iWQFJofG7XDCFtz2j6qx7SMLJPWsTBU0VpDjeLZLJAtt9 WIa2CnWQbytBJXr2GeNi4NojsFv8BP0UIeNSRSdEnlZVeLbtVwY9IjxMN0YdQuh+Jr3bnQRaDoOP3 z13gm1fUpj9IW9lXX7HFwN6j1TpytORMCQMsm1iU83Rr8kxH4D6QeMA34NCGgCZPSLLmQiDQ4Y5dK 0y1aLIugH/vWrjyO4w0UPyJ/ZlX+nX9oIR8vds9yQWjlw8o05B+GLTLK3MNzW/upzCFqY9tERrmY+ SoxH1KMlFdHdIbmqEctg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjjo-001H9w-Qs; Tue, 05 Apr 2022 14:00:53 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhI-001Ftw-HO for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:18 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 32C98B81C6D; Tue, 5 Apr 2022 13:58:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B1F95C385A7; Tue, 5 Apr 2022 13:58:11 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Alasdair Kergon , Mike Snitzer Subject: [PATCH 04/10] drivers/md: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Tue, 5 Apr 2022 14:57:52 +0100 Message-Id: <20220405135758.774016-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065816_757751_F14EC21C X-CRM114-Status: GOOD ( 13.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Alasdair Kergon Cc: Mike Snitzer --- drivers/md/dm-crypt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index fb80539865d7..b6929dd1d283 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -3250,7 +3250,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) cc->per_bio_data_size = ti->per_io_data_size = ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, - ARCH_KMALLOC_MINALIGN); + ARCH_DMA_MINALIGN); ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); if (ret) { From patchwork Tue Apr 5 13:57:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0E760C3527E for ; Tue, 5 Apr 2022 14:02:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w7FO+VQF97vc5Vw7QGdvSD2ROV/x7TYoK9M+LaKxvQM=; b=hbCHyYnnTUSQ1C lD3MKEXfSh8hpSW6T1bW+oSddDUmsLVUTbSE+fE4wKaWFsZsNieyleSlYKCx/RtYSHauQVetrEW/p V/oEvLp0GnVHr4i8CvfPpa29kQWh7JPuxLmkxJrtiGtlpIk8QJeVV0Aw4D2uqEN9frCot6rAf9ZlQ RdeYF2wfmwqePdtNKQxqf08flcy0Zt3DV/e0UpWWRkCMa9XsTgC6maPcFYAl68UQG/KF5aWsUvHAq ndii8C0Zqiv368wNlbi8nZRqb593GTyFZ7x/xlsxHD97LCRpWpRmeowgByap0ybejmhY3g6YZCKy3 HorCsaone2gUegfJF8kw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjkN-001HSF-H2; Tue, 05 Apr 2022 14:01:27 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhL-001FvV-72 for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:22 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A592AB81BA9; Tue, 5 Apr 2022 13:58:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E544C385A1; Tue, 5 Apr 2022 13:58:14 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Brown Subject: [PATCH 05/10] drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Tue, 5 Apr 2022 14:57:53 +0100 Message-Id: <20220405135758.774016-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065819_468824_924EF622 X-CRM114-Status: GOOD ( 13.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Mark Brown Acked-by: Mark Brown --- drivers/spi/spidev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c index 53a551714265..4baea157949b 100644 --- a/drivers/spi/spidev.c +++ b/drivers/spi/spidev.c @@ -227,7 +227,7 @@ static int spidev_message(struct spidev_data *spidev, /* Ensure that also following allocations from rx_buf/tx_buf will meet * DMA alignment requirements. */ - unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_KMALLOC_MINALIGN); + unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_DMA_MINALIGN); k_tmp->len = u_tmp->len; From patchwork Tue Apr 5 13:57:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E5FDC433F5 for ; Tue, 5 Apr 2022 14:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kavBMQA/fuPNCsObF6YPPSwNz3JpbLwI2NknMJb10A0=; b=EtKZy4KNth5OYT g0Lh9+0nuNeHr5KlxGycrAc2CctIO1e7ulnunexkapCl/LAzebXqQdCB0t9tXzzz+7g77F3KAnAWF Wcm6jIYlRq4LkMgqHXkZ2lSOGrCfUzlE5iFb7m1OZlO6f/IzXRYXdFeH5tXITBwASwXK85WpYBsuz 7lx4opX3I8QjOdpINihb87/TZ0NTyoqeYnbJL2iJHfXiisWkX06jPoKv1ToPg2nyJY1qhiPN4h3ZG KmQVTHFcJ7DQ0humbSdLd9M03ayzBstQjnkB5AMIkptFTvVoZc+0xqD/IQPRVKpEIMQPpj00TXHsz SF/ZEfB8QJX6r0rBuImg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjkx-001HiO-Ec; Tue, 05 Apr 2022 14:02:03 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhN-001FxJ-F0 for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:23 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1CC2EB81D2F; Tue, 5 Apr 2022 13:58:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4C5BC385A4; Tue, 5 Apr 2022 13:58:16 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 06/10] drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Tue, 5 Apr 2022 14:57:54 +0100 Message-Id: <20220405135758.774016-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065821_710445_A6D92F1E X-CRM114-Status: GOOD ( 16.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Greg Kroah-Hartman --- drivers/usb/core/buffer.c | 8 ++++---- drivers/usb/misc/usbtest.c | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c index fbb087b728dc..e21d8d106977 100644 --- a/drivers/usb/core/buffer.c +++ b/drivers/usb/core/buffer.c @@ -34,13 +34,13 @@ void __init usb_init_pool_max(void) { /* * The pool_max values must never be smaller than - * ARCH_KMALLOC_MINALIGN. + * ARCH_DMA_MINALIGN. */ - if (ARCH_KMALLOC_MINALIGN <= 32) + if (ARCH_DMA_MINALIGN <= 32) ; /* Original value is okay */ - else if (ARCH_KMALLOC_MINALIGN <= 64) + else if (ARCH_DMA_MINALIGN <= 64) pool_max[0] = 64; - else if (ARCH_KMALLOC_MINALIGN <= 128) + else if (ARCH_DMA_MINALIGN <= 128) pool_max[0] = 0; /* Don't use this pool */ else BUILD_BUG(); /* We don't allow this */ diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c index 150090ee4ec1..e13f7f6904ff 100644 --- a/drivers/usb/misc/usbtest.c +++ b/drivers/usb/misc/usbtest.c @@ -378,7 +378,7 @@ static void simple_fill_buf(struct urb *urb) static inline unsigned long buffer_offset(void *buf) { - return (unsigned long)buf & (ARCH_KMALLOC_MINALIGN - 1); + return (unsigned long)buf & (ARCH_DMA_MINALIGN - 1); } static int check_guard_bytes(struct usbtest_dev *tdev, struct urb *urb) From patchwork Tue Apr 5 13:57:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A079C433F5 for ; Tue, 5 Apr 2022 14:04:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=m57ZWelSnv4HfSdrR/nM0CC1HFkqIt3A1ZCGEElH0Jw=; b=c0XXDrVNLDYMCA UPCk6NLec0xYOSWeAoyu4m46j8eFGTPfEyo6jmFSP6LeL3CznzJvNCVkIW+UI9gLU30yKctj5OExH nA65R5DmrqmGKwtdEc0/nYk08g+3y6m2vd+nLkgQczkx8NTeMyjkwruqNQ8vfLGO4uFxg4O2whS2v qaA4f2GJTTNniE45oPApS9mjSyzZ67nqKytSHX8H01DM1poBv7DHbK5Uye3DlHDejidS/t4J4ZGKl r1j+7II/CLcOWG8fBPl263wJXm602kjV2a249xPNqoQhWkCuNbRYB8fshjFG+6XAhtTky4gkycQsS PhCyhB3hfPfY7tQOF5EA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjln-001I5H-Q1; Tue, 05 Apr 2022 14:02:55 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhQ-001Fyl-4t for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:25 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id BF2F5B81D69; Tue, 5 Apr 2022 13:58:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 312D1C385A8; Tue, 5 Apr 2022 13:58:19 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Herbert Xu , "David S. Miller" Subject: [PATCH 07/10] crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Tue, 5 Apr 2022 14:57:55 +0100 Message-Id: <20220405135758.774016-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065824_425075_C12E66A0 X-CRM114-Status: GOOD ( 13.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Herbert Xu Cc: "David S. Miller" --- include/linux/crypto.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/crypto.h b/include/linux/crypto.h index 2324ab6f1846..654b9c355575 100644 --- a/include/linux/crypto.h +++ b/include/linux/crypto.h @@ -167,7 +167,7 @@ * maintenance for non-coherent DMA (cache invalidation in particular) does not * affect data that may be accessed by the CPU concurrently. */ -#define CRYPTO_MINALIGN ARCH_KMALLOC_MINALIGN +#define CRYPTO_MINALIGN ARCH_DMA_MINALIGN #define CRYPTO_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_MINALIGN))) From patchwork Tue Apr 5 13:57:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8D05DC38A05 for ; Tue, 5 Apr 2022 14:06:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9HX6llSBOidNqqVCfj7Q4Vo/5eqIsZ9TQrW4jri+ZIs=; b=xpEa24te0EKUwJ zIy+ODF7L73lMQUn3X+J+X9GdW8bSL8B8c3b/nfyxMWi7FrYX+HaLnjzUVLIIg1ACccl9ZdYkzRz8 QScokahBW/FaGgNPlxJDSyUQokxoDvr7syKh8ypM8ACg5CvXpnIlJPkyFpVX1xHXcMq8rYga1bOAt 5ui1Y3krUjhu7zRJLlniebydmhdxnJ1JOo4GHvr2JbJAvqDbk0AEqrqVHTSCU6lwIbMC3I2+jpqmO q3m0v9lVd7sGVd5p1WyBXc5tZDHIDbpmeR7ITckzQOADo9rOzCmVzFyEIsVOT59z+cJbJ41hFiqG8 2nZZk5ndGFjSyX+fe8Tw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjng-001JAA-U7; Tue, 05 Apr 2022 14:04:53 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhS-001G0c-FR for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:30 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 01D99B81BA9; Tue, 5 Apr 2022 13:58:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D18DAC385A4; Tue, 5 Apr 2022 13:58:21 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/10] mm/slab: Allow dynamic kmalloc() minimum alignment Date: Tue, 5 Apr 2022 14:57:56 +0100 Message-Id: <20220405135758.774016-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065826_882992_4843EB35 X-CRM114-Status: GOOD ( 18.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org ARCH_KMALLOC_MINALIGN represents the minimum guaranteed kmalloc() alignment but an architecture may require a larger run-time alignment. Do not create kmalloc caches smaller than arch_kmalloc_minalign(). Signed-off-by: Catalin Marinas Cc: Andrew Morton Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- include/linux/slab.h | 2 ++ mm/slab.c | 6 +----- mm/slab.h | 2 ++ mm/slab_common.c | 33 +++++++++++++++++++++++---------- 4 files changed, 28 insertions(+), 15 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index d58211bdeceb..2137dba85691 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -332,6 +332,8 @@ enum kmalloc_cache_type { extern struct kmem_cache * kmalloc_caches[NR_KMALLOC_TYPES][KMALLOC_SHIFT_HIGH + 1]; +unsigned int arch_kmalloc_minalign(void); + /* * Define gfp bits that should not be set for KMALLOC_NORMAL. */ diff --git a/mm/slab.c b/mm/slab.c index b04e40078bdf..4aaeeb9c994d 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1256,11 +1256,7 @@ void __init kmem_cache_init(void) * Initialize the caches that provide memory for the kmem_cache_node * structures first. Without this, further allocations will bug. */ - kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_info[INDEX_NODE].size, - ARCH_KMALLOC_FLAGS, 0, - kmalloc_info[INDEX_NODE].size); + new_kmalloc_cache(INDEX_NODE, KMALLOC_NORMAL, ARCH_KMALLOC_FLAGS); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); diff --git a/mm/slab.h b/mm/slab.h index fd7ae2024897..e9238406602a 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -283,6 +283,8 @@ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, + slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); diff --git a/mm/slab_common.c b/mm/slab_common.c index 6ee64d6208b3..594d8a8a68d0 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -838,9 +838,18 @@ void __init setup_kmalloc_cache_index_table(void) } } -static void __init +unsigned int __weak arch_kmalloc_minalign(void) +{ + return ARCH_KMALLOC_MINALIGN; +} + +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = arch_kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if (type == KMALLOC_RECLAIM) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -851,10 +860,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_ACCOUNT; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags, 0, - kmalloc_info[idx].size); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags, 0, aligned_size); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for @@ -904,11 +920,8 @@ void __init create_kmalloc_caches(slab_flags_t flags) struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i]; if (s) { - kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache( - kmalloc_info[i].name[KMALLOC_DMA], - kmalloc_info[i].size, - SLAB_CACHE_DMA | flags, 0, - kmalloc_info[i].size); + new_kmalloc_cache(i, KMALLOC_DMA, + SLAB_CACHE_DMA | flags); } } #endif From patchwork Tue Apr 5 13:57:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C9DCBC433EF for ; Tue, 5 Apr 2022 14:07:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XJ7WKoCBMxYfsfMcxenFf5v0ZBLyOBqlvspk4bl7xVI=; b=lrWXd36Fcqr9Wb stjLRK3nKK+XS1XFDx/ppPdkai+lTOEj8oER/vEUXLtjdr4olicPNmBtpEEnKh/gn9qZ+rOClP5tL MwcWKGwGFix/EAVXZEgT0gF/NwjS8so0hZNCB7M2Pw6d1QWjoN0ku6REqMXCUADc5IxjSK1x19yBB MmYKfi7qjSyvFbozVVKVGEGxisBbacjK6F6ytUkfypa+Mgm0tGcB9cX/ffPD3j/Hwt9EZxMbPpT/H 0UncrdLg+Jo7tQxU8B/GJHhfv185mro+GPNDVDzl3UWLQiY8HZ8QSbv/0m6IDJXubk/Mp5iz1M0R7 Urswd0kZI/af4LGk2jaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjog-001Jbd-ET; Tue, 05 Apr 2022 14:05:54 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhT-001G1i-BU for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:30 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A1BD761910; Tue, 5 Apr 2022 13:58:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A697C385A7; Tue, 5 Apr 2022 13:58:24 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/10] mm/slab: Simplify create_kmalloc_cache() args and make it static Date: Tue, 5 Apr 2022 14:57:57 +0100 Message-Id: <20220405135758.774016-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065827_528075_C95A756C X-CRM114-Status: GOOD ( 16.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org create_kmalloc_cache() is now only called from new_kmalloc_cache() in the same file, so make it static. In addition, the useroffset argument is always 0 while usersize is the same as size. Remove them. Signed-off-by: Catalin Marinas Cc: Andrew Morton --- mm/slab.h | 3 --- mm/slab_common.c | 9 ++++----- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index e9238406602a..d334b5e6d7aa 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -280,9 +280,6 @@ gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); -struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, - slab_flags_t flags, unsigned int useroffset, - unsigned int usersize); void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, diff --git a/mm/slab_common.c b/mm/slab_common.c index 594d8a8a68d0..3a9f7a1e6490 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -655,16 +655,15 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->refcount = -1; /* Exempt from merging for now */ } -struct kmem_cache *__init create_kmalloc_cache(const char *name, - unsigned int size, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize) +static struct kmem_cache *__init create_kmalloc_cache(const char *name, + unsigned int size, slab_flags_t flags) { struct kmem_cache *s = kmem_cache_zalloc(kmem_cache, GFP_NOWAIT); if (!s) panic("Out of memory when creating slab %s\n", name); - create_boot_cache(s, name, size, flags, useroffset, usersize); + create_boot_cache(s, name, size, flags, 0, size); kasan_cache_create_kmalloc(s); list_add(&s->list, &slab_caches); s->refcount = 1; @@ -868,7 +867,7 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) if (!kmalloc_caches[type][aligned_idx]) kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( kmalloc_info[aligned_idx].name[type], - aligned_size, flags, 0, aligned_size); + aligned_size, flags); if (idx != aligned_idx) kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; From patchwork Tue Apr 5 13:57:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12801570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4E815C433EF for ; Tue, 5 Apr 2022 14:08:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FfDMSKxpzqKyBNnhaSeYHy9+e0o+GkUQGf8G454PiL0=; b=l/D4mqkdhX7Gw5 o+ZLi7l9zcEVxyE26wfHFNyXKTJZ/iueg7RTF0ZGcO/3TkYEWbYrZJLNpsgVZjdLDMqk1YCWpkVy5 p4d0pxe+mtmaIdieVsBEJBefRKJpnssLK0/LMP5CxvBsuJg4JAMCS5RHMcFw8hOVjHITRdXH0FrZG uNXkjaC4pzvbveMhMuukgbH09lk5QCndRTYwo2xXeupEUfK95hs7/rkCZxQNfF/JU2Q63MsqnndXO F2vPJabjr0VhEVe2fJsVp9gdAtiR8nv9C9zi+oy801HmHuwRpXXjR89JYUMee2OhhLLUEDssqmcKF n3nwhAuQr4p2pThsChhQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjpl-001K92-R9; Tue, 05 Apr 2022 14:07:02 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbjhX-001G3m-82 for linux-arm-kernel@lists.infradead.org; Tue, 05 Apr 2022 13:58:33 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B19DEB81C9B; Tue, 5 Apr 2022 13:58:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7941AC385A6; Tue, 5 Apr 2022 13:58:26 +0000 (UTC) From: Catalin Marinas To: Will Deacon , Marc Zyngier , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds Cc: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 10/10] arm64: Enable dynamic kmalloc() minimum alignment Date: Tue, 5 Apr 2022 14:57:58 +0100 Message-Id: <20220405135758.774016-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220405135758.774016-1-catalin.marinas@arm.com> References: <20220405135758.774016-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220405_065831_581146_5D527012 X-CRM114-Status: GOOD ( 11.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Define ARCH_KMALLOC_MINALIGN as 64 since this would be the minimum requirement across most arm64 SoCs. Define arch_kmalloc_minalign() returning cache_line_size() to set the minimum run-time kmalloc() alignment for those SoCs with bigger cache lines. Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/cache.h | 1 + arch/arm64/kernel/cacheinfo.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h index a074459f8f2f..0bec986c9d51 100644 --- a/arch/arm64/include/asm/cache.h +++ b/arch/arm64/include/asm/cache.h @@ -48,6 +48,7 @@ * the CPU. */ #define ARCH_DMA_MINALIGN (128) +#define ARCH_KMALLOC_MINALIGN (64) #ifdef CONFIG_KASAN_SW_TAGS #define ARCH_SLAB_MINALIGN (1ULL << KASAN_SHADOW_SCALE_SHIFT) diff --git a/arch/arm64/kernel/cacheinfo.c b/arch/arm64/kernel/cacheinfo.c index 587543c6c51c..61211cd597f7 100644 --- a/arch/arm64/kernel/cacheinfo.c +++ b/arch/arm64/kernel/cacheinfo.c @@ -97,3 +97,10 @@ int populate_cache_leaves(unsigned int cpu) } return 0; } + +#ifndef CONFIG_SLOB +unsigned int arch_kmalloc_minalign(void) +{ + return cache_line_size(); +} +#endif