From patchwork Wed May 31 15:48:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 961F2C77B7A for ; Wed, 31 May 2023 15:49:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 37E6B8E0008; Wed, 31 May 2023 11:49:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32E7C8E0001; Wed, 31 May 2023 11:49:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F6518E0008; Wed, 31 May 2023 11:49:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 11D6F8E0001 for ; Wed, 31 May 2023 11:49:02 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8461B120286 for ; Wed, 31 May 2023 15:49:01 +0000 (UTC) X-FDA: 80850983682.18.E2AA8F6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id D428740010 for ; Wed, 31 May 2023 15:48:59 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548139; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0mqLCBdpgkfqDUcqHwwIym+BakJfQ9CLI5FzmbFJWTk=; b=yJ/kFAUArFGd7RqCqx8E07Q3TAgWO8KiqEb+0t5mt++2Lo/i2XwvHMIW5Q8oKBciWDWYpz TRahR7pdBacF0+fiVztgRDOv/FM1YmvS6O/IJ7OGiKRjMuwLE9rkG+ly/LPE4dqoiJyyyn 9fFcjpcOOgLYjR8u58qWr+GXbzhi8HU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548139; a=rsa-sha256; cv=none; b=tEb3U6wKvWpXSi+KMtg44rL1nLHBUjxK8JK+yldyIFSHSVKcL0mzwv8cxbZ/C5twKzGqa7 dwfFHWht47swvTMl/WmPJUio6wMMWy0sbXVf6w4DkWjWfh5v9fsCPB6+tmMnX4eQBsPlR5 QC6X0FoNQ+rCZkn0MlT8/kYQ6JuEPEM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EEBDA63D92; Wed, 31 May 2023 15:48:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACC7CC433AF; Wed, 31 May 2023 15:48:54 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Wed, 31 May 2023 16:48:23 +0100 Message-Id: <20230531154836.1366225-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: g8w6nauwzgaerxecgb8q8e94f1ic4m9d X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D428740010 X-HE-Tag: 1685548139-708932 X-HE-Meta: U2FsdGVkX1/IBEoS5sWg1S1PAh8LtGWpEJqgQfG/2Sphyh5fFPbE3YKuOyIosW3OzFns8u0R+vb6nur4aqpzt21E0O1a/B22gm4FA4Mlt3ZWl2KLbyzoF5dHrGsO27F2u2CB98x0KuE5QXX/xDjR6UmuerYmchFeNemPmbiebnvHFYZl7hv1+F7VEHVIjcJ+8tUF+vLZY2RgrReHsGXOwOUWA+XkYF5AInOjUUHxljcTwU3t8ON4IUF1Gv1R5oJ+akqavI051xIK34hMu/gBjb22dDRYztpKLDjC9LlN99U6VqUQZf0t4h7R8+FIYnNZ/Yu/Uqkxg5todaLgS02goD+AC0vEFYaioaO/r0Ds7VMFv+JYe8GhSpVwOOk2e0D21XzuIXTjDP/EPp19YEF2wsMabz0ChEa4chm8kLiPTcaqV9pVuZyX4N7CEXd49lRPItLNdjrqCIKlxUP6CBMWW8CozPY3YX9YCH3ah5GHyUaBc/mPA/t4Pzi/6o54h8O38chkwGUK2uk79MfuDOQi9amh2GHdFNMMsZierdh8PRAfkqqq9a8azMQ57abzMwJtXmrWyeyH+5CIhB/spoa7ii9kxyE3IOoRy4Tpo1z3gV77FRK2DzS0BbOSSyJkexygHw85CNfm5y9sljIGeqoluTIrpJD06djAjDIgr1q37MmhXFt6MSDIBe5y5y5CQSDR9LF2m7mhwhf6oQAz3nxekOEcB1AvKDKDxAWIzC5+Rl97Kijp1thcvl+8VZg0kt7FBb6xyCR9W04fDyk0bx3c1tL5n8ECvp5N6yvxBbS5nZxjBT+jrZbT2o0fIVdffaEJN4at4ENqi3gtIwAWLHt/Z/Oaoa7HafST9crbKt+HDcrntUdL/iDZVs40XSKCKrpqUtd795Ucc890sZt+GTFtxwJHcgeIJS0Z5XZ6n1gnGCGQrioF2m/VnhohHvij4m1j9c/QdMg0A0WjdNJGcgP xx6yjGsi WHNKy1FspEPYaKaxb1Wo/bRLGQsJwETVH61rL+Y4dKP0ptYtXupdLO7h25j5qxNVoAVQwPokHBoHObmwjsx5F+Glyj3teWuKZiuh/AEz5RJKtwFu4WRSv+ROjtySIMR9jIMbm2qGc2WL4LmO3blvOnuwytb+E81ERmjzDGJpI7Cs9I60s406ceLHl/yTGDtjK/2/c7KUPXed2r2c3bKxjKjvGlHd+xbIPQbVrTUQRLxjKwejJNa8amSKPG8F5ghbHqQI5D7RPM+itZj0Ulj8vWljY5s4j1hBxZ43lNYGj5Y3fZl7DD39Rn6fk0G9mvA7HHjaRzxaPUC8s5/onGYMB1h9ccddSqEUtJb1VM7kfZaZnYMUli2A2g4BYK9Fwhm8u2LV1/2VkYSlaaq09LsZwv4WmtgD/shFtjaHwxpaKbdsHpI6hkglDva0ZmOyolUJadK2FCh6Jr0JDsOXbdvQNyfHJNFWXu62A8IHDOz8fAR7R7AYcuB6cAYAIMQ7JO2s4SEO8knZkoITsbRLYEz4demMOUr816IcadoQCcOEPrgfSwfVH2H+alvxsCOvedHn1HFr1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy Reviewed-by: Vlastimil Babka --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for