From patchwork Thu May 18 17:33:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39C1FC7EE23 for ; Thu, 18 May 2023 17:34:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C97FD90000A; Thu, 18 May 2023 13:34:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C48F2900003; Thu, 18 May 2023 13:34:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0FE190000A; Thu, 18 May 2023 13:34:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A2663900003 for ; Thu, 18 May 2023 13:34:28 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 77823160227 for ; Thu, 18 May 2023 17:34:28 +0000 (UTC) X-FDA: 80804075016.24.DFAA24F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id CC99A14000B for ; Thu, 18 May 2023 17:34:26 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431266; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0mqLCBdpgkfqDUcqHwwIym+BakJfQ9CLI5FzmbFJWTk=; b=uoPXaQrtScjymt7TfHzvR1jnvdcKgp3oPHwBjNWjsL7Xz5ViBwxf3TlNJc6SbncabVAbkz j6PB0IJ6uuZONqxoOzK+B6f1L15Y3wVbqAtcAckn1sdY+1svilz0eUVijG7yLmfKr+KBtq Wbjwv/X4ts2Bt9CMmrdEshSNfCd025Q= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431266; a=rsa-sha256; cv=none; b=zv7Rnvl0yPAKMQTFhbCkINRVVzy0yWqEKfskEOfCM0I7IS7cOKvhpJx0C6chQ25WMh0QiZ DmsNZ4FdcIX7zjE6lb0PHwxnz8G5vaKnP95F37Do6lI0k87Ab/vJdGjLnvlOcKaquEIUK6 yGGOGGDTxyf9b2eoKoW0DOioQAyqW0Y= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E5B6565153; Thu, 18 May 2023 17:34:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AC1C9C4339B; Thu, 18 May 2023 17:34:21 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 04/15] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Thu, 18 May 2023 18:33:52 +0100 Message-Id: <20230518173403.1150549-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: CC99A14000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bzk5bkmgwi8dzpiu3ko8x8uutuy98fxi X-HE-Tag: 1684431266-742692 X-HE-Meta: U2FsdGVkX1808b38ZZSsPdP/L/s2cYc/gM/kDEtkpn93cKwQiIsCHW/+UwQy8E0aBpRuY0yYp9CT8LI0Gtfo+oXUXmbbvAktr8pouNaWlUPWFsl2MvFMWFJt0SU/je/vsotTXvyBa4D0U0wdNUP7GeRZIkuJjHpT5QcrC7XQkLdqjlEYSuasoehouVZOyxFp+gfI8LLEp77DXAENGE9QLQU3w1n7V17GQ5v8cs16CJoGysfuqUisp2e7or64YRPBDUAo0LItYb4nUb8FAxTelar5P/E01jqqGI+Uq86EcF0FhCXSj2Bgh2wQcj6sfjVXpCxllfXArSs2NeUQbd7pm8nAnCQHTcV93Cn2LKtyHmb00eb7PCJQFr3ePqYPmfrxJ1QnoXDxIsFVahIxwbwVqeL9dvjdxvL9zA1H6eq1nQqxkiM7j6TedDmNtJ1Pqh2QVetearAc7oK+Mm9JmeHMEzvMA4MTsp33PPsAV9uwC9VYurliLjJ23VuZhY1nKO/c3vtBEdxHTw6SqjW2UmLDV97sWhxE/akt5yFiDBojVwAjJeOHG4FT2P1F3tCxLNTh9BGETWZRk8uOF+2GsrNGB1T72riIAXHLO7IgUVRdT4Hn/9MD0UeWfWgzW+aqOeW24JJR4Z4FX+AiLVpdx8NZoTuDJ6ks4/vwTar/A280Qbrk3KqxhKUF095rg7u+xXCW1iTg407KWPgMxxRixXxIEX0+9XDjDggMy3DJSikF7ryTr5LXZn1njtL/qUm70kSjn/OaCwd/KK2pl10mNMVaAinu0MgVtsNaItYxMkBEGizeNZwqp/ofFvMFLh2IsUqnPeuE++7sNIRBa+JmkRtjuEZCbumOCcmLuS9aIh2t9YDFitMNdeBqg5oSQsMeXPF/uysaRno/2hCoX94ydPGIpgOiAAnDESV0500KWnHzdkmjkNM1GTJABvcooazJhCIEACU3et2hz7RGEVDNu32 ryeRCnSB pVErEUmWeGscYdfIV0BxQZfw3YXIr/7tNJP9dW6AaTnDqfn1sv64uXrSJIxZ3o9bSWVb9udEUXbbxUG7Swl9bIe7iN/BER5nEQS15yfvnGW8M2KsS5IDaVZmNMy74pv/UmHLWrrxPlE2qa+Cr36KLqbactdFQAJFQWzThHs1bsQgT/AxuAR6yJ79LlkzUB7xvOe11DGz3ZmxMrLXL7WWbYBm+WpseQgoIskrenKhAUnVnUiPAJeUUx8TafV6hYivDZdsqot53DgjRFBP1uK5TTpnTQDNnU8unucgYcveg6/ybJVRY4+/kNkpnBFTsPuis1vno3qglOp0AUkT/1HbYLXxxp8En+tX2WucdH+G3LGxDfsP4iJH4kJ92JK8li+yBKF46y+lqPs9bjFS3a+rSfA5fC2nRCEYBqoUJCPKlukVPdcpk/DznEPLNtZdMWa/QE9i2VmXoRKiXPgQDASs0PEmjwEj0P8pePMiPPGme2G6Cf1j5t3F/srPDrlXkBylwb8sf7zJcTfnFcxUGBQsfQcJtmf2xHazDBcDrFxcOB6p6BQrp9yspubKrA6u8EJ7bmL0V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for