From patchwork Thu May 18 17:33:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B753CC7EE23 for ; Thu, 18 May 2023 17:34:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5BA29900007; Thu, 18 May 2023 13:34:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 54462900003; Thu, 18 May 2023 13:34:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E4B8900007; Thu, 18 May 2023 13:34:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2D9D7900003 for ; Thu, 18 May 2023 13:34:18 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E60E8A092E for ; Thu, 18 May 2023 17:34:17 +0000 (UTC) X-FDA: 80804074554.05.A13D7C3 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf13.hostedemail.com (Postfix) with ESMTP id 8337920005 for ; Thu, 18 May 2023 17:34:14 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf13.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431254; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=54tOsqc005inVEd0dWDBdtkpjhyjeg1kQ6gzKb3D0+8=; b=24+pFR2Mt4xN6yb0c4J/zvmIImcxfpyjEYXQbpu4ZvgKY2sePKk6gIs9RSu9uhUYKqI2bh EkvWFuyYNndxporh87sAFToUak9/+bczdkVlA+mZ3l4Uo2EvKPwUIsYB7YvEPjhgdEKcqK fTlprfAtWpf3kzyJTRXvyzezCjxXD2w= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf13.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431254; a=rsa-sha256; cv=none; b=jYuj3BJf4NPqw5kqc/E9J1687GZQlHKZxdf+BYQ/yVFvvAVXZ43oASBomfHS/mCbxcPCx9 6sZxrwqu59RrXyG5wyWzkbkWFe9QnCXalsBBgeW4qXKEFPmrR2Stor9v0+YrJAgJ/mBfe2 VBSnasP6A/aIyNvK/U9siSGzMp9h+oY= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9FDD865151; Thu, 18 May 2023 17:34:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 65FCDC433A7; Thu, 18 May 2023 17:34:09 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 01/15] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Date: Thu, 18 May 2023 18:33:49 +0100 Message-Id: <20230518173403.1150549-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8337920005 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ozsn8txioikm34i4j6m8qr5nxyfkqag5 X-HE-Tag: 1684431254-605421 X-HE-Meta: U2FsdGVkX1/uw+h+k5xHhCYZQzH78Ow+FWgfi7tJWsy0GgEgzR2uEEuIy4FhgBzXcTxDV+7AJTRe8gU5iwKii9lHjjLOV1ME8oDsLf/QoYxuAIC6RNOjQw+Kf2oF3S7CkbeL99TY7ezIjHD04J/mzUfluKhGBVGw8WyIC4Iht9pCd5Z1vVFoUF6RcGo/6gosx5L3pwrL9uHKRRIhpNIK1PmrZtxrWlWz2uEs0l1zZZsoCkUSYRrxfN/vxxxCeaLNpafT3jpFiZpYIIdOooOy2SijfaKuZ+0TCMVs98Bk7Vf8uihjyUzb1COsyF0V7+Y5xIAts/cog6aHR6KU+EXZUrTms06kI9DpVpfzctYazGDQtW/iUC/HFOwCbhYw5lZekb905o0aGgQJT3YMwh6tz2921zbx4erx1o1wxL7CumHxI6auEau1HxGdERPJto3FPzpiZYynyRA6WlvcWN7i39jsT+f8WlKEP/R9Unvhsx7BacqaiYe8IX+eIdkXCm4G4HmIVSsu2dEJhkuuC1Qv21u4AkzslJHwKlJ7pBIhCVwpP6NbzkQUKKmCSdEiVkTdXn4BdI2T8yNGsZCjUM1UjUyImFE6k1A/YWie7A2J8/HGN0C75APJydtAB8xho9XkFrIPq0YxhJtC5rSp7kLzM/W6Gytfl7yQXwHuHsXgglSZvWIk4qrqHVMYVG0bjeFqNPrgK1QpYmOqTYLgCXJMZN5cqdL0TYIW+83KN/WaO4toyVZy3w/n0FRxY+TBEWaZMq4j8N+1Lp2p56ZHMUeVnjnePN7NyLeZ0yyL1ChxSOHl/SG55QvXUKPOmQ4ClrAOpEa9oe76HrIpR7jb0AKHcDu3PXcshQl72wU0c8uJ2iajZFjQUHF/VHlhn60n12/wkmJ14nswT99GlW5z8YE5Ckh+bhGPw6IEpkTWaGISXpeXIlaq7hnfXM/IZLHJ++y9SLCNOrIT3Y1OJpp4Mxh joG+QeBs DRi09trs8Nakc5hq6GLc7VVKsgHPoHUNkXp4O60AtYRcyDz+R/DacbeXlMd+W0DfbO+PdfPakG0z3U1yEf02sl1lTKnBc3xDm4s9J3KBnwezcgNmwigrswIV40LerAnB8BrojQSYZW0fGzQh7tFHqEmO+gpjF5Z9fgHsdzrsnVm7NTVG55vGjbz3Lktf0bqtdBBuHvWfa2wOB1rdHRBC5nzDgZ2Hb5DvtZVkuj1TMGVLgwQ25SR7pE5SK1twnRrPAc0o12k5oWGVzOY1ZRB6zaS0Xz8/yI1VOcew807H24NdvtPKoQ2jnFvdDIO9PVHI1n3z8IOWYwRISKbxRxlUDpMWUXtPoNAFQavsMPFD6IdbSZoELios7tvMWCQ6HG1RA7Q/Y0IvZ+donGqe2raxr2Yht/63WJpX4UTEmj8oBP65YZxjUA5WLAqnRqWCfMCP4BE6Bax2Fcj8QZIvY3Wfzdns+fpO1CLfIqJKBKb+bQ/Q9xS0dLaOiJp1xUFfitcc25VA+WKKMQEM3poqpd3fMISQrCR4f9/d9jtw3XsXc/zTXxQN7AKmcPF+JgTY3hA9BqDZU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for supporting a kmalloc() minimum alignment smaller than the arch DMA alignment, decouple the two definitions. This requires that either the kmalloc() caches are aligned to a (run-time) cache-line size or the DMA API bounces unaligned kmalloc() allocations. Subsequent patches will implement both options. After this patch, ARCH_DMA_MINALIGN is expected to be used in static alignment annotations and defined by an architecture to be the maximum alignment for all supported configurations/SoCs in a single Image. Architectures opting in to a smaller ARCH_KMALLOC_MINALIGN will need to define its value in the arch headers. Since ARCH_DMA_MINALIGN is now always defined, adjust the #ifdef in dma_get_cache_alignment() so that there is no change for architectures not requiring a minimum DMA alignment. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- include/linux/dma-mapping.h | 2 +- include/linux/slab.h | 16 +++++++++++++--- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 0ee20b764000..3288a1339271 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -545,7 +545,7 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { -#ifdef ARCH_DMA_MINALIGN +#ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif return 1; diff --git a/include/linux/slab.h b/include/linux/slab.h index 6b3e155b70bf..3f76e7c53ada 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -235,14 +235,24 @@ void kmem_dump_obj(void *object); * alignment larger than the alignment of a 64-bit integer. * Setting ARCH_DMA_MINALIGN in arch headers allows that. */ -#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8 +#ifdef ARCH_DMA_MINALIGN +#define ARCH_HAS_DMA_MINALIGN +#if ARCH_DMA_MINALIGN > 8 && !defined(ARCH_KMALLOC_MINALIGN) #define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN -#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN -#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN) +#endif #else +#define ARCH_DMA_MINALIGN __alignof__(unsigned long long) +#endif + +#ifndef ARCH_KMALLOC_MINALIGN #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) #endif +#if ARCH_KMALLOC_MINALIGN > 8 +#define KMALLOC_MIN_SIZE ARCH_KMALLOC_MINALIGN +#define KMALLOC_SHIFT_LOW ilog2(KMALLOC_MIN_SIZE) +#endif + /* * Setting ARCH_SLAB_MINALIGN in arch headers allows a different alignment. * Intended for arches that get misalignment faults even for 64 bit integer From patchwork Thu May 18 17:33:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7721C77B7D for ; Thu, 18 May 2023 17:34:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C2DC900008; Thu, 18 May 2023 13:34:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 14C7C900003; Thu, 18 May 2023 13:34:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2EF4900008; Thu, 18 May 2023 13:34:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E3ADF900003 for ; Thu, 18 May 2023 13:34:21 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7A17780904 for ; Thu, 18 May 2023 17:34:21 +0000 (UTC) X-FDA: 80804074722.27.D24AB96 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 9C7E1180012 for ; Thu, 18 May 2023 17:34:18 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf16.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431258; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GFrMa2owsewxfwyo+ZMqMWw5mqNbHDxBwlBBT0yhJWE=; b=eRUZyfRQjzGR/zCTezJmzDML1W/yCwOo94YJvS+jZOP5vzV4o6EO/7FwUJWvY06EUO+2dV r0lzYX5rkiOH/T0g2S64f4dr7v1sRZsQMy83PNmHRtUqgHGR5W5aaBd7EM2djm2Vee7GSE UhnZkMkdkp7qNA58gbXyQYQxmGqLKoQ= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf16.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431258; a=rsa-sha256; cv=none; b=4tmIJxFOMQPM+kqcDAteyXjhL5UIPR8jzO8+nQ4VKIJ4ARD5v98Lp2/wgQupak+2EX7TkP N6dvCLfIA2+eAZcFMVk8ClT6eGCjgecn5wv8+La+jh82uqQM6Jj+ifxctop4yIT7gbfDGP CxzE8bxnnQm5IkB9rljKOOJqE8hX5xI= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A6B25642F3; Thu, 18 May 2023 17:34:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 71A72C4339C; Thu, 18 May 2023 17:34:13 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 02/15] dma: Allow dma_get_cache_alignment() to return the smaller cache_line_size() Date: Thu, 18 May 2023 18:33:50 +0100 Message-Id: <20230518173403.1150549-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9C7E1180012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: o45qetyicwkq5c73qpfamn7tdrqr3s4m X-HE-Tag: 1684431258-979533 X-HE-Meta: U2FsdGVkX19l0GMroA95+DmXvfxl7kEnNF1EUACG/263RD6kPvU90SFpx5y7cQEi+MhJPOQ43qfiXPp6NR8w4VJdYvnj9w11XCUTDHQMV6FMNcuDQEo/4fY/RIunssinKJ7ZS1PnQlKWoJM638wAkA/P2WrE60Ru5TzPE3Vl6ZfAFQlBm+EygzRFSCaMZ3kOrcThfnHqtBNhmZyCSU0USAnPowTPBhksBh75hcVgYvt5n0XTbMoq/bYxBWbhoDtZkr6PlOjaPybHZDv9aoQ/2CBA6Vb2RyxPZfVYFjlVEYiys8nP+Sfi3D1U+UWMCChI232siU3AAajbaiOVRzQAW6SDU2131u5tdXB8mP1mVhwbyu3yN1aYa39SB759EEFiQsz5sRBRF/4sFk8R9G0+kR6YhXEjbHLVbseRoasBAqgEf5iufJnogu0oFKxQxcDzKCL0f24gE9tQLpoFq9LnL5KdexcHaT1U4Lb7ENK5fkWsfkUhEGD7OWkymCGh54fgU//rEfTWaM79J7RXGJ+plXuC87j9JpLOn5R8ZRKWR0FONDVc8gnO4HC4+Wgj0lrHVGiPcNMT6hER5GN9isilAjcHKwBarKC+6g9bd7AWib4k1mN0f8L1ctOut9mif8lY/2IfqR/S+kyKGJpnGNOGSscH9iIHWf8PRwj5oLO9NLiImRTHk3hmln685PVENGwpHpc+kAYoTZ37mtlzJBb3piTlaGv92ZnnGDbWYKgZIrMMPVkDjPqh4rSDsOotDZnlIKs3cIhKweBGjMmWDlfw2Du7lVktO04MYZZBWyY8RjkYNXAZUJkkdjRI/1qD4+m3QLxI4C+D+k61kXWQBcl13m1cn3Yzz9v4AcBdbxT0N5/5zPs4tlo/kPaiN9jpyhdcR9J4FsBaGVg6ghRgl7UpK+Lj+lMjcjx6dxiHQcMNlpAOaRJP7e7jCrK3matCsl7/g2UKWdw5lNoKrORY7wc XzwLo5yP YLfiUXSmvKDMkclqA+De31eGlkqnZhhkVTHToEgDIrvDaTE9GQU3AA2rxChIzS0Qa15FMkmAKDCEmyuAGIgUzO+5y/rGnQ3kFZci049hNIz8NHbDFFeHTDXWidi9qsxXYDZChwqD8Io7gZ/YcvMb0YG0sbsPg7YSH3eb/l+dcOpJ+dbcTN+3lb0Oc16BzPjRh4EzehFcgC7HKzayuleD3XyU6nJD/5TeyGL7lyqZAqskEqg5RtEYTPI6GMAkw1nM3ZNY4286B8lYrXfXpfqaP1KhUXjWgzeHCtVSayDREsEhyLab3qJn9SefaFiZ1aHRdVt36dD8OhmZBNAAaTkUWRenTK/IYidajcO77YIrQLxh3nZ45FhV91MS+gutNcoBHjG+hF3z3izcQPtTXL0FUK3ITMxY7uomsdl4zOu/bvl/7cCEz38u1Y8ui8TtkKPfS67Zws7KpM42mpEgOupYT2mXjGGX/XWBizvOkfOVUudTISEaWCQKop+5J737gC20+kL5/aSFkXV0LpzW88WshJ6ZNZQmxn9YrRgeS9O/iSQ1hyvIgrCNQO5K+ZuvIluzi71Na X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On architectures like arm64, ARCH_DMA_MINALIGN is larger than most cache line size configurations deployed. However, the single kernel binary requirement doesn't allow the smaller ARCH_DMA_MINALIGN. Permit an architecture to opt in to dma_get_cache_alignment() returning cache_line_size() which can be probed at run-time. Signed-off-by: Catalin Marinas Cc: Christoph Hellwig Cc: Robin Murphy Cc: Will Deacon --- include/linux/dma-mapping.h | 2 ++ kernel/dma/Kconfig | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 3288a1339271..b29124341317 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -545,6 +545,8 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { + if (IS_ENABLED(CONFIG_ARCH_HAS_DMA_CACHE_LINE_SIZE)) + return cache_line_size(); #ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 6677d0e64d27..cc750062c412 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -76,6 +76,13 @@ config ARCH_HAS_DMA_PREP_COHERENT config ARCH_HAS_FORCE_DMA_UNENCRYPTED bool +config ARCH_HAS_DMA_CACHE_LINE_SIZE + bool + help + Select if the architecture has non-coherent DMA and + defines ARCH_DMA_MINALIGN larger than a run-time + cache_line_size(). + # # Select this option if the architecture assumes DMA devices are coherent # by default. From patchwork Thu May 18 17:33:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B29AC7EE23 for ; Thu, 18 May 2023 17:34:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBEE8900009; Thu, 18 May 2023 13:34:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C481C900003; Thu, 18 May 2023 13:34:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B110A900009; Thu, 18 May 2023 13:34:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A1AFF900003 for ; Thu, 18 May 2023 13:34:24 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6FC5380902 for ; Thu, 18 May 2023 17:34:24 +0000 (UTC) X-FDA: 80804074848.02.1613AF2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id C5628140019 for ; Thu, 18 May 2023 17:34:22 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431262; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=35xPKuIoBzBBtPc7oLgirz1u4PdrAIKlSY99QdYwnw8=; b=BJylibUD/TFBPfgEXMkQ0zJN7b7tnZ1XbHgF7jM6kLbz5gXezz8xw4t9slrEJ0zfqYgZ56 CwSpKmTwGC96rGKteLUDIYD0/J6FchKOGYI9VYJhy7Mpko+YIWRHh9a8Pl8+21y6+qqAyl 2OEIXsz3g/taIY7GCfQzNMWmkRG7uao= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431262; a=rsa-sha256; cv=none; b=xYGP5joLUXskYxJfNqjWXH8gaF4f1Epapm0xtDh22j8z2ZxP12Jkc+LqjWZC3Df/zdLSas /SdJbj7296RvnlgOrvAtxC3F/haPgneoQy3TFLvQE0dTn46knZrzz843OedK8qHICPNMR0 w4ckhhQ1iSfgnZUeiYvzYFxBry591Aw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D6CA3642F3; Thu, 18 May 2023 17:34:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7D7A8C433D2; Thu, 18 May 2023 17:34:17 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 03/15] mm/slab: Simplify create_kmalloc_cache() args and make it static Date: Thu, 18 May 2023 18:33:51 +0100 Message-Id: <20230518173403.1150549-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C5628140019 X-Rspam-User: X-Stat-Signature: shhyro89xe633k9p1bwt3h3hqmt7pit7 X-Rspamd-Server: rspam03 X-HE-Tag: 1684431262-153188 X-HE-Meta: U2FsdGVkX19u5Xfwf/75YGxQAaS7mAITIOk1TCkK9P2Fl2oMbCNi7J+WE9Y+KlTGhHzfUAtDDwZFYkzpUx5xa7HyI2ELlmhq/O0ewe7aaItsTTENMWEztRClAyCNyTTQmBjVt245ykPodTrumiW3gH3hhs0m3rFYWMipDKyCCpuhZZr6Xj3D+6zkzi7EuAaSmipwnPArVtNFBTloC4dzY/dwCPA8vb7znc6KyWSwtvIaRaELGHdAsvdNX3ZsHKvojdncAAhaJpa+7FEY599Q841z3MwS4gwZZ2fkb//k+gyiCMLSkr5q3loMYBnttWwZE3FrvvU+V1k/oJf8G1w1sgruy9CwU2tFmm/pkb7WUsOINzijVATkUDnc6O4DgHZrJ4dOtjheM9ZC2U+wfXl3uNwUO58VLiJcMosgg27Pfv4de4rkoukvpEM024OKAgKUZus9ao2roCWES4yskJa/NrKnGkxY7T76hAhNnai4mhtxA2lG2I6Km2djncnn2Dkc0VAz50rfroF6SUQZg7qRCRdOYbkeAzW5zNQuGR3Tu/7JS+EQxYy3Iy29HKQ57EYp7Rp+/V16hBltUgkYS9gA/EPuyZZKiFotBo8PbtCSzbDM6XVTpEK9jAqRxK/KP+lkOSk4fuq4vxXAWqf8cZVkZ+KpD4thbdSFF9fj936XoO2fVbwAdixkU0njMKhpIh5W5z0C16AtSe+d375EtOoNPv6usH1hyplD3jSkKtKROuOmksNZT3KSKNQfgQRYQn4O+vconYUiAqmgYlq9W7M0NdDMwJ0Q0Bcg1uSNEPOa4nShzTa4i3ZS8tMHejovtb17BfK3Mr32WIirS9WWbdgB1QDrvKFjsnPu78X5vPpeKDKqLrwv28rUY1w83ifw7GuibXpRVOOEYpuH/kGUrJyFM0wKZd9eaOlA3k5N/Wv+6OURVeeS82gwsuY/C23Ou96mwjmhIgG5naX3iZKpVVw +jcVSvDM mSkb09CpyOX4xJr9LzJVEbUId9EfqiQPDnJDPhkBdK9UjJgyAcdv59kQcVJUm6XROCF3xwNG3TtcNydSaDBTs+ADlxvSrWrcmOowyBOEGak1I0woyj3+VotKR6t3pIzj34xxNm8GB4X0AXit5ciEnc9aVfdKeISVaKjsMvAguDf3v+5lZ9EVh1zpzYKvSG6qAcP0WMcnVU/1nUNfqXpYmMlY1qfrIGhy4wv3LRRPXv6M5gXRIyfqotx478v1rZCMhh6lJ83pt29NL9PHGZNxC1H9Horvjn6cJq3HNab+QzgGREV++ZB8+3NR+8i63zn98RDLzsMwoIdBY0sHVco83+INnA4iITCahaBNRQJ5eo/StOgaQBRwSW4zbXRkKPcXNA3nIMSnO3eIxwL+wMzdltpMS53JuztrYPHZjj91J1tgBsMqh8kk4u4DkVaFUq/EbKtHZo3j6w33NRkWLMEdX9tavgqkvBVTUQ3ZBhKzusKD36FxDrNP+lVQB5LcrmmYexwwvUXG6IxLE+nUk7c2TzVZVxktDeJbGroIrfo9hnRaFSoKaUr97wbmv2hvG1b2QCHqy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the slab variant of kmem_cache_init(), call new_kmalloc_cache() instead of initialising the kmalloc_caches array directly. With this, create_kmalloc_cache() is now only called from new_kmalloc_cache() in the same file, so make it static. In addition, the useroffset argument is always 0 while usersize is the same as size. Remove them. Signed-off-by: Catalin Marinas Cc: Andrew Morton --- mm/slab.c | 6 +----- mm/slab.h | 5 ++--- mm/slab_common.c | 14 ++++++-------- 3 files changed, 9 insertions(+), 16 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index bb57f7fdbae1..b7817dcba63e 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1240,11 +1240,7 @@ void __init kmem_cache_init(void) * Initialize the caches that provide memory for the kmem_cache_node * structures first. Without this, further allocations will bug. */ - kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_info[INDEX_NODE].size, - ARCH_KMALLOC_FLAGS, 0, - kmalloc_info[INDEX_NODE].size); + new_kmalloc_cache(INDEX_NODE, KMALLOC_NORMAL, ARCH_KMALLOC_FLAGS); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); diff --git a/mm/slab.h b/mm/slab.h index f01ac256a8f5..592590fcddae 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -255,9 +255,8 @@ gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); -struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, - slab_flags_t flags, unsigned int useroffset, - unsigned int usersize); +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, + slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); diff --git a/mm/slab_common.c b/mm/slab_common.c index 607249785c07..7f069159aee2 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -658,17 +658,16 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->refcount = -1; /* Exempt from merging for now */ } -struct kmem_cache *__init create_kmalloc_cache(const char *name, - unsigned int size, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize) +static struct kmem_cache *__init create_kmalloc_cache(const char *name, + unsigned int size, + slab_flags_t flags) { struct kmem_cache *s = kmem_cache_zalloc(kmem_cache, GFP_NOWAIT); if (!s) panic("Out of memory when creating slab %s\n", name); - create_boot_cache(s, name, size, flags | SLAB_KMALLOC, useroffset, - usersize); + create_boot_cache(s, name, size, flags | SLAB_KMALLOC, 0, size); list_add(&s->list, &slab_caches); s->refcount = 1; return s; @@ -863,7 +862,7 @@ void __init setup_kmalloc_cache_index_table(void) } } -static void __init +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { @@ -880,8 +879,7 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) kmalloc_caches[type][idx] = create_kmalloc_cache( kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags, 0, - kmalloc_info[idx].size); + kmalloc_info[idx].size, flags); /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Thu May 18 17:33:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39C1FC7EE23 for ; Thu, 18 May 2023 17:34:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C97FD90000A; Thu, 18 May 2023 13:34:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C48F2900003; Thu, 18 May 2023 13:34:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0FE190000A; Thu, 18 May 2023 13:34:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A2663900003 for ; Thu, 18 May 2023 13:34:28 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 77823160227 for ; Thu, 18 May 2023 17:34:28 +0000 (UTC) X-FDA: 80804075016.24.DFAA24F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id CC99A14000B for ; Thu, 18 May 2023 17:34:26 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431266; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0mqLCBdpgkfqDUcqHwwIym+BakJfQ9CLI5FzmbFJWTk=; b=uoPXaQrtScjymt7TfHzvR1jnvdcKgp3oPHwBjNWjsL7Xz5ViBwxf3TlNJc6SbncabVAbkz j6PB0IJ6uuZONqxoOzK+B6f1L15Y3wVbqAtcAckn1sdY+1svilz0eUVijG7yLmfKr+KBtq Wbjwv/X4ts2Bt9CMmrdEshSNfCd025Q= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431266; a=rsa-sha256; cv=none; b=zv7Rnvl0yPAKMQTFhbCkINRVVzy0yWqEKfskEOfCM0I7IS7cOKvhpJx0C6chQ25WMh0QiZ DmsNZ4FdcIX7zjE6lb0PHwxnz8G5vaKnP95F37Do6lI0k87Ab/vJdGjLnvlOcKaquEIUK6 yGGOGGDTxyf9b2eoKoW0DOioQAyqW0Y= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E5B6565153; Thu, 18 May 2023 17:34:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AC1C9C4339B; Thu, 18 May 2023 17:34:21 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 04/15] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Thu, 18 May 2023 18:33:52 +0100 Message-Id: <20230518173403.1150549-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: CC99A14000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bzk5bkmgwi8dzpiu3ko8x8uutuy98fxi X-HE-Tag: 1684431266-742692 X-HE-Meta: U2FsdGVkX1808b38ZZSsPdP/L/s2cYc/gM/kDEtkpn93cKwQiIsCHW/+UwQy8E0aBpRuY0yYp9CT8LI0Gtfo+oXUXmbbvAktr8pouNaWlUPWFsl2MvFMWFJt0SU/je/vsotTXvyBa4D0U0wdNUP7GeRZIkuJjHpT5QcrC7XQkLdqjlEYSuasoehouVZOyxFp+gfI8LLEp77DXAENGE9QLQU3w1n7V17GQ5v8cs16CJoGysfuqUisp2e7or64YRPBDUAo0LItYb4nUb8FAxTelar5P/E01jqqGI+Uq86EcF0FhCXSj2Bgh2wQcj6sfjVXpCxllfXArSs2NeUQbd7pm8nAnCQHTcV93Cn2LKtyHmb00eb7PCJQFr3ePqYPmfrxJ1QnoXDxIsFVahIxwbwVqeL9dvjdxvL9zA1H6eq1nQqxkiM7j6TedDmNtJ1Pqh2QVetearAc7oK+Mm9JmeHMEzvMA4MTsp33PPsAV9uwC9VYurliLjJ23VuZhY1nKO/c3vtBEdxHTw6SqjW2UmLDV97sWhxE/akt5yFiDBojVwAjJeOHG4FT2P1F3tCxLNTh9BGETWZRk8uOF+2GsrNGB1T72riIAXHLO7IgUVRdT4Hn/9MD0UeWfWgzW+aqOeW24JJR4Z4FX+AiLVpdx8NZoTuDJ6ks4/vwTar/A280Qbrk3KqxhKUF095rg7u+xXCW1iTg407KWPgMxxRixXxIEX0+9XDjDggMy3DJSikF7ryTr5LXZn1njtL/qUm70kSjn/OaCwd/KK2pl10mNMVaAinu0MgVtsNaItYxMkBEGizeNZwqp/ofFvMFLh2IsUqnPeuE++7sNIRBa+JmkRtjuEZCbumOCcmLuS9aIh2t9YDFitMNdeBqg5oSQsMeXPF/uysaRno/2hCoX94ydPGIpgOiAAnDESV0500KWnHzdkmjkNM1GTJABvcooazJhCIEACU3et2hz7RGEVDNu32 ryeRCnSB pVErEUmWeGscYdfIV0BxQZfw3YXIr/7tNJP9dW6AaTnDqfn1sv64uXrSJIxZ3o9bSWVb9udEUXbbxUG7Swl9bIe7iN/BER5nEQS15yfvnGW8M2KsS5IDaVZmNMy74pv/UmHLWrrxPlE2qa+Cr36KLqbactdFQAJFQWzThHs1bsQgT/AxuAR6yJ79LlkzUB7xvOe11DGz3ZmxMrLXL7WWbYBm+WpseQgoIskrenKhAUnVnUiPAJeUUx8TafV6hYivDZdsqot53DgjRFBP1uK5TTpnTQDNnU8unucgYcveg6/ybJVRY4+/kNkpnBFTsPuis1vno3qglOp0AUkT/1HbYLXxxp8En+tX2WucdH+G3LGxDfsP4iJH4kJ92JK8li+yBKF46y+lqPs9bjFS3a+rSfA5fC2nRCEYBqoUJCPKlukVPdcpk/DznEPLNtZdMWa/QE9i2VmXoRKiXPgQDASs0PEmjwEj0P8pePMiPPGme2G6Cf1j5t3F/srPDrlXkBylwb8sf7zJcTfnFcxUGBQsfQcJtmf2xHazDBcDrFxcOB6p6BQrp9yspubKrA6u8EJ7bmL0V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Thu May 18 17:33:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81755C7EE23 for ; Thu, 18 May 2023 17:34:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 237B290000B; Thu, 18 May 2023 13:34:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E8B5900003; Thu, 18 May 2023 13:34:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B0F490000B; Thu, 18 May 2023 13:34:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id F08E2900003 for ; Thu, 18 May 2023 13:34:32 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A86A0140880 for ; Thu, 18 May 2023 17:34:32 +0000 (UTC) X-FDA: 80804075184.13.6810FD8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 01FB81C0012 for ; Thu, 18 May 2023 17:34:30 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431271; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dm+d7Y6ryj2dE1YxB8WadOx3Ilyh2lsHCQVm10XuuU4=; b=sZl1/ZejVvzzaVrkIbsap4bH1lPoQtjNeL/pBhQy7hnce0++ViRmtc4vyBiG0cGRibstTX okXHPpRzIL1ChCbfds9TW147XU2OPQ8Y+2hX9jbzASdpXcyJ2PfR2NS3vyGb7MijyGyc5n HmARAxb1nx4/kP9VL12WzuQpco6IWcE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431271; a=rsa-sha256; cv=none; b=Kr0hojksy8ApogI9qTiNWRaZYXy5BdhWXZY3FpxSG+Nwxr3+WJFDEQNBesKxViKKxCqYY5 hMnjJbr9iMLhcf0rjIrEmZKlSRmYrlfEB8yOgTtV6tDoBmT4y/oTO7303m8FKncBj3/BW2 upQwIf2M+4AzvklAVJeMyG+aL5om3I4= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 17FFC601D7; Thu, 18 May 2023 17:34:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7C5EC4339E; Thu, 18 May 2023 17:34:25 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 05/15] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Thu, 18 May 2023 18:33:53 +0100 Message-Id: <20230518173403.1150549-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 01FB81C0012 X-Rspam-User: X-Stat-Signature: wtnytgw4jsueno3ywucjc3fntzgu7sgh X-Rspamd-Server: rspam01 X-HE-Tag: 1684431270-920346 X-HE-Meta: U2FsdGVkX1/61YnNirnvGAuL09ypGEIH4w6Jewr49FQxr/WxBv2VZZ8uo1ZdkdYt3W2ZyeuJvbnSI0S4iwx+YyeRnbzC54MAldWaYST7uAsRM8WSHdX6NtieUZhNx6uCeRScwowAAph1/VaShjW55zMnA3liLyJ6O7T+OOzcjQGOqfPypyh6YcL2rK5ns3ICCYflhDSgW1T+EIU5bi/lf9KjlEkmR5mbSXDF2cJkYtxPPX3a5TvPPrdCODw3eBZxv1E+s4tF88zg8uO2ioRKr/++vqeWqZLGp/JP8WZLipBi7/s984dHoXIsRGUwDnF2ndMnqHhSebUHU8m+VPLJpgZvjW71KJKzqTtduFwBXt+VJZ18FnOONzeAZmHDOq3te6ctZxZfFxxw8alKWa/dbqrYIYcqKwbo2bBbpJJ6YuWiYMVsvvBQaitdSs7rlSxrh1xqmExtMWAtdMgwcmx8RfeZ5ZVEanmKpi5KEsgTdmtL23DU1cx9VCWrRt1PAVakNj63WcUa/pkScUJPJqzHNt6zJWj4tpZouG9GtBuo3ZwwJW7jpTCpcjDBVQQP7BuVwKisW9vRizWs2+HyQFvSBjxg3ZB3IljZVypjRLxqIuR6geNwpOXk1vA8ntdFjNTUFOthoE4NB8F/lTRIJ5RFawxmeFaVSogX01Gus9omCki+ue02LMJnt6nIDfYez6Fd86vuOOVkVtDmNKeb1obpeoAWPi4vcAHNjuI3T7Vqlw1mwRIy5lS+5BqXlAdJyCeMnxrVNgK59ViaTXC2eNQnL0A8N5rsVRuaM2tLj3hNgM7oTBiMkxKyss5BNX63SX3/FBN/5hmxRGLMRULCEA8Q/v9ETa7VmWVmcY+FlnGwAD1OShtp4T621bEszrvuSlJ7FVSjktZ5bgB9ncZmsxqflmtOiI2MhlCxWOEs+Rk/VkqMrjYJ9+5aH81fyCFHz0Ppw9Tk8uZm4x1BFPrutb0 9LAs+hAn LEgQl3saU9O3oTA2Df+0TshIsq7qQ2RxNObQ4PZA0iytedXCy2KHOdhTGMsEwTl7q1lGCE2eANq5gIGem87dHPyZ6MT0LdNEmvKDcli2/VkfIUQmkLK3Fyak4p63KltrPZtVJdf+o1MCj8hCJlTSdmD+zQd/heKgnSoo78bUWm9QkBoYD0tx43cUwpA/OsbBq24J7KdJcVRrQAOqwYBaJbYUXRvaMSNYr//SO27aF39ZIeRZegdlurBLkaIPdsgKplAnyl6DKenghsGUoWaT1nVd7lEl7rppodih3+TaVaPXRH6An7vCpFEmOXE6lC/rptIq7DWcnxobGsGc7ddBlizZp5CU6chHljg6c/49hz+/ShcBzRBi0h+24cayF3/IdE4txbMotpJk2Xw2i2JMjc8aF28Rs82Kg4g9cNNv/lovm3bEsp8hy9gVouFd8gttA8mqW7FlN1pOvrEAYOs9uc/dFeRPNTAyCcJyG0oolQJJexMcuf79pnwFrQQWWx4uq2C2IL0s9zhNNWF7FFiez8JsvMqAIYgwPzapO/Re2ptvkW+J6ng+25ICXF3tNw3wlqKK/aP92+Tf9coe255d73mtNNacdKuRZiJdIhv05KmBJwFB/k2Sy53y50A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Acked-by: Greg Kroah-Hartman --- drivers/base/devres.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index 5c998cfac335..3df0025d12aa 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -29,10 +29,10 @@ struct devres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct devres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; struct devres_group { From patchwork Thu May 18 17:33:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C943C7EE23 for ; Thu, 18 May 2023 17:34:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B1F990000C; Thu, 18 May 2023 13:34:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0613A900003; Thu, 18 May 2023 13:34:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E453090000C; Thu, 18 May 2023 13:34:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D6007900003 for ; Thu, 18 May 2023 13:34:36 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A26D0A093B for ; Thu, 18 May 2023 17:34:36 +0000 (UTC) X-FDA: 80804075352.11.551BDBB Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf13.hostedemail.com (Postfix) with ESMTP id 0190520002 for ; Thu, 18 May 2023 17:34:34 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf13.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431275; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S59+slMdeGHVr9gGv1wpxhrhYrhnmUIhONDI6UrLqhg=; b=3Zfg7nSZ02RjenyT7TPvDWhMeYiOooKFIqWNAGFsoTZ46yZhoA8y/aynNSfYRgCpUe8rgv VM5BhLX6+Vn0ZMIsg6wP/rO9XEb2dAOEBeZzuE4ooVLxfHXr8p1544Rf8cntjpVR4sCPCT 8Gx9CiMJBb41ycikuohypbBjQ3W1a0E= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf13.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431275; a=rsa-sha256; cv=none; b=FeMCjUDQTuAGHJgf41i5SI9r2vfcRuU/xRnd+LUQos7pFn8I4jZNR2qAGfb/ydrxA1PvOF x+ITh0Cd+HIjmlkuTR3BDkNzboOfeq4xkVydHLKc3mTkxlC9X4geguKB0z79SJEm2xASpy Jk0WkmIj8KDLMk1elz0jxNvFlk6crPI= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1C9D665155; Thu, 18 May 2023 17:34:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D58C5C433B3; Thu, 18 May 2023 17:34:29 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 06/15] drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Thu, 18 May 2023 18:33:54 +0100 Message-Id: <20230518173403.1150549-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0190520002 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: hifqobd5yns8fty3wap7w3o816eus9bt X-HE-Tag: 1684431274-890578 X-HE-Meta: U2FsdGVkX1/koQXqJL70nCie9t0wdxjHOBfiAb4t44RDhVrDSRNaTKEzGszCY7dVF5l/6vMclWjWZG5zs2ArbwdS6AfZWGElyPGtsE1FSM2sf6P9hTnRAcOFaX9zoI7CjDbl8Z9x7it5myqUWSAljyseYZIlctC++3UzOvBt7PGWs+jRt/nIEHC+1H50FQf2Vw5mI6K+WSZo6OIEH9u1s4kRI4Ck10P8fYJYS0CZOw/Oepfn4AQw9PpCSZf3k9utZ1zqoTrk/W9Ac3uyOR79zFMCjiH9M0ZgIbVDsX5GszDoqMiPWR2FVeacJnpcBaUA2S0lPuwaAHxAotv7d1z48W/LKYF5XJ4bJQa0mB261GtyduQBaDK2R7Lf9SzSi0UsYc4j8794aPRmKLPkfH+QuFR3SJvh5lraSZ2MO5ugGUVQMfUUu6YHLWnhoWY+KuFiu7+rm4p6aVQPTAq8pLFczthNidjZizx0Gve/nz7YCXRRwenaZX2RwFpe1rWkbTPH+a8UEW5FObH0rfFb8yqDA56XA8yNvBqc9oV/hYk4bvoKUrAOqEiajYq8TohS+1WchiptWgzwAOmv/aVlBTrPk8OpkbM9CIXTLeb+meUhW2/MHjj/DEAUt/4t29/gJIdoJT+jDKWCJr9+zLAsPJD0pFBEuCsEb3sCwMQd//FyDIe/Ki0LI0oIJENmdAzcF1owHnyh6GMqtuMEDcEiI72GXgGaCB3FtEvUddEFQXNqrhL9IAXYK20i6isv+jzBORgY4Bm7tLOOgIF5MJCL1KAPTaUsbJFZxXZopwF8cVYliAbLnm5/mmXD1EWKlbRX3Vm2EUQn/JMtcCqqs5liddJMoxZaF4pa2cMo7w7NSIDHrIkQVlPAGXMXFtpqous8eppisqndQhFXGRKTA0Wgn8QGD5GcU3oxLfPn2+JOkOO2aez6RI0k7PElvUaxlm2gcDM0fgb0J4Hb/sfqp7L310f 5lHsRC2W FLFRtibUa/EBvrAtC6AWnBXNDLYOx2v3zpji6J2g6SIdY2u9yWm/rsjo8fwxuC6mnGEglu6ctsPgtUla0LpLafrg3eYohj5zPE/Knv+JX2wlkX8Pthpn4n0szIPZo+iOTfvIs/fyLhjR4qjI39LFlfsbcN5srmOqX6vKxitZLdrFiZN7v3jsNUgCp+dj31fKbCTleq0FCn//Vh0PRJO/ANZw8c8q4OLOy/uPCqTDqxiCPR3LocSaDL8FkVkAk1LesgRSADwtqykft+YPZKBwhkBEftrzPbnGtl7wG6c6cjbaT9qo3YYrFxIWEMW/bOVSJEdn2EGQ2s8XyvhXJaRnl4dhDfDM2X8AEJpTUiz7ffwZ6nO+MJKiLJ32CXjogyWc8dZk3LUcMPcpmCP0A5U6yxixxrMEPg2PMN99CBVRJ9xmfpQuHM6eY5TjD1VnpExsNuE/7OcHRtL4khXv8xjwcvWQKtE92MufK6phk+r5SnCoxEbDGQR6wpDektoxFVQ/FfdUU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Daniel Vetter --- drivers/gpu/drm/drm_managed.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_managed.c b/drivers/gpu/drm/drm_managed.c index 4cf214de50c4..3a5802f60e65 100644 --- a/drivers/gpu/drm/drm_managed.c +++ b/drivers/gpu/drm/drm_managed.c @@ -49,10 +49,10 @@ struct drmres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct drmres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; static void free_dr(struct drmres *dr) From patchwork Thu May 18 17:33:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52ACBC77B7D for ; Thu, 18 May 2023 17:34:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C717590000D; Thu, 18 May 2023 13:34:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2196900003; Thu, 18 May 2023 13:34:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE89390000D; Thu, 18 May 2023 13:34:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9E303900003 for ; Thu, 18 May 2023 13:34:40 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5E87FA0938 for ; Thu, 18 May 2023 17:34:40 +0000 (UTC) X-FDA: 80804075520.03.7959404 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id B0C4C1C0012 for ; Thu, 18 May 2023 17:34:38 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431278; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rq19jP8KoV6cfikLmjA5jQ41EC5pXyhnoRUI2FjO7tU=; b=K4OS6SIKdYqrTmyGcPNNJrwpjujvZwBn5ecmDZbjDPdGkg628CoZj5OwM/bTqwkvvFdPGp inOUJUXn4MUSQAISNLFdyi6IyxadAFch7kSW54jHB29hptYO2ZXKIhYLia3dPEQFWJntuY jwCixEg7mc/clrGV9Cc18jZG4Fm+dI0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431278; a=rsa-sha256; cv=none; b=xZP9I+2Ej9/EPmmrmZeldE6O6XjLD/LJ4Qfym1m+xBGnb39GmXwBpm7GSFyUonqr6mQirv FHyxx1sBcnJBXMH2M6Bwgg2dPbIomAsce8zkY/CBo+xwvGAWVz3F6Zygqpr/89LwkryPFN 5oc0irc5XNpkyWJ+LpOgugYNhz86L2M= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D1B336515A; Thu, 18 May 2023 17:34:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DDC7AC433EF; Thu, 18 May 2023 17:34:33 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 07/15] drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Thu, 18 May 2023 18:33:55 +0100 Message-Id: <20230518173403.1150549-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: B0C4C1C0012 X-Rspam-User: X-Stat-Signature: odzgayo48cyz3gtcwzr8kharyj3rykuy X-Rspamd-Server: rspam03 X-HE-Tag: 1684431278-999077 X-HE-Meta: U2FsdGVkX185Uy7UyXm69mMzyJ3eqncP+qmzLjBAAt3w5OMe5MZA9ApVi0Nfwie5iSLvXQFTXEeDt2g9c1GWAaKjX04SmTMU7E2P3lLLh0e1Fuw5vZ8CkFCelJeJYFhvIB6+Hh6UQ6KAalEVSpuvtKM91pSWxWZ0RyIPia0yhi4eeP2Ct02Av1gflNQno4iKPgobLCrWb02CWPrGApX1HPxqLvc93aNpa1czvVjdYOOyQksh658JKzhzWEAhOQaLRgBOWZ4jcEIQiuTnlcsZ1B681hNqRHSNY/07zNmsKhAuSGUCXTi8cdvYL3eTj4MADW9zxL/ytMcOY5989KVBazZeMFYO8LORcuxEKjP0BM7TjSO4yzucvOHAEeVQX8/4Yd65DSbOuuwBh44igX8w89g0fsqnAZEy7vvydyvcjAmlPqX6iUTFJdKvpXCPNvCVTQD91KHT1so5pyTdJ+oSHdJM+uIf0eRgdw/GrRwd+YldWkT1QgSgVcDNNMruZcef7BMqRYWZO5t6oLCOKSSJAfc9knUh8Hq1YmmATtLDCV2F4gWCVD5QebCpD+DZRfLEesMT2edeNZCSaWnyuDp3xao9TgATALzQXsRtYRR9cweLdiTr3+f6PaZzFnb5hxGXye9jUU/7oIzPK27bHpvEOK0iGo+kG0wJFAs4SKykd32C0MRuDqOUbx7FXMtKjQy5/Lc05IQEgX8TY20KL0n/dY2w1dqc3JDfvnjH5Dt/VLzN66EsYmsk8dzkBFMts+TjMBEvz4yNA0Pv14Sga4+e9s6QfPZqzEKal4CMBe6QjxTahahWlUH9xkUWbaV4l7hbQN7oYRAcIS/FD9WFDaeAfTdUDqr8fAR/zuQjV/FFhzgGbXUhfygNjWyao1zCBJqNHQ27Rli49ZUeuB+x1JPBchok6Az46ThxyqW3c3Byk1Z4MvVFu2SRcA2xuGBa5A9D6i/pFuFP2yQ77Jlf/oL 2N3exXBG urji6TM9Zvs16Ha+Yr+Z7NbTFFS+poIzn/eoB+pr6f+OVYnUCh4YT+yvXaVd4Gdo/jDQLevUku0QGX2HKl7CnQxMMU4CG6u2GzUlmAkXHq6PM3eY+kAEdh6DkItTAxEfEZUu2YMLR2tRe84laMC3+4K0pTb8Vck06pmUxJbtlBYXl968MBa7zGXCykPc9D8kDB5O1fuyw/RyJAXZkPDKNtbOyWDJkk2rGbYDyAhThyUEbYGitikitgS45UTkLN76zAO2oYP0f2Xq+h2h6qDpROJ10VT8Rzup4ilCvEtyKxxIvLjJRWH7mdJsATSClZhN+igN5Lr9r6uoW0Nf3N5YQrLCqHipxc94FlGXGjvMi1RxTtEVysJNyPRXvjKS6DoOC+QAgIdkadXGG+RK5MKwhleVVwEIsNdOq1fefGbIh+duFVUaqUUhMCW5ZHIqfED286LRQl+SGttnpqlqNFEP+oGqkFBEp7QGA7uDEq2S4SMQAbjTSnD4ckl61YGDDdtW4DMpWwLgm8CvyZreuXvC14rQuzrRE94u9jjQoJh0j6Bk3rTOByRqOx4xDtAO5yAGMulKm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Greg Kroah-Hartman Acked-by: Greg Kroah-Hartman --- drivers/usb/core/buffer.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c index fbb087b728dc..e21d8d106977 100644 --- a/drivers/usb/core/buffer.c +++ b/drivers/usb/core/buffer.c @@ -34,13 +34,13 @@ void __init usb_init_pool_max(void) { /* * The pool_max values must never be smaller than - * ARCH_KMALLOC_MINALIGN. + * ARCH_DMA_MINALIGN. */ - if (ARCH_KMALLOC_MINALIGN <= 32) + if (ARCH_DMA_MINALIGN <= 32) ; /* Original value is okay */ - else if (ARCH_KMALLOC_MINALIGN <= 64) + else if (ARCH_DMA_MINALIGN <= 64) pool_max[0] = 64; - else if (ARCH_KMALLOC_MINALIGN <= 128) + else if (ARCH_DMA_MINALIGN <= 128) pool_max[0] = 0; /* Don't use this pool */ else BUILD_BUG(); /* We don't allow this */ From patchwork Thu May 18 17:33:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2BE9C7EE23 for ; Thu, 18 May 2023 17:34:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F48790000E; Thu, 18 May 2023 13:34:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 67BE2900003; Thu, 18 May 2023 13:34:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5438790000E; Thu, 18 May 2023 13:34:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 46ED6900003 for ; Thu, 18 May 2023 13:34:45 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 254331608AB for ; Thu, 18 May 2023 17:34:45 +0000 (UTC) X-FDA: 80804075730.14.1964A17 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id 70D7A1C0007 for ; Thu, 18 May 2023 17:34:43 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431283; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+fgUtZ7z2bEcsaerci/jffnaSF7K1ZtF0r/2TxWeNAQ=; b=aXWqQflSpxKMyMXkzwyUSiddVEbQigcrb1eIZ0nvOuRExsc6bBJtCn55yEcNop8xJW5OIz /1xo1ine1l3Gxtdcx4fqQBk0/p6s+tz5gHyV4zGVe+f224FwUYsHkKK9kTj0zuG+qsg+RZ ODS2fba/0tNc/zIHyXnMCZkl4zYAqR8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431283; a=rsa-sha256; cv=none; b=u26a+PflVI4Sipi1bfwsiQLL05QisTyZAjOv/D0qT/A1N+i99CPQ9Yoxt9ItoUXg3gzOqR 808OnG+MUGTlv6Nr4Z09e7BRuTHCC9v7o22wiICvG/QbQxF+WtoiPvEpm4f0atWN+rVJ1N GX93BS/m2BriblhrVqhNuv1wuMwRDfA= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4B80F65146; Thu, 18 May 2023 17:34:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10313C4339B; Thu, 18 May 2023 17:34:37 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 08/15] drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Thu, 18 May 2023 18:33:56 +0100 Message-Id: <20230518173403.1150549-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 70D7A1C0007 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: zuh85tc67qduepr66e8ttpxsr19cptbn X-HE-Tag: 1684431283-692471 X-HE-Meta: U2FsdGVkX1+OPM6e9czvbuNCJK64n5Zv0dj/7wtYloIq9Hs/0LWntvRzKEK/OedcllCxFz5QDfk7Mufcxvu0gikZdlyP7xPpXBs+lm4NeZB7nKM+ZFM00qm+YhlW8nSxEot75IRxRSQ/BGxcN/IzNs2Duw+qfc7yDBXIkCiK89zw1ZyExwnD8k4eBq4jAwOD6Kim2QDPio2WFXLTMDJuefmgw51SlVVPUGafptobb0usKbOiCbZghSauxis/oC1StCKl0JV19a4gpmJF3FYdXK4+ncdNS8ZfnF52BpEVyIQr01m1l8FX9oZXIZzicO1pdwvWhgbo29Eatl+JKJzdbYPC7INqyoHz2RRCbhPspJc3Xg5IxS1DmRsUuVBy7WFttaWTQn+8ehJ/NxQcr6dvjWOoc7aOox2yBIwYgFlByRW2DZglT0NQZ3XWnN28dAAiku9GhybcTFQuzGRVnOPsdA6q+aIdwDVLvJB53jZxR+2ZBO0Zt0I8sl3tCrTABZ91Wyp5Sywz2ziYLvwInnL0CJEAioSA4nwgebypfuDhMzyraNM5t/Kjui3Mcb05SkJ6c7EGU/B6gdxfx6vK5I4f52YV1czwLU8Go9meUjfZCHMt23DbgCHXRQ3QW1kKGnEiNUVhEEuTVO7P0TI4eR7Vvv6wrt/WM6kup8ZbccmXY0c5l4Mru+gihudk7iVuK8IVh2l1DoEgmpZE/8+ZUyjEHEdyDvR9t9HB1ARbMZE/SqvdvxdgJMBUkF2oDDZ/1PGlbIHeabsfk4umIpm3pizUZEtTyEX2bdfW/3I+G8vi4IX3SZ1xaAivgTd7xWZiHdcVLEIo5UYY4EzcP+G2uc2xo4beXd8st/b3PMuFoFp8ygxZPTtG9ueUN7GwnUdLvWvj3qqbe/M5P/OTCpjNxPTRorNOwqRzZ/9DdzBTcIYDOCwvOHbOs7jKmmqotgzWsg2hSI1XWL8tNAiPuhTHT+G qaaJgm7Q OdJexyHPfE1YnsqjJ+2Y35EThYMRcrXyOSxVwaoIhU/U+8usUPwYjQirQ+ZPIq08wnlevNSs+0Z5AsAHPbfwb5GIr0NuM0Qc/Ui1Qv5GSJlQnRd9hNb4CtoZn49NePYfQW+zmLbR17zG+NgowfY2cD0DOLDNKEqbex6dcIknkkubhFDgcQgCqHqzlIFMRKIf5ynbjgLdO8PqLSRgKxKG1zXrPtstK6wSbYC0vJ0We/GomarwBEau8YBUhpTGwuv8zhw+xPljCrW/uwFsLDk/iusyENt+/iYSy4+bBJ2JXn2rEQrHsQZiBZkrwHUQmv6RVvmEFKR5nOzQiTf38s2LrYD7h8O57ikJ5k/SNezfM214UDUMYMqwtPmlPqICjndL4KOu8p9VnCYF18ssZmcA6cI7M4s+agxREQF/XoTc++vSixaaV3r7Ay0DkxED1nGhWNBUNbQ5D4RiRO61O8V/8cx+Ysc6K58/3JJnI5DKfvS2vszf3tWvo+D6cVagPDC40q8f9fn8GtPRSKYAWGM7qGWRn9yaDlyupChTf3yphkCmf7KAj6NVwKw0zqpuKuXEfVMUm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Mark Brown --- drivers/spi/spidev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c index 39d94c850839..8d009275a59d 100644 --- a/drivers/spi/spidev.c +++ b/drivers/spi/spidev.c @@ -237,7 +237,7 @@ static int spidev_message(struct spidev_data *spidev, /* Ensure that also following allocations from rx_buf/tx_buf will meet * DMA alignment requirements. */ - unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_KMALLOC_MINALIGN); + unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_DMA_MINALIGN); k_tmp->len = u_tmp->len; From patchwork Thu May 18 17:33:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACE55C7EE23 for ; Thu, 18 May 2023 17:34:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F6D190000F; Thu, 18 May 2023 13:34:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A6C0900003; Thu, 18 May 2023 13:34:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36EC190000F; Thu, 18 May 2023 13:34:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 29D61900003 for ; Thu, 18 May 2023 13:34:49 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E922B1608AB for ; Thu, 18 May 2023 17:34:48 +0000 (UTC) X-FDA: 80804075856.08.D183140 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf10.hostedemail.com (Postfix) with ESMTP id 57C74C0012 for ; Thu, 18 May 2023 17:34:47 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf10.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431287; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P3GKCRqSal+FoaPUZC73tQFjQIUPiL6KpjJp1J/vlzY=; b=51AuFCaMasWNjQ3r/m3f4dZjZUvxVbHWG/kaeVfsLC4AF91RslAqy6p7rLPoXXFLg7RVT7 F1IkI4BZykwmpBTA2XwNcnX388V826NXxKG0QFrVmLrxsw5VDAxUvxMI2QvqgqVWYe3zKh WcS0vhFXaTBn1uH7bS1jPoYfZUrIErc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf10.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431287; a=rsa-sha256; cv=none; b=UOkn4SUUT052yrHfKzd6CXkT9kj7SQjgbiuPKVdvGIYThpPoWW33L3uGU+qroDSLUukFGn GPXc0pPcCJZbFC1QMvjP42PvPOv2zM0F+iR+vV9z0QfSgZETSfnXXyVjeT3CgL3HGVNdKP uZRv3pZ/rFDq+TuFpE8BIPI8Y7tkbHQ= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5524C65159; Thu, 18 May 2023 17:34:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1CF81C4339C; Thu, 18 May 2023 17:34:41 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 09/15] drivers/md: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Thu, 18 May 2023 18:33:57 +0100 Message-Id: <20230518173403.1150549-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 57C74C0012 X-Stat-Signature: 3p48ukdtdrio8qsnign8gsyf4mpewt9q X-Rspam-User: X-HE-Tag: 1684431287-6643 X-HE-Meta: U2FsdGVkX1/nVzQaxRFL9tVJH3jRf9WQlbpvuqgA/ytlsylspv4f1pz7Kv0lfLmknCq+9WOlJ9/fAgvel7iNOSeeoqed8gHj9l2FECJUIgPs9u5F9Fgok4NvYNb7Ji3Usu2jzjvRMj2RegRDOEy1mI32jdATysY7h57AOovuwrtP3WrHwfVANjngAl3M4/KTjXxenlkX45TOasZmzj9rU+E1MhS8KleHC3iymYgU4PdSK1v/uwQwNAxp7xiv/SLCsHP1M4LihSkInYy+DXOBSVU3WK8wnlnOqVvvcmmTjB4JmqbNIGhiMoJxKt3YipAjVhUSsp3iENL6lxgy83RLfBdjLlon1yk19NN0ua4ggWLGt3BWp+yq1h4hjY/zz5tu1X7ZYUMyfVO5cfVyxooJ0b+MDKbMRCqdzOOkNA6jfyR30WVZYn0MDhrh3xObtjY1gw74s+D7WPsYMSVuhV2XUvVQwAfQAxJRdr3m1BjC8Jc3Ga+p3h8j0ZeAUruSUda/HNoifIGBHg01oy8VvIwZp+0mpl9CLH+eGnuT2+DMi0u7XQHeIy06FDxqJ+8Jl3VHThsWMusLRioQkttvAdVqh0M4qNW7ThCGPxHQckhf11ixVlOttwTFn4/X8/3Cukpf2SXmbcDtproq4jXxgx7Il/b5KKEwTPQR21e/mzvBkarqeFvWldsPh/v/jb1yW6grYG/o99zHo8AbuFVd75vHCsYY+KeKc6f5uzZfjQgkO5mDKaMnbgeWGJ4vxVhAuPsYkVWvg/Fbctq/0RPv/voqEfK/FTMKuLyA8yC913DKlrxzWI3FFc6g2iB/gLDrZjBxE3Wt4XiWk6zlGxIX4C+3K07e4Cbyvi3RKizq2jaGjnmcRTtZSmAhrZPnAI3IEzJCQG+ArfXN9U00kSdClShIOBeS0X2GK36crsCo8j+WtzXOPOSqwpLc2QDX+dtMqZBH3VD4ty8DgFsnXrxf0un aEX8pZ4A 3sasgogwzuy+j+sujHJSU/cmG4GCAa/glR2Ix4tdc6pvmdnCpk8OrRLK2vezc8C+OkBcr7gmp/6+EQoi9SFztA2uwN9uIHHm9Xi9fllPKKYAITvx7iM6RgsZ7P7TqVjLB2i0tfgDyafhjKRru/VqLJYKzVBpTlbHHtYDnZc1KZcNFXjKbx886aGca54pFGzfa3T591418ZUppbMi5fa12WETJjqQJJa+eaLWIYHYbZGAIA5kyOGv6sos67bwz/FYknbypklxh+jo92m/wjYZXP6t/Fk9aH7g0GvS3oU0mDCQQTR2NWK984z90sfMXdgJWcBfrjpXZVM4yz6QkXvIVrPWheicxZxpUCx7138G31TBmntFhGtE82DgD2GvIQrIDBYDgLLs6gBf7CJJvJ/eNXl7o15pUIfNrXYFz+WyKyuSSfDYwaB7KkKFRSewJi7B8W4ettARQBh6H95YNzNCIzucNzWITL1JPpqo+RCigCf3KUqrz/Ta0drEwuxEx/Rw5f8EGLAte/eis+f2JKIcGs1HvdyHbo8gAAo7zO2hBGzmQyo2wzez3eTSCXHQ1tfMPEzzZ9le4hzHpxCgbAuRZ+RblLQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Alasdair Kergon Cc: Mike Snitzer --- drivers/md/dm-crypt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 8b47b913ee83..ebbd8f7db880 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -3256,7 +3256,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) cc->per_bio_data_size = ti->per_io_data_size = ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, - ARCH_KMALLOC_MINALIGN); + ARCH_DMA_MINALIGN); ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); if (ret) { From patchwork Thu May 18 17:33:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6A3AC77B7D for ; Thu, 18 May 2023 17:34:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8EF26900010; Thu, 18 May 2023 13:34:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89F1B900003; Thu, 18 May 2023 13:34:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76758900010; Thu, 18 May 2023 13:34:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 66D88900003 for ; Thu, 18 May 2023 13:34:53 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2A313408D6 for ; Thu, 18 May 2023 17:34:53 +0000 (UTC) X-FDA: 80804076066.02.186F1B4 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf02.hostedemail.com (Postfix) with ESMTP id 696A68000F for ; Thu, 18 May 2023 17:34:51 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf02.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431291; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gh+3BJM2SL1SQbFVnuGwYNETki1VdSuu83rEIeI6iP4=; b=u2idiKCpOtPlmpVKigiy5GBc7COEyJqKCP3CDMgLx/p2Ip3DKJu+76ccHxiU6N3qprhFbF iuO8CKhp8XSGVcxiEZfHV0heHqNOMv/cHVU/ycWuOrxVPd8v9u3Qp+H5F+W16ZWZiAri6j zlW0rjHg/pqD+88wav9AaBD37lW70IY= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf02.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431291; a=rsa-sha256; cv=none; b=66IKTgzjjnefhy4RoGMwGSCcBWOTNTJK5x9ZZ4N5G7ErtGp8+YZXF5zaqCWWVFjJFYI11u ZBCjUaGEZUA6V2t2yCpdPQNp2pasUeBpo6GJaJvYOHxRCZZ1/hUI6jZBz5YeqIekxbYKin WXg7hPg+zHnp9JuMpjLoAmc7BxdCjuE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 67D686515C; Thu, 18 May 2023 17:34:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2686AC433AC; Thu, 18 May 2023 17:34:46 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 10/15] arm64: Allow kmalloc() caches aligned to the smaller cache_line_size() Date: Thu, 18 May 2023 18:33:58 +0100 Message-Id: <20230518173403.1150549-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 696A68000F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7r8nsidw1ypcrxnxsrcakirkrie3tuq7 X-HE-Tag: 1684431291-978642 X-HE-Meta: U2FsdGVkX1/yTD+xD1gS35+KbQ/V/zB+l+nzDIJ61JabnMTpF4fby1rU1Mihs8QbyczhKjL2p7YLZ2jvPdT3FaDQGySaKULxRNDIY+hZpabeLSgvoWJfKo3LwVbRthPXaqCEEOjAvo7XxPROsc5nZB96wwq8iz0PUOSseggTJoWwrcMYQYa98fR2NfZk1AgYkosnrL3WrSZ9B4LcYT7TzmP3ofdW8VVSTAsx+jstcorNKaI5/afmptGNWKan2vwUZ6RAGdvgPlye7IhbSuZhhYVJ3xHOiNTSwW2BOCiKZ/SqLtlK9t7pP3fE5a8gV3YU7L53ySnH5GYeS/txRQRAbb1JzS1LxVeAde9cSYQBSsPM4JKz2Z+d2RC+R9nNC1vfOe2u7n/wLK4xAYH4d/BHjjzs9wDEPHTTBPFwb9KmyaOfAK4mVEiqOpTQ0vig2BDWTszalPJjvNNd9Z8jXq8fR5RUflCGviPVAGWNdj+2I5Sm57j33luWkO3r8ta4u09o9MAHYujHn8bOXqIWM5Q24qMAHLJ4aS6w/8+cEQQviCDn6rLTXbsAHCyzj7m++6v8VPONdZaSTkcjEXMSNIsgQHm2M9lNKANq17I2Ng1kSWjzCfN3MiJovR2HIelREjOV6XC6Up6fFb9knxoU83rYeEpaPk/G76v8mIH1Akdum5BfgltoEhVfwrKpaOpGEB4y7ttTDaLuDFZCquV33CO2I3rkGMDXVbdAh78XXbzsYs3Y4m6mt9In5Z4DTG5DCbudnm9Jrx8bIABGs+WFF+/BJe9WwLnMOSpIKonIYlhJvot09VkdLnU4CfZYBd8ZKJL6ONVsiMl+JsRY9jkrTZSnCeE3NpKkUe5f8T+fiuwkLZ/auTMz1M4GvT6P/tU+M0Jj10pec4o/0q1jjQztLAQ3yvLW0LibxH/jKEFQz2np1eVnZmDghdR+DYXgm2QMaCJebDb62ZKTj3/37Qiuy2y 5EPTntJC vupsuNFCEzmxC7u9jAmcZMoJgU5vxSqF0RoDczolY+L47YhgDHYLXpsloH5AW1kFWpZE+f1WRq6eFp80y/Oup8mu5qM9fBxQuU2iUQIemHm3Qt1YkR91Ln5DXa7/DkRct1oYsDC49TEnFjiGmT2XfZXLSeJaVKNMozoUkhg/hfDXRZyLPVkYJs4SqNaNytdoHgj+1c3XwQ3dDdKXlJuuHzoCEHf5Ob3XW7ApUttrZui8yWgxABD3guLVtFE857TX2JEj2YcCooJoY0Ev7BEnDM6x9tiwQp7W0GTnwxPAT5nPt/7k3gX/C/R38OLKrZlI0kR82WeUir6acpwNuZqfYXlTILoJ4fVkC4eazebjzYTYwcAx7ULLQjtFflQJ26Q2W3HNusKgX/J3SdeBJ09XCp9gEDLpqJaWdf7Gs8JqNh3/T41IHQhO06UGmsEEa5zcDBkmoJgNK3A5NK07o2tqCmb5FQfEwIfGL+1AKVObhRMO2sOftDK4zYmXF2dXYeBC7fuGQ/D80s5Xs6RhfWpruH26IX8vGYtatSlZDWJr91W1qvomEXTGVFXHFOmTHdCLCU0k4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is 128, larger than the cache line size on most of the current platforms (typically 64). Select ARCH_DMA_CACHE_LINE_SIZE and define ARCH_KMALLOC_MINALIGN to 8 (the default for architectures without their own ARCH_DMA_MINALIGN). The kmalloc() caches will be limited to the run-time value of cache_line_size(). Typically, this will allow the additional kmalloc-{64,192} caches on most arm64 platforms. Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/cache.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b1201d25a8a4..d11340b41703 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -23,6 +23,7 @@ config ARM64 select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE + select ARCH_HAS_DMA_CACHE_LINE_SIZE select ARCH_HAS_DMA_PREP_COHERENT select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI select ARCH_HAS_FAST_MULTIPLIER diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h index a51e6e8f3171..e24c10192636 100644 --- a/arch/arm64/include/asm/cache.h +++ b/arch/arm64/include/asm/cache.h @@ -33,6 +33,7 @@ * the CPU. */ #define ARCH_DMA_MINALIGN (128) +#define ARCH_KMALLOC_MINALIGN (8) #ifndef __ASSEMBLY__ From patchwork Thu May 18 17:33:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6ED6C7EE23 for ; Thu, 18 May 2023 17:34:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E1A3900011; Thu, 18 May 2023 13:34:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 591CC900003; Thu, 18 May 2023 13:34:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 45901900011; Thu, 18 May 2023 13:34:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3608D900003 for ; Thu, 18 May 2023 13:34:57 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 06739140271 for ; Thu, 18 May 2023 17:34:57 +0000 (UTC) X-FDA: 80804076234.29.0192D7C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id 48D4340013 for ; Thu, 18 May 2023 17:34:55 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431295; a=rsa-sha256; cv=none; b=vJ40EeYksvC/y+Y0dqoSoJtwpCrS20EKew7zoaBTUREKF9mN28q1yZ0a3q7LhNg6fIQ4v9 BbWcZ6h1pXTWzCiQXDExj4q/zqmf+KBX4VURlEfmRVPhgusyhd6TT8LpNVoLL1roZukGvG JWLvjk+eHGoKdIkPdlIrXRZ5DaHgMN4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z/m1w3e8Po7xonc6XvWY7SoVtTJgRC1UKGOvdrhNCrg=; b=TABABr3Mg41JdWenY+7/jNzWOlI7rrOXEL78KDX0c2sXFEmhrxkz9MhbmMcC3+MxWkF5uZ iyzDDC7OTtjQGIXieMuOODkl9I5muCE00U6wQaOmPC+kSOcOQOT52OCRIuTWPTDJFH76Ve TliUV4U2atX3Cnp6/7uoS9RYhovtBHA= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 71F7665097; Thu, 18 May 2023 17:34:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39C46C433EF; Thu, 18 May 2023 17:34:50 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 11/15] scatterlist: Add dedicated config for DMA flags Date: Thu, 18 May 2023 18:33:59 +0100 Message-Id: <20230518173403.1150549-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 48D4340013 X-Stat-Signature: gst3kqoz4umboae4upqoejiqgntorm1c X-Rspam-User: X-HE-Tag: 1684431295-622609 X-HE-Meta: U2FsdGVkX19QXEqp8F8cFeDCTMzEgdElm/06G00OvUpR60kYPkhBY2vLUNRnTGs+FC5DTiGM/nUK5sEFfM918hmPziVI9AQPFnALMwNLaFKiNC8hXt2+b2cly3YQgRwNpN2TexKy96QxTB2sxzfx0v3aZOlNW6cGeNZv3wtcbynPWJ/yTrs7b63bOkuKHGPqnZ2sf1laYdjbeJfVlQMtyuN23ror2IYdw8vAD7Gml/TRz9lPbm/bPrwybyGoonbIzMhaTtlwBDufgS5QqqnNz33vYDnGyTSrP1cRbf0SQGmbl5HspcVypQwfZiWV4S9hSMp9L8iC4QT1mTh1MglyA0vpjrrfdJuWyICeLSFgWyWuaj+NmBsRMWB4ajrYxxAX/diDR0UXdOXSoSDNyvSw6dIaQlzQdvH5+pnoMh+3jcayx8hE25kGv7bQ19H8tAJ5+ysuSPP+ApFayc6MBYlaXMroolekOsVVidzV7gpKVPfGbfBlTVnZwuEQd6pLmYTO+Kgv13JI1DXVlJcDs8ObHJVlmtXGFi570X7VT7SYfZVw2XBiOnKTz1flokAjU8U26hsI80jeo4nuHoIzZnzcQl3Mj9qY6NJu+U+SSVAgXOyshboyWQ6/yk+xoB+IUlMuQpTiqnI6wYlPD6euLZHYWymMceq1e4DL8Z+1FhyUKcivVfWW8is1FwKVmsFWkGJOtN8GBrkGazwTruF0/wrWqLxI0PzBI2+BxJwwlnyUIcxwygvHdozWD9ZurwVjLQktWMnGyDV0DJZkWDBMefssZNsBC70gbrgGcgahXuEqgFuNq7OgnT3rjh0iAo+CgzT3lHPPMaFbkq/gOGUPIe4nTWakgJb2l0YexZnoxrZDmW/2zd/1DwGfI7dwT6zpbcSaA5U3ROMJsRZzMl1WYLSUkEtWLjE5HqUo6zYJVMrVjetH9k4tR1a/DU8Yf9jYM90ABzKOC3v0PwtOlizRU+c 9oySPbKf UI8ejFHix2qIUdL1h30/iqFC4pKkgwEjPdTifxMigDLCBjO6Zyvtuj5K4903N6kHZ6XrDJyg28MroWo95j/Z+z1StNIXgh6EogQRNNjx4LQ2OB2UgvC+6TTkwffNOYSE5FjjXVIougU97ADt88ozaWdJkKkhzD/5XKwrBqbp9LS3JcJJb1lZZVrtqR/XmfiA15aY5ffv5ZjNmvjzp5OIwUCu9pmpA3ydDWqa5CmUdgwjc/MKFY+oeM11zIz6qEHxxbfeMYVZykp1Zz/Dw6IMbzJprcL/AiSoMEdf7+ZQeTKlvDpEKCQ17zxYeS06up+KdRhktf/nlPvHUqVJPXlVJOCbUlD0omz4fApY7MtjpxmYV0kz401I4azB0CwRXsHhdHupV/7yYBqW55re6rUdVC+gRd8OBHAVla+HfNEU7MR9i7dV3Z/JvHkhNEDTKQWxcGXkpeR+0hlzp/tzArPoCc7F1C+H7X0uo+3PohRwo9y3jCyUIqKl3WT/dWoyR8TRa0oiD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Robin Murphy The DMA flags field will be useful for users beyond PCI P2P, so upgrade to its own dedicated config option. Signed-off-by: Robin Murphy Signed-off-by: Catalin Marinas Cc: Christoph Hellwig Reviewed-by: Christoph Hellwig --- drivers/pci/Kconfig | 1 + include/linux/scatterlist.h | 4 ++-- kernel/dma/Kconfig | 3 +++ 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 9309f2469b41..3c07d8d214b3 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -168,6 +168,7 @@ config PCI_P2PDMA # depends on 64BIT select GENERIC_ALLOCATOR + select NEED_SG_DMA_FLAGS help Enableѕ drivers to do PCI peer-to-peer transactions to and from BARs that are exposed in other devices that are the part of diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 375a5e90d86a..87aaf8b5cdb4 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -16,7 +16,7 @@ struct scatterlist { #ifdef CONFIG_NEED_SG_DMA_LENGTH unsigned int dma_length; #endif -#ifdef CONFIG_PCI_P2PDMA +#ifdef CONFIG_NEED_SG_DMA_FLAGS unsigned int dma_flags; #endif }; @@ -249,7 +249,7 @@ static inline void sg_unmark_end(struct scatterlist *sg) } /* - * CONFGI_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes + * CONFIG_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). * Use this padding for DMA flags bits to indicate when a specific * dma address is a bus address. diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index cc750062c412..3e2aab296986 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -24,6 +24,9 @@ config DMA_OPS_BYPASS config ARCH_HAS_DMA_MAP_DIRECT bool +config NEED_SG_DMA_FLAGS + bool + config NEED_SG_DMA_LENGTH bool From patchwork Thu May 18 17:34:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247178 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69AF2C7EE2A for ; Thu, 18 May 2023 17:35:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B0D9900012; Thu, 18 May 2023 13:35:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 061A5900003; Thu, 18 May 2023 13:35:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6B7B900012; Thu, 18 May 2023 13:35:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D915B900003 for ; Thu, 18 May 2023 13:35:00 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B0588C0858 for ; Thu, 18 May 2023 17:35:00 +0000 (UTC) X-FDA: 80804076360.25.AAC00CE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 0B93E140005 for ; Thu, 18 May 2023 17:34:58 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431299; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Po7IyUSvGYuNcwSRGXH+yWudEHSJDb5jHBbVqoilbOw=; b=NmoYtcV/xeO68kt1NyiJ+na12qbYb7EdD59MEZzvQEvf0V1mos3meaCFU5/75mJyUMwEii couScUJtUiUHbAABHxQ4wj/hcAHUf0ky36AnuWSiPN+VhOc2FyD/vHQT5INJuu/9gDoCV8 r4E2+K/2puevoNJu6/pItDktdgJejxQ= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431299; a=rsa-sha256; cv=none; b=0YAtZRFkSFXmREzOdvm9ab6c68m26K1KaCzGdQ4CAuBrGP8fXKMbYNVP+2Va7Oa3e/WzZq KUksa36rANswRpQSSZzAsW7DlPWWJ6HtD615Sr3nWlO7prO0HVxVu+6n5ZdTNW5aRYLSWp RI5caQUqnZqXYmKyUe+Y/6h/4zI/r1E= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 206CB642F3; Thu, 18 May 2023 17:34:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 440A6C433A8; Thu, 18 May 2023 17:34:54 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 12/15] dma-mapping: Force bouncing if the kmalloc() size is not cache-line-aligned Date: Thu, 18 May 2023 18:34:00 +0100 Message-Id: <20230518173403.1150549-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0B93E140005 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: kjabt8j8o3iuuyygep7jjhfnyoswwnf5 X-HE-Tag: 1684431298-263971 X-HE-Meta: U2FsdGVkX1+Vp3KABr1KHik5XYFrml4JDgmD7A+6kO73EXEAFU/0NsIAmOC/6uDeOBquyu24lvJtO/ZSpR4uS3TQYGJza0Lx/c99r2MdI3o4rTNCc84FelkSHtcW+HWIixOWPVpRY0nTpxXVKNYNHUzrWFnkPEYISr7Kl3hcMxaMg5Mr5gYf0AatoIMhRu1ebACihGZhOuJXMrHOUM/l1Ar7gs4En3Lh227rQLXPzuCNjX2zu5+D/tmSJJIDUO6BBCCifl1Ic2T/dyyQW3HZYcLu1IHV+npbzqGuIded1CufE8FhiINWkiKBrGpDQ1ZmrY7eaOHSqH+8odEz6a8NSwTC5w2mjei0+CN6cKCd8YYzPEyyrBc6ObpaRDoKZKNdLdwjfxCtzPqJ7BVF8OvnuUY7i/yNMHbSD+rlxayA7etFyOsKtyR/Ui/D60A5pCNmtPkgWnaFnUjx7scm6bIfMifKPK0hvA4py3dbE7IheUVq8U5XUG8EnLX4lYxbPoJ2vWwG4DQXtSdDcf71NspNRChfDaS4BEhENDq47TiUHNLa99jDsai270RFxdDqcEcHJ116eg3E7Td1UBIwbTneaLRKJrqw4TPnXvg/U5L6bbZxb8GmpDzM1BYI6TORnZHVXKliiEKRI6a/tZRXEcA72XtWZk73wivHo6/+8Ss2/x7pddzkU45oYHd0oH9phLBFrB3yjctkN1iplTvK7MGzqRxqC77+vxEzIkzQ92IPHcapDXhs8u5DEO92uGcUA2rnwio3jluzSSikPmcU5qPqc0aaj+56+3CmKLDa4pflRQfWwCMcwyA78/TmVP1Essw+u+yM8vTq7eStQ7JQADv8n8089kfl1Wq32TCBuCFUSs2qYboNQv/LMSh200mkWPoP7eeUmVxjOVRjPPgPxaE1WJB38RCtbNEiT7+lMJ6fZDlNQdt7zMvgk0xDNguhgB6ydfiwGb3bnvsZix+h6YO LSujV97F p+Ethr33eZPIoEgNLhAsxn6kMG846DfmOMi8uZRZQ7QvL10Bua+jsSXhpnXV3cDyhe/V6ns2ULxtCucpa2jqal458y1BT2cgZ8yGIQ590QEHboCnY/PhJjzY47x+93ZP2s5qNPrmqiKnRVLOAyD5WNrekVa00pPJOjpD1W00S6Zy1eeo9J7TxF/9ZjDdQawmnfwz5UDGcFHSM45n20G/7XRNg6Ea78LjdbpxyTh/k/SYcG/dWDB6oncCjc1Uf5wNMQqLhYaz3of8l0hvQenhvhGdTYsxcKWvMfIBwYrf69bK2YPnfT6rISGuSn6+zjJCguZojE491FLvtjcdxLgyKsG43O7cO1D/t1NSYvlfGkiFRGZTKtYFvDyyNYb2CKgL1+DI2y6qIBbXBJyRdmHMEKrrYtSXgNXdczMmyGaVAUmEZsBwTSduE1TkedC9EIOSTvqtoWonO2DREUJiIbJgNE6G2+yFxSO/SlILEyC8IYePTJrWuqBvoBSnrU+PAoBRo7U4S X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For direct DMA, if the size is small enough to have originated from a kmalloc() cache below ARCH_DMA_MINALIGN, check its alignment against dma_get_cache_alignment() and bounce if necessary. For larger sizes, it is the responsibility of the DMA API caller to ensure proper alignment. At this point, the kmalloc() caches are properly aligned but this will change in a subsequent patch. Architectures can opt in by selecting ARCH_WANT_KMALLOC_DMA_BOUNCE. Signed-off-by: Catalin Marinas Cc: Christoph Hellwig Cc: Robin Murphy Reviewed-by: Christoph Hellwig --- include/linux/dma-map-ops.h | 48 +++++++++++++++++++++++++++++++++++++ kernel/dma/Kconfig | 9 +++++++ kernel/dma/direct.h | 3 ++- 3 files changed, 59 insertions(+), 1 deletion(-) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 31f114f486c4..43bf50c35e14 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -8,6 +8,7 @@ #include #include +#include struct cma; @@ -277,6 +278,53 @@ static inline bool dev_is_dma_coherent(struct device *dev) } #endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */ +/* + * Check whether potential kmalloc() buffers are safe for non-coherent DMA. + */ +static inline bool dma_kmalloc_safe(struct device *dev, + enum dma_data_direction dir) +{ + /* + * If DMA bouncing of kmalloc() buffers is disabled, the kmalloc() + * caches have already been aligned to a DMA-safe size. + */ + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + return true; + + /* + * kmalloc() buffers are DMA-safe irrespective of size if the device + * is coherent or the direction is DMA_TO_DEVICE (non-desctructive + * cache maintenance and benign cache line evictions). + */ + if (dev_is_dma_coherent(dev) || dir == DMA_TO_DEVICE) + return true; + + return false; +} + +/* + * Check whether the given size, assuming it is for a kmalloc()'ed buffer, is + * sufficiently aligned for non-coherent DMA. + */ +static inline bool dma_kmalloc_size_aligned(size_t size) +{ + /* + * Larger kmalloc() sizes are guaranteed to be aligned to + * ARCH_DMA_MINALIGN. + */ + if (size >= 2 * ARCH_DMA_MINALIGN || + IS_ALIGNED(kmalloc_size_roundup(size), dma_get_cache_alignment())) + return true; + + return false; +} + +static inline bool dma_kmalloc_needs_bounce(struct device *dev, size_t size, + enum dma_data_direction dir) +{ + return !dma_kmalloc_safe(dev, dir) && !dma_kmalloc_size_aligned(size); +} + void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs); void arch_dma_free(struct device *dev, size_t size, void *cpu_addr, diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 3e2aab296986..18dd03c74734 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -97,6 +97,15 @@ config SWIOTLB bool select NEED_DMA_MAP_STATE +config ARCH_WANT_KMALLOC_DMA_BOUNCE + bool + +config DMA_BOUNCE_UNALIGNED_KMALLOC + def_bool y + depends on ARCH_WANT_KMALLOC_DMA_BOUNCE + depends on SWIOTLB + select NEED_SG_DMA_FLAGS + config DMA_RESTRICTED_POOL bool "DMA Restricted Pool" depends on OF && OF_RESERVED_MEM && SWIOTLB diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index e38ffc5e6bdd..97ec892ea0b5 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -94,7 +94,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, return swiotlb_map(dev, phys, size, dir, attrs); } - if (unlikely(!dma_capable(dev, dma_addr, size, true))) { + if (unlikely(!dma_capable(dev, dma_addr, size, true)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; if (is_swiotlb_active(dev)) From patchwork Thu May 18 17:34:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D955C7EE23 for ; Thu, 18 May 2023 17:35:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2FAB5900015; Thu, 18 May 2023 13:35:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AB41900003; Thu, 18 May 2023 13:35:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 173DD900015; Thu, 18 May 2023 13:35:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 07DCB900003 for ; Thu, 18 May 2023 13:35:07 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CB125A0915 for ; Thu, 18 May 2023 17:35:06 +0000 (UTC) X-FDA: 80804076612.29.3378346 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id DBC854001C for ; Thu, 18 May 2023 17:35:03 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431304; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sVNhDYozvMQiV5O03sPIWzZhD42mGtaSbkIbs8wrJEY=; b=XSJK6JCRxVKwEjVIWB0FYBYp/Sns+tPxTSF4HIGy5fSfkAeV+dMzjKx4QCg6xfMHhFPSNh nEkPxbW5LU+k4rTfs20L7yuLTsNeHk4YLfxgXvpqCR9G7lsNVe63ssYUfJ2RjwzdWYSAsM pwgMBBfO6l7erera8oGA0/Fje4BQxRU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431304; a=rsa-sha256; cv=none; b=EfjyFlKPimFYxjHB2Lcty51vshOeRke+izllpDmc7W3i7j7ZITbH5RGIln2ssCyNX3sB2p NwBuovVqff1lKgxf6S/qYeL37ChLX2mWWsdSCl+FWn67UVe0uk80bhqqupwp17XB3MeBZG 2jToDu3BticBmdOY7JX8rJ/GQbcUkck= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C82B06516C; Thu, 18 May 2023 17:35:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50B11C4339B; Thu, 18 May 2023 17:34:58 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 13/15] iommu/dma: Force bouncing if the size is not cacheline-aligned Date: Thu, 18 May 2023 18:34:01 +0100 Message-Id: <20230518173403.1150549-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: DBC854001C X-Rspam-User: X-Stat-Signature: meyfaj4z1irgy51re1weepqegpm9hji8 X-Rspamd-Server: rspam03 X-HE-Tag: 1684431303-786786 X-HE-Meta: U2FsdGVkX185tgS23H/2Pkkz8iVRZYrbv/LFHzZxmHEMyQ+k6yfGSfeQMRZmRKLl16/GI9uSvnEEWqULxLlln7UACRskUlIXGnptgK5SnhrDMYPQmDPUGntqKku54rjgvmCNaHf7D2eS18HjHg6rHTX/cdpaGjiADURhkoXmAAx7DZdx+TG6c1U5vxnrJ8SbGDOjwZ2JHVMJcNTfC+tW0uK721Dr41b6oH3Ldi7P7wqc+e72MYlMBU3aOe8tyWm5VcaHjc4Fi4WiO1PjP9nCRE7h3jndhnFcORCibuySozqAvkR8D1ahXVgg8QRKO9vPUo/4x83mRshlIqxxFK5xpLvrsTudFQUKQ9lmj01UKO0e+0w2NRlG8JyqHXUoFGLttEAigY+7mXvxKbElvIFBefZS/sRKx1eOCa4I2XQj9dhyH4O4Xl6cqhXE9vuVNzllgRz/H1XIKyJU0MnlXc56YjkiKvT2Ql+mdn8MsQMKnzC43ZwDnmjfW0hiaWYogfmlAY0xCBqpK9fnWdOZ/4yID4BK3VW5XnOMYtGZtnglvo4zuQDWy2FJ2MjibsNu8R3BK5CwutmRPkzdsbeNqZu9yL1QX/1h1K9Rg6Z9QSUjvU1JS7us8V/WIrw4qa+25wtzyKZkohAoYQAN4f/Gsz1FlFqnFnueGN35hH0UuPGowLFdtKOYk2GfKKRviSqYGgE2aK0jKaFm1xRairZzsKn5q0e/IBazeC124Os0rjQR6pE3Ur4gbgF8qoH9+yDi5iKZq03fScFneq9nv8m5YvWc9yXvSUktwcu/yYkBjwPth6AEx5qMo93hsQn1N25OBXVU3IHd5AzVXxMgeegisIp/7HRniJBopxOjyznZCFBAZanu4rk+8kfdTSs/+HHzcVmiucyqrkZI9NywM3nsorssffi1vGYleCuL859tW5lCt5pVJ04vLSRTu2x0vlZ5ShJByQx7etbH5wdVVwpjjCU fwHnVsBS a63Z/70biYnflAQoQ9WBkX2QZ9avsWRhtQiGvsJeLkdVZFrGbnbS8ZZSSX15tm2HTSOrBjeL2kZA85PYF7ycGlraukMOHLr4x5XSZXhH62qibL3tbQaWHN5ceGbWJe2be8zvEpiPaCgnWfUOul1F3r4ZPyPmdHfD34loZxFeMtdbAReAN/xsrnhAO9z63I0iLEte/S5YIzurpx/uzwqjxv6OLMbgjduQuQc3LYjHf1d7wH94QwOk9AZrYs8cBn4GWIBo3BZTCa2ihmsiuhBKSC1rmQSZGrmafueaClKm9X7blVJwk1xC4tW+IKNlbLgx0RxRBrAvUi6xB/4raUUjItAUUkrnDrQ5n/NfaYPaf7zgJjjVRtrHpSfUy/3Hne9SmKNrqe60wMlx5mrC2MO+b8fEWy7bQL691eFpA78Gx3kjQ1dGFaTVAr9UTIsiRTPC1dm1oddwQnKihE1L5EcEy+c1MQOELSOC2cQp3ySm6Eh63CK+LSFtBSDaLXrCe5QTG8VSOt0GWo77YMlIXuHlbqNuZkQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to the direct DMA, bounce small allocations as they may have originated from a kmalloc() cache not safe for DMA. Unlike the direct DMA, iommu_dma_map_sg() cannot call iommu_dma_map_sg_swiotlb() for all non-coherent devices as this would break some cases where the iova is expected to be contiguous (dmabuf). Instead, scan the scatterlist for any small sizes and only go the swiotlb path if any element of the list needs bouncing (note that iommu_dma_map_page() would still only bounce those buffers which are not DMA-aligned). To avoid scanning the scatterlist on the 'sync' operations, introduce a SG_DMA_BOUNCED flag set during the iommu_dma_map_sg() call (suggested by Robin Murphy). Signed-off-by: Catalin Marinas Cc: Joerg Roedel Cc: Christoph Hellwig Cc: Robin Murphy Signed-off-by: Catalin Marinas --- drivers/iommu/dma-iommu.c | 25 ++++++++++++++++++++----- include/linux/scatterlist.h | 25 +++++++++++++++++++++++-- 2 files changed, 43 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..ab1c1681c06e 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -956,7 +956,7 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_cpu(dev, sg_dma_address(sg), sg->length, dir); @@ -972,7 +972,7 @@ static void iommu_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_device(dev, sg_dma_address(sg), @@ -998,7 +998,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, * If both the physical buffer start address and size are * page aligned, we don't need to use a bounce page. */ - if (dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { + if ((dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { void *padding_start; size_t padding_size, aligned_size; @@ -1210,7 +1211,21 @@ static int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, goto out; } - if (dev_use_swiotlb(dev)) + /* + * If kmalloc() buffers are not DMA-safe for this device and + * direction, check the individual lengths in the sg list. If one of + * the buffers is deemed unsafe, follow the iommu_dma_map_sg_swiotlb() + * path for potential bouncing. + */ + if (!dma_kmalloc_safe(dev, dir)) { + for_each_sg(sg, s, nents, i) + if (!dma_kmalloc_size_aligned(s->length)) { + sg_dma_mark_bounced(sg); + break; + } + } + + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sg)) return iommu_dma_map_sg_swiotlb(dev, sg, nents, dir, attrs); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC)) @@ -1315,7 +1330,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, struct scatterlist *tmp; int i; - if (dev_use_swiotlb(dev)) { + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sg)) { iommu_dma_unmap_sg_swiotlb(dev, sg, nents, dir, attrs); return; } diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 87aaf8b5cdb4..9306880cae1c 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -248,6 +248,29 @@ static inline void sg_unmark_end(struct scatterlist *sg) sg->page_link &= ~SG_END; } +#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_BOUNCED (1 << 1) + +#ifdef CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC +static inline bool sg_is_dma_bounced(struct scatterlist *sg) +{ + return sg->dma_flags & SG_DMA_BOUNCED; +} + +static inline void sg_dma_mark_bounced(struct scatterlist *sg) +{ + sg->dma_flags |= SG_DMA_BOUNCED; +} +#else +static inline bool sg_is_dma_bounced(struct scatterlist *sg) +{ + return false; +} +static inline void sg_dma_mark_bounced(struct scatterlist *sg) +{ +} +#endif + /* * CONFIG_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). @@ -256,8 +279,6 @@ static inline void sg_unmark_end(struct scatterlist *sg) */ #ifdef CONFIG_PCI_P2PDMA -#define SG_DMA_BUS_ADDRESS (1 << 0) - /** * sg_dma_is_bus address - Return whether a given segment was marked * as a bus address From patchwork Thu May 18 17:34:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE335C77B7D for ; Thu, 18 May 2023 17:35:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62EEF900016; Thu, 18 May 2023 13:35:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E019900003; Thu, 18 May 2023 13:35:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A877900016; Thu, 18 May 2023 13:35:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 344BC900003 for ; Thu, 18 May 2023 13:35:09 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 101661608BC for ; Thu, 18 May 2023 17:35:09 +0000 (UTC) X-FDA: 80804076738.08.D60F4E9 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id 7954B40009 for ; Thu, 18 May 2023 17:35:07 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf27.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431307; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bHWY9l8OJoLo8hdZQBx3YOP+S/Ayh/jMX0+APRJNrik=; b=TywxEVXikBNO5iawu5OWLjBi5/hLqcBW3bofjf60NyYZ74lvLj62aG32hJfOi//yQMvcgj HrxE5s0/bBz3P8JLzo5LSo2EzCpJ6otKpNpG+N4SM6YAIIRd/fPYMCkXcrN3bTM9J44rZP nBanGObkZhOrL00SzkGx4tHW+AXyg1Y= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf27.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431307; a=rsa-sha256; cv=none; b=QKyeki9wHHVi1TOXq4DW0WX2uuKlHMdbnx+ZO3Lojb5FCyNM4gmQVrGGUV1HI2QtF8Kd3u wjtdoHc2yD5dS6/DISGaW+//LFRo9hKecd3A7MH63fNsXjPxYDJVGDnpadG/3ksDeEYDa8 JS+mrKNq5fWvrbLIMDN+r/+bm//MLHc= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A72FE65152; Thu, 18 May 2023 17:35:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 61FC1C4339C; Thu, 18 May 2023 17:35:02 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 14/15] mm: slab: Reduce the kmalloc() minimum alignment if DMA bouncing possible Date: Thu, 18 May 2023 18:34:02 +0100 Message-Id: <20230518173403.1150549-15-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 7954B40009 X-Stat-Signature: czstrifoh55yptks9oiwh3mga86dxwbk X-Rspam-User: X-HE-Tag: 1684431307-75098 X-HE-Meta: U2FsdGVkX18L1CfmWyM3OUkCFDvi5yJYVWi983CNNEFj9dgx5lXb55p4Q3tWTpMlLm+RCHQg3w/B/zHdcz3KdF4t5Bjm2tW1FlYVtCqXjMt5zayLImdnoOwlx8NNMPC1wpvXDNIFgX+bVZ5b5AsmLctyunOaXeCdbg4JPuu78TrycnLaVr/BCIxWH5g7fRWefk6KR9o5Wh9vot2HPEQZ6pVioZJ/dobD2SDhISCqNLIt/9mHb1d+krI9eg4uD5rKWaZNMvKFX6yIQglKPUIrLt7G6c52zlOG1FumKP9mPF+mxZiHImnA34HWYCrRo7zQr2xAdsoAFDOgWLMn+OXgFkNEpiqj9++Ag2XF8KJF18sNda8LbsDHDkrwpk9sAXQ6s+k/xN8BzXqL2Tm3JRhZBTATsVXY8GpLAEg7eigpLoasLWhJ8bJffvxo1tHtwhAq2C3qHZ7U2dGlq+4KDZnwzs7y1G8PYgBXj4ag9O5fqsLSDIFGzDRv600lVjVe9+YyPajzz+Js4N924oWSKs9xN4heaKTyXzQT/xaZK3ZWWX5/ykT8aMJAkrhLSe/ZntKzdqDRZPnp52cBygMvyVy0VFcErVypsS1s+rx/Rr6jHYEfQ/uSuURv/nym9EKz8RPvhoDZBStf7lf0VbVAtHh3hSWej9rbm73e/4RwkG8mO7YFa/m70hpxhIgV6SoHWyLrqK8Kg8lt6TYb0NKzTuy7tov2aSqIUh/fEIdq0Kp61bkvCFLlmD3zU4zBypMRZ60QSSVSkEE+yvF6+TlCiLpiUJqabwY9W7fXt+heM0lSxAf5ndDVYQ1JrtOBL8FTtszJ+cUoaOndYCIGt6N62KQJzWkTWzkuPGTkqVSoXvi+zi7eJvM8RCvrszzedJHN3WRHEHDer6F8xMUlAPVrEoU7UnOrqIzaCdPXhsey5DU/3JYygpZE4rK27blH7Hz/9G3H6XrWiJo9Shl8sr61O5J g1Io/JuA zWf/FI4NLAUkSRXEj2evUfJwOiCO1RdrofZLHUCByIQlwxcrATZJhTX8UBYWI+QPKNFQJ3YPokqSvvX0ev9tBAHDam6DUf2FwILKQVDHjMddf3v63WCIghuWnh9TBiyKo6c0lh606J4lvpuom13tFn6T6OPXGkBAw1o1HnIksXOb2dlxywVM1TKQ/CMWufaoFioBlkzfmR5Hnb7Vb6Vz+iJm4z44E9Yp2yp7SBVuWSPTZ2Oj5WROJKVtEDw+EJOY/LWDES2FGthiSoQ4YUshQlL3VoxxdvV4QswvEbWZpQoxb0MoJXB22Adh19NyUjuyU23XnNzpPnyR5lQ13nE/Q1OhHvqz90FvfInyzOEqJAoSS12/odhIRM3/O/vbxU4r3XjYtCANN9pkr8lNexZ/hQFQBTGVnWcDwPQtlVgnYDXSETkOFkDzrq6KNQ4R4tbV7IQaT/ifZpV5rsmIFJNV6P8w0MT9lOCA0vlvRrilFRL1QfRYlLbJjPvoNuNovOl0/+3YkPx0vDHCf8fbhgbLhbVtsCg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If an architecture opted in to DMA bouncing of unaligned kmalloc() buffers (ARCH_WANT_KMALLOC_DMA_BOUNCE), reduce the minimum kmalloc() cache alignment below cache-line size to ARCH_KMALLOC_MINALIGN. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- mm/slab_common.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7c6475847fdf..84e5a5e435d6 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -865,7 +866,13 @@ void __init setup_kmalloc_cache_index_table(void) static unsigned int __kmalloc_minalign(void) { - return dma_get_cache_alignment(); + int cache_align = dma_get_cache_alignment(); + + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) || + io_tlb_default_mem.nslabs == 0) + return cache_align; + + return ARCH_KMALLOC_MINALIGN; } void __init From patchwork Thu May 18 17:34:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13247181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F26EFC7EE23 for ; Thu, 18 May 2023 17:35:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9636E900017; Thu, 18 May 2023 13:35:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 913CF900003; Thu, 18 May 2023 13:35:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DB96900017; Thu, 18 May 2023 13:35:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6E999900003 for ; Thu, 18 May 2023 13:35:13 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 37479408FF for ; Thu, 18 May 2023 17:35:13 +0000 (UTC) X-FDA: 80804076906.26.B839D64 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 89E1040005 for ; Thu, 18 May 2023 17:35:11 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684431311; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QH5r4yeZLShWNILLwf1McZXq40oIioPqDsZRYQypELo=; b=Uzy0nedF92scuzHIYchIG2/6nHXL5TYK8rDLHHGwcF5VM6M4Op0Hh0QowsH0yg/sz9fuqM D4zXk6+QrVo4yh9WAY+4lKEjjylB9SIwoeqRXgAduIJmG+P/1Lxk3pRZkYACschxAmi+vU KaZJVRQbDsKZd0KLCb3pUC2cFJ8wr78= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684431311; a=rsa-sha256; cv=none; b=aliout7rSKM9FtFDVj/PFKipIc3GkYRZ3qPDx/mCyEi0bVgenYldqViXtN2LyeZCt8Lrjl A53c9wAIVpga8kFx4Wj2Ur05H4otcSTBho5veRjHzer3LtGUTzR0CBBjM9udAENnkOQ1Sg 21tni0B4sCauZIb7OqbnkXVTnjsinJA= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B781265161; Thu, 18 May 2023 17:35:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79802C4339E; Thu, 18 May 2023 17:35:06 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 15/15] arm64: Enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64 Date: Thu, 18 May 2023 18:34:03 +0100 Message-Id: <20230518173403.1150549-16-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230518173403.1150549-1-catalin.marinas@arm.com> References: <20230518173403.1150549-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 89E1040005 X-Rspam-User: X-Stat-Signature: hgzkq4yen4nqe94qzrbmy9o4fmc7cbr6 X-Rspamd-Server: rspam03 X-HE-Tag: 1684431311-85732 X-HE-Meta: U2FsdGVkX1+y8+eVb0L8dWfl2YGncPl9+kpFKTVs1FQju52G9MsLn0yz5Si3UqortbbPdl0nYY8qhmgpDwd18SBWE5LDZLDczTdR/kRy/WZPKVngSrP5IWjSjTKmjf7GiPTz3aEvj39utrMq1XF+X+g6cWXGmvxVzU5Up02dqpKili7gBOKfOEPZQy1K/CJ0I4V2ujPZBhMqFHjGmI9hjeBM+YaR2FXJ/SJAuQ1c4V7azRWY7OESP79wAJfdHh8zqi7HAirQOcikmt8pbt4ipkw75TluPsgVrLLN1Tz06qEKAbmocrXr61z0MO5igbgJdVtQdPgGDI+llp+Rmf0IHvIEfucWJPiKewRgcbweo29k1ihTmthz6WrXPt8tFpvLXdeEfDqp0hE5feqkMzOs7KQV+5VqSJXAyLbOBDeYSwUCllfozHuisBgNs7nVDNk4ztZy5Xx5+QA37Du4MRHTiyQGP+XPjLJcNZ8oHuI8c+HnmLIqIs1GsBDJYPd8hlNxRYfxOWcrEnJ9PjV8Ul6leAadoSgmCPPR72JJT0WAodrLH7eYcU0IxTB3Pg9U1VlYT65cxTo5QRGskgUmLOI2mtRmWCDo966u1k/8bdl12EQgZQtnyWgmztax0fM1qMQIGsP0q9aZ3yavX1kUIZoCiiyp7QWJr0RjFCcLYUSM9JpiQxaIOYHg4fdN4pFwNe/HkU6gZGrlrvFGcppgxLqkqs4bZmVflJimelUta9f/tzzDZ/g4ze8/W3CvgXjX5+fKeNMikXFdDXynxyg6maex2RrepZwDTvT3EI+jFc9KX5qe17O06eVNBwqYn/zYuHayES+2Q5431+KttBeF4sBZCgrO5GVYJ4jlZUQp/7b0Icc2mJECEIK3gR9WbqxzNoVNWN+Gtsb+ezaFJlDcDaOXSlVXx764t9jfzowD6LLcUBNnCLqi5InIVw1nyravUnzOaSvDAAIbaMcGFiGtwKm +XrR9qfJ ri9jBC5onfBziMnG20IHQlW2Ibt064+nu8JQP06pYbNelyyBt2u0+pE2suLGj0E9nrLI8nfv8CWVYWUyg2ThXM/75ED7xsuRdtdbwRmxKmy4uZS19qeF0tkDzWQBF9aSAewPY6bFwUbNvPwb1VR9v79nINAm9mPiGGTELfrNclMTjx6VzDSe+0NvxxcFr9SEeUHcxJGKu+3p2N1ev/mcA0Vl/yJlv5jhGpeWzN4q29vEqCbbA7l0gadcapMKgd4aI64nWS3F/HUbEz2V/4NQNGJfRvyh62oQg1XoHW5LCTiJNeYMic/C03YHzXxulZoILKua9ynU0iMB/SoxoU/e9Ag/BoiJ1oPnE7AAYym3pK2AgQ0Spx1rQsfGkdU2MI1XJeutuwBStapeZiR+cmnY9CLof/lI/ua6yZ9cNE1juR3e4k5r6uOWCh86r3bwF0rXt9WswmIqRIHX1j6si2ibFjGzDavqJRO4dlLrQyufSARAnDG5PAiX5bQnLDaKfgGMuCv1QG0S0/EDhqdwM3nZ3ZUA3bh+wsxTjoeUhvbMkW6loZzFiifYXTy4X7AM2imChWjKq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the DMA bouncing of unaligned kmalloc() buffers now in place, enable it for arm64 to allow the kmalloc-{8,16,32,48,96} caches. In addition, always create the swiotlb buffer even when the end of RAM is within the 32-bit physical address range (the swiotlb buffer can still be disabled on the kernel command line). Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 1 + arch/arm64/mm/init.c | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index d11340b41703..1dcfa78f131f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -102,6 +102,7 @@ config ARM64 select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36) + select ARCH_WANT_KMALLOC_DMA_BOUNCE select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_NO_INSTR select ARCH_WANTS_THP_SWAP if ARM64_4K_PAGES diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 66e70ca47680..3ac2e9d79ce4 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -442,7 +442,12 @@ void __init bootmem_init(void) */ void __init mem_init(void) { - swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE); + bool swiotlb = max_pfn > PFN_DOWN(arm64_dma_phys_limit); + + if (IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + swiotlb = true; + + swiotlb_init(swiotlb, SWIOTLB_VERBOSE); /* this will put all unused low memory onto the freelists */ memblock_free_all();