From patchwork Mon Feb 17 14:04:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13977897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B041C021A9 for ; Mon, 17 Feb 2025 14:04:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00C3F280055; Mon, 17 Feb 2025 09:04:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ED81228004D; Mon, 17 Feb 2025 09:04:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA027280055; Mon, 17 Feb 2025 09:04:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BB67F28004D for ; Mon, 17 Feb 2025 09:04:54 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 77A26AF0F8 for ; Mon, 17 Feb 2025 14:04:54 +0000 (UTC) X-FDA: 83129607708.30.00F3298 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf20.hostedemail.com (Postfix) with ESMTP id B4A141C001D for ; Mon, 17 Feb 2025 14:04:52 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739801092; a=rsa-sha256; cv=none; b=mAovw8TIHvmsSHak7iI9GlRsX9qvL0TWcEPf/BxRr9LFeI/4HK7dO4HOkCBotV6/LDdALQ S88ZbtBjnHLFbkWaUQJpJePuUdeODiv/qucvwmfHRLEb1OLCi1YxweSsbtbvEQwkgOiI8D wKerT8jUvvHDseG8T/5mCaFXuI3Pi2w= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739801092; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+9mwnZQpzod0UnINf3LqRbHm7apaqJ7LUOpjc4gUVy0=; b=IHxExHSbkjJQfc8GV0Gg7f5a66tJ2JODvD0k6zP/Q2zPlOawSiJW8BX17FOp3gcdoPML8/ W/CcgyVG1nXKuVlsFPBcu2GWDfoLp4l0MkarVxk5bGYG5vTbR5nXOG5GsP0O64Z0YDeRB6 VXzCRD1RQpY60W4YTv8aBorrcEj4KnI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 10677175D; Mon, 17 Feb 2025 06:05:11 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 905E83F6A8; Mon, 17 Feb 2025 06:04:46 -0800 (PST) From: Ryan Roberts To: Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , "David S. Miller" , Andreas Larsson , Arnd Bergmann , Muchun Song , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , David Hildenbrand , "Matthew Wilcox (Oracle)" , Mark Rutland , Anshuman Khandual , Dev Jain , Kevin Brodsky , Alexandre Ghiti Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v2 3/4] arm64: hugetlb: Fix flush_hugetlb_tlb_range() invalidation level Date: Mon, 17 Feb 2025 14:04:16 +0000 Message-ID: <20250217140419.1702389-4-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250217140419.1702389-1-ryan.roberts@arm.com> References: <20250217140419.1702389-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: B4A141C001D X-Stat-Signature: itrnh86jbehkhngrnrnfc1py9ihsjsrg X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1739801092-597512 X-HE-Meta: U2FsdGVkX1+X6ObOr6nSJHnTkbMQXIuO1hVRbLVS9XVSZG7aw3MKriP4xoaSZfF0oBnOrIIqLduvCGwEArz80PfpevwFJIWuc2y63SRWI6Jc+Dm1shitknulQOl2w/aPf0/EzpIdvDkIrtoJkJEBiZIHsKJ9YaZVPqf1N4vR+M+TZps0DJ0hGbki2phNSXVO8M1XYiiF6BcJCjbOmBCTyhJt2xeBm9Gy92fWJWT2jCuXAfAMKEq2a++h1RLegYkeCfvuaOR0QRD6QzMjRLC7Bf+nHVhOMOmAJ2c8CU2ql7+PHSBdOwDRvJPr21eDTYpdCDdHs4BYk4FHpCfETVTPg4W+0NO+Eqku08u1NMUtBnjGVNdRBc3B8UXR5uUuXSjkULcXMHGAGGVf/x9Wsdj1tOWTGRmH6KjC7a4IbgjYrQoo4fEqV/xgan4RY22RBiHeI9kK8ZW/tYzuhz7bZWGVwA1PtIEw/HFhAtxU/eh4zOHIdr/85FD3V/cejCzfPoXXsDEvvE4OUBsf6PlC8i0QUZIvVjRWYn49aeOVUq+1f6FCUaGUc8aJIHsuhqJv+6Nh6LhviiQaYs5gB/nPTVuKI3sMya9pA3Wu7srZB7V4A9+LzISPI5xJNYeHqdNry3l6Py1TUv3TbS+Uy7Kom1m6Ru20+AcaxuBntMD7e29DlX/KUU0IeK8jb5/xqdtqtL1NeV+xXmEY2Lgi8R5ttymK+bXZwA2fZIN8TCvB1SyLNo+INv0y5a4h0iFhYbSs0kNX2UFAZEdtY7AtlzZhCBb6oNzSSmJb3+V9W+YPG9QoYVuDj3ZMZUMTBPNxL465CbpRPI5QOIdH78e3lXy7gsGUDJWe3WKLdojCXEZnEm1vW7D4UurFYpM9CZJ3K0JVYPiYK37IaqVeffzz/e4kFRYCP5igdHwA+ph14XnN6ygxvuV/3vie65rvLv17ip82nboRuRsYflwQS1NxAMwFu9P tdIsQp6Z mX1k4hkSedaed5XQyful+QGfdqhXxTuymWfBfNijEjzD3fAMwlBUwIi/D1GPHkrMy1Hrl/Wx+0oAo+X2oiYtSDN1Jnt6Sp37JifMsaCkTyXlI33XVJVgd50THIIjXeulx8S+MvBk60YNCOWniPGstdebXt8UtwH4pSebun8Z/GQk2SMhU27NAXfSxXQZAltYdarOa+1WPenE2SK2bbSYWZZ6FMYUppf2KSuZK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit c910f2b65518 ("arm64/mm: Update tlb invalidation routines for FEAT_LPA2") changed the "invalidation level unknown" hint from 0 to TLBI_TTL_UNKNOWN (INT_MAX). But the fallback "unknown level" path in flush_hugetlb_tlb_range() was not updated. So as it stands, when trying to invalidate CONT_PMD_SIZE or CONT_PTE_SIZE hugetlb mappings, we will spuriously try to invalidate at level 0 on LPA2-enabled systems. Fix this so that the fallback passes TLBI_TTL_UNKNOWN, and while we are at it, explicitly use the correct stride and level for CONT_PMD_SIZE and CONT_PTE_SIZE, which should provide a minor optimization. Cc: stable@vger.kernel.org Fixes: c910f2b65518 ("arm64/mm: Update tlb invalidation routines for FEAT_LPA2") Signed-off-by: Ryan Roberts Reviewed-by: Anshuman Khandual Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/hugetlb.h | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 03db9cb21ace..07fbf5bf85a7 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -76,12 +76,22 @@ static inline void flush_hugetlb_tlb_range(struct vm_area_struct *vma, { unsigned long stride = huge_page_size(hstate_vma(vma)); - if (stride == PMD_SIZE) - __flush_tlb_range(vma, start, end, stride, false, 2); - else if (stride == PUD_SIZE) - __flush_tlb_range(vma, start, end, stride, false, 1); - else - __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); + switch (stride) { +#ifndef __PAGETABLE_PMD_FOLDED + case PUD_SIZE: + __flush_tlb_range(vma, start, end, PUD_SIZE, false, 1); + break; +#endif + case CONT_PMD_SIZE: + case PMD_SIZE: + __flush_tlb_range(vma, start, end, PMD_SIZE, false, 2); + break; + case CONT_PTE_SIZE: + __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 3); + break; + default: + __flush_tlb_range(vma, start, end, PAGE_SIZE, false, TLBI_TTL_UNKNOWN); + } } #endif /* __ASM_HUGETLB_H */