From patchwork Thu Mar 20 17:41:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 14024223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5982EC28B30 for ; Thu, 20 Mar 2025 17:42:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 999F228000D; Thu, 20 Mar 2025 13:42:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 949E728000C; Thu, 20 Mar 2025 13:42:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8122328000D; Thu, 20 Mar 2025 13:42:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 66CD928000C for ; Thu, 20 Mar 2025 13:42:27 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 535A9BAB23 for ; Thu, 20 Mar 2025 17:42:28 +0000 (UTC) X-FDA: 83242648776.01.9F6C5F8 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf23.hostedemail.com (Postfix) with ESMTP id 44B90140010 for ; Thu, 20 Mar 2025 17:42:26 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf23.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742492546; a=rsa-sha256; cv=none; b=rHPEtnthbqjQAn9UECUE5ol4hnWrksjx6qvoEAFMjEaSK3ylX1A1nNJCVO/zsQ2SPqNukk yAz6Xz/9OKYjua++qaXvyk77YlQG0MTzH2AOwHB/gFZhni2RiZjmzyqY7UfM+Pmk2kBd0Y 1w0M4NkOcNMe1bLsXL2KJlrmSrvS25E= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf23.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742492546; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8NtAANCe4QtIKO9m3X/Y53rt7yHSwtrSDMfLnQ8LDCg=; b=0FAKQu+sMphffMa1ivOAg74Wkxe+Xpf9+bUxu2vJJHb4kTCi1CuRoBqAcB5eiB8XyIHASc vpIvnxy3Sza4//U6+V0vwkCwq+c3Erio4u7f2OYxJbEJ08Z/H6AwszUCM/jQpIzPtd6BlW vteIeyqNpBfhLZ9Qjj/2yfh+heS3hIA= Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4ZJXr66Rdqz6M4js; Fri, 21 Mar 2025 01:39:02 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 2F93614050D; Fri, 21 Mar 2025 01:42:23 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Thu, 20 Mar 2025 18:42:22 +0100 From: Jonathan Cameron To: , , , , Yicong Yang , CC: , , Yushan Wang , , , Lorenzo Pieralisi , Mark Rutland , Catalin Marinas , Will Deacon , Dan Williams Subject: [RFC PATCH 2/6] arm64: Support ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION Date: Thu, 20 Mar 2025 17:41:14 +0000 Message-ID: <20250320174118.39173-3-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250320174118.39173-1-Jonathan.Cameron@huawei.com> References: <20250320174118.39173-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.122.19.247] X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) X-Rspam-User: X-Rspamd-Queue-Id: 44B90140010 X-Rspamd-Server: rspam05 X-Stat-Signature: 6z3195esy1fmftb8sp6uzauuqkgwf6gt X-HE-Tag: 1742492546-459555 X-HE-Meta: U2FsdGVkX1/gkyTK+5Q2zkkYCg1WHGRYecvCPvApFChPVTXM082Nap2TPUUth+8VKdWIMW0nSvnbYpXuTGLI9bj7KFQWLdxOWm+kLr+NuEornI4Cj+wI/poBnPRWLH4+tR0D1pbH6AwgF9X28vL5sI1kpJlB9SEpekdkhaLRJ95f/rFL6y0DdByfRFUDsRERZ0hk4tGlrotsP9wiCYPkszri8wN/C/aoemjGHrDoTqmt5Jn3JLn6AS7Y/Z3dHXyg4MK7Xxb5AmvMyGlaHoCmVj3EuWhLolRSMQ9oxXPLx9pbRZtP9gAZDqjfJMXx9Dn/RxHOB9bEqtKh1Q4wVfQbm6z3+CB7UQofEopJ7hJvnQSequtDX1KDumOCgpSCTdMMCNZ73FSS1s8jw2SZLZl7TZsFWCENho9QSTS4iH4IdxBbsk0gcBOLyo57cX5nKo/R73oFmXZO01Op44hrloZV+dqyYGbpe3e80MtcyAeHiMxQaqXMJpofvLG2q1IdoteyBmTlTn7DnK6xgnOmkfGxVXPsMfTDZgwK1Xh08ozk9gybI0gCacvIDWwpsRJeR/+C/3H6mPCF+Q6bCEqgIxFoepA5syJCCx34x42NH4h2hLEmjNgsOV7WVtvIucLnYLxbU2eTtXHHsNSxewDq09GhkNr98Gd9tbad2O9XOR3F7QDm9itTDpwlkpXFdBixLvPcZW1BR8JPa/YsftvM0A5BIDaySWr/1sgLZHd3bCgRQGQ3ucGgbu9YmgA/tFVXmkACW3dW9GTzy6BUePAtN+uZXwmsmADnbX60nO/Wjk+Zz9tUp3Xo8XnarcfNlyO4Fkh1iwgOydj80h1aHnE0AhLUysVQhOe7sIDPoZrYAZtJMqxIT4S2hm4jurFb3ATPLfVMujVGDmmOM1aLAOTA9HeAIXpp+yit5ER93/Y4hW0kL3c/fGsu0coEaKuelCmtwDD5djQSUA5FUynNbu5cpvX 2XXqPsvN VqZqR3mJaIGqmq5Cl22FDT8XmP4OfAFkE2lX/4njOb7X98I/noMJvI52dLQ3eh/bongnVsoMATL3Uqz6jW2Itv7UmFkp3B6eYoJqZV1A9IYWjOavENwGw/AM5Gw423YwgMSuWkN2noyQjOaBY1beTMUNIsdbmeuMHIEJ75P/9iOSGo5gDgW33X+cw592xAHheolX8wYwLZTYLXWytzXzAAJxvNq2dUnY4fv7gZAbxzbUlPQQvYPeYDO7xcA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Yicong Yang ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION provides the mechanism for invalidate certain memory regions in a cache-incoherent manner. Currently is used by NVIDMM and CXL memory. This is mainly done by the system component and is implementation define per spec. Provides a method for the platforms register their own invalidate method and implement ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION. Signed-off-by: Yicong Yang Signed-off-by: Jonathan Cameron --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/cacheflush.h | 14 ++++++++++ arch/arm64/mm/flush.c | 42 +++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..11ecd20ec3b8 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -21,6 +21,7 @@ config ARM64 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_CC_PLATFORM + select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION select ARCH_HAS_CRC32 select ARCH_HAS_CRC_T10DIF if KERNEL_MODE_NEON select ARCH_HAS_CURRENT_STACK_POINTER diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 28ab96e808ef..b8eb8738c965 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -139,6 +139,20 @@ static __always_inline void icache_inval_all_pou(void) dsb(ish); } +#ifdef CONFIG_ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION + +#include + +struct system_cache_flush_method { + int (*invalidate_memregion)(int res_desc, + phys_addr_t start, size_t len); +}; + +void arm64_set_sys_cache_flush_method(const struct system_cache_flush_method *method); +void arm64_clr_sys_cache_flush_method(const struct system_cache_flush_method *method); + +#endif /* CONFIG_ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION */ + #include #endif /* __ASM_CACHEFLUSH_H */ diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 013eead9b695..d822406d925d 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -100,3 +101,44 @@ void arch_invalidate_pmem(void *addr, size_t size) } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif + +#ifdef CONFIG_ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION + +static const struct system_cache_flush_method *scfm_data; +DEFINE_SPINLOCK(scfm_lock); + +void arm64_set_sys_cache_flush_method(const struct system_cache_flush_method *method) +{ + guard(spinlock_irqsave)(&scfm_lock); + if (scfm_data || !method || !method->invalidate_memregion) + return; + + scfm_data = method; +} +EXPORT_SYMBOL_GPL(arm64_set_sys_cache_flush_method); + +void arm64_clr_sys_cache_flush_method(const struct system_cache_flush_method *method) +{ + guard(spinlock_irqsave)(&scfm_lock); + if (scfm_data && scfm_data == method) + scfm_data = NULL; +} + +int cpu_cache_invalidate_memregion(int res_desc, phys_addr_t start, size_t len) +{ + guard(spinlock_irqsave)(&scfm_lock); + if (!scfm_data) + return -EOPNOTSUPP; + + return scfm_data->invalidate_memregion(res_desc, start, len); +} +EXPORT_SYMBOL_NS_GPL(cpu_cache_invalidate_memregion, "DEVMEM"); + +bool cpu_cache_has_invalidate_memregion(void) +{ + guard(spinlock_irqsave)(&scfm_lock); + return !!scfm_data; +} +EXPORT_SYMBOL_NS_GPL(cpu_cache_has_invalidate_memregion, "DEVMEM"); + +#endif /* CONFIG_ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION */