From patchwork Thu Oct 10 03:50:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bibo mao X-Patchwork-Id: 13829446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB2FACF0437 for ; Thu, 10 Oct 2024 03:51:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2CDFA6B0089; Wed, 9 Oct 2024 23:50:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 234AC6B008A; Wed, 9 Oct 2024 23:50:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F26186B008C; Wed, 9 Oct 2024 23:50:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C8F276B008A for ; Wed, 9 Oct 2024 23:50:57 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 00F56A1B4F for ; Thu, 10 Oct 2024 03:50:51 +0000 (UTC) X-FDA: 82656316554.12.5462E31 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by imf13.hostedemail.com (Postfix) with ESMTP id 30E8820002 for ; Thu, 10 Oct 2024 03:50:53 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf13.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728532105; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rd4ZkujWkFK413qElnLYwYdjOhfxwEaEsEBX+Qwnza0=; b=Uscam//ILFFkVNlpiueSqBpZNZRBvYMpPdKtVoF20abT7VqmTxn5XinT3gjuBKNTze/5m9 bSCjDia/s3TL9z/nNnkmL9v8VYrO/CIX0NGqnNgDYN5AQJ+hDCWq8u9ZG1duJwIWlsQRwJ JVhrwU/zdhuRaUzGFCaSbK0rUlLqiR0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728532105; a=rsa-sha256; cv=none; b=UfsxE5vGmBe0BY9iCtSWcUB9yzs0sfYZW0cLuNGCXnc/0ay4DtYdYb0GHnqVfwotxuEC3N tNOeaoraQwOvzP71U76TLmuMDr0zho00gBx2KwkP8AA6+nOwS43QhbBQ46AR7ZloruHOAp tV3E6FbevbolnizymtnrTCUU8hd1/8I= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf13.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxrrMaTwdnxrsRAA--.26960S3; Thu, 10 Oct 2024 11:50:50 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMDx7tUZTwdnFP8hAA--.52915S3; Thu, 10 Oct 2024 11:50:49 +0800 (CST) From: Bibo Mao To: Huacai Chen , Andrey Ryabinin , Andrew Morton Cc: David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 1/4] LoongArch: Set pte entry with PAGE_GLOBAL for kernel space Date: Thu, 10 Oct 2024 11:50:45 +0800 Message-Id: <20241010035048.3422527-2-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241010035048.3422527-1-maobibo@loongson.cn> References: <20241010035048.3422527-1-maobibo@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: qMiowMDx7tUZTwdnFP8hAA--.52915S3 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 30E8820002 X-Stat-Signature: mf8cafz1hm7mbu33au8dgpcukfs9rbm7 X-Rspam-User: X-HE-Tag: 1728532253-423611 X-HE-Meta: U2FsdGVkX18sf8tzWetFN8qR2RbjnGUEiIG8ZLrW9wk7TkXvIn7fh9TuLSEyx57TTdBkpUOfVYm63/dEKY5sNWUrIjwm4ye5gEwhhNvxekMVdVZgu3UZ4+HZAtVSEpldiah8pGYR/Bik6IyJA8xHafsB04WhgamwPaiv97tYqdhbty1FSjYU8f1ghGtCE0hkZl2UjREVDcjzE5PJKgx/06mmKRtxtrASyM3fENGEdo4jxZVjheO+zCh0AV8jyOI8urxC8SIxrdZO5A1+jkVei7JExVWNDOc+4RoXQNy3Ws1950y9Hb2czEyQL6v7iSKGLkEt/KQhei1uC4Jixq8o0aqYpUJ9bTB0wphgYJHjwUfmnl81+GcqXCcrVVu2Viky/NbPNrk8Dd+CBqDJqqN0PExKl6plZkYAeKVCLZhW8bOFooevXKg6YF9dvacWs1Xevzh7bEX3oxb3xgMn2kASDrUYOoH0NCsa1QlTr+LkXSAWy7DiNvf3aEqcAzTUjXDiMvV+4Q3nsNnrZ6VNMM9Gd52ZeCn6Eym7aJvAbHVl57SOuPIneXkPrKNFMuNZsQJ4fTzgKiYra20bOXfNxHhmPGdAL/uRxXC+kkcn+BbY8HJ7n7Vd/6yhd/PeksLmlU1Y8rqFqake3DZfQRaACdcK6CUNq7ctD3VurXa+T1E0umSDmuGg3Rg7st/Wkt+7BjqR8YqUyYg/Ar5Yh/KkQalCtTyGoVBJM0sr8WGBTof2zSuyEvuHzZgFTFyGOsmgz59em8u6FQmS7wkD0ophhNMf+Vdt4yBkvxpGkltzxtkyW6VlnVWBtqRJZ5Ctuig6keUwzw4MBcstp6xMKpXZgwSf8dtCX/jKO8tZnS6rH8lIXX6CqRu2G7TGiJTsssZtqWKJhkq7RuqHlYGXxqj4DOlXhWoN7WOjq6g80fHXH2ZXSkkLNQSOOjQllxR+BFrmwXuzP9zWozyEwRVKKGMcy+v gzaFnMwl U2+PHPCO0hKzl9jpa3pfrP767/NXagpgiPTpPjZo2cAGc19HibnXzd41244f+fj3XcF0i3XdEP18NcnvmbUHMGCRxLISogUy36kJoQOrKf6PNgadCWfnyMNAmIwgA265bKjk1+lDxx4shqulgmAje9N/ZeA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Unlike general architectures, there are two pages for one TLB entry on LoongArch system. For kernel space, it requires both two pte entries with PAGE_GLOBAL set, else HW treats it as non-global tlb, there will be potential problems if tlb entry for kernel space is not global. Such as fail to flush kernel tlb with function local_flush_tlb_kernel_range() which only flush tlb with global bit. Here function kernel_pte_init() is added, it can be used to init pte table when it is created, so the default inital pte is PAGE_GLOBAL rather than zero at beginning. Kernel space areas includes fixmap, percpu, vmalloc and kasan areas set default pte entry with PAGE_GLOBAL set. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/pgalloc.h | 13 +++++++++++++ arch/loongarch/include/asm/pgtable.h | 1 + arch/loongarch/mm/init.c | 4 +++- arch/loongarch/mm/kasan_init.c | 4 +++- arch/loongarch/mm/pgtable.c | 22 ++++++++++++++++++++++ 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h index 4e2d6b7ca2ee..b2698c03dc2c 100644 --- a/arch/loongarch/include/asm/pgalloc.h +++ b/arch/loongarch/include/asm/pgalloc.h @@ -10,8 +10,21 @@ #define __HAVE_ARCH_PMD_ALLOC_ONE #define __HAVE_ARCH_PUD_ALLOC_ONE +#define __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL #include +static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) +{ + pte_t *pte; + + pte = (pte_t *) __get_free_page(GFP_KERNEL); + if (!pte) + return NULL; + + kernel_pte_init(pte); + return pte; +} + static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte) { diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index 9965f52ef65b..22e3a8f96213 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -269,6 +269,7 @@ extern void set_pmd_at(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pm extern void pgd_init(void *addr); extern void pud_init(void *addr); extern void pmd_init(void *addr); +extern void kernel_pte_init(void *addr); /* * Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index 8a87a482c8f4..9f26e933a8a3 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -198,9 +198,11 @@ pte_t * __init populate_kernel_pte(unsigned long addr) if (!pmd_present(pmdp_get(pmd))) { pte_t *pte; - pte = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + pte = memblock_alloc_raw(PAGE_SIZE, PAGE_SIZE); if (!pte) panic("%s: Failed to allocate memory\n", __func__); + + kernel_pte_init(pte); pmd_populate_kernel(&init_mm, pmd, pte); } diff --git a/arch/loongarch/mm/kasan_init.c b/arch/loongarch/mm/kasan_init.c index 427d6b1aec09..34988573b0d5 100644 --- a/arch/loongarch/mm/kasan_init.c +++ b/arch/loongarch/mm/kasan_init.c @@ -152,6 +152,8 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, phys_addr_t page_phys = early ? __pa_symbol(kasan_early_shadow_page) : kasan_alloc_zeroed_page(node); + if (!early) + kernel_pte_init(__va(page_phys)); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && __pte_none(early, ptep_get(ptep))); @@ -287,7 +289,7 @@ void __init kasan_init(void) set_pte(&kasan_early_shadow_pte[i], pfn_pte(__phys_to_pfn(__pa_symbol(kasan_early_shadow_page)), PAGE_KERNEL_RO)); - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + kernel_pte_init(kasan_early_shadow_page); csr_write64(__pa_symbol(swapper_pg_dir), LOONGARCH_CSR_PGDH); local_flush_tlb_all(); diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgtable.c index eb6a29b491a7..228ffc1db0a3 100644 --- a/arch/loongarch/mm/pgtable.c +++ b/arch/loongarch/mm/pgtable.c @@ -38,6 +38,28 @@ pgd_t *pgd_alloc(struct mm_struct *mm) } EXPORT_SYMBOL_GPL(pgd_alloc); +void kernel_pte_init(void *addr) +{ + unsigned long *p, *end; + unsigned long entry; + + entry = (unsigned long)_PAGE_GLOBAL; + p = (unsigned long *)addr; + end = p + PTRS_PER_PTE; + + do { + p[0] = entry; + p[1] = entry; + p[2] = entry; + p[3] = entry; + p[4] = entry; + p += 8; + p[-3] = entry; + p[-2] = entry; + p[-1] = entry; + } while (p != end); +} + void pgd_init(void *addr) { unsigned long *p, *end; From patchwork Thu Oct 10 03:50:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bibo mao X-Patchwork-Id: 13829443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68EB6CEDDBB for ; Thu, 10 Oct 2024 03:50:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C40626B0085; Wed, 9 Oct 2024 23:50:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC8E56B0088; Wed, 9 Oct 2024 23:50:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A69A06B0089; Wed, 9 Oct 2024 23:50:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 889C76B0085 for ; Wed, 9 Oct 2024 23:50:56 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 41B05121865 for ; Thu, 10 Oct 2024 03:50:53 +0000 (UTC) X-FDA: 82656316470.07.BB27BE9 Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by imf16.hostedemail.com (Postfix) with ESMTP id 72B79180008 for ; Thu, 10 Oct 2024 03:50:52 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728532118; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ojkToVyZt+6AV6/9LXFFZWf07ErRU8/WaYD6ufYZxvY=; b=ANIO0Z1IW3PW9jLbdtODAOLNUb5Dddj7NnLZoEC6P75itxeuRodUAzeO2jOmsrbtsoQjpi AjGaXs+GdCIYvLLM9T32LkAopUp6Zq1PuIT+WL2z1feFydi4Q85Yvo+G32R4Fk3mSAwfyV e2uTKk80U/d5FeMEKte7WOQbHnwu0y8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728532118; a=rsa-sha256; cv=none; b=vkk4zR7P3k3df2Qq6yt5J2PO6llKFIMJM17JEHG7HS/1yNVtdWDUqTYLn3wsWsa0YwIUvf HNfNGVCMKsSoTfOwQ4Tg8Sgy7CRKagPaEzcR8vVnVawu6USxZEED2bA3ZvDGSvtL4ONwWF EmWtKAaBUh234vbp4ZBTSkVIg8J5Y6M= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8AxCGoaTwdny7sRAA--.25768S3; Thu, 10 Oct 2024 11:50:50 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMDx7tUZTwdnFP8hAA--.52915S4; Thu, 10 Oct 2024 11:50:50 +0800 (CST) From: Bibo Mao To: Huacai Chen , Andrey Ryabinin , Andrew Morton Cc: David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 2/4] mm/sparse-vmemmap: set pte_init when vmemmap is created Date: Thu, 10 Oct 2024 11:50:46 +0800 Message-Id: <20241010035048.3422527-3-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241010035048.3422527-1-maobibo@loongson.cn> References: <20241010035048.3422527-1-maobibo@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: qMiowMDx7tUZTwdnFP8hAA--.52915S4 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== X-Rspamd-Queue-Id: 72B79180008 X-Stat-Signature: nikabxnp5zohm4wx7mumnqqdegizbgqm X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728532252-130832 X-HE-Meta: U2FsdGVkX1/BoxLoKLvwilMuhy937IeemzOy94z3S62/A04cKkEnEnB3PAL1x2GmMGjKTRD1FArE5RPbFcL9s9X1wY+Zs83IjEfFoGZD2+TkysoZ4iGL3IeR6dCzRMSZ4atdwhSGYCTcn6xlnXQvW7hGIkTcPvlsbQANxM/C+1vFOJqj/bvwEgFT3Uhn3KYogMxCb6+fhm1mO766J75tziyFP2naxN7U8jFVYUkCT9FlY5psdVxEVmq/1G4oirP9QOMadp5/a23w7l28uRldZUeiJhNGz1x2ebqc+tufq6GHysyUyP9lhWHJfz2CMJyhz/jjWjFujVLcI4PuN/K9bF8Kj1PW9DEXKFR+0d7v/C7gt/jWxbreN/u55WLuWNyhuuIwS5Z7tci2dPkRGzYBDmH6xc7lu06mayjif1RgjoXsERvMFnRDMo60Cum61Y1Xd4RKdMnM5b729uo9ENYxkT/JUt2OqadhT+j2QVfjEzk/GU4Gumb75ggiBregw0iKPiOiiY6kz+xsU4YiKtrpgiqShFBlvaQ/4gcEGw0OtxP/34zpceCRzIjRiHLSb3Yb+Ia+4OxCbxjUoFwV8B/KcquounBZSwf933hucKwaIUenFGSiiy3IHYnyeOagV4vMiNP0OpfCiOxrO/tXkT07xMt0smulM4eIhWGIovgGQUp7WsnZUVFpYoXYo4x+XBP28fSZvDQGajD5f4r/j0gex2O5AgcpSDxsObEBreMydhgRMZWF3tzKpFMoTxhufDPhGOA2Kj/v2nkafmzcGlRc66/JVOGPS89dPc8PkKBVp2favknsQyiGHRbfZRAvTrUc3AcmCbam++nMmtoXDiBU5FQNuwXZ+wZJ4hQx7/C53zEXdJAw5ltvY5ng4hXSf8HYedpwy+d+IQTSIcOVM/zypujoeSADLq5vYiAq/YgG/PBYMtAXBPsNSf+h+GnX1HdTayQcu06dXxJ+q02iXN3 xmaxM+nh IwCzosGdTJF2G/pe8X2qJgBlsmpv55+yVp+itGJUmgJrdPS1tD5eyyPezu044am4dSIJBnfjAUlQEqX62xn7t7fDq8m6O8N/7amyipnZAnwxqqmbHJ9G26nYvfQpsVN8od6RqfNnd3dukjIJN3yIiz/kMvA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Like pmd_init(), a weak function kernel_pte_init() is added and it is only effective on LoongArch system. When pte table is created for vmemmap kernel space, function kernel_pte_init() is called here. It has no any effective on other architectures. Signed-off-by: Bibo Mao --- mm/sparse-vmemmap.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index edcc7a6b0f6f..c0388b2e959d 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -184,6 +184,10 @@ static void * __meminit vmemmap_alloc_block_zero(unsigned long size, int node) return p; } +void __weak __meminit kernel_pte_init(void *addr) +{ +} + pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node) { pmd_t *pmd = pmd_offset(pud, addr); @@ -191,6 +195,7 @@ pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node) void *p = vmemmap_alloc_block_zero(PAGE_SIZE, node); if (!p) return NULL; + kernel_pte_init(p); pmd_populate_kernel(&init_mm, pmd, p); } return pmd; From patchwork Thu Oct 10 03:50:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bibo mao X-Patchwork-Id: 13829445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA2C2CEDDBB for ; Thu, 10 Oct 2024 03:51:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2AF76B0093; Wed, 9 Oct 2024 23:50:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC0346B0092; Wed, 9 Oct 2024 23:50:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D364C6B0089; Wed, 9 Oct 2024 23:50:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B67416B0089 for ; Wed, 9 Oct 2024 23:50:57 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 63EDE1A1A5C for ; Thu, 10 Oct 2024 03:50:52 +0000 (UTC) X-FDA: 82656316554.16.662458F Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by imf05.hostedemail.com (Postfix) with ESMTP id ECB2A100003 for ; Thu, 10 Oct 2024 03:50:53 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728532074; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Iw/wAbEu0SHxzXukkARToGn17qUMEhn7+bxpypLMb0=; b=Qme2dgdr/C32zTOaPROLZUYJKGYLjWdS0Bw9BLfgOn3nM47Hn8RzvX2xMbt29gmZ/+ANZM ulQEeSB25VfwMYSdlsDhicRTOfq3x6/5Wber5hhA915laxKyfEuMlBAH7zjqfKATM3u/Nt pYSNOEflHWd1xT6fINt0RfdXnxItt6I= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728532074; a=rsa-sha256; cv=none; b=7XT49707xYoqb+mKKQqN+2d+vtdvZC8wbwmCwe7fIXW5P7jYB0PvLZURCnMdVl6DWyd1AA eodiO+BrGXvoPNUlHB18vilydKNM3bw+WDdpzgqVtIMdWKDcUBgrWwc2AxZ/byEIZRh84r 6hsFp7RCd6U9LFGTR2vRvEgZK6IhSO0= Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxrrMbTwdn0LsRAA--.26964S3; Thu, 10 Oct 2024 11:50:51 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMDx7tUZTwdnFP8hAA--.52915S5; Thu, 10 Oct 2024 11:50:50 +0800 (CST) From: Bibo Mao To: Huacai Chen , Andrey Ryabinin , Andrew Morton Cc: David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 3/4] LoongArch: Add barrier between set_pte and memory access Date: Thu, 10 Oct 2024 11:50:47 +0800 Message-Id: <20241010035048.3422527-4-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241010035048.3422527-1-maobibo@loongson.cn> References: <20241010035048.3422527-1-maobibo@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: qMiowMDx7tUZTwdnFP8hAA--.52915S5 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: ECB2A100003 X-Stat-Signature: z7yripc9susp885asupoft9yj9dtywh1 X-Rspam-User: X-HE-Tag: 1728532253-630459 X-HE-Meta: U2FsdGVkX1/+6V9QKxrP1RExrRzuIGosgHFUVPEDY+Ylc2pPxWzyACOwAvqvaoKpd8SR4GCeabFv/MoK2XDNyefhLoHHKzJ+Gi4bU5diKDmAxfTEE6lYbF3Z8OcTzsJ25oRMMo46A2P4/2U404dLwSDqTV5e0Tfn2sY2k5fBL79tIMez/AKBOI4gcz4czwzsFZ+RLdtzAdgVA4DT0N7T96qe4aS29l4zqT0V4Tmvsv/X5OoA/hqlcJln7YzR/50NIwXf1/UWIBzCuqifat+QB/ppuLMeU5SKWiBiXQHxkcLVL4oAJ6nPthWWSizNLjhvuAeEbwQQ89obDpAIseMMzD69iUYbyP+MyuGeHlVFqUt9SeEpuJRb+a4r8PoDTqHPsBBeUOxO8QaBUg33urhk5DXFSlGeyTTSGpIMysG//YGTvVs3ilF9kniav6QJOThgVtMDs23fKAG7pirBCxvA5TXMdEuSi6lIEbtTtMzF7XvBTbhZLvMZTEXfUBHgSbiDKkvPfBwWupTjHM6itzI8/KIodEEKeq28SzdkmOMGUBUuID/0RB3AviASeoeydjLTF8VdNgomF4gE8BJtJheIkIkcV+nYcL6ZiCsdRQOADhtEy2vEVXxvUUmERKkxOLL5ggdQi3bVC6uI8+UzB0qusQ5NXtPgkyPfGCPQgjGTuotBVF5vQZZ+4NwSmwxuU8scRMAWDa6SIEP8tCl8uky7VP+tTq8pH4upqpAXkGhNQqshKbezm8WNUj1XPIllEqDWYrX8k+EBt3brCEVOEb+AtWTRjx5IfKXxXJ5fwVgmQZb+SnhtXNIzBCl94qCif+SstCuFewiYLE5xUquYJiRA0MzoHW90ATx8Qr2K12PobicAF2NDbvzCEpqJKC+7w9bC6BDQvNJWjXuTpbBrbKvyqmpjBGR2U8qC25BR7snr+fKa9ij9AS0qbKTarIjxhoZFzSaLC6jvUQL+kJVoJQ8 HBGYCRnG aRYreGDflWO7h3pyOFKGBHVCBu1bm9Capbn69Gi84RtrE54oWWSpiN1PKWH6jdxiposjuljVqlzAphokauNwT0MZ6xhhH7HlSV2G4UI9ivnTPnkg2HOxcZ5uul/0mXDCMsNWN0kiJkLFNhDvuyv823ALBXg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: It is possible to return a spurious fault if memory is accessed right after the pte is set. For user address space, pte is set in kernel space and memory is accessed in user space, there is long time for synchronization, no barrier needed. However for kernel address space, it is possible that memory is accessed right after the pte is set. Here flush_cache_vmap/flush_cache_vmap_early is used for synchronization. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/cacheflush.h | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/include/asm/cacheflush.h b/arch/loongarch/include/asm/cacheflush.h index f8754d08a31a..53be231319ef 100644 --- a/arch/loongarch/include/asm/cacheflush.h +++ b/arch/loongarch/include/asm/cacheflush.h @@ -42,12 +42,24 @@ void local_flush_icache_range(unsigned long start, unsigned long end); #define flush_cache_dup_mm(mm) do { } while (0) #define flush_cache_range(vma, start, end) do { } while (0) #define flush_cache_page(vma, vmaddr, pfn) do { } while (0) -#define flush_cache_vmap(start, end) do { } while (0) #define flush_cache_vunmap(start, end) do { } while (0) #define flush_icache_user_page(vma, page, addr, len) do { } while (0) #define flush_dcache_mmap_lock(mapping) do { } while (0) #define flush_dcache_mmap_unlock(mapping) do { } while (0) +/* + * It is possible for a kernel virtual mapping access to return a spurious + * fault if it's accessed right after the pte is set. The page fault handler + * does not expect this type of fault. flush_cache_vmap is not exactly the + * right place to put this, but it seems to work well enough. + */ +static inline void flush_cache_vmap(unsigned long start, unsigned long end) +{ + smp_mb(); +} +#define flush_cache_vmap flush_cache_vmap +#define flush_cache_vmap_early flush_cache_vmap + #define cache_op(op, addr) \ __asm__ __volatile__( \ " cacop %0, %1 \n" \ From patchwork Thu Oct 10 03:50:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bibo mao X-Patchwork-Id: 13829447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BABBDCEDDBB for ; Thu, 10 Oct 2024 03:51:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D52E26B008A; Wed, 9 Oct 2024 23:50:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CDCE56B008C; Wed, 9 Oct 2024 23:50:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A90A36B0092; Wed, 9 Oct 2024 23:50:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 822836B008A for ; Wed, 9 Oct 2024 23:50:58 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 05922C1A28 for ; Thu, 10 Oct 2024 03:50:55 +0000 (UTC) X-FDA: 82656316596.08.82DB07B Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by imf28.hostedemail.com (Postfix) with ESMTP id 6AD79C0004 for ; Thu, 10 Oct 2024 03:50:55 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728532119; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GhqZgEmhO01NNuYz9LQ6ht12zOqcVK4S3QCK4DG4Y1Y=; b=18V1iHSYNqOBDJThb4CVflgg92gKJglDLJnLI4yPYEuR7HJb184MMBtYinYVWlNE1cjqrS UVBfbLf1BNCGc5VHE7/IM4ts0m2D0cKL7kMumzB5Z1a+bdRNNcEm3FFME8RVQoNdcdQLbn YdQVzf8moekPcHUszWyrKcrf60OR1hk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728532119; a=rsa-sha256; cv=none; b=eKcYoyzjmUgwccJXP8IrZy6C4KIjEk6M0QahvjT7k7swn8LbTR1v970dkQDCZKbTuXhKUF lqK1MTmfDQKW69pdngYjEvXVXhzHBVXYb/6BKgskLPqi+z7DnRQv4P6O5Q7pEAInHEyZJM vSOckEaONO37r6mi87cmuQfAtrJZQWI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none Received: from loongson.cn (unknown [10.2.5.213]) by gateway (Coremail) with SMTP id _____8CxbWsbTwdn4bsRAA--.25946S3; Thu, 10 Oct 2024 11:50:51 +0800 (CST) Received: from localhost.localdomain (unknown [10.2.5.213]) by front1 (Coremail) with SMTP id qMiowMDx7tUZTwdnFP8hAA--.52915S6; Thu, 10 Oct 2024 11:50:51 +0800 (CST) From: Bibo Mao To: Huacai Chen , Andrey Ryabinin , Andrew Morton Cc: David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 4/4] LoongArch: Use atomic operation with set_pte and pte_clear function Date: Thu, 10 Oct 2024 11:50:48 +0800 Message-Id: <20241010035048.3422527-5-maobibo@loongson.cn> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20241010035048.3422527-1-maobibo@loongson.cn> References: <20241010035048.3422527-1-maobibo@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: qMiowMDx7tUZTwdnFP8hAA--.52915S6 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7 ZEXasCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnUUvcSsGvfC2Kfnx nUUI43ZEXa7xR_UUUUUUUUU== X-Stat-Signature: ms8z7ksz357nnzk4aq3kk8nch8osu883 X-Rspamd-Queue-Id: 6AD79C0004 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728532255-516521 X-HE-Meta: U2FsdGVkX1/aOSbm5HvRUz4vv0Vgmc161UXWp9BlVH+5E+pYc4DeQcDI+DB4v4kICJic6YzuXJhadWP4bB9/SzViHsadJEK+Gl9lWllR5M2iNuKZ/EegzFvWexwPYhu5fvLRDkTh4AIMS+tdnMu9Pbg8RN1gGOkwLUbQb1moYHEEVUmqnR9HR9Z5Zz1AoKlAZ5FTkazkbbHHrJAApH4ufAnTXy8hiBRAnAz0J4C2cZKESg/hwtdKTUYEzuTMxJi/Q+DpiEi3GdB/RR55UeW+h96/a3DZFpdu4HPp4xpDqXaTwN7FlnPpxcUvkvUDbStO92pS0kFD1ws4BmCK9JMXnh1JGUmYIBo8vE0//03W9AMlQE0mG0JTo5Ow2UwSjiVU5y3Ji6TNKsNfIfcSwhDWsBw2N+s/9+1n3Y6FbnuigIQIF7hOk4Rj7JwLMf93czgeA5nfPx3NVlfFXrJUTeemp05ut0/eR3sqjWXPVe20zlbSJ1ATYXx74cz5AETQXUrj5JoSRhNCFGyhabwgv70iTuX+4/mzZfxdpz+VeDYbJkt0bki4KdLYgGAbf/kXocVIEB/jjgQKqrH4CB1Fm8qAHx8b5Q8+ki1pALpJ7I7+KhOE8DWTPpFzapD/aZtoPHuikFco7Js/9JM+KSX7bxaRnVDVtp15YvrAcTqoC5UUsWYdlzZKoil1tjPf0mZ4C3ZFnWVIcP/JYwhF+MO9Lfq0i+LfDi3DM3pj7EZvp+ip2RBQcX8CxeX1z+39+uzltKO/Sjn7U7CS/K7wwPvBxLuHbIElQyZnacmKsnKIpuGQBASM61Euy/EuFkILhsohmefGZsA9PonzZfGZRWT6oDaKbKhe7bUsxQhV8gPbxSOt5R2XLIMlRon+HzUO2hHbjLIvu+XQMAtKCNRskf4SaCG0bMTiZd4LoOvz/p9RRVofBq9+YoW4xxWA7Fj8d23ZdzJR/a+1j0+5opFdF0xrsow bW8PMQXz EFRYADFbEzbhOs5O3D3xTdkfvrLSSr9bf01XiKQX/ntdXHi2UJKGKS4XJdwueKaBGaaudxZCzg8VoZSLxbiTzOK00aqPmeAk0FSj/rjj6ytmE1RYtSiuhOpZczVLBE+E1UsmA2GJrhklMAQKcBjz5gGvelQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For kernel space area on LoongArch system, both two consecutive page table entries should be enabled with PAGE_GLOBAL bit. So with function set_pte() and pte_clear(), pte buddy entry is checked and set besides its own pte entry. However it is not atomic operation to set both two pte entries, there is problem with test_vmalloc test case. With previous patch, all page table entries are set with PAGE_GLOBAL bit at beginning. Only its own pte entry need update with function set_pte() and pte_clear(), nothing to do with buddy pte entry. Signed-off-by: Bibo Mao --- arch/loongarch/include/asm/pgtable.h | 44 ++++++++++------------------ 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index 22e3a8f96213..4be3f0dbecda 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -325,40 +325,26 @@ extern void paging_init(void); static inline void set_pte(pte_t *ptep, pte_t pteval) { WRITE_ONCE(*ptep, pteval); +} - if (pte_val(pteval) & _PAGE_GLOBAL) { - pte_t *buddy = ptep_buddy(ptep); - /* - * Make sure the buddy is global too (if it's !none, - * it better already be global) - */ - if (pte_none(ptep_get(buddy))) { -#ifdef CONFIG_SMP - /* - * For SMP, multiple CPUs can race, so we need - * to do this atomically. - */ - __asm__ __volatile__( - __AMOR "$zero, %[global], %[buddy] \n" - : [buddy] "+ZB" (buddy->pte) - : [global] "r" (_PAGE_GLOBAL) - : "memory"); - - DBAR(0b11000); /* o_wrw = 0b11000 */ -#else /* !CONFIG_SMP */ - WRITE_ONCE(*buddy, __pte(pte_val(ptep_get(buddy)) | _PAGE_GLOBAL)); -#endif /* CONFIG_SMP */ - } - } +static inline unsigned long __ptep_get_and_clear(pte_t *ptep) +{ + return atomic64_fetch_and(_PAGE_GLOBAL, (atomic64_t *)&pte_val(*ptep)); } static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { - /* Preserve global status for the pair */ - if (pte_val(ptep_get(ptep_buddy(ptep))) & _PAGE_GLOBAL) - set_pte(ptep, __pte(_PAGE_GLOBAL)); - else - set_pte(ptep, __pte(0)); + __ptep_get_and_clear(ptep); +} + +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +static inline pte_t ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + unsigned long val; + + val = __ptep_get_and_clear(ptep); + return __pte(val); } #define PGD_T_LOG2 (__builtin_ffs(sizeof(pgd_t)) - 1)