From patchwork Fri Mar 1 09:14:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E42C0C5475B for ; Fri, 1 Mar 2024 09:16:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F2D66B0095; Fri, 1 Mar 2024 04:16:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 77DF56B0096; Fri, 1 Mar 2024 04:16:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F5986B0098; Fri, 1 Mar 2024 04:16:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 4926E6B0095 for ; Fri, 1 Mar 2024 04:16:17 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0F0FC121294 for ; Fri, 1 Mar 2024 09:16:17 +0000 (UTC) X-FDA: 81847913994.29.8F4BAB7 Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) by imf09.hostedemail.com (Postfix) with ESMTP id 3CC4314000A for ; Fri, 1 Mar 2024 09:16:14 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=2JUr1Ski; spf=pass (imf09.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.171 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284575; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3HmYMQ4fCaWre3Z0n4nCWU3axYoW5M2HfRVImcv9VqU=; b=yNtWrpZb5gMu/cAIW5NRzE6NubR85hLUiuOCbs1SXq0kY1vnFtIDbvQkgk0Z+sZh/E/o2M h6ICewbzmHSFwRnZfbr/+8YT7dXUQ404GwSKNSsoDYFVOXkbmGfx2RPMy/IRSzIRLtqJqv VlF0uutGW1bwwpDUp6ycG0XL9A1RPmc= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=2JUr1Ski; spf=pass (imf09.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.171 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284575; a=rsa-sha256; cv=none; b=McE7akcuOwAFGRcPa6RqndfekRPj64L4bu7kvo/6D5oK1TR1kHZ/jOQMbuw8ao98jND0br y93gV0ahtZdYtT93VX6YTt0DG5/8zp7UhV/Td4ilUGBWH28E39i4Uozl7sfCiw6ENqo32P AuPEKYCF0XxKGzXHh/6ATrcyGrWrYwU= Received: by mail-lj1-f171.google.com with SMTP id 38308e7fff4ca-2d29111272eso22395181fa.0 for ; Fri, 01 Mar 2024 01:16:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284573; x=1709889373; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3HmYMQ4fCaWre3Z0n4nCWU3axYoW5M2HfRVImcv9VqU=; b=2JUr1SkiGTfVkCbDNHYLcTAr/+5ed5TVgPcotX6uTkW3qTrwA8u82OWGg2Msegw9Q2 tRELLX76PdkXuWImvx20zMB3sS1DLTEhbhKxrHaJCl9FL3dqFhljQBR/fqbh+5p2Y+VO r/4zA+WMxjkNB+zQqr8/nv9qfrTqTFJDMkKLB1C3dxFoO89HmMVSYyri0eScmCcnvtKP 7gUOOirDzjEk8ogLiVJVKb3vyptr/wfOSrb7SDDiWjsVIXETaZQXnPuEmuMPpt26B0L3 E+1MFZ6dI7yCkuFmi5PzivIi7nbADF03WpL09V0E9an3pX6yVeRT7tP0l2AQEo2hHk1M dzHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284573; x=1709889373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3HmYMQ4fCaWre3Z0n4nCWU3axYoW5M2HfRVImcv9VqU=; b=tvtu/4ffgyqQrLl0VPSi4xKiHF8iwHMXk6LMw/q/L4vS6asbUyqrr+HwiiR33h2cMO vZItfCgl1gPVRyxmcECYFzO9WoOo0NuhVskMadKJFUoOTXUAkaF4wI7nSfzaqXO9u6GU cQjAvQvGWfsMFW71CpsPCrjMlYHFBIbL1K9IPMIoET8NOd02bm84S0RnjFMkwej9Hiba 4WGxrs2CIrUqblC5mlcyxg2uvCe7qQMHvujWbzQDkuf6kQGu+Rdye+phHSxvceFpXds8 NZ7jEcCWIwx/mZjbbQ2NsjFFcb/BuQK2GK+knGYfu91uEVxC+mCUceXNgW0fPKqNVARe mirQ== X-Forwarded-Encrypted: i=1; AJvYcCWnpK5Pt5vF+QR/OcXeOq10QjLG+FCKeeHfBQe/o636Y5acnlV4cJTQTf2cZyZO+xr2zEQBx7HiKVOL6Uyurh9+Oxk= X-Gm-Message-State: AOJu0Ywjry6PtGzlk2Gr3MNa2RF0zKuYjWFH2ORE/bsygBDbE0S0pfKD hD291/n/4F37aR3wUQbeY4Jf8GdYUExzWKBPHCsTFt+jbsDeh6zvogKyoGDmy4E= X-Google-Smtp-Source: AGHT+IHQVU7ZdAsKvA/XTSgIwwIMQlWsmY+5DOvluKjOjYHWuq3/2JdBJ53cUSY5g2ynNIYVyWu0Lg== X-Received: by 2002:a05:651c:10cf:b0:2d2:eb8c:b3a6 with SMTP id l15-20020a05651c10cf00b002d2eb8cb3a6mr619531ljn.40.1709284573125; Fri, 01 Mar 2024 01:16:13 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id m21-20020a7bcb95000000b00410b0ce91b1sm7777825wmi.25.2024.03.01.01.16.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:16:12 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 1/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code Date: Fri, 1 Mar 2024 10:14:47 +0100 Message-Id: <20240301091455.246686-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3CC4314000A X-Rspam-User: X-Stat-Signature: rzdsjiset3mz5i6xjhmift8fewkn4epr X-Rspamd-Server: rspam01 X-HE-Tag: 1709284574-602519 X-HE-Meta: U2FsdGVkX19M7b9SgDxHuAdvxV9IruXYshrBWFJPQxI6hOyk1twiY4h8+uB3ay0CyHuua8bwQm1FwztwookViR4rSFbDHZxV39nhg2+cQ56nLzoxmHbU5+UvM4id5hEcvrS7I87hYAdkhkzVrvS2cpDivxPe9IoVKKfsxUcpsh1yEBlzuPzddXc5fbGVdgJ4V1+d4jvEzJqwTiwvcWiryfHS4d9bbz7Vh7kd66DrMBFywdwW9X67RjXsqZDhYKRkZNE2AdZ08XLoncw7gqpA0Rb9utNpdknmgE0UaOyEYxgbQeVYub9UO+UL16qO51SORSpTE9XisvHBLwQ16iQi4JSHdd5GQ0hnDQBNzLegkKX9fmiwe193oaaoBs4PpUB2t+fHyUStMK24Q9OApCTba3w5GzXByDGh7eMgWBRUX4a5hjOlRygSY/bNFiT7/yxkpnnC0XCIEXQ6V4tOrppupCHx3oQkftyOk+DsmwmvKv3wR4ij8FCTYSzEUZ6ZQ3uMzxmWGCvCI0JG3PrMi0wHpU0M7sDKYKZV/zy9wv4gYOddIxekVayr002F7Mst6VRM1bmjh7xpXd43ECNE0ic1w5HFImnKuFAbaHYcAv4p8j3+jpWfXW1jD6NcBzcaUT7NUi46vDxFAXsBY7sYVqtCwWWcAW2djW7j6e+fvO8Bf0SgFAhzDnrWC5eoqJSgxfEzW6Yi3yC5W64hclCDIl4n5Kf7m/yVKdmOREhZwPi3pPmBd7gKnYfvAke2EBMP7kmoWr0b0JhmhA1CRpmQSYwP/aNystgw7fEpQjiZmUBYlOjNsCVqV3u27OfQhB9BDicHBDzjwiO4aIkh5N57+b3fUB2MDO8RjtLioJ0wy5X3E3rQkfUpxeM6EUs+2kYX6veF4V2SPDmsghVdF03Xh6KeDOkTt+bf90nZwkkVlW6sUi35fPJovoO60qYkFqnW14SJMJifbZzApSOf+dEwfcE DYRl6IIq a2Sh5iNKM5m31m9D8BIpy+PiqVrjUh35Jv+C2P0XfBDy7VbsgdxzzVLalgmEqowc2EXcKYLrmOhSC7+sWTOBe9UTSF+IkWUmUYVc779uVcnQBnL3J/KH1uQ4dcF9LyLGFQnnqa+7DAlfRAzJkVzxmHSvXgvvRx4f5Q9Eu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The core mm code expects to be able to extract the pfn from a pte. NAPOT mappings work differently since its ptes actually point to the first pfn of the mapping, the other bits being used to encode the size of the mapping. So modify ptep_get() so that it returns a pte value that contains the *real* pfn (which is then different from what the HW expects) and right before storing the ptes to the page table, reset the pfn LSBs to the size of the mapping. And make sure that all NAPOT mappings are set using set_ptes(). Signed-off-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable-64.h | 11 +++ arch/riscv/include/asm/pgtable.h | 105 ++++++++++++++++++++++++++-- arch/riscv/mm/hugetlbpage.c | 38 +++++----- 3 files changed, 128 insertions(+), 26 deletions(-) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index b42017d76924..a0f76c3071a9 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -106,6 +106,17 @@ enum napot_cont_order { #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) #define napot_pte_num(order) BIT(order) +static inline bool is_napot_order(unsigned int order) +{ + unsigned int napot_order; + + for_each_napot_order(napot_order) + if (order == napot_order) + return true; + + return false; +} + #ifdef CONFIG_RISCV_ISA_SVNAPOT #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CONT_ORDER_BASE)) #else diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 0c94260b5d0c..951f3ceb5529 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -303,6 +303,8 @@ static inline unsigned long pte_napot(pte_t pte) return pte_val(pte) & _PAGE_NAPOT; } +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) + static inline pte_t pte_mknapot(pte_t pte, unsigned int order) { int pos = order - 1 + _PAGE_PFN_SHIFT; @@ -312,6 +314,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigned int order) return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); } +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ +static inline pte_t pte_clear_napot(pte_t pte) +{ + return __pte(pte_val(pte) & ~_PAGE_NAPOT); +} + #else static __always_inline bool has_svnapot(void) { return false; } @@ -321,17 +329,14 @@ static inline unsigned long pte_napot(pte_t pte) return 0; } +#define pte_valid_napot(pte) false + #endif /* CONFIG_RISCV_ISA_SVNAPOT */ /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - unsigned long res = __page_val_to_pfn(pte_val(pte)); - - if (has_svnapot() && pte_napot(pte)) - res = res & (res - 1UL); - - return res; + return __page_val_to_pfn(pte_val(pte)); } #define pte_page(x) pfn_to_page(pte_pfn(x)) @@ -523,9 +528,91 @@ static inline void __set_pte_at(pte_t *ptep, pte_t pteval) set_pte(ptep, pteval); } +#ifdef CONFIG_RISCV_ISA_SVNAPOT +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + pte_t __pte; + + /* We must read the raw value of the pte to get the size of the mapping */ + __pte = READ_ONCE(*ptep); + + if (pgsize) { + if (size >= PGDIR_SIZE) + *pgsize = PGDIR_SIZE; + else if (size >= P4D_SIZE) + *pgsize = P4D_SIZE; + else if (size >= PUD_SIZE) + *pgsize = PUD_SIZE; + else if (size >= PMD_SIZE) + *pgsize = PMD_SIZE; + else + *pgsize = PAGE_SIZE; + } + + /* Make sure __pte is not a swap entry */ + if (pte_valid_napot(__pte)) + return napot_pte_num(napot_cont_order(__pte)); + + return 1; +} +#endif + +static inline pte_t ptep_get(pte_t *ptep) +{ + pte_t pte = READ_ONCE(*ptep); + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + /* + * The pte we load has the N bit set and the size of the mapping in + * the pfn LSBs: keep the N bit and replace the mapping size with + * the *real* pfn since the core mm code expects to find it there. + * The mapping size will be reset just before being written to the + * page table in set_ptes(). + */ + if (unlikely(pte_valid_napot(pte))) { + unsigned int order = napot_cont_order(pte); + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + pte_t *orig_ptep = PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * napot_pte_num(order)); + + pte = __pte((pte_val(pte) & napot_mask) + ((ptep - orig_ptep) << _PAGE_PFN_SHIFT)); + } +#endif + + return pte; +} +#define ptep_get ptep_get + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr) { +#ifdef CONFIG_RISCV_ISA_SVNAPOT + if (unlikely(pte_valid_napot(pteval))) { + unsigned int order = ilog2(nr); + + if (!is_napot_order(order)) { + /* + * Something's weird, we are given a NAPOT pte but the + * size of the mapping is not a known NAPOT mapping + * size, so clear the NAPOT bit and map this without + * NAPOT support: core mm only manipulates pte with the + * real pfn so we know the pte is valid without the N + * bit. + */ + pr_err("Incorrect NAPOT mapping, resetting.\n"); + pteval = pte_clear_napot(pteval); + } else { + /* + * NAPOT ptes that arrive here only have the N bit set + * and their pfn does not contain the mapping size, so + * set that here. + */ + pteval = pte_mknapot(pteval, order); + } + } +#endif + page_table_check_ptes_set(mm, ptep, pteval, nr); for (;;) { @@ -533,6 +620,12 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, if (--nr == 0) break; ptep++; + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + if (unlikely(pte_valid_napot(pteval))) + continue; +#endif + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; } } diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 29c7606414d2..3d84fbc5c572 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -256,8 +256,7 @@ void set_huge_pte_at(struct mm_struct *mm, clear_flush(mm, addr, ptep, pgsize, pte_num); - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); } int huge_ptep_set_access_flags(struct vm_area_struct *vma, @@ -267,16 +266,16 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, int dirty) { struct mm_struct *mm = vma->vm_mm; - unsigned long order; + size_t pgsize; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); + pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + ptep = huge_pte_offset(mm, addr, pte_num * pgsize); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); if (pte_dirty(orig_pte)) @@ -285,8 +284,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (pte_young(orig_pte)) pte = pte_mkyoung(pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, pte); + set_ptes(mm, addr, ptep, pte, pte_num); return true; } @@ -301,7 +299,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, if (!pte_napot(orig_pte)) return ptep_get_and_clear(mm, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(orig_pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig(mm, addr, ptep, pte_num); } @@ -311,24 +309,23 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, pte_t *ptep) { pte_t pte = ptep_get(ptep); - unsigned long order; + size_t pgsize; pte_t orig_pte; - int i, pte_num; + int pte_num; if (!pte_napot(pte)) { ptep_set_wrprotect(mm, addr, ptep); return; } - order = napot_cont_order(pte); - pte_num = napot_pte_num(order); - ptep = huge_pte_offset(mm, addr, napot_cont_size(order)); + pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + ptep = huge_pte_offset(mm, addr, pte_num * pgsize); + orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) - set_pte_at(mm, addr, ptep, orig_pte); + set_ptes(mm, addr, ptep, orig_pte, pte_num); } pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, @@ -341,7 +338,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = napot_pte_num(napot_cont_order(pte)); + pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } @@ -351,6 +348,7 @@ void huge_pte_clear(struct mm_struct *mm, pte_t *ptep, unsigned long sz) { + size_t pgsize; pte_t pte = ptep_get(ptep); int i, pte_num; @@ -359,8 +357,8 @@ void huge_pte_clear(struct mm_struct *mm, return; } - pte_num = napot_pte_num(napot_cont_order(pte)); - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); } From patchwork Fri Mar 1 09:14:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CD4AC5475B for ; Fri, 1 Mar 2024 09:17:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8BCA6B0098; Fri, 1 Mar 2024 04:17:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A14FA6B0099; Fri, 1 Mar 2024 04:17:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BF766B009B; Fri, 1 Mar 2024 04:17:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 752806B0098 for ; Fri, 1 Mar 2024 04:17:17 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 57A6BA190A for ; Fri, 1 Mar 2024 09:17:17 +0000 (UTC) X-FDA: 81847916514.10.15A1801 Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf21.hostedemail.com (Postfix) with ESMTP id 7567D1C001B for ; Fri, 1 Mar 2024 09:17:15 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=HGfdlAVE; spf=pass (imf21.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284635; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=celeezlUFfnCgdabriQL+vY9x87nEUlib9jbsT9nzg0=; b=kXjMICP40G+JJw+nJZMUQPqwJZrrAqg2zwlk10LHGmA/73C/AgIJy85SbSPmZDKfKbf6Kl 09hJHu51Y7U6h4Ztwtb5ED/mBqmZO4kH5UgQCRPBWBnAYykc9HlLIrW4n2MLGBaIBTm3dN 1fSQBTrVYNArgcWLcDjkpTSIUxZpgFg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284635; a=rsa-sha256; cv=none; b=ShJ2HpplMEVvCzpBDxvzlzL2GvWXAeWEe6r34tTaPx3EzrIw0m9gzMN2vVgvr6f/grCRoq Cmj3VBVjiSx3R2P33K8Piyso7KH3KzkgLs6O3h/hwX2X4tJRdGMVDgzEFRMP0waV+07hjw xYxf3w1uYBbxopPSRpv09m0A8f4p1Po= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=HGfdlAVE; spf=pass (imf21.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-412a3371133so13602335e9.2 for ; Fri, 01 Mar 2024 01:17:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284634; x=1709889434; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=celeezlUFfnCgdabriQL+vY9x87nEUlib9jbsT9nzg0=; b=HGfdlAVE1vVjqZs0tqQTqy2MTb8PyUTNhorG8M2l7Mz6z7kZZDUs0tFIz6YqIF/zHB U80oeiGr5NMznF3ehVSE7/v+pKh41vMktkoc8miiR+GaTUgBHwwgpCDzCOGBcQK9U9f8 Cm7Nah9mdPsL36jX7z/kmOVmFFGX4veTvBu4chT7Q3YlVmGT4nW7h3WPDZDgMcYJp7Ss YC3A44ntkQNRcNOkYD1mo0ej51SCwAxV3GvuElwBpg+2Jlm95iOc5yJUVZdDOsHiFlMN TxVOMUOnnlsDFdCyKPJfnlGX3km/AwaGE8KAs0imw72xtHyFD6Viwe6V697ho5TEY2PO /kcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284634; x=1709889434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=celeezlUFfnCgdabriQL+vY9x87nEUlib9jbsT9nzg0=; b=FVTe1fh1TMB3BRA/1c4SqtwRyLako4Li6qNWILT0hQFV6IifUoeX/sVGIyH4YbtwoN 98SBHhsH/KBT1ekTO75aRuVehZd0w5w/Fta9Pb9/hhspS2r0zeCW0QwOyM5U9t0RgVth xo+2NhM+/b7KKBjAhDfQ/B7Cwu0e3dPEWEnJAkazm6LJAc5QUD62HyvCoPEfT+ozHRkt LwMglMryvE1pBhnyA6YzVORkzPqZcR/T5HXd2gTyX3G9IKVvMXU80qSL4ZKWEocyJH7n NvVwgHPU3Kk+0qoWp8WFBrVKjhibOBiPzeGKC4iilxetCtwjhWG5TWuw5DlphO35YAfO QJAA== X-Forwarded-Encrypted: i=1; AJvYcCXFK6anAVe6eC1erCsAKrEIbNIDdQgeP9ImnJcHYtd5vAcVpHzC5naM1FU2vySJJpT/XMw6FIYPN1yo/hxWMHNyQ0A= X-Gm-Message-State: AOJu0YzBe4qRrJ5M6BvAJ0AWDkVaHp6/RfGlBKrz5Sr8wSaoXsYy4zYB G5f0K9i2DKipoPZABKl89ctaXrdrHfefoU9RF4L0YDNia5pDSFJlJCgR+EfUZY+gDCL9LJKRA82 p X-Google-Smtp-Source: AGHT+IFB0pFFxU/+IPg3Lb+DpLxmFSKXoGv4IrPSAZpcxiUzICaAY8iLg+9vVvkiiGxUNIP5Ue4QKg== X-Received: by 2002:a05:600c:3b26:b0:412:c810:fd18 with SMTP id m38-20020a05600c3b2600b00412c810fd18mr995837wms.18.1709284634158; Fri, 01 Mar 2024 01:17:14 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id l33-20020a05600c1d2100b00412ca88537dsm358022wms.0.2024.03.01.01.17.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:17:13 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 2/9] riscv: Safely remove huge_pte_offset() when manipulating NAPOT ptes Date: Fri, 1 Mar 2024 10:14:48 +0100 Message-Id: <20240301091455.246686-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: oqkhpsxyiew7ms67z41wfjskey51ciwt X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7567D1C001B X-Rspam-User: X-HE-Tag: 1709284635-247314 X-HE-Meta: U2FsdGVkX1/8StLVAOTn+Smwmy/wP5n2kizkmPcwdC+e4+dNRSRhB6rUS5RfQZMqPULid+v2w+oyLWSeCAf/mIOqgeYUu1P9Qq7PS7hHgtNFeiYpHvU9QtYjMvsYVEgYt4s6JPvIMTdYkJNap7IReYOuzx3LDs62Hn52N/cE1vzHIgnlVKRG7lfFZZ+/+33pGMCahVVRmr4s3r+q8onVPIuftRVyhFeO1vrFKWIWknI+sXUDm3hYJiMYSCkX7bd6dsHfNMVw4toKBAE3wfbEyns/sMysiGqJaXEEzMkuNGdL27cVy2JF81CxkxH74uINjtpLA+0Yz7ICAVfWEwprn3vzRqPj8AfsE31bqCi6bn+wYY1R2FU0HFR3ipgywU4ZPzQH9wxgH0Ruyr8OEgqtHok+O9sfBcHjjr3FPEDLSXHgm9lQK0ALobFkxkcJODUW/6OdYNsCSWF4CfcshAnTrQ4yWzjgoZN95HwRUM3adpaRsrAjvLvXaxJypmVEnDtSHoQtHLe4mJlrnQvIoJ1HvQzHViI82Mp8Pf7Pl/g5DS+vR2ecyh2nqHFcOOKYQgOx6XwkK2RE6FftI6i3d07cUNhJfwUOwA9jfkiWdEK0NdpTUCpIHSY7nJfCKrE9xgVOaEr5M/sATLvHJjncxKaKYnpPqOKrQTaTLC2gP9xs+LoGfimeS+ymfJSpKl9O8egbJxqdCPovx7HTwV8rM6f26465irn/gNejDxx249v6/7umS56HNk+MojYP1aI7I0DspvfNhPYUzg+oIkaJ9CQTF6sD/uDmcGXjEFoTEpLW3/3r+Jwnuxphn+aP5xVLUDUEfUjAn8Njnhai5Ye69fOzn2BbbNoFXd4Cbwa9boS5pyUX33WbUOOA6mAzvUaO5qt7yFem2NvzpVMXuK13hapL07ILyBK5e28vMBUcObQEFlVkwIFp/JQ/onCHThcSnrNZWQ3b+Ka66nc3//ydtdv XTTNj7xq bbKPu+zAjVGQQ4J7FBZUeZbs82bDEayoF2EQiqZYh5JjZD3W1DMtFg86HYW2NPHrShQSEf4e0tDjS6svqCDgc7cGVK8NT369neBY3ky7gk/7iPmFoCOV84df/R6AK5dELx7dl2+yCqrF6KpD9dD9qlFmQU2NZC1KdJzBhJU41aFxK9j+CRx9Ki/F8844E5bCrKER5RZtpLmQMBTM6d0SA+Zmz7pmjISIto+ZLpiT+ab/2dNArF/UeCTBZnIzQbhA6K+CdrAA9yBUyEaz9cRPaUl60mPoNopWcYeHQWtXpU3RyJE1clZe7lsWmObC+Lk7lU197p3Aems5YrYVUPtxinUxboJWy8Cqmld93Nk+Kv1NfaD59Mv6K2nerggZbVP4OU1QuFFOeCxaeRGc78lwnQGqB1Zbb5eAawUVJplDGsjoSbx6WvfxAwU4ChT+hZgqVpu823ug427rSz1g7sqL/vk21kvDEir37Ji03A66YPxpy16oqAohpmmZVrQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pte_t pointer is expected to point to the first entry of the NAPOT mapping so no need to use huge_pte_offset(), similarly to what is done in arm64. Signed-off-by: Alexandre Ghiti --- arch/riscv/mm/hugetlbpage.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 3d84fbc5c572..2477d20c1497 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -274,7 +274,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return ptep_set_access_flags(vma, addr, ptep, pte, dirty); pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); - ptep = huge_pte_offset(mm, addr, pte_num * pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); @@ -319,10 +318,8 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, } pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); - ptep = huge_pte_offset(mm, addr, pte_num * pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - orig_pte = pte_wrprotect(orig_pte); set_ptes(mm, addr, ptep, orig_pte, pte_num); From patchwork Fri Mar 1 09:14:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C99D5C5475B for ; Fri, 1 Mar 2024 09:18:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3AECB6B009B; Fri, 1 Mar 2024 04:18:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 339036B009C; Fri, 1 Mar 2024 04:18:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D8996B009D; Fri, 1 Mar 2024 04:18:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 094F56B009B for ; Fri, 1 Mar 2024 04:18:19 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AF6691C112B for ; Fri, 1 Mar 2024 09:18:18 +0000 (UTC) X-FDA: 81847919076.22.B744F3A Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) by imf30.hostedemail.com (Postfix) with ESMTP id D4CAF80021 for ; Fri, 1 Mar 2024 09:18:16 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=cBYWqzky; spf=pass (imf30.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284697; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=f1tNyFTcsXe5j/Xz9tdKUfPkAneqDazbYHB6YPwtZWk=; b=tOVqK6iVrH0Azsl4HiBEVTlsF2LlM2zhoV0/G2MwHrwTDxz/b7os2TxeN/nAwE2MSifagf WaXsCnVTkFgyfVYWQiu35Morb7XtsTCgWbjMNeK546lWDZ3CtS0hNKwrVTVJb7eIRB++TC /N9nlJcQo3fuCSuLTuuJNuGEyp9dHVk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284697; a=rsa-sha256; cv=none; b=6lnOUdlbPGXpcP+6s9RoGYnOumDG/2T3VvnZLrmd5TcBg5Dpi9d/gQI327IFSA0zp0HiDK FZSoA4qyB5raPRMWqyJa+SjtIHZ/lDQZkvTD2L5BgDUswpDByNkscPl0ztwiaGmsXXdKxL jQlFBjeb311/8PZ9cPYhc9k7UbWaCrw= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=cBYWqzky; spf=pass (imf30.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2d23114b19dso21770501fa.3 for ; Fri, 01 Mar 2024 01:18:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284695; x=1709889495; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=f1tNyFTcsXe5j/Xz9tdKUfPkAneqDazbYHB6YPwtZWk=; b=cBYWqzkyE0YPul7RYreeruvx+7PE/JPHHTYO/X2vnVFrVEDKH43arP4V8Bm8ddEYJl 6VlwPESYKdZF5uqxRO+Kpq5pp5D5qklSTfdT1B20swIRYTCSVbaG7vx3ZspbfOf63gZd Eu3zBQ35ZBz5qfHUYwS+U/39r9Sf8n8tvMOLtBA3FsUpZltperan0J0VqtpCgiE5Ga9a hNYdD23BLN4UpGa1Vby0LBkzfOGAlGZS0hqdD+wfw9clzjZPLGUAn8O8WvbP98yEoKOv Xbol3gFVEquDVXrJ028tLPdNBueafZzsb6JVdXxkBxwflI4hQM1xWRkUjjY9979HMzzV Fhjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284695; x=1709889495; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f1tNyFTcsXe5j/Xz9tdKUfPkAneqDazbYHB6YPwtZWk=; b=pxENXql4jipg+UV20SLJII+P/n2Rry8MVeAEBWxHWnUks4VJDmy/g4qT01szlo23AR b184qcSlN9t+nwKXnWZZfZzY36KfjoSnT9V4fH6rfWTj1jrXDdUszl6o+iN+ICs4Epj8 lfb577z24qbFyFqlDbvK9ns3VBQBNKR9+vhRInnGdvAYLGbKr27AASqRIgdTd80EBNJA Zv5q83a4es1p413z8vc4Ol0aMA7RNNOcCGDvYlED/xHccYsFZyyadzbM0ETEd4ZdB0bX m1kzCZcp1nGojuEN0ENNf7InoYNxz9EkeMty/Rd+Wv3PVJWBxAzakPkbvqw7wnUiyqza e7/g== X-Forwarded-Encrypted: i=1; AJvYcCXI8p/7idMdyTCgK6jIWUJ7qakRcaWZMFU/FoLFIHeRbPgOHv/YSSRc53BHxAhu2DQhZcgbU7hgYSilSm8mXp9hiUk= X-Gm-Message-State: AOJu0YwGrtGK3sp8gvHqgjlaIicavWazXby+8Ro3xV/bqajJCtoK839Z mDtAKncM0B4N+4NHn49LQgJIkXtvoa2F3P9BMEJs88aiPWxJFqKGsdTAmJXw7g8= X-Google-Smtp-Source: AGHT+IGpqFyv1mrLhJ3nW6qi9rLC4OBLYLmhrJ+wNsqKGTezYuRrvThKSI3wD+ZkJxl8NSkgNWNgvg== X-Received: by 2002:a05:651c:b1e:b0:2d2:cb34:2e0c with SMTP id b30-20020a05651c0b1e00b002d2cb342e0cmr1088079ljr.10.1709284695243; Fri, 01 Mar 2024 01:18:15 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id m20-20020a7bca54000000b0041290251dc2sm7744159wml.14.2024.03.01.01.18.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:18:14 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 3/9] mm: Use common huge_ptep_get() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:49 +0100 Message-Id: <20240301091455.246686-4-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D4CAF80021 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: kxeuwgnjtgoasufoa3fkiuceytq6pudi X-HE-Tag: 1709284696-166412 X-HE-Meta: U2FsdGVkX186zHJh57mFITkb+rZB86BD0cbINrV0ir7fklakRb0KLJjeHF0mH6kceGvqiVEtdAr6Z5ROIwJaVi9/rfetGYGGhcatWtKzZJ68Ojlk2ZbBXfw71E5cFTHvicVuDucPhzX+7KryzAxUBItao9XDEnSbcrYLgk6EiIm+MbRT5Jn2vzuMwEYXH66FOUaVDPrZYLKwKeth6+3/jIx5yK4rZyiNJ9YVgZzlONdyBoLw8kJgBqUIe8Frp6D953uOUxKuLVCNnReA8CVJXZIWPtlfASywGoNtVcucg87imRK47RWHmirPXSlzsJikeolcgu3FAC5zzhLbwnAyP6sQKuhl2nJWXc/n9HsSldFfMvl0TL+M5GBnnL3LSjPRrwXob40eHm9+8MyXTGRNUJj+Nvju03axjBQV1ANntgxlVALjTtUr11KeqsmB9GwjHmafVs+hrl0ICj+G/pX4mBR+3ARg2HgvJmOU2v8mPCmtgle8Vspk6L4c/kmbdFMhKf8CS63FwYNdcaLFzj3ySXQddtm/2DCmS8vzaSmMi2thjwjQjhe+m++n7ORnGf+nFUJjPsFxZ+ZWC24tlLeh9n5xnjHPiopzrch+Tzr2jDJoL0RkqFYLuEKaBaEaKvYootJmrJzGnv0E+jkL9jyt0wy2D79w1ohI13fbmDdmLc5UUG40HM4kREiJR/IrOWpIypQYZ3UZAluJWcWUgM3unbs3upu3I2w3yPrNE4k+oE5x6ueUUzpMS9uVM7u5IpFQBMPYJ/dfrqC1CSg4UWi+d+5Qg9VyOjc0ljaA8ULnS7JHg0cCZ6p9gmu1QrNQpjPglHFGiPrfCLEbQPFbxqLrUEluPVh9BeqGOo6eiyyq/d7MIEuhLYeUnS4k/M3IBmL3QvRY0N8rtvHHVQlzK/PsVFwkOBMD242MavmdYNShb12ZAZ6nGe9ziEgK+jTD5g9kRujnAouxFiNKYN4FqGY 2FxXk8kL tffhAEWLSzEFnziycu/fhwtIpHir9Aea5OGUPOA/APFzD1g7tgoOtCXS3OdMFfLcgWcnhiuiNh0BPeg6Ilwd0TAt+3pC+Xew5lAbHeE/5yZbBH0TZILf8pGy9uevmiPxzX5v9qcVnzoocW8A4u5F22+rlezG6oqXsSyrE/5xb56yF5u3VXs1KNZbulSnGda11vYTrzgyZBYVSi6Z2Gb00XMqXWw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For that, we need to introduce: - a new config: ARCH_HAS_CONTPTE, - a new arch specific function which returns the number of contiguous PTE in a mapping and its base page size, - a pte_cont() helper, only introduced for riscv since we keep the arm64 naming (contpte) which is more explicit than the riscv's (napot). Signed-off-by: Alexandre Ghiti --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable.h | 31 ++++++++++++++++++ arch/arm64/mm/hugetlbpage.c | 55 ++------------------------------ arch/riscv/Kconfig | 1 + arch/riscv/include/asm/hugetlb.h | 2 +- arch/riscv/include/asm/pgtable.h | 1 + arch/riscv/mm/hugetlbpage.c | 24 -------------- mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/contpte.c | 45 ++++++++++++++++++++++++++ 10 files changed, 86 insertions(+), 78 deletions(-) create mode 100644 mm/contpte.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index aa7c1d435139..5e6bd49169d7 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -20,6 +20,7 @@ config ARM64 select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_CACHE_LINE_SIZE + select ARCH_HAS_CONTPTE select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 79ce70fbb751..3003a10547de 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1124,6 +1124,37 @@ extern pte_t ptep_modify_prot_start(struct vm_area_struct *vma, extern void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t new_pte); + +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, + size_t *pgsize) +{ + int contig_ptes = 0; + + *pgsize = size; + + switch (size) { +#ifndef __PAGETABLE_PMD_FOLDED + case PUD_SIZE: + if (pud_sect_supported()) + contig_ptes = 1; + break; +#endif + case PMD_SIZE: + contig_ptes = 1; + break; + case CONT_PMD_SIZE: + *pgsize = PMD_SIZE; + contig_ptes = CONT_PMDS; + break; + case CONT_PTE_SIZE: + *pgsize = PAGE_SIZE; + contig_ptes = CONT_PTES; + break; + } + + return contig_ptes; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 8116ac599f80..6b61714d7726 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -119,57 +119,6 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -static inline int num_contig_ptes(unsigned long size, size_t *pgsize) -{ - int contig_ptes = 0; - - *pgsize = size; - - switch (size) { -#ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: - if (pud_sect_supported()) - contig_ptes = 1; - break; -#endif - case PMD_SIZE: - contig_ptes = 1; - break; - case CONT_PMD_SIZE: - *pgsize = PMD_SIZE; - contig_ptes = CONT_PMDS; - break; - case CONT_PTE_SIZE: - *pgsize = PAGE_SIZE; - contig_ptes = CONT_PTES; - break; - } - - return contig_ptes; -} - -pte_t huge_ptep_get(pte_t *ptep) -{ - int ncontig, i; - size_t pgsize; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_cont(orig_pte)) - return orig_pte; - - ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize); - for (i = 0; i < ncontig; i++, ptep++) { - pte_t pte = ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - /* * Changing some bits of contiguous entries requires us to follow a * Break-Before-Make approach, breaking the whole contiguous set @@ -250,7 +199,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, unsigned long pfn, dpfn; pgprot_t hugeprot; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -397,7 +346,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = num_contig_ptes(sz, &pgsize); + ncontig = arch_contpte_get_num_contig(NULL, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index dba28a756e63..121183768d1a 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -19,6 +19,7 @@ config RISCV select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_BINFMT_FLAT + select ARCH_HAS_CONTPTE if RISCV_ISA_SVNAPOT select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL if MMU select ARCH_HAS_DEBUG_VM_PGTABLE diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index 20f9c3ba2341..a431a0c0e0fa 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -47,7 +47,7 @@ pte_t huge_ptep_get(pte_t *ptep); pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte -#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ #include diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 951f3ceb5529..ddff4a56e12d 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -302,6 +302,7 @@ static inline unsigned long pte_napot(pte_t pte) { return pte_val(pte) & _PAGE_NAPOT; } +#define pte_cont pte_napot #define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 2477d20c1497..51ec80cf2028 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -3,30 +3,6 @@ #include #ifdef CONFIG_RISCV_ISA_SVNAPOT -pte_t huge_ptep_get(pte_t *ptep) -{ - unsigned long pte_num; - int i; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_present(orig_pte) || !pte_napot(orig_pte)) - return orig_pte; - - pte_num = napot_pte_num(napot_cont_order(orig_pte)); - - for (i = 0; i < pte_num; i++, ptep++) { - pte_t pte = ptep_get(ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/Kconfig b/mm/Kconfig index ffc3a2ba3a8c..71d92e6c50d9 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1001,6 +1001,9 @@ config IDLE_PAGE_TRACKING config ARCH_HAS_CACHE_LINE_SIZE bool +config ARCH_HAS_CONTPTE + bool + config ARCH_HAS_CURRENT_STACK_POINTER bool help diff --git a/mm/Makefile b/mm/Makefile index e4b5b75aaec9..d5aa9326fc80 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -90,6 +90,7 @@ obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_NUMA) += memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) += migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += huge_memory.o khugepaged.o +obj-$(CONFIG_ARCH_HAS_CONTPTE) += contpte.o obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o ifdef CONFIG_SWAP diff --git a/mm/contpte.c b/mm/contpte.c new file mode 100644 index 000000000000..c3f4b8039b19 --- /dev/null +++ b/mm/contpte.c @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 Rivos Inc. + */ + +#include +#include +#include + +/* + * Any arch that wants to use that needs to define: + * * contpte macros + * - pte_cont() + * - arch_contpte_get_num_contig() + */ + +/* + * This file implements the following contpte aware API: + * huge_ptep_get() + */ + +pte_t huge_ptep_get(pte_t *ptep) +{ + int ncontig, i; + size_t pgsize; + pte_t orig_pte = ptep_get(ptep); + + if (!pte_present(orig_pte) || !pte_cont(orig_pte)) + return orig_pte; + + ncontig = arch_contpte_get_num_contig(ptep, + page_size(pte_page(orig_pte)), + &pgsize); + + for (i = 0; i < ncontig; i++, ptep++) { + pte_t pte = ptep_get(ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} From patchwork Fri Mar 1 09:14:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1C6EC5475B for ; Fri, 1 Mar 2024 09:19:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 49CE66B009D; Fri, 1 Mar 2024 04:19:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4257F6B009E; Fri, 1 Mar 2024 04:19:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 276CD6B009F; Fri, 1 Mar 2024 04:19:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 13BF66B009D for ; Fri, 1 Mar 2024 04:19:20 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B55DFC125E for ; Fri, 1 Mar 2024 09:19:19 +0000 (UTC) X-FDA: 81847921638.10.0DEF8E9 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by imf17.hostedemail.com (Postfix) with ESMTP id DD44B40012 for ; Fri, 1 Mar 2024 09:19:17 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="oB/UuS8t"; spf=pass (imf17.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284758; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mB8cpSFUHI2v28tFlR/YUHSb5lk45+n/9IYk3vd50wk=; b=EHKnbpOC8QTOX2kRHJOl3ACZ7xFqz2pm3ijx5juPUYk+HSRjQJf1xpWiwZQ7YL/I3c/dL3 BYhoAowApEd3kQDVMN1hFKyBmsFI/X2Lhcr8PLDTQ5RxrFCmawHEiNTrVKYqTwS9jDHw7h RfUuS8piq2lYOyFsMnCYh7Tr2uoLyTM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284758; a=rsa-sha256; cv=none; b=0o1+tV5I+LgZBCF9jdSblxbI/hDvCRt0jJsTFwEOl5Dg5EbqB/G9LuVI7FjmllAOkn1/bt 9fB8Nf1atDYgofOsEKTQ0fjXb55boZp8umyFnooLtVOgWN9OfuOz+M8MfybMdhe8lV4o8k SMbkWyJdwVJ0ETye1oUfe2s+e+KhyoU= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="oB/UuS8t"; spf=pass (imf17.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.45 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-33e12916565so847754f8f.1 for ; Fri, 01 Mar 2024 01:19:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284756; x=1709889556; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mB8cpSFUHI2v28tFlR/YUHSb5lk45+n/9IYk3vd50wk=; b=oB/UuS8t/ggi3HMYS8+eKu+6AtVfQjjIk3HCys6DG0qSyY1pPF8jCHoltvnUlpXRSB qQJSc9QoxeKeepQj5/KWq4iAIjuPPpXq4SyKsaJ4M+aZO8j5Sp3pOxjzZcfjpmWX/JBk sYVdf5RCmfYKnJhLGBotDN8fOAcG0aA1NyiCD/FJsKrK08aT37hteqEERc4YuU95A9pn uSA5dMuBmJu0i2q5xt7olRQoCpedBhMexU7MuDe1pv5aJc7sswILytdCwlDbyxGGXE9E vYgjcXD2mReePZiPPu7uq1MT013g4NMPma1cTLs8dcfraoDBWE6GZ//Qr605ALUkt9O6 Cdwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284756; x=1709889556; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mB8cpSFUHI2v28tFlR/YUHSb5lk45+n/9IYk3vd50wk=; b=Ans3k92pVqtdHb9flaUrpSIkZXqiAdUoe9Sc8uFxXVj70u00wZP9vOo7ZUwaiemGMc HnORrYSfc0+AxxzIhyzSmdSSbZ/Q1hAuxnVnJn8Fcpve6jnSaHmAx6lJkCqwoXT/HYEd QxjX16UJxohNo/SiPBD15MtjkJMqaxNroI6hiKC4BykXxrOL9J+4Koyfa2we+svJfoG8 xds5V4MbceRTv1Qh33k2Pdfn2TNhw88H3dnMZgQd138UOdIKgmcsdKuPVNQJHhsvC0SC p61eI7sWgEJSM+GHbK7D/jD3BOvp/Yyu495I4p4CKMtwkI2CwUo3VDLPoPK53wp5k87c bSJQ== X-Forwarded-Encrypted: i=1; AJvYcCUHrbZgTPFupXZ8bWgUdTdQzyjQrEgobB67Z36nUVRMN2Q35nByEwf2FMpvpvOtVaLd1gHJ/YzzAz6OA7BowJkyCFs= X-Gm-Message-State: AOJu0YxsV0PWjAvXBLDLWX1ou4luwF9Em7DkU5xpiwDoMKSJhqKUf8v7 GQiYfS/0UEwmKFGCWZ6aEDn0OxwOqoxVvwDURgw1kuEHahSS/Ck7Qh0nejxTuzo= X-Google-Smtp-Source: AGHT+IHpEyM7pTJk28YugLtMmnzHnlZqFOrfhIPrp2GM/yv98BwZmlX0jtBa8/Iqo86IkfuCuT/7uw== X-Received: by 2002:adf:9dca:0:b0:33e:1a3f:44b4 with SMTP id q10-20020adf9dca000000b0033e1a3f44b4mr1239957wre.26.1709284756335; Fri, 01 Mar 2024 01:19:16 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id bv16-20020a0560001f1000b0033e0523b829sm4144440wrb.13.2024.03.01.01.19.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:19:16 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 4/9] mm: Use common set_huge_pte_at() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:50 +0100 Message-Id: <20240301091455.246686-5-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: 5ohi4qh6k6qjqrtho6ju1stjagqcpuqw X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: DD44B40012 X-Rspam-User: X-HE-Tag: 1709284757-104180 X-HE-Meta: U2FsdGVkX18cGsFSPJM+DLHDqBQBUWlsAKcx3i439nAlZk7oRoGq0G9kiaE9a705N56/p4ACDkOOky3CHZZ9OhOliSFIIUMe6mDi3b0kfSzPO7I78kn9SWzBGYLMIZu20pF9825gIO552uHPOz/I1UAE3iCM5/NWi1jZ+r3mTKsTi8MonetBzr0rW8Pz+H+4fThD8fzez0sapYeBbDb/8mjPxsx+74gKw/RLgxeveMHEd7vqyCEbKeReVJ4uedZxhmau9sT8Z8wiiVZJghlDmlxoHdZ688vV/Ldi9cGOY8NKqQYIcLrCTkXVLdxByzD2LsRk4iG1v8GIsoOlhQi7pPjoaYLL48W8Sv6rtZcyrRAu5zt40Uci0/YZ0h/fj/SPebwBXw6P8QFLhE+5BV+ysm4NeWFskbhYnTpcKTeA3cRnD1oTm5wzePL8mxAuTro/Wxh7JPNWhsUkLDSjexf8WEgKpiWmcmItSI6CCoQExotOb/HRUFJtZboMNLmVKVIyyk0Yk77VIN6fPzdQL+29AfE1KdDs2A80FuJMqMN03NjVlHzsa8HtmH1Jm0lwVqVdccX0H5O1LzGvOpbYr7mfLt1ZJgGiJdRVJxIfiDcMVei7sut8gvboF/9kS6edPAjdaA9uP9dAXHZ1f8ifloXf5SG0521E7XhhC5myC1OquVPgI59P1V/gCAKm9RVS1pEB6KKzQMa1pG1nA163WaEJ+DJn1wyrh4bgjV/2JqJDlEOXYdYn9WPipjhcfD9xMHUmoz0nXwUOTGN/2X2BqOZZnycr4blo2ko22/5UHXp7VCmtelVgFnqYtvqGpgTYIMnhHNIYo19mM5egu1bwN8klRnpcaJEudCfCbsTfA0ESkFPlj4X5XfxwBRCMHZg5TBrahRY5JBNut4hKdCyVMg06ETkgk81fCjqx1AG2sUtzZafdhT9WescNm5v1WdB8ZIJ+pBfvB7+Urx72i4lxRBU /UVeUfOB D+smCExHCfwOd09pmSx5itzbmKASa5bu/HVIOZF2N+PCYdEIrwU/USmwtQxmoRKYxGPn5B2BV3+LGNq0b/6bEi7c53OWDOCirQw5viSJks0NBM0cpBnW6Iue0dv09NZY5IEbSPTs90lGaA9KF1Z05E6TPPBRYvIdJ6jPpeLOupf7K4FWbszIZ30x75wB4iQN0DNzvK9F40kyv0uChFHVxMe0lnA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/pgtable.h | 16 ++++++--- arch/arm64/mm/hugetlbpage.c | 56 ----------------------------- arch/riscv/include/asm/pgtable.h | 26 ++++++++++---- arch/riscv/mm/hugetlbpage.c | 62 -------------------------------- mm/contpte.c | 58 ++++++++++++++++++++++++++++++ 5 files changed, 88 insertions(+), 130 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 3003a10547de..437e9638b2b9 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -341,9 +341,10 @@ static inline void __sync_cache_and_tags(pte_t pte, unsigned int nr_pages) mte_sync_tags(pte, nr_pages); } -static inline void set_ptes(struct mm_struct *mm, - unsigned long __always_unused addr, - pte_t *ptep, pte_t pte, unsigned int nr) +static inline void __set_ptes(struct mm_struct *mm, + unsigned long __always_unused addr, + pte_t *ptep, pte_t pte, unsigned int nr, + unsigned long pgsize) { page_table_check_ptes_set(mm, ptep, pte, nr); __sync_cache_and_tags(pte, nr); @@ -354,10 +355,15 @@ static inline void set_ptes(struct mm_struct *mm, if (--nr == 0) break; ptep++; - pte_val(pte) += PAGE_SIZE; + pte_val(pte) += pgsize; } } -#define set_ptes set_ptes + +#define set_ptes(mm, addr, ptep, pte, nr) \ + __set_ptes(mm, addr, ptep, pte, nr, PAGE_SIZE) + +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + __set_ptes(mm, addr, ptep, pte, nr, pgsize) /* * Huge pte definitions. diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 6b61714d7726..4da951e81bde 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -166,62 +166,6 @@ static pte_t get_clear_contig_flush(struct mm_struct *mm, return orig_pte; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step for use cases where the - * original pte is not needed. - */ -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - ptep_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz) -{ - size_t pgsize; - int i; - int ncontig; - unsigned long pfn, dpfn; - pgprot_t hugeprot; - - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); - - if (!pte_present(pte)) { - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) - set_pte_at(mm, addr, ptep, pte); - return; - } - - if (!pte_cont(pte)) { - set_pte_at(mm, addr, ptep, pte); - return; - } - - pfn = pte_pfn(pte); - dpfn = pgsize >> PAGE_SHIFT; - hugeprot = pte_pgprot(pte); - - clear_flush(mm, addr, ptep, pgsize, ncontig); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - set_pte_at(mm, addr, ptep, pfn_pte(pfn, hugeprot)); -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index ddff4a56e12d..03f8ced8b26a 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -533,29 +533,39 @@ static inline void __set_pte_at(pte_t *ptep, pte_t pteval) static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, size_t *pgsize) { + unsigned long hugepage_shift; pte_t __pte; /* We must read the raw value of the pte to get the size of the mapping */ __pte = READ_ONCE(*ptep); - if (pgsize) { - if (size >= PGDIR_SIZE) + if (size >= PGDIR_SIZE) { + if (pgsize) *pgsize = PGDIR_SIZE; - else if (size >= P4D_SIZE) + hugepage_shift = PGDIR_SHIFT; + } else if (size >= P4D_SIZE) { + if (pgsize) *pgsize = P4D_SIZE; - else if (size >= PUD_SIZE) + hugepage_shift = P4D_SHIFT; + } else if (size >= PUD_SIZE) { + if (pgsize) *pgsize = PUD_SIZE; - else if (size >= PMD_SIZE) + hugepage_shift = PUD_SHIFT; + } else if (size >= PMD_SIZE) { + if (pgsize) *pgsize = PMD_SIZE; - else + hugepage_shift = PMD_SHIFT; + } else { + if (pgsize) *pgsize = PAGE_SIZE; + hugepage_shift = PAGE_SHIFT; } /* Make sure __pte is not a swap entry */ if (pte_valid_napot(__pte)) return napot_pte_num(napot_cont_order(__pte)); - return 1; + return size >> hugepage_shift; } #endif @@ -631,6 +641,8 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr, } } #define set_ptes set_ptes +#define set_contptes(mm, addr, ptep, pte, nr, pgsize) \ + set_ptes(mm, addr, ptep, pte, nr) static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 51ec80cf2028..ebc735f5d325 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,68 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -static void clear_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - unsigned long i, saddr = addr; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - ptep_get_and_clear(mm, addr, ptep); - - flush_tlb_range(&vma, saddr, addr); -} - -/* - * When dealing with NAPOT mappings, the privileged specification indicates that - * "if an update needs to be made, the OS generally should first mark all of the - * PTEs invalid, then issue SFENCE.VMA instruction(s) covering all 4 KiB regions - * within the range, [...] then update the PTE(s), as described in Section - * 4.2.1.". That's the equivalent of the Break-Before-Make approach used by - * arm64. - */ -void set_huge_pte_at(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - pte_t pte, - unsigned long sz) -{ - unsigned long hugepage_shift, pgsize; - int i, pte_num; - - if (sz >= PGDIR_SIZE) - hugepage_shift = PGDIR_SHIFT; - else if (sz >= P4D_SIZE) - hugepage_shift = P4D_SHIFT; - else if (sz >= PUD_SIZE) - hugepage_shift = PUD_SHIFT; - else if (sz >= PMD_SIZE) - hugepage_shift = PMD_SHIFT; - else - hugepage_shift = PAGE_SHIFT; - - pte_num = sz >> hugepage_shift; - pgsize = 1 << hugepage_shift; - - if (!pte_present(pte)) { - for (i = 0; i < pte_num; i++, ptep++, addr += pgsize) - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - if (!pte_napot(pte)) { - set_ptes(mm, addr, ptep, pte, 1); - return; - } - - clear_flush(mm, addr, ptep, pgsize, pte_num); - - set_ptes(mm, addr, ptep, pte, pte_num); -} - int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, diff --git a/mm/contpte.c b/mm/contpte.c index c3f4b8039b19..f7bfa861c6a1 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -12,11 +12,13 @@ * * contpte macros * - pte_cont() * - arch_contpte_get_num_contig() + * - set_contptes() */ /* * This file implements the following contpte aware API: * huge_ptep_get() + * set_huge_pte_at() */ pte_t huge_ptep_get(pte_t *ptep) @@ -43,3 +45,59 @@ pte_t huge_ptep_get(pte_t *ptep) } return orig_pte; } + +/* + * ARM64: Changing some bits of contiguous entries requires us to follow a + * Break-Before-Make approach, breaking the whole contiguous set + * before we can change any entries. See ARM DDI 0487A.k_iss10775, + * "Misprogramming of the Contiguous bit", page D4-1762. + * + * RISCV: When dealing with NAPOT mappings, the privileged specification + * indicates that "if an update needs to be made, the OS generally should first + * mark all of the PTEs invalid, then issue SFENCE.VMA instruction(s) covering + * all 4 KiB regions within the range, [...] then update the PTE(s), as + * described in Section 4.2.1.". That's the equivalent of the Break-Before-Make + * approach used by arm64. + * + * This helper performs the break step for use cases where the + * original pte is not needed. + */ +static void clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + unsigned long i, saddr = addr; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + ptep_clear(mm, addr, ptep); + + flush_tlb_range(&vma, saddr, addr); +} + +void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + size_t pgsize; + int i; + int ncontig; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + if (!pte_present(pte)) { + for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) + set_pte_at(mm, addr, ptep, pte); + return; + } + + if (!pte_cont(pte)) { + set_pte_at(mm, addr, ptep, pte); + return; + } + + clear_flush(mm, addr, ptep, pgsize, ncontig); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Fri Mar 1 09:14:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE326C5475B for ; Fri, 1 Mar 2024 09:20:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEA916B00A1; Fri, 1 Mar 2024 04:20:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9B0E6B00A0; Fri, 1 Mar 2024 04:20:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C62956B00A1; Fri, 1 Mar 2024 04:20:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B29296B009F for ; Fri, 1 Mar 2024 04:20:20 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 841A816106E for ; Fri, 1 Mar 2024 09:20:20 +0000 (UTC) X-FDA: 81847924200.27.D5CBCF7 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by imf07.hostedemail.com (Postfix) with ESMTP id C348540014 for ; Fri, 1 Mar 2024 09:20:18 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=CY5QjhmR; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284818; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rdWlfuEJdB5bVZdrcri04Zmq/yhRfMcRDdftEw7qNFA=; b=ppnnuqyOd/ZsToq2RMOLQsKzGaH7p0cnwkmC1BAGKjcZX2B9ft/DAEX2jtxxeyYMiu4lxD NDvRXrW3Sz8H9mNoLAq8sEMVKN+J/ldUSuW9w28jSSft/u2hvU1xSImHdmbhukyJ4D9yyL vJqKVlqjH0phk5pWGZ20OasiOKLXzaQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284818; a=rsa-sha256; cv=none; b=Yz/5Pqk9CJqBFhw942BYb3FelDyfNtBV7NJk3sIbrFEFFLeR99Z5V9wvNu1bHgoUlx2lrA iU4F9DbqFWDecQyKGnn09/z/eD4yQEfrZm8KrPutmPVUED/J7kjG6beIBepa5Hh2JLIVPJ HHJW9U6x+sf8AbuRSwj92pN5UWGFlkA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=CY5QjhmR; spf=pass (imf07.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-412bb23e5c5so10031075e9.1 for ; Fri, 01 Mar 2024 01:20:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284817; x=1709889617; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rdWlfuEJdB5bVZdrcri04Zmq/yhRfMcRDdftEw7qNFA=; b=CY5QjhmR5Wgy4JH6hhxyPii6FtmI1o3PGz8OSOjKRwb+/Id6LBBcCYZWUNXYkkBbjr aCAg6dasIkcwT5WDiJHNvKk7KswTeiXUeGk7fKCs6hSAHN4R6G14ROByu/DqC5qMiUcb gwAS9qIPU/OI353fMEhGT6R/MXURxvtc+9kEJEVoi5+YUwGKNxOYGTK8Hyr8GSmS4GUU O4DJduUUQopfW1Pcau2k4EFQNOf7uvGRAapgA+PJzdXlnWs+gUNP9O82dyVvxbZjM2Rl q0FUyzOSWq0bVvJ5DmVhcrXXCv7JTPF3lny+qkZ2mlMIWbcZBPxWmgTpNJYjx+9f2eVC Ov0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284817; x=1709889617; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rdWlfuEJdB5bVZdrcri04Zmq/yhRfMcRDdftEw7qNFA=; b=AF9jvZwDz7nkfO8s+DY47YknvvTNaQXaivvl0SAhLroytVqYiPWo3Fzm0zL6W/ZrCQ rpvFJkbU2St6z1VmzUBqpeC29uNG1p0xKWvreJAVU3/6rLZBiYG11b7yhaEmz81FdDDT QogzT6t1Sh1XUWQn7pOsWkFbfb9BKrnivHqjQaqvOKrJmQmwORRITvjpGRx1rA83X3ld nRVada8QKrHJR4y9yuejYZ6yh7nbtUABSXkVrwTaNI3lWg8xuoeDCFy6lBdF0/z21kYJ gwFBEdEO+0nohK/mnE/lLNDmkvFg+jjAhQYfc1Dco20nvR4QnpGQ4/5SwikAvzZW7Oo3 sjZQ== X-Forwarded-Encrypted: i=1; AJvYcCVklxZxjUr2h1YD06jYmeCXx3UUvtZ36JaVkol6bfPXM1LKt2U/7JopNrUzTBRgbwOxKlFqyoHwX00DEIYC1k4/hnk= X-Gm-Message-State: AOJu0YyPjWOHwGkl+J7nUTF89tKfhd2WsKv9f6fqQH3gmrDPRiP85Weq Ig9/0zcdWRVPn90HRNEx51YqmYwL/02tFWeIsRK8J37/7JX9pEduyHjXAliOxGU= X-Google-Smtp-Source: AGHT+IHirYUWWAQ3EzQ3D3sM4w5kbNmWdiMfyHww76VDivA5fk7+UlmhKkRo6vORNs4FOsSe8ylbjw== X-Received: by 2002:adf:a3c9:0:b0:33e:21bc:bc6f with SMTP id m9-20020adfa3c9000000b0033e21bcbc6fmr809159wrb.11.1709284817327; Fri, 01 Mar 2024 01:20:17 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id l9-20020a056000022900b0033cf2063052sm3994161wrz.111.2024.03.01.01.20.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:20:17 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 5/9] mm: Use common huge_pte_clear() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:51 +0100 Message-Id: <20240301091455.246686-6-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: coyg8fo4zoiqm5s77frteu1swxr1ne88 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C348540014 X-Rspam-User: X-HE-Tag: 1709284818-21644 X-HE-Meta: U2FsdGVkX19OuDy0luEE4lK13jCarJp0NA41GgmzmANzWgZqd+ot7SUa/gQCVcbShkG/YQ/uz0pvcuPLUGE8F7hIkFK2sQt39S/qsN07obGtZ0bFKRZW9oAWypBAUn7J8d/Gj0jkj+CRX/tU3WUlsachHTD99OO6DSa/ndvUKrV2LouikzHCF/7ALsreBmPZq8jTbkx5nHbS9QZ4OBsz/uwJPOmp4J0PLwBgnfv/3LQvlP9z/qEZpBQlS+8mTGSFiO/BjDOGu1gGNhlJBVrf/rTmhCeJjIimg7H1Xthc9F3R7Apa3WS/8/m4CfL7HI4I+sd5utAUva8zZ1IdKZ4w6ih7+U//pZIxtK7yJrhlFCRj9lyAYlBweo/gRnK+7cab49/GGXt0+CRiiY0De60BXOSy5gE/RMRJAIY/RSWHYzvbR5S6zf0dHLAvuP/viOgZBX9xn1uiXSJcLjMq2JnoSZgy0tk0kNtuPxzketd2qSKB48xQJq9mO3SlMMmVNlu+Y/9Tys5ciIskcDx8JsKEW6j6iVKLPaRLjgTRHyY011/7IUYrQMNNvqw9AlAMmVHLrhlqMonUn+WMmZDukfx/lhLGhtLLvzigMYE+Ad1/ptVbek2uCn5oXMBb9QLrTA7QDcJyJtBeAF7tMEEzEAdKggTCMQQc11Jx/25WgJ+K+hG2XnUOR2e9ghzEgISIBaQmExXajEyknI0Xu3utZUVQS1SVBEdaouNc3+BjJO1Maisc8KQ7RT/2KXVxs8zlQUJtabWO3/QaIUcziLo/1hM73BKRLqFmaY7VYv4tTZPqYsLlHMk12ZNuJTuAezshZz/na80Jz/T2xjhLiFQ9+YnsIJkNcKevcHuva6NRwxCP2ojq/xNxVr0BEXsEQnbI7z+aInioqiwhJSY1AXyGZusJtDt/u5kUdl/a42QEJCrmOK3vl2SUgwy0iw4yKZw6TY/3yBfZN4SPWBtLbhwEcnS 0aAH0mVu OyQJWfzZasQblXqRmdM3k+B/6vJGLoNgFd8g9TBKls6frxdfHBNXgMBH2tPJ16cIDxGvMFLeGtDadmfSbrhDGg2znFohJsBQ9Bb2yBk/azEKVRl3Ay/m8RoxyQZO0WgL2pj0TF6cVMIl6x2EULJp8Qmjm6HR3LRV5iBVce89uAo0ph4MlsDp8vB6PXWbK1ITwTu21gEp0yXXsO1Bmkg2j3w5M2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have the same implementation so move it to generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 12 ------------ arch/riscv/mm/hugetlbpage.c | 19 ------------------- mm/contpte.c | 13 +++++++++++++ 3 files changed, 13 insertions(+), 31 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 4da951e81bde..09b55bac8c7c 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -284,18 +284,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) -{ - int i, ncontig; - size_t pgsize; - - ncontig = arch_contpte_get_num_contig(NULL, sz, &pgsize); - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) - pte_clear(mm, addr, ptep); -} - pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index ebc735f5d325..01b1403dd4cb 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -254,25 +254,6 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } -void huge_pte_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long sz) -{ - size_t pgsize; - pte_t pte = ptep_get(ptep); - int i, pte_num; - - if (!pte_napot(pte)) { - pte_clear(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); - for (i = 0; i < pte_num; i++, addr += pgsize, ptep++) - pte_clear(mm, addr, ptep); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/mm/contpte.c b/mm/contpte.c index f7bfa861c6a1..5200c234404d 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -19,6 +19,7 @@ * This file implements the following contpte aware API: * huge_ptep_get() * set_huge_pte_at() + * huge_pte_clear() */ pte_t huge_ptep_get(pte_t *ptep) @@ -101,3 +102,15 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz) +{ + int i, ncontig; + size_t pgsize; + + ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) + pte_clear(mm, addr, ptep); +} From patchwork Fri Mar 1 09:14:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EB32C5478C for ; Fri, 1 Mar 2024 09:21:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02FA86B008A; Fri, 1 Mar 2024 04:21:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F22246B009E; Fri, 1 Mar 2024 04:21:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEAB66B00A0; Fri, 1 Mar 2024 04:21:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CF7D86B008A for ; Fri, 1 Mar 2024 04:21:21 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A69BFA11F8 for ; Fri, 1 Mar 2024 09:21:21 +0000 (UTC) X-FDA: 81847926762.01.D98F82B Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by imf16.hostedemail.com (Postfix) with ESMTP id E0BA8180009 for ; Fri, 1 Mar 2024 09:21:19 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="UTe/JdCA"; dmarc=none; spf=pass (imf16.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.47 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284880; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=roMginZSvi1o0fjo+EUJOo1pu0LVGawrxcxtyn24YPc=; b=vi9RSyr+Aj7U76207zsUDxWUGhJFeKtdCvll8NMlW6fP9Sh5KlFbwFi4Po6HHxDvWctq7L rdglhZo7XbnaMfsBc4ipacQKnoiU8dCfmf1Lk7mSma3PlAwR+8jdAZA1yjKdgmPKlw0b5r EoA5kYoTERPIqSsH6a7Uo8EcP39F8xs= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="UTe/JdCA"; dmarc=none; spf=pass (imf16.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.47 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284880; a=rsa-sha256; cv=none; b=pM2F2n1z08F/vNgvqo5nq3Rh/IQ4X381JYuqDRrPp/2Fg/3g+0F2IfVgNg8oETK4LDiTP+ iAGCuXUNFQzzrfmLcPvHdvCIfYjA3C015yA1QMVLvxB3cKmhVWpIJ7zEe7AATuUGNaSEM3 CVXrfEwWPzkpgQ6300K7M69Gv1sWtKY= Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-33e17342ea7so527403f8f.2 for ; Fri, 01 Mar 2024 01:21:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284878; x=1709889678; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=roMginZSvi1o0fjo+EUJOo1pu0LVGawrxcxtyn24YPc=; b=UTe/JdCABqY/FsPQw1gnaRYs5DBYWmVurX490RR4nn+BDG4TGIcBGP7+BhJ1weQeTO wvC8A5xGSb7H2AYm1CRZtZKN/XpfF3A+0Yh6q0NDbfKUHKux9/9SPxFopOlFf+Zeyw+3 +uLmtsOJyMgaTG1BdshrfkwcJxrp4k44Ps78hQqfbpq6cgLjKvPQ50hUaZUV76dtTq3Y eEnxo3h0k+9rLf8zoGItmc6izRSpUWbQFLJZJ6oMSYLr48pF2lVv9bvZFAEFOEAwUUvq QDFbzLmPFYycd55+hLlxcjxiJY2+TVmeMziN6PddEw1vzIo+ruDlzitexJCSFicInOEo 4LVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284878; x=1709889678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=roMginZSvi1o0fjo+EUJOo1pu0LVGawrxcxtyn24YPc=; b=WQ6HnZPeGH5tBVDFfJmdobE6Vn8H8NLwRow8RXriaJVDlB5u51pFE2eaya3YJDZg5U juAmGXSfJOQsioybN0/DiJMtgaTjjcUkyCHOaXD6NMgIOMLoiNIvqQLr/bjur3s1DiZf OCXbxmbstkw/WRaMV14TD7rbUcI4fsTTVHfkESt7JLBVeqJERo0ZDhiMsXNfMMLsDyZ6 UVO4TOsA+97JLnZYFoAJAapgRZ9sDkEy/ZSUPAiRatImNPEvvtXS5NnI+YvK97CtGqe6 k/GBr3D+PCS0A6VHfYbxrT/Q47M+7L+RKFeCS0C9rp7sRnfEYBuPcHDSo8Wdif8vd/nx bbxg== X-Forwarded-Encrypted: i=1; AJvYcCUUPgUWP9DsiXmpx1ELKXjZQSOuDdqGX9UbvmAh/DkPPdYPqIpYzV5eq3KRjUN00nU1itBPNchjQCT9fvlsFcptaIk= X-Gm-Message-State: AOJu0Yx1e4PJob/7Pm/wDAmeaZ1UAAxEc8KUXmvzcfIv/UioQG4a4w4r UQxSgaa6N8yv95SSvviyMIEgel1XIoppCpRJUsblPPXtt7bsfQ+oqWPPng2OtPA= X-Google-Smtp-Source: AGHT+IHE36Cx0pzkf7/PwROG0sFnOnPILtKg/5tjC2XGRavDiE3m4OmzpuOVj7dxMVbSTb8/7tKiwA== X-Received: by 2002:adf:f24e:0:b0:33d:c5c5:9bc6 with SMTP id b14-20020adff24e000000b0033dc5c59bc6mr893274wrp.54.1709284878472; Fri, 01 Mar 2024 01:21:18 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id i6-20020adfb646000000b0033e033898c5sm4044660wre.20.2024.03.01.01.21.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:21:18 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 6/9] mm: Use common huge_ptep_get_and_clear() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:52 +0100 Message-Id: <20240301091455.246686-7-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: E0BA8180009 X-Stat-Signature: du3hicqpq3kig8a3e4mj6fh7jrrhcwae X-Rspam-User: X-HE-Tag: 1709284879-937886 X-HE-Meta: U2FsdGVkX18Xj4PVNBI8AI8uHuNGdOA5v1lXaQLOVswolcxP2KcSzu4QuqwJfwqdcOGnYClrI8bUq7T890WPZDzVGT4bFhxtPgXSWi6rj5PuDsoQHQyMKlx5f2KFR4XQyauGKk77/61O0VjzfDHJcNS2FrgEyoTZFqKtVs4yrTBbK57+un1YD5f4hIKNFrWBP9ICwgqbI8OxSy9VevvpzCXo6uwXzG+4N0g7USCcVFcjmJns0hGBE81ULnvkzJWXfL6+yG+fn2T/4i8Dwc8wCVd7945VeBdxDCaz3pyhJmh8BZCBTNv50QtZK372mI9aPen+mG3Zdt/Iuwufh9lW1iBXBBBDS7U6M3r171HepDyrt0789RhfKGusnUkJuditegfzN0drTiR61CLc2YVJuVv3wJHnFQpG2Zu5XOi/9Fa1wWMyoqbACkufCbafovMrNPt1Xcmh7BkGU1qKy0EFFdgKwJQoVXi5Wke4brlpMt2rQpQ7RS6+BV3UdGR/g6iEOMmqObNhYTGjqk3LC96YTfncSueuWdWG5rt0w9GfDDPuLpBSzhx2X2rkG2PwiXp91HFw6ZtejqnEDkgsjxrmB35pr6XuAwRi4z1KjVDyNDruypEDouQme2tV+L8OIhBUMVv4dS7WkxGQ6hqw2v0JwdiLevjqjY7SpKc5HswfVQy7JPKgv2mkd4e7cuyLV5jvVRVZQbzvqz9GKKicERY7zQLhDZAvKtrQz93LDU8b4QSvcM+Dh4spCUI+yAA6evAqoOQq188rNlCOebPV0HuyxS2KRBYYSwGLXG9BvzqUlq0ApKiR3Jn7ARXyYeRoyV5lpIWyRVW74OMo6FbxX1MwlSTPz6nHqeGAo0y/f2VUCioITUqfUAx7QcFIVmOwLtKgN/F4R2xwsBM7w4TU6zE0wvhuLpFenlOBM/q+zXdgnT702eA+zjQcInO5OyC0mL+g/q4YQR1IYF5Ps36VdHt XvJSG8KM PXKVnJa8be0xdL7jmujis5dx5uph9FfkkEfUsqX3SFwf0wNpxYWqtYJYmY2zSPiWL/llZKHIL4L23CzvygoCx91P49AR+x/Pclj3rcSpVid3excxd2BERqC4t0vNV/WATiltBZSdXJtGf1IXVqwdA7/1wsBBvtG9EuzZzTqKTuepl12GL+JusLpDVXzSGfDvZtyzxAoZxYxhcMM3Xg1kxoEigbw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Note that get_clear_contig() function is duplicated in the generic and the arm64 code because it is still used by some arm64 functions that will, in the next commits, be moved to the generic code. Once all have been moved, the arm64 version will be removed. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/pgtable.h | 14 +++++++++- arch/arm64/mm/hugetlbpage.c | 19 ++----------- arch/riscv/include/asm/pgtable.h | 4 ++- arch/riscv/mm/hugetlbpage.c | 21 ++------------ mm/contpte.c | 48 ++++++++++++++++++++++++++++++-- 5 files changed, 66 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 437e9638b2b9..b5caf7a9d03f 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1131,11 +1131,23 @@ extern void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t new_pte); -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize); + +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { int contig_ptes = 0; + /* + * If the size is not passed, we need to go through the page table to + * find out the number of contiguous ptes. + */ + if (size == 0) + return find_num_contig(mm, addr, ptep, pgsize); + *pgsize = size; switch (size) { diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 09b55bac8c7c..a2d33cbc7da5 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -100,8 +100,8 @@ int pud_huge(pud_t pud) #endif } -static int find_num_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, size_t *pgsize) +int find_num_contig(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, size_t *pgsize) { pgd_t *pgdp = pgd_offset(mm, addr); p4d_t *p4dp; @@ -284,21 +284,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - int ncontig; - size_t pgsize; - pte_t orig_pte = ptep_get(ptep); - - if (!pte_cont(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - - return get_clear_contig(mm, addr, ptep, pgsize, ncontig); -} - /* * huge_ptep_set_access_flags will update access flags (dirty, accesssed) * and write permission. diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 03f8ced8b26a..1cb877f0d0ce 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -530,7 +530,9 @@ static inline void __set_pte_at(pte_t *ptep, pte_t pteval) } #ifdef CONFIG_RISCV_ISA_SVNAPOT -static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long size, +static inline int arch_contpte_get_num_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, unsigned long size, size_t *pgsize) { unsigned long hugepage_shift; diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 01b1403dd4cb..4c441664db8a 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -187,7 +187,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); @@ -202,21 +202,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return true; } -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t orig_pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(orig_pte)) - return ptep_get_and_clear(mm, addr, ptep); - - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); - - return get_clear_contig(mm, addr, ptep, pte_num); -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) @@ -231,7 +216,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, return; } - pte_num = arch_contpte_get_num_contig(ptep, 0, &pgsize); + pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); orig_pte = pte_wrprotect(orig_pte); @@ -249,7 +234,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, if (!pte_napot(pte)) return ptep_clear_flush(vma, addr, ptep); - pte_num = arch_contpte_get_num_contig(ptep, 0, NULL); + pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); } diff --git a/mm/contpte.c b/mm/contpte.c index 5200c234404d..8d2ab391e0d8 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -20,6 +20,7 @@ * huge_ptep_get() * set_huge_pte_at() * huge_pte_clear() + * huge_ptep_get_and_clear() */ pte_t huge_ptep_get(pte_t *ptep) @@ -31,7 +32,7 @@ pte_t huge_ptep_get(pte_t *ptep) if (!pte_present(orig_pte) || !pte_cont(orig_pte)) return orig_pte; - ncontig = arch_contpte_get_num_contig(ptep, + ncontig = arch_contpte_get_num_contig(NULL, 0, ptep, page_size(pte_page(orig_pte)), &pgsize); @@ -85,7 +86,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, int i; int ncontig; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); if (!pte_present(pte)) { for (i = 0; i < ncontig; i++, ptep++, addr += pgsize) @@ -109,8 +110,49 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, int i, ncontig; size_t pgsize; - ncontig = arch_contpte_get_num_contig(ptep, sz, &pgsize); + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize); for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) pte_clear(mm, addr, ptep); } + +static pte_t get_clear_contig(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = ptep_get(ptep); + unsigned long i; + + for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { + pte_t pte = ptep_get_and_clear(mm, addr, ptep); + + /* + * If HW_AFDBM (arm64) or svadu (riscv) is enabled, then the HW + * could turn on the dirty or accessed bit for any page in the + * set, so check them all. + */ + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + return orig_pte; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t orig_pte = ptep_get(ptep); + + if (!pte_cont(orig_pte)) + return ptep_get_and_clear(mm, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + return get_clear_contig(mm, addr, ptep, pgsize, ncontig); +} From patchwork Fri Mar 1 09:14:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3310C5478C for ; Fri, 1 Mar 2024 09:22:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 317D76B0074; Fri, 1 Mar 2024 04:22:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C8E86B0087; Fri, 1 Mar 2024 04:22:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 169FF6B0093; Fri, 1 Mar 2024 04:22:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 04ABE6B0074 for ; Fri, 1 Mar 2024 04:22:23 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AD76AA16CC for ; Fri, 1 Mar 2024 09:22:22 +0000 (UTC) X-FDA: 81847929324.20.C70B2EA Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by imf25.hostedemail.com (Postfix) with ESMTP id ED620A0006 for ; Fri, 1 Mar 2024 09:22:20 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=g+J7fmMG; spf=pass (imf25.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709284941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fHzJQXP5BEIeqfOJsZUcjaIxOTpiaXVWcZpI+r0iAkE=; b=oLKveQXJpafUoeIUIsHnfncYtMBbdVlyuqaBK0ikhnFark5cAz6RD5435j8gvleza0lx/D qWNtqryKiw9MNjWs5P6Sc7pmZtkEZiXWybQqMua4dlLwFNroIZmLDiu9sKk+GxkdyHHkzs aY+nslFsxc1xb+y+V0EphCJlzqY7CiM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709284941; a=rsa-sha256; cv=none; b=wkG26GFXhzXUTrr6jkut9be8rO0diJoacmwsTSR6vbXjd4xgN4IbY8EPWuFu7eb8W8gj4e Md7XHiDL2mbeIHXPewCqGymp9SOgaGbFKCXvZ6B5/6Oqd67q3KbOih5vf4mwavi4rx+ZvE 3n6crCqe3wibhxOFY6H9YhbzeY0Gfbg= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=g+J7fmMG; spf=pass (imf25.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-33e17fc5aceso535685f8f.0 for ; Fri, 01 Mar 2024 01:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709284939; x=1709889739; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fHzJQXP5BEIeqfOJsZUcjaIxOTpiaXVWcZpI+r0iAkE=; b=g+J7fmMGDzr4lj5STiJvKdLKTu56mPJOShDypqAwK9WSD17mQaEPUmDvF9KAwgY5El JXHYmioDqvIuT69qs4Ci0rrY1A/2844J8V4vGh+YJx7UFp4xNLJicVQqZ0KWjTBa2Q9U GF5E2g1u3dpA7LskOL6TNaqQvt+AYj5lZZ/NWofCBRpcYfq9qQW7Ojy8moKc9yE73LSQ A2ZBKda/n16OYynaB6M7hTcWJAzxiCHNobjKhIDzl73PkQUMGGwR7iRV+haDUu5jfnpb PR0W0dYBBG4ZP5o0xKfBycgK4bJajppxj94I5hHKmMVAKZxBJPnVsuFE7BMAkgGd2hqH J3Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709284939; x=1709889739; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fHzJQXP5BEIeqfOJsZUcjaIxOTpiaXVWcZpI+r0iAkE=; b=dcsLzT7wzekmirhGxXC8oYzTQzluDW4boXKYtOqfB3WvXO85L7yEjV6PSZlyJ5ObyR MfdCXAMizEKr+UbDJGYgSPvYec34lT0q08z2bNTWM8gHyWQUSNNK4DjpnCanprt+q6TS clTnCs2eSX2UXIHbfxOMjQYsJQqc5I4N5oOwJEwsUUDObG1elE3xaT6irOu2CdZvnc3L /H941ZKZXorVZpvZPniSPwh2AViI+Cf0N61q1t14ISCsXNGoHK6/Ref5gk6Gy7uvuAoq Qrzt44zrpd3kgOy9weeQYlAbv9cRx/Z46yUUUUvalJd/QwAwIh4b3bshezJOossyUh0u r33A== X-Forwarded-Encrypted: i=1; AJvYcCU654IZxEB4LIGUQ3PrxUADj6bHaAUReZiYVvsIpzCrC4rSvUT/zVfY8gfJbEm9Vynwe/IJZJKs1YaC9vtX0e8+t/k= X-Gm-Message-State: AOJu0YzL5of5xdgx0qWEwr1zuYERL7eJUeBhCbD958poVgJNqbUArerH G5D0sfMzMA01s2xZ8HoUeWcDuIWh5CSNhcsc0uz1fsZtabekwBmYCaIOu/9Lbd0= X-Google-Smtp-Source: AGHT+IETq5VRJHPuwhCoVEqwX4u/EPRIvAhXp1XnZCRkR4LhBkbxP9zDwDTrac3tIekTrNx6Id8PTw== X-Received: by 2002:adf:f84c:0:b0:33d:39db:a0f8 with SMTP id d12-20020adff84c000000b0033d39dba0f8mr1044349wrq.7.1709284939471; Fri, 01 Mar 2024 01:22:19 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id e13-20020a5d594d000000b0033e12b2e567sm3302518wri.35.2024.03.01.01.22.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:22:19 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 7/9] mm: Use common huge_ptep_set_access_flags() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:53 +0100 Message-Id: <20240301091455.246686-8-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Stat-Signature: bafu14fcre463dysmdtejhf3ekgsn115 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: ED620A0006 X-Rspam-User: X-HE-Tag: 1709284940-256916 X-HE-Meta: U2FsdGVkX18pc74qQNJVlQFd0dd0OAJ3Wbgz4jEtf297FB3lKp0QFpmNGxts1AzXdse6h7ucFv9AZPQ/E7VdZaf3lmz4bAI+BXlq2eKWCbNyYDP/9B+Y5DP9AmZMoXPp3tN7RBhLj0XPvjzXf+d9aEAeBNbD7fRQDhdsnVcw0rv8BPP6fDl9x46ZW2ISA70AQ43BgX0qC93wSgJhqisBTfEVhtmsaextKJljX5zTrINUpYG+q/neHIhppoLla6TPaB3klzBAYXFe7C2f34eUO5/S1RGUduxSzA8YHQFwy0y25SvoGj9x0vBLx8TfrF8Yy4F/g8HSsQ6Bm4nrAmd3ZttpE1VqnPMJ+ZfXGkuJq2dXArHhqIRfAYhaS2LmkkbappSy9jj2aZf95EK8M/TUWxAKAkCIonSB6BjjgRZPo45fFH1aLTCXmS2wOXAVs8IZYBbyPJxB++aCVvI6XW+DcIzyOhfGS4ievS6fxn88DjpEqcPUfzQooZV6CS4dlm0pAv3HqGhao+sV7wes1V+GhNK02jJ1SvW4ZbQX0Uxs/Yl4KalpOFf1R9XJ0K+Cqfh1wO+yJ61wU8Vk1HwJinsYHymvY0+mViQ3szh8dUFjh9x+Pt2qf/jcw7KYyQLce1g2J2YBPWzewiNNT4WNflnQxgEzfdTUyoaESjyCf0JqgkFwQdf8Cc7vxKi1WsLuOQKkf4RsvRoKdhO8EhohmbsB7ERi1+0wWnu5b2V3SUeT+HryZq432942PCE7aUey8xfJd1sxUb/k4tEHEX87YdOKrO6oEC2+fA8GsZ5kIYfU0TCoffUhqsn7Dn1NO/1FD4reRoNLDY4Hkd1nu5OZnPY+SpWBe1yE+vsTJDIIY73q3SrZwaER5/iscIw/2UjlHug3++mQbiYvT3Zktb/PP5kYQgtt/8Yj4R6qIiiDSCkUg3BlFjTPu8fg+xFDYN2xyd/MOVYD9F8f9Duouzzm8QR 9qB3bRMc XQfgIddPcDqy1ee3Yg3g6EhWgyD2s9PD3uTIhr1VWwIJjc+HBg0SNtxe6QsquV21ipbsj/rBnNdLcKXHdD1FKsxvuaCz+gkoXyNoJLgshGBmLbm1X0K1eQTrcITdEGu0rHa2JiS5PPRzMPAKNcZgpbjWleesUEGqUW2VqPN9EHZ5zv/YWWUsTc1qLm8JGcbzQ1R09c2XtxNhP8igBsR9m5J6JDg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have almost the same implementation: __cont_access_flags_changed() is also correct on riscv and brings the same benefits (ie don't do anything if the flags are unchanged). As in the previous commit, get_clear_contig_flush() is duplicated in both the arch and the generic codes, it will be removed from the arch code when the last reference there gets moved to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 65 -------------------------------- arch/riscv/mm/hugetlbpage.c | 29 --------------- mm/contpte.c | 74 +++++++++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+), 94 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index a2d33cbc7da5..b8abbd5ace4f 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -284,71 +284,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -/* - * huge_ptep_set_access_flags will update access flags (dirty, accesssed) - * and write permission. - * - * For a contiguous huge pte range we need to check whether or not write - * permission has to change only on the first pte in the set. Then for - * all the contiguous ptes we need to check whether or not there is a - * discrepancy between dirty or young. - */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) -{ - int i; - - if (pte_write(pte) != pte_write(ptep_get(ptep))) - return 1; - - for (i = 0; i < ncontig; i++) { - pte_t orig_pte = ptep_get(ptep + i); - - if (pte_dirty(pte) != pte_dirty(orig_pte)) - return 1; - - if (pte_young(pte) != pte_young(orig_pte)) - return 1; - } - - return 0; -} - -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty) -{ - int ncontig, i; - size_t pgsize = 0; - unsigned long pfn = pte_pfn(pte), dpfn; - struct mm_struct *mm = vma->vm_mm; - pgprot_t hugeprot; - pte_t orig_pte; - - if (!pte_cont(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - if (!__cont_access_flags_changed(ptep, pte, ncontig)) - return 0; - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - - /* Make sure we don't lose the dirty or young state */ - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - hugeprot = pte_pgprot(pte); - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - set_pte_at(mm, addr, ptep, pfn_pte(pfn, hugeprot)); - - return 1; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 4c441664db8a..fdb633844d4b 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,35 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep, - pte_t pte, - int dirty) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - set_ptes(mm, addr, ptep, pte, pte_num); - - return true; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/mm/contpte.c b/mm/contpte.c index 8d2ab391e0d8..3b65c49c6d4b 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -21,6 +21,7 @@ * set_huge_pte_at() * huge_pte_clear() * huge_ptep_get_and_clear() + * huge_ptep_set_access_flags() */ pte_t huge_ptep_get(pte_t *ptep) @@ -156,3 +157,76 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } + +/* + * huge_ptep_set_access_flags will update access flags (dirty, accesssed) + * and write permission. + * + * For a contiguous huge pte range we need to check whether or not write + * permission has to change only on the first pte in the set. Then for + * all the contiguous ptes we need to check whether or not there is a + * discrepancy between dirty or young. + */ +static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +{ + int i; + + if (pte_write(pte) != pte_write(ptep_get(ptep))) + return 1; + + for (i = 0; i < ncontig; i++) { + pte_t orig_pte = ptep_get(ptep + i); + + if (pte_dirty(pte) != pte_dirty(orig_pte)) + return 1; + + if (pte_young(pte) != pte_young(orig_pte)) + return 1; + } + + return 0; +} + +static pte_t get_clear_contig_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); + return orig_pte; +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty) +{ + int ncontig; + size_t pgsize = 0; + struct mm_struct *mm = vma->vm_mm; + pte_t orig_pte; + + if (!pte_cont(pte)) + return ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + ncontig = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); + + if (!__cont_access_flags_changed(ptep, pte, ncontig)) + return 0; + + orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + + /* Make sure we don't lose the dirty or young state */ + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); + + return 1; +} From patchwork Fri Mar 1 09:14:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA912C5475B for ; Fri, 1 Mar 2024 09:23:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75F526B00A0; Fri, 1 Mar 2024 04:23:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70F806B00A2; Fri, 1 Mar 2024 04:23:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D7D86B00A3; Fri, 1 Mar 2024 04:23:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4EB2C6B00A0 for ; Fri, 1 Mar 2024 04:23:24 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1268CA1B15 for ; Fri, 1 Mar 2024 09:23:24 +0000 (UTC) X-FDA: 81847931928.15.5B7C629 Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by imf01.hostedemail.com (Postfix) with ESMTP id 445D340014 for ; Fri, 1 Mar 2024 09:23:22 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ieOlSQ+r; spf=pass (imf01.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709285002; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+Bwm42ASg+Pn4jchqsA5u8tprYh0PuExgujdSk24Zd8=; b=M+4gk9i342UWLa1p9idwbk+Za/1OlXWiUk9J+2l3M89Jb6CP8X9+g+MFyFyNTw9dsSfuid aL/U9eLm3LS3T8G7YcSQ3kypUqbOf9n7aj9OMTXVKvQMnlB2mgR6JiMQ3zvJZ0Pof3eNgu x93y5BUK/cj2jRccOWgPQNp6Y4PYWz4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709285002; a=rsa-sha256; cv=none; b=jdTyfyw8I2OpbJakVnE+CNo7RPsXLfsKi5JefuRZFZgoiLl4CtT8EKxxG7f6pmYhr0zWS6 pWacTjAJSOdOBn/fYDl/GSzMw2iIBlsSFFJDcj8H7b5WyOLpUqwr92bsBeSZT0o1xjDZxV mzuSkDEpE0pFbTa4AbhLaJgTzRnPAx4= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ieOlSQ+r; spf=pass (imf01.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2d28464c554so22607121fa.3 for ; Fri, 01 Mar 2024 01:23:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709285000; x=1709889800; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+Bwm42ASg+Pn4jchqsA5u8tprYh0PuExgujdSk24Zd8=; b=ieOlSQ+rs23ApNfVtlCx/buFmrSho5/hCC1Nz+JgpymP9unrZNYra06VdUCRE4VwHB zLIiCWfYGz5LO1VsIvJ0kd4yue17W9tQabsfsM26UWYW/v4MiWQHvg93a3vO65UzbHfp Wy32nNgTWaIGHtsQadUzIcutDuMzM6voaW+bvy3Hvwqh7W7ZMz5vDSD+/51EgorM+Ogk 63n8WSOqY14UKlkEimcI06KAw4ic6+oXq+lF39gzb5Ni0Y6A0Ui9YL8DDWrCB1UFOo0M FHppqhPG2M9gzz9L5Z0Ew4ztvoY8T8oLLyY81kFjGa6fwXmclvCYNky2Fm4Xv3fEvOd1 VKdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709285000; x=1709889800; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+Bwm42ASg+Pn4jchqsA5u8tprYh0PuExgujdSk24Zd8=; b=mqjR+L9Fb+VYB7rC/ZQMue6ti5ccZfm6OdRXkOnqC7V99s21eFzilZzBFf0QSF7vSq sRH/htwwgXt5XcCYxr1vw394bAXHoH7g8Rf3gOvHc/f5eaWUml4K2Mp/F+Mg+mvwho3d f0TFtgt0emtXhFuRRUg6z2t0ai4jllDda8VMJqvUdfku8w/OHvqLekjKkxR1CK/84V68 xMbX6z38TtDU6hkyCRBCqL0RY7a8Fo/ji3mSeLE/+wzmOkVk7YFXwO8QgS7WXrs2bfly 3XTnKpUMCXqMeW1nzrLi0rzHJUXc4JTxtkpk9gMPEBLLA/wk5u7x+lFwjpIONKDiJWvv 1A5Q== X-Forwarded-Encrypted: i=1; AJvYcCXfo705k4D4DVnnKwpw39ldfzxnxEUOo6TkjMjL23jcGjeftYUWJJPmGJZJzPg4NMYVl2rfHPSvkV0e0mOknuzBTtQ= X-Gm-Message-State: AOJu0YzgLETtrSQHLIuRQnXN/DnzPQERxViaCVhnZy6EQ63qOB5mT9sy pFGtBRQmWC8n2FcNHXdhzEFNTslnsSxzX9nY7JZmdc0O++2u/pHL+USMd8WShVU= X-Google-Smtp-Source: AGHT+IFkOrueHwu+dCituNGudpmJQSWFJOG3RRqM9+lTLTigiW3MgJH56JMq1RpUEmcE9Sa98B2xOQ== X-Received: by 2002:a2e:a4cb:0:b0:2d3:365e:9d02 with SMTP id p11-20020a2ea4cb000000b002d3365e9d02mr635537ljm.28.1709285000535; Fri, 01 Mar 2024 01:23:20 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id l33-20020a05600c1d2100b00412ca88537dsm374810wms.0.2024.03.01.01.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:23:20 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 8/9] mm: Use common huge_ptep_set_wrprotect() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:54 +0100 Message-Id: <20240301091455.246686-9-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 445D340014 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 94qbogueerkxtfeaisrgjoiwu34ryd5z X-HE-Tag: 1709285002-318335 X-HE-Meta: U2FsdGVkX1+aEp+SwObHH43yT/BT424f9gNQckZTdweShezL/qxpzC73/J4EjMiijAZ5z1HEf4xrk55RmavSkz3P/5YX1B66W/hZUACCy1jMsdP6zSSjw+SbnDVEzKpIqm3DYVqw9+AaSivvLTe9lYq2DlvAo9WPfiLOV/AnIVqBn/CB84HfVyICdr1LMKMludg2BIDCTWBDZFMcUd/6VJxVQmdEoJ2Pme/whJVKEAnaaQ5cirRvVS/1oBvhoebycR/PjncPa8ZHVZrySriSF5or4BA9r1nU/GHHLSCKljOnTTPxfGOshjMfE850d3+9/JNlHOI+igDf/XXvTyzJbcN+2AK+yUbJwqDot92eMz02i7ZJRpcs+CtXVOJ9A5twGHyQknaXN582R04tOnuSRN0z00iGV5xu9QbYK5TvuVOz3SHDhzOPBp/zoqsJUfKkAds7h2K3Sw/rwH0dZ8Cb8pvWjWYiVN37QP1IRGRjwgusddmdkQbV+ZQkQ6BN1jQg9JUM8Z8ObBVjfMUZXlqaKRtbXIRGrfikNdeei7nSDVl69XPmnA3anwX8/0m6DLYBEGFTUAAblt0DScsUtIo2aWYgN7TsjZLkcgwENw8oDqzHgmHGvYkbkjQ2UwNgylmOc6dgo+QwrfwJeIBqNsvzEMZa4mtB5Jlk8zs+bI0EFZeMDeGIeCyc1FHpu3bb7vW8W2ZMqGjlSiB2dIBjRbFzCAyxHUMH0ue07XV1jJY80fNeJKXvdfPjqvPASlxZfiSg771u4liee7WGiFLhSbTtm+ArWYJuvF+PT9QBy8PXTuoOeV3WbqhTT5/c8IDUP2mSCVg4GTlCjIHBFNNdnnVII+y76JeDtEUhh0p3DAGttSABkG7aSOJJCcya1guFhLf5jwYSswP76K23fp7PzxQVu/JXv1y99jngT7LjeT4/vbqpdHDxI+GvWRlDQBpyfy84OPcXajcxnu4A1JBNplr TDBcL3Pd jkW/x3xOprufTWhJYMP52ztlJy2Bgc2fNQKGz28/uRuGUXodTapAojoIJvDn9QD31jGO1oTbnYGRLYlKbXB1nhX/Nt9CVLlTmGK8AbzV3Q0ASwbHSN3hyQmccpGweTRKrQNa5NcEuBwNLuLLawdNLp2mqc2hAmwhQBMJ147ZTd1FHEmMjazKNNeeZL/tdzk57QVc8zca+2zKrZ+lamtEU9HZcgQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 27 --------------------------- arch/riscv/mm/hugetlbpage.c | 22 ---------------------- mm/contpte.c | 21 +++++++++++++++++++++ 3 files changed, 21 insertions(+), 49 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index b8abbd5ace4f..d6ddaf282a94 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -284,33 +284,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) -{ - unsigned long pfn, dpfn; - pgprot_t hugeprot; - int ncontig, i; - size_t pgsize; - pte_t pte; - - if (!pte_cont(READ_ONCE(*ptep))) { - ptep_set_wrprotect(mm, addr, ptep); - return; - } - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - pte = pte_wrprotect(pte); - - hugeprot = pte_pgprot(pte); - pfn = pte_pfn(pte); - - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - set_pte_at(mm, addr, ptep, pfn_pte(pfn, hugeprot)); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index fdb633844d4b..e6cbb6fb2904 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,28 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - size_t pgsize; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) { - ptep_set_wrprotect(mm, addr, ptep); - return; - } - - pte_num = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - orig_pte = pte_wrprotect(orig_pte); - - set_ptes(mm, addr, ptep, orig_pte, pte_num); -} - pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/mm/contpte.c b/mm/contpte.c index 3b65c49c6d4b..f7f26d2cfa23 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -22,6 +22,7 @@ * huge_pte_clear() * huge_ptep_get_and_clear() * huge_ptep_set_access_flags() + * huge_ptep_set_wrprotect() */ pte_t huge_ptep_get(pte_t *ptep) @@ -230,3 +231,23 @@ int huge_ptep_set_access_flags(struct vm_area_struct *vma, return 1; } + +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep) +{ + int ncontig; + size_t pgsize; + pte_t pte; + + if (!pte_cont(ptep_get(ptep))) { + ptep_set_wrprotect(mm, addr, ptep); + return; + } + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + + pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + pte = pte_wrprotect(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); +} From patchwork Fri Mar 1 09:14:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13578215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21A8CC5475B for ; Fri, 1 Mar 2024 09:24:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A95126B00A5; Fri, 1 Mar 2024 04:24:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A45066B00A6; Fri, 1 Mar 2024 04:24:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E5D46B00A7; Fri, 1 Mar 2024 04:24:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 79E036B00A5 for ; Fri, 1 Mar 2024 04:24:25 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 19D64803B4 for ; Fri, 1 Mar 2024 09:24:25 +0000 (UTC) X-FDA: 81847934490.06.583BD06 Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by imf05.hostedemail.com (Postfix) with ESMTP id 2C85010000B for ; Fri, 1 Mar 2024 09:24:22 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=Nz8b1kiB; dmarc=none; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709285063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=K12kuYfal65hYkuTm7CnsK9zS+VdU7mNn9HZDxR8IYY=; b=nfhznasb1dC4OUmpZuWAyNdYOIbvV9CWjV+299Ouur9ZC9OyrDQ9z1EYsJuwlxj9Pfti4P 10fZ7pxWPIaS1bcGoc00OZ9rBekcjasNcF3jgklibKig5WyZ6FHkmPD8bO3xqsQG2MCQN2 Cn3rrCbbGpGV8oKt4Ae7yXgNPzg+kVQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=Nz8b1kiB; dmarc=none; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709285063; a=rsa-sha256; cv=none; b=rlTfVlxpbdEjtfIeLbbcsRQEzYxk7HcuflhOa4wajl31yR5clReRWvXJRjiUJWe2HixBcT z42lEIcJQgl22KSqrlkRBBDvvNHyWewbIw/x5tpzdboefdi1TMyapLC4YFpaAuxTSgDSZU rwUzOHys1TGx0hI5wiPxSh5Hw0xLxLY= Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2d28464c554so22619191fa.3 for ; Fri, 01 Mar 2024 01:24:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1709285061; x=1709889861; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K12kuYfal65hYkuTm7CnsK9zS+VdU7mNn9HZDxR8IYY=; b=Nz8b1kiBLOuqHUDiuvYIEM7dK8sk9ygb8t7HHJhxnkuBNqW3CCCepAnN1u6Pl+5CYG R6SHzaCqPlsCGSh0Ye3+p3+I/9hHAT0xT3+SQVLU8se4LA+K9CUTh+0k9/VSHH49wuuV OW5G8xAApPN8i/YO7wgknMTq0Tkdb1pQ2FC17WoFj/a9KnEejcwN7NSq1ou7ICz6tb5/ dE6kO+mFl7U5ullXorsKpbNQW+gl8UhxnGaitzcqtf74xsEjxhSFogpt2j4HWg8YUJDr jmPmDMKVOt57HXm3tdECcC58t/kIGgvoc/54fn1DOLn2cKTp6CyqqylPQUdvANrf4MAg j17g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709285061; x=1709889861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K12kuYfal65hYkuTm7CnsK9zS+VdU7mNn9HZDxR8IYY=; b=vJ5cMDGA4e1LcKOFYe393WH4uHlzjk3715qgVO2dj8sqCm/O5RrEYSFNIk2LQ1sdPE 3YXwbFI5FKFWXVEP4aTN4JNgqnmygYFjma+8zRETvJPd67TGQQGSEBJerS9mMTOVPB42 g44DrbvGzcd2PpxmX4P7vhWsUXMHF9vlV++FvX8EkBXFWdaZnLQBCn9Yt5o724yI03gb aMyXVK36y7y+xwqKyk8CPlYu4dkqm4D7Wm8V95TtN0B7BXZtvBJDdWCPx3sA/RlMe75i Nk5wdb5wEwfjdfS5aeH6Q324Gc62SoXau/yNaXdEOsK4Z61y0noDO9frmOsMtcUWdJel 6aBA== X-Forwarded-Encrypted: i=1; AJvYcCVwbVeOtnIW97XIlhj0L/qON5MvXblSON9QeJ9aOBPAvzFge4x7atHzJsMhQm9Rnlnwdhph0JP+cHzmTBlUwjRR2H8= X-Gm-Message-State: AOJu0YxFqC6BaYwb94plXwlJDxGwn1fHCe5uGvL+PxvsHhpAhB01OOCc PDDf9hzLAVTBE8kd3BNtx9kWCbkTrYxz+tD846A2X9gYyeQ4U07onC5O/y5LoII= X-Google-Smtp-Source: AGHT+IFt+kCs8abRI8MZlthQ5rKNhpeiYCQtIBul4mlcyfPmRwi1Z8kSKZm+NO/gjQvAypId4axWxw== X-Received: by 2002:a2e:9909:0:b0:2d2:92a2:9a84 with SMTP id v9-20020a2e9909000000b002d292a29a84mr731669lji.43.1709285061585; Fri, 01 Mar 2024 01:24:21 -0800 (PST) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id w15-20020a05600c474f00b00412ca1dc2e4sm649267wmo.7.2024.03.01.01.24.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 01:24:21 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 9/9] mm: Use common huge_ptep_clear_flush() function for riscv/arm64 Date: Fri, 1 Mar 2024 10:14:55 +0100 Message-Id: <20240301091455.246686-10-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240301091455.246686-1-alexghiti@rivosinc.com> References: <20240301091455.246686-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2C85010000B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: oepc6bf5t6d8udn3fccqquz7aroncosc X-HE-Tag: 1709285062-698138 X-HE-Meta: U2FsdGVkX1+gnvvBK1C3zCngrdTwBt1wRbrDnoziYJt0Er4yEAHgoogU3PgUDgXwaYCL4tBHDkbZ0GaokpUOZRFEnBYxgypfF2TAStbiG8dloBpTTfRU+3tBMFQJbxSSsHdxqdbmslIWpovtOT9gdDCV8hTo6yegb6w593GsqGjdceFmpaoNPyXvST78bS4Wz+B0Xa5oslSaESt5BMV+oL8/H7uQZ5lMbq1JM3wgNfhTroFY/GbqCCCngZWxwWKGy2TgAOSCaGV15yvdkosB+apWZcFEfD8VFMCTtlTexyOhiv/px159oPXKaLGkRafsH5/64H4lCT9eqPf2ZwP3PdinX4q/FcS/RQvisH1+KGspwoO81uojzLx7YuS9kOQWnfZgB76GJliveRi1fh2IPgXPtMNJuGg4wOYD8qhs98B5W6vPuDQNLuYVVMecR4Vqs/7VibmS5ePFf2Z8NNoQX4qe5TSf/5rLt7Yt92S6kuKiYHABW38p+Hl7Fpcro6nhAzVAXaunLxDtGF2sHc17QVfkxCXcXZ5/aXsfP4irrjmfTKfo8MlauHAPNvRS7jLKOJ8nvGb5hjl2lAaTiZbBOya8NtKNJN2Fa7dy1cscm2M5PQPau/+c8jcF2se+NpURiZ5pCFmiSRwIgE9UhhvWFAmz/xtf06oWlgP6gq9RUIHCuFpJipdy6UbOggZ0oeCCHuRdKrgoCuwie1X+PhHzkwANxwkBrOO5WZ9EUKL6rNZ+utdKZaKYVhV4rx2k9qQJiTJjRuwA1z67TYHpaXcsc5/Po4bdWPzw8MVojXHQmPVTgLwqiShslAW/gQTEAbli/G2YrJ4Qr85VtxmlHYpyGAgPI2RxnvTeUCQTw/MsCOcqFcu+oz1BVJqoX6Jm+wKO2FX1QhT1238ygIPnImM3wNgNqt1pBnM7Zb/IpfO7Fmg5x/KdGdE39gZzxn9vzeVLhSzS3JPloX8ChH/xTFF 63gFORp+ nVr78yMllbRlWApVKKI7+29OpsQOREaS8l4vhDbTsajVbKvTpYO5X8sharKmPD5cNRkfsHDVdrXpXvSr00Va5SNSeIvh2Y14V/PnRl0mrwxgW96g6In71e8e1ctXWZjrxPv8kuXnESq4eKJyAUwXz6O8JQ9m4fwJTqT3MEKlso7Cr/T8OwBeJnRDb84ksAqjaFD8MnjoVfGlBJbg9kOBslZvgqheovOYDpkw5f4irlLb6tce7xQTIjiUOtUyCLASf7miCwmwYOwlEpThNwwzNlIJoVXWE+TDbmkREGsg2+qXhKCzymY5l36q+JjWPJEZ91DPUyB5HzR66J3st5Lgc3erzNHupgi+HDvz7CQpicL+ujh5J0zO26dM9zYS9MD0YSJjf+Wue4m068nNdTMzoupwaCYa9Myf95tXaehj6AQMqAQ1P461hDM9I/5z1to7XYF1BvaKrb1gEw2DZa3Y7z+YFFUTS6/eRkShUThlImRWdDLko7aOVR6t0hg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: After some adjustments, both architectures have the same implementation so move it to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 61 ------------------------------------- arch/riscv/mm/hugetlbpage.c | 51 ------------------------------- mm/contpte.c | 15 +++++++++ 3 files changed, 15 insertions(+), 112 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index d6ddaf282a94..8a273b9816d7 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -119,53 +119,6 @@ int find_num_contig(struct mm_struct *mm, unsigned long addr, return CONT_PTES; } -/* - * Changing some bits of contiguous entries requires us to follow a - * Break-Before-Make approach, breaking the whole contiguous set - * before we can change any entries. See ARM DDI 0487A.k_iss10775, - * "Misprogramming of the Contiguous bit", page D4-1762. - * - * This helper performs the break step. - */ -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = ptep_get(ptep); - unsigned long i; - - for (i = 0; i < ncontig; i++, addr += pgsize, ptep++) { - pte_t pte = ptep_get_and_clear(mm, addr, ptep); - - /* - * If HW_AFDBM is enabled, then the HW could turn on - * the dirty or accessed bit for any page in the set, - * so check them all. - */ - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pgsize, - unsigned long ncontig) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - - flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); - return orig_pte; -} - pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { @@ -284,20 +237,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - int ncontig; - - if (!pte_cont(READ_ONCE(*ptep))) - return ptep_clear_flush(vma, addr, ptep); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); -} - static int __init hugetlbpage_init(void) { if (pud_sect_supported()) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index e6cbb6fb2904..caf1db6f8f20 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -121,42 +121,6 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) return 0UL; } -static pte_t get_clear_contig(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = ptep_get(ptep); - unsigned long i; - - for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) { - pte_t pte = ptep_get_and_clear(mm, addr, ptep); - - if (pte_dirty(pte)) - orig_pte = pte_mkdirty(orig_pte); - - if (pte_young(pte)) - orig_pte = pte_mkyoung(orig_pte); - } - - return orig_pte; -} - -static pte_t get_clear_contig_flush(struct mm_struct *mm, - unsigned long addr, - pte_t *ptep, - unsigned long pte_num) -{ - pte_t orig_pte = get_clear_contig(mm, addr, ptep, pte_num); - struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); - bool valid = !pte_none(orig_pte); - - if (valid) - flush_tlb_range(&vma, addr, addr + (PAGE_SIZE * pte_num)); - - return orig_pte; -} - pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { unsigned long order; @@ -173,21 +137,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep) -{ - pte_t pte = ptep_get(ptep); - int pte_num; - - if (!pte_napot(pte)) - return ptep_clear_flush(vma, addr, ptep); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL); - - return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); -} - static bool is_napot_size(unsigned long size) { unsigned long order; diff --git a/mm/contpte.c b/mm/contpte.c index f7f26d2cfa23..445e5ebe46b4 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -23,6 +23,7 @@ * huge_ptep_get_and_clear() * huge_ptep_set_access_flags() * huge_ptep_set_wrprotect() + * huge_ptep_clear_flush() */ pte_t huge_ptep_get(pte_t *ptep) @@ -251,3 +252,17 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, set_contptes(mm, addr, ptep, pte, ncontig, pgsize); } + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + struct mm_struct *mm = vma->vm_mm; + size_t pgsize; + int ncontig; + + if (!pte_cont(ptep_get(ptep))) + return ptep_clear_flush(vma, addr, ptep); + + ncontig = arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize); + return get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); +}