From patchwork Sun May 26 09:22:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13674211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A0A5C25B75 for ; Sun, 26 May 2024 09:22:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC3D76B0085; Sun, 26 May 2024 05:22:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B758A6B0088; Sun, 26 May 2024 05:22:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3C316B0089; Sun, 26 May 2024 05:22:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 874826B0085 for ; Sun, 26 May 2024 05:22:32 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 304AD160B91 for ; Sun, 26 May 2024 09:22:32 +0000 (UTC) X-FDA: 82160006544.23.3CD9562 Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by imf01.hostedemail.com (Postfix) with ESMTP id 0795B40008 for ; Sun, 26 May 2024 09:22:29 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=csgroup.eu; spf=pass (imf01.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716715350; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=AvHOpL06OsfSUyFd4DIfbtnZ8kjN6KROAgH6FAYsjTc=; b=RwLclv4Zu6YTgidD5aEi23WmWyNPmz06qEeonSDixtDHC5T8KM2Cc9AMXbcYQ4Z8A6BL0W 1dS9/bjp0Zad59TLCDin8FoO3PObxDBwpyBRpLQXpfs2e6v46yRyV4URUcmOUK1uhi7kpY YjgklZfPDDktGJQWOxVW/oO9mLV6YbU= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=csgroup.eu; spf=pass (imf01.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716715350; a=rsa-sha256; cv=none; b=NyyxuI4vqPTroCdNUPntzqGPkBINTSn6bVmlQTdLJMEWQ4OjV0basvT75bN0OptzdhM6ym 5GJ4mXTANcffBmCeaNdJsCxHCMrNASown0lcPoAC5xXldlnLGIR71FMIF1W/muwdsWJBMD 6YO+90LUsWsvbfta6NIarBNfp4ghdTQ= Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4VnCwg5Qnlz9scH; Sun, 26 May 2024 11:22:27 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5fo1q1rQEO3k; Sun, 26 May 2024 11:22:27 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4VnCwg47nzz9sx4; Sun, 26 May 2024 11:22:27 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 856478B774; Sun, 26 May 2024 11:22:27 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id YDpGNZfzXcwu; Sun, 26 May 2024 11:22:27 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.233.45]) by messagerie.si.c-s.fr (Postfix) with ESMTP id EB2488B764; Sun, 26 May 2024 11:22:26 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Jason Gunthorpe , Peter Xu , Oscar Salvador , Michael Ellerman , Nicholas Piggin Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH v3 00/16] Reimplement huge pages without hugepd on powerpc (8xx, e500, book3s/64) Date: Sun, 26 May 2024 11:22:20 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1716715343; l=6917; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=yEceD/hbpdIAELHf8uEaHdWd6UwZeeDcHUyMYFNuong=; b=5G7rRyOZLOxROPS9SmrKrxRmd20BM59A1ayCeawDYbnfoaVKrzWhAMJEc7/FZUbtPk61UGZrZ IKThwUehpKTB6vDGZ9keT4HXJkY1eg4g3ft38eV1WpBcOAPSDzCfn33 X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Stat-Signature: iqhw4aah8agcqok9yeqiz9rctrwqzyem X-Rspamd-Queue-Id: 0795B40008 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1716715349-473080 X-HE-Meta: U2FsdGVkX19TzyotCi6R73t+emobTANUDnFTnRCsSEAE4lmFJddmq0MdQBTD9GXc/lM3jXprln1TlxKN5ViiNGNDccmayxV7B7woHdgh4J/hUODPKMUbiy6//7j59SDwWCgRfFN8SF7Lf9Y+eUB32MbnyJj5weW6tBg5sbOIaBouIaZW5CSBW/pC56Yh67RPDZxzIHBN6gMXG74ECnUd59dmfXPp3dlq9viwDLKsWAPTRMNNfXWaZSXu+rl5PxhgPd/f5TVy6t+yoQ8QFA1Xs2cfAVEI8MUxPoenKNrgW00CPaczlOIdPdOCovLoYer++itEyADScaz6A+U03LrtNDlMi07BHVX7ktSV21jE5Xa4p68PQX81d98FSB1qho+aH5ZrwxksGP5C9WrJ/etJXPxqGoOWW70ixLHvp1aCLSjgT9GIjFtIN1HHG5HZb3UQ153IR2pBJgVL4H25VruqWAtnW4N3FduL01Zse9Qs0suIpSvEcw6JzkWPNpDznv9JmWQiJ98eCkAf6w3b77YuSchFMrl8uPAcZCZZ8caZa72B7WW3joH3+tE6N3QVqmo0miibs+4Ls+fCH3alPGeRwORDTvN12EyWldP3w0aQXgSBhr+m8bSrTKqi99WDhAZ2DyoeQS5SaIZaDPG4f0u4oFzsYXUF6+u87qKfX6pOIa4QgemgYa7kKAaUXwR/WkcXHwg4DzJeF8VxxfcmTntT8LGY6QltC5M/m40LQ7yQncTmbRxy9Z+H8KWvp+kvS2ALgb6QUqgcem44GlGDt2iuLq8BoFVulxnoLK2KQ151TeLmKiJjUEtcBJUCatlTEmePsdfzu5b2FpbXbrscht7MlH/GxK0FOQp0yc0z85fADrY5TzkXfj+Lbitu5LEHqHQgi50+JXaWmuF2FixhFYYywokxmPkG8U0Dnn4wQ34iplBauyp+XKfbjJlk5/buc8PYOCev0Uiin40J6tpXnLA d0JQ5uai Z71xJlTueCYjTBeOhPGmQLbWv4w+4oEC8VSUXyiKLuv597KvX2AiunWXhX/Rb8C6zmUqUrcFfIWfa/RrtSYmTfWaIZg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is the continuation of the RFC v1 series "Reimplement huge pages without hugepd on powerpc 8xx". It now get rid of hugepd completely after handling also e500 and book3s/64 Also see https://github.com/linuxppc/issues/issues/483 Unlike most architectures, powerpc 8xx HW requires a two-level pagetable topology for all page sizes. So a leaf PMD-contig approach is not feasible as such. Possible sizes on 8xx are 4k, 16k, 512k and 8M. First level (PGD/PMD) covers 4M per entry. For 8M pages, two PMD entries must point to a single entry level-2 page table. Until now that was done using hugepd. This series changes it to use standard page tables where the entry is replicated 1024 times on each of the two pagetables refered by the two associated PMD entries for that 8M page. For e500 and book3s/64 there are less constraints because it is not tied to the HW assisted tablewalk like on 8xx, so it is easier to use leaf PMDs (and PUDs). On e500 the supported page sizes are 4M, 16M, 64M, 256M and 1G. All at PMD level on e500/32 (mpc85xx) and mix of PMD and PUD for e500/64. We encode page size with 4 available bits in PTE entries. On e300/32 PGD entries size is increases to 64 bits in order to allow leaf-PMD entries because PTE are 64 bits on e500. On book3s/64 only the hash-4k mode is concerned. It supports 16M pages as cont-PMD and 16G pages as cont-PUD. In other modes (radix-4k, radix-6k and hash-64k) the sizes match with PMD and PUD sizes so that's just leaf entries. The hash processing make things a bit more complex. To ease things, __hash_page_huge() is modified to bail out when DIRTY or ACCESSED bits are missing, leaving it to mm core to fix it. Global changes in v3: - Removed patches 1 and 2 - Squashed patch 11 into patch 5 - Replaced patches 12 and 13 with a series from Michael - Reordered patches a bit to have more general patches up front For more details on changes, see in each patch. Christophe Leroy (15): mm: Define __pte_leaf_size() to also take a PMD entry mm: Provide mm_struct and address to huge_ptep_get() powerpc/mm: Remove _PAGE_PSIZE powerpc/mm: Fix __find_linux_pte() on 32 bits with PMD leaf entries powerpc/mm: Allow hugepages without hugepd powerpc/8xx: Fix size given to set_huge_pte_at() powerpc/8xx: Rework support for 8M pages using contiguous PTE entries powerpc/8xx: Simplify struct mmu_psize_def powerpc/e500: Remove enc and ind fields from struct mmu_psize_def powerpc/e500: Switch to 64 bits PGD on 85xx (32 bits) powerpc/e500: Encode hugepage size in PTE bits powerpc/e500: Use contiguous PMD instead of hugepd powerpc/64s: Use contiguous PMD/PUD instead of HUGEPD powerpc/mm: Remove hugepd leftovers mm: Remove CONFIG_ARCH_HAS_HUGEPD Michael Ellerman (1): powerpc/64e: Remove unused IBM HTW code [SQUASHED] arch/arm/include/asm/hugetlb-3level.h | 4 +- arch/arm64/include/asm/hugetlb.h | 2 +- arch/arm64/mm/hugetlbpage.c | 2 +- arch/powerpc/Kconfig | 1 - arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 - arch/powerpc/include/asm/book3s/64/hash-4k.h | 15 - arch/powerpc/include/asm/book3s/64/hash.h | 38 +- arch/powerpc/include/asm/book3s/64/hugetlb.h | 38 -- .../include/asm/book3s/64/pgtable-4k.h | 34 -- .../include/asm/book3s/64/pgtable-64k.h | 20 - arch/powerpc/include/asm/book3s/64/pgtable.h | 18 + arch/powerpc/include/asm/hugetlb.h | 15 +- .../include/asm/nohash/32/hugetlb-8xx.h | 38 +- arch/powerpc/include/asm/nohash/32/mmu-8xx.h | 9 +- arch/powerpc/include/asm/nohash/32/pte-40x.h | 3 - arch/powerpc/include/asm/nohash/32/pte-44x.h | 3 - arch/powerpc/include/asm/nohash/32/pte-85xx.h | 3 - arch/powerpc/include/asm/nohash/32/pte-8xx.h | 58 ++- .../powerpc/include/asm/nohash/hugetlb-e500.h | 36 +- arch/powerpc/include/asm/nohash/mmu-e500.h | 6 +- arch/powerpc/include/asm/nohash/pgalloc.h | 2 - arch/powerpc/include/asm/nohash/pgtable.h | 45 +- arch/powerpc/include/asm/nohash/pte-e500.h | 35 +- arch/powerpc/include/asm/page.h | 32 -- arch/powerpc/include/asm/pgtable-be-types.h | 10 - arch/powerpc/include/asm/pgtable-types.h | 13 +- arch/powerpc/include/asm/pgtable.h | 3 + arch/powerpc/kernel/exceptions-64e.S | 4 +- arch/powerpc/kernel/head_85xx.S | 33 +- arch/powerpc/kernel/head_8xx.S | 10 +- arch/powerpc/kernel/setup_64.c | 6 +- arch/powerpc/mm/book3s64/hash_utils.c | 11 +- arch/powerpc/mm/book3s64/hugetlbpage.c | 10 + arch/powerpc/mm/book3s64/pgtable.c | 12 - arch/powerpc/mm/hugetlbpage.c | 455 +--------------- arch/powerpc/mm/init-common.c | 8 +- arch/powerpc/mm/kasan/8xx.c | 21 +- arch/powerpc/mm/nohash/8xx.c | 43 +- arch/powerpc/mm/nohash/Makefile | 2 +- arch/powerpc/mm/nohash/book3e_pgtable.c | 4 +- arch/powerpc/mm/nohash/tlb.c | 407 +-------------- arch/powerpc/mm/nohash/tlb_64e.c | 314 ++++++++++++ arch/powerpc/mm/nohash/tlb_low_64e.S | 484 ++---------------- arch/powerpc/mm/pgtable.c | 92 ++-- arch/powerpc/mm/pgtable_32.c | 2 +- arch/riscv/include/asm/hugetlb.h | 2 +- arch/riscv/mm/hugetlbpage.c | 2 +- arch/s390/include/asm/hugetlb.h | 4 +- arch/s390/mm/hugetlbpage.c | 4 +- fs/hugetlbfs/inode.c | 2 +- fs/proc/task_mmu.c | 8 +- fs/userfaultfd.c | 2 +- include/asm-generic/hugetlb.h | 2 +- include/linux/hugetlb.h | 6 - include/linux/pgtable.h | 3 + include/linux/swapops.h | 2 +- kernel/events/core.c | 2 +- mm/Kconfig | 10 - mm/damon/vaddr.c | 6 +- mm/gup.c | 105 +--- mm/hmm.c | 2 +- mm/hugetlb.c | 46 +- mm/memory-failure.c | 2 +- mm/mempolicy.c | 2 +- mm/migrate.c | 4 +- mm/mincore.c | 2 +- mm/pagewalk.c | 57 +-- mm/userfaultfd.c | 2 +- 68 files changed, 754 insertions(+), 1926 deletions(-) create mode 100644 arch/powerpc/mm/nohash/tlb_64e.c