From patchwork Thu May 11 13:21:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13237922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 980D3C7EE22 for ; Thu, 11 May 2023 13:21:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 354BD6B007B; Thu, 11 May 2023 09:21:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 304766B007D; Thu, 11 May 2023 09:21:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F3F96B007E; Thu, 11 May 2023 09:21:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0D2956B007B for ; Thu, 11 May 2023 09:21:27 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1387916097F for ; Thu, 11 May 2023 13:21:26 +0000 (UTC) X-FDA: 80778035772.06.EEF67CF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id DAAA0180002 for ; Thu, 11 May 2023 13:21:23 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683811284; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=XT2il+uD6+IJXosawb0fjTyQiRQ8N3Tm4nPIfzQEaKM=; b=teR2d+3oic38voNThZSNN/uiWZ+iEg1wWSGlVrFZauLnwIEhX5RJi6P8sN+m4XxVzVhSv6 /N0/9kwSZjXow6S87514Hx/B9dWQl4C+v+ZYzZjJ11IvENJ0I/zNz4Bo5pe7cFDlJRI8/A UpvnF9e9xfPP2dZTp9a+Jh+hJW/AwFE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683811284; a=rsa-sha256; cv=none; b=YNkeUQy0ip+OYqgf7oK+EnsYxfMIgiibcXI7O/SZd2HgUcl8H1/+hVwHZX1nT6hqeaEICT pt4giniC8H6Kpv68zJelDvpSjBtpGR8MpmumZ0ymTCVe1CJTLoMIJ66QjHgBcQ9JAKZbQg xhjtjn67nO5ugZhPclH8DXkyAvxHfZc= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6EA72113E; Thu, 11 May 2023 06:22:07 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C67193F5A1; Thu, 11 May 2023 06:21:21 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , SeongJae Park Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org, damon@lists.linux.dev Subject: [RESEND PATCH v1 0/5] Encapsulate PTE contents from non-arch code Date: Thu, 11 May 2023 14:21:08 +0100 Message-Id: <20230511132113.80196-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Stat-Signature: h7xzmgiamfkegahajzyc31qwyyucesx4 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DAAA0180002 X-Rspam-User: X-HE-Tag: 1683811283-621308 X-HE-Meta: U2FsdGVkX1+9r9saNmXkvaxO6O3KZTPsRcq5MDPNXUQrKlqAvyhJS6VA3WwnBetrk2VMOf+dpo/0XTMBa64ZPs/h1z1JwyQGXb1lz0B7cwyY4TrTbuDB5r/fR+IMdS+Zg8qS8DRNYqfCNyLTN8UFPrMdgVKHH1/ZOgnvxhUpXz/mjDc+a4ftikXgzl3Gy5gpTQ/FsEig5nPgCPOrvoO15Sl2wiVlQsw7E8trKuXHTi8l//iAGXrjUXkGQckHVxkDCbus5UpKrvUhUtRFUbacNVnT0pweNdLZUBQ2bpskDTc/S28d/uY1PRJF79bm2SnpDQXKg0C38Ib+HtcRANBI0o3JHGKlQQKggV5AeEhaE6o8C+SuMS+xwrek8dgFOhzj3zmm98c8VodxUtbuq2vBFILcACg/iBm3nqAm/2C9ofsLUheG4cIxGLq28aisoNO6gTOCO7pN4qWKfayDm+14uEMmpog5I/A0kjGYorSwmvtFCgYa6JpM/Uv95XGV/bTU7fymiABsPOUysnzwT6MP+WM1P3tSeTHXGtE8n52wLU2zeVo6daqc5NTPwYSyZ77GcywGAEkvgrCxSTUmJ7RcfjIJqh+phtWmoP9F1ethmG3BSG2kRRb5aLmZNvsrMSOU8h1VPwhjyOsK2L4qtqpKLrWAuYzZYSmr34M7G6cN22ebhZZbCgEMkuMefYSJwjl/2bdSfs3rj6h2J5U12JEwCMLwHM7fCFVcuepuMBsRvod1SRG+Hl+SAF5ley8Q3eUR0kofF4yQZ6hmYL7Bfv6jQWYqvCimDc6Mf3VSA3Eo1+pzvs6rsR762EZO8rDDK5dzUHQ7CjDv7/YjCIw5nZ7ZhdBuvdOj9+MMb7Ub6wfMbNmPsG9w/lPih3QTyh9LVfG21Q3bG3BBv7T1ik9FutiLpkT7b0utdAb/SYR2cAZh/xMHA3B081BB2rL5NjPxYIMoHocTHyiQhBr87CS74Mm SPJT3S19 rsOLT/mbOViiW6j2f/8RNhYbKie74Nr2geGw+S03GSsPtL5T1s/pGRZkOUU7zwQr+0vuCVmIfYr4K4JKoLuFBorEyqX+Xy3FmBG8e7d714n/UbmgsxfQ1G6ntnZ3zfW20WNxQd5Rx4vvqhK2CmeyRviZoV5Sd9YZ46AkD0SrbiGSy1ZWi4olvgFvM1ttqkvGJdi94FFwoLJWp5OWVw07YXJMGNQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi All, This series improves the encapsulation of pte entries by disallowing non-arch code from directly dereferencing pte_t pointers. Instead code must use a new helper, `pte_t ptep_deref(pte_t *ptep)`. By default, this helper does a direct dereference of the pointer, so generated code should be exactly the same. But it's presence sets us up for arch code being able to override the default to "virtualize" the ptes without needing to maintain a shadow table. I intend to take advantage of this for arm64 to enable use of its "contiguous bit" to coalesce multiple ptes into a single tlb entry, reducing pressure and improving performance. I have an RFC for the first part of this work at [1]. The cover letter there also explains the second part, which this series is enabling. I intend to post an RFC for the contpte changes in due course, but it would be good to get the ball rolling on this enabler. There are 2 reasons that I need the encapsulation: - Prevent leaking the arch-private PTE_CONT bit to the core code. If the core code reads a pte that contains this bit, it could end up calling set_pte_at() with the bit set which would confuse the implementation. So we can always clear PTE_CONT in ptep_deref() (and ptep_get()) to avoid a leaky abstraction. - Contiguous ptes have a single access and dirty bit for the contiguous range. So we need to "mix-in" those bits when the core is dereferencing a pte that lies in the contig range. There is code that dereferences the pte then takes different actions based on access/dirty (see e.g. write_protect_page()). While ptep_get() and ptep_get_lockless() already exist, both of them are implemented using READ_ONCE() by default. While we could use ptep_get() instead of the new ptep_deref(), I didn't want to risk performance regression. Alternatively, all call sites that currently use ptep_get() that need the lockless behaviour could be upgraded to ptep_get_lockless() and ptep_get() could be downgraded to a simple dereference. That would be cleanest, but is a much bigger (and likely error prone) change because all the arch code would need to be updated for the new definitions of ptep_get(). The series is split up as follows: patchs 1-2: Fix bugs where code was _setting_ ptes directly, rather than using set_pte_at() and friends. patch 3: Fix highmem unmapping issue I spotted while doing the work. patch 4: Introduce the new ptep_deref() helper with default implementation. patch 5: Convert all direct dereferences to use ptep_deref(). [1] https://lore.kernel.org/linux-mm/20230414130303.2345383-1-ryan.roberts@arm.com/ Thanks, Ryan Ryan Roberts (5): mm: vmalloc must set pte via arch code mm: damon must atomically clear young on ptes and pmds mm: Fix failure to unmap pte on highmem systems mm: Add new ptep_deref() helper to fully encapsulate pte_t mm: ptep_deref() conversion .../drm/i915/gem/selftests/i915_gem_mman.c | 8 +- drivers/misc/sgi-gru/grufault.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 7 +- drivers/xen/privcmd.c | 2 +- fs/proc/task_mmu.c | 33 +++--- fs/userfaultfd.c | 6 +- include/linux/hugetlb.h | 2 +- include/linux/mm_inline.h | 2 +- include/linux/pgtable.h | 13 ++- kernel/events/uprobes.c | 2 +- mm/damon/ops-common.c | 18 ++- mm/damon/ops-common.h | 4 +- mm/damon/paddr.c | 6 +- mm/damon/vaddr.c | 14 ++- mm/filemap.c | 2 +- mm/gup.c | 21 ++-- mm/highmem.c | 12 +- mm/hmm.c | 2 +- mm/huge_memory.c | 4 +- mm/hugetlb.c | 2 +- mm/hugetlb_vmemmap.c | 6 +- mm/kasan/init.c | 9 +- mm/kasan/shadow.c | 10 +- mm/khugepaged.c | 24 ++-- mm/ksm.c | 22 ++-- mm/madvise.c | 6 +- mm/mapping_dirty_helpers.c | 4 +- mm/memcontrol.c | 4 +- mm/memory-failure.c | 6 +- mm/memory.c | 103 +++++++++--------- mm/mempolicy.c | 6 +- mm/migrate.c | 14 ++- mm/migrate_device.c | 14 ++- mm/mincore.c | 2 +- mm/mlock.c | 6 +- mm/mprotect.c | 8 +- mm/mremap.c | 2 +- mm/page_table_check.c | 4 +- mm/page_vma_mapped.c | 26 +++-- mm/pgtable-generic.c | 2 +- mm/rmap.c | 32 +++--- mm/sparse-vmemmap.c | 8 +- mm/swap_state.c | 4 +- mm/swapfile.c | 16 +-- mm/userfaultfd.c | 4 +- mm/vmalloc.c | 11 +- mm/vmscan.c | 14 ++- virt/kvm/kvm_main.c | 9 +- 48 files changed, 302 insertions(+), 236 deletions(-) --- 2.25.1