From patchwork Mon Feb 10 19:37:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13968593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43A11C0219E for ; Mon, 10 Feb 2025 19:39:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C419E28000F; Mon, 10 Feb 2025 14:39:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BF13128000B; Mon, 10 Feb 2025 14:39:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A6A8228000F; Mon, 10 Feb 2025 14:39:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 851F428000B for ; Mon, 10 Feb 2025 14:39:01 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0AE5680515 for ; Mon, 10 Feb 2025 19:39:01 +0000 (UTC) X-FDA: 83105048082.21.62D35F1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf29.hostedemail.com (Postfix) with ESMTP id C6852120014 for ; Mon, 10 Feb 2025 19:38:58 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KnvVN5IM; spf=pass (imf29.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739216338; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mwHnciP+OvnJG9alUSG9/jEypyVF3ZPTXQLV9dVcfyg=; b=svyfhMss+ELEIaYrnJBhLxIGStdkbLZadyNNFR+Vv3ikir10Nm2BExs9ABngxeRTFG3lBm vRGky9XQPkNb1WQM8vt4tSB1Vga2cLlgzaI4k8A65qP9f4gh/oIsHjfnnbxMHkFme0NkoF w1/NLCAl6BURobVtvdkhnOqhIaFLn/U= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KnvVN5IM; spf=pass (imf29.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739216338; a=rsa-sha256; cv=none; b=1eL7ScwFzpR/3ropSueW8oEj59bIa6DUenYEIZKbksS+/vtLu6JIGYUpOqGJZbSrt8L3ss KD9/gxOe+x8ZDlPfN5jl48ae36RKfiP8UaO9qpF0W6g+KceDmOInVXh+tm+RFP0Z9116Di aDoNYn2eL1u6y4q74i+6o2yRyRhLfUY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739216338; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mwHnciP+OvnJG9alUSG9/jEypyVF3ZPTXQLV9dVcfyg=; b=KnvVN5IMxtkcDDowFZqh8kQ42q8NrSJE/w9c6lx64bRs/ftMieqW5Fku5dXR6/AXHzliy9 g8GFOHvit0FYelXwJteVtcIbv8VG8gbgbzJMAmyu9Slv+QzYhyRU55gGdwfpvPdDO1MhfU zksoefqjEMs0O5gRtjGL4O9R67iDvvY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-121-0dl4y5PCMW6riv9Z_zUIrg-1; Mon, 10 Feb 2025 14:38:55 -0500 X-MC-Unique: 0dl4y5PCMW6riv9Z_zUIrg-1 X-Mimecast-MFC-AGG-ID: 0dl4y5PCMW6riv9Z_zUIrg Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-38dc6aad9f8so1835252f8f.1 for ; Mon, 10 Feb 2025 11:38:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739216334; x=1739821134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mwHnciP+OvnJG9alUSG9/jEypyVF3ZPTXQLV9dVcfyg=; b=bI/tYL7b4dqXQGigdMgdTi5Gjnnbs6LsNmX8X1zXFd1gkgQAHJjX8mnmi6k+wGqs+i dF1Bf4aOKE5niBjFmWegkDyI/VY+RsdgtVXP/Yfeicg6+gN1YnHh/eGxnlMivcK0BrML vw+uAFK8wTvfbl8sn/WZiIE8UHLy9M4RZ2TG9359Oli0toF5lLkVSbvOEkm4MTAuzFRU 7rEWM8nvyQ3hmXlEw4uSAfhY5AfW7DMsNRdjOXp05H/dcziYKkTXeuHIjTliKi6YQa/U TBFo61z5sDElVkH+kQ7pZjUyNk2vd2wShM+iWC3KXjjcL1wrxXOOyn3MB/IJbCm0peq+ cJrA== X-Forwarded-Encrypted: i=1; AJvYcCUBINRx2GNLqJzbfkOSkfV24KrFhtJuurwyt78Xjy1h66IOAZ97bnR7+a2lwEZ3fFY1+OMHhWM1Xw==@kvack.org X-Gm-Message-State: AOJu0Yz+VPMojPS2PjDVuZekOSaTUV4VjAfPuyRi5o4Fn4YpMeGiDw8b 590Ot3IUkYt2CBHLbQqaf5H/WSdDy9nJlOOLilTMEx+RzoHpVSnROitEBNtdiO7gphz/kGqt9Aj vMmCUjGKMpJ4BCAzHQg9t5+lkJGG8mSxuiDtopA9c7yBM4jYv X-Gm-Gg: ASbGncssroXTkpBIjhVbuUVjVRUcgPzfHCN9AsP/uEXXNJ6951kyM1AGOOWL5jd1Zqk Un1m20kohW68aMVP0CQeG86+BtaxJP08g6ijnBg7iHLIytHggkaVpzay2gVG1LwBd9bp710m6jt /YwZP0neHglOX3fb375XbLEU3rtYokF9lfIvEAk8gco3I3slIjyxidVrJkoSjUYBfEX2qEQo4cy vq7CEbkMuEgx/M+mZ5qA6cTPrsUS7ith1OMSJm7clCM/po5OEP0VrhZ+ljRIzGYNfWuLRzbyI6R licSAwMPzOMYrLY/tWC/gWu1ciUULs1LxcWtJOqO6vbahFeSnURVhKOqRHfN1jkbPA== X-Received: by 2002:a5d:5f42:0:b0:38d:df15:2770 with SMTP id ffacd0b85a97d-38de432d90fmr568617f8f.0.1739216333904; Mon, 10 Feb 2025 11:38:53 -0800 (PST) X-Google-Smtp-Source: AGHT+IEQ8jI3Pr0PhgIrX5XFi88nBpwoaMc1TLFqexlyUt/iDFsCN9k6Hzlrlovdfu0/+3mYproMTQ== X-Received: by 2002:a5d:5f42:0:b0:38d:df15:2770 with SMTP id ffacd0b85a97d-38de432d90fmr568579f8f.0.1739216333460; Mon, 10 Feb 2025 11:38:53 -0800 (PST) Received: from localhost (p200300cbc734b80012c465cd348aaee6.dip0.t-ipconnect.de. [2003:cb:c734:b800:12c4:65cd:348a:aee6]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-4390db11200sm187831345e9.38.2025.02.10.11.38.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 10 Feb 2025 11:38:52 -0800 (PST) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, damon@lists.linux.dev, David Hildenbrand , Andrew Morton , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Jonathan Corbet , Alex Shi , Yanteng Si , Karol Herbst , Lyude Paul , Danilo Krummrich , David Airlie , Simona Vetter , Masami Hiramatsu , Oleg Nesterov , Peter Zijlstra , SeongJae Park , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn , Pasha Tatashin , Peter Xu , Alistair Popple , Jason Gunthorpe Subject: [PATCH v2 13/17] mm/page_idle: handle device-exclusive entries correctly in page_idle_clear_pte_refs_one() Date: Mon, 10 Feb 2025 20:37:55 +0100 Message-ID: <20250210193801.781278-14-david@redhat.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250210193801.781278-1-david@redhat.com> References: <20250210193801.781278-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Twe-AR8wHI6pEjPbTI6f1WrzteQlg5tCkqYxMqiSW6s_1739216334 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-Rspamd-Queue-Id: C6852120014 X-Stat-Signature: zes1n6auf7prwf1nu4pny3dbife5dh8i X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1739216338-697116 X-HE-Meta: U2FsdGVkX18RYD4XQh7wBVgd4NIk6kYpOuG8Pt15kCE3bItG+763bizcbINOMhdW3AR09GEd+TldkJWWmy3YXIfKaw2j3qjzJYr2Udci4bOXIE7Qeyvdz5xDnkzZqy7VmYOdAkvMh2Rel8VopuPjks9J4IvYaM/NAV/gUcuLAwa/vrwlciBK47eTdkrnQEdklDkJUz3WJEdsiTGokf4G462kpdg1Yj0gKkhwSB0iRMYZ4baLrvD4ur7/qbivCTN71nAKufNp3TiHJVkXPFynF+ak7zKoUHOYtyBayKB6/9locdePbMU/FMUX1v+jpvXlG2tOSw9S6phF5ZLX/d8ItnDfQR+rbubOTdceIRDfwNIObLYVkxKb/FVoONqPWM39KsnkD2uYTCL5lGpYDAqYvrk+GFepfEDOK2rN9qWzalnN6zNaM8s1a2uxe02DBhJLvTlC2qzSc2qGCDHQMuOa9rsYwLVv5NLOuDrWUdGo2ausCCrXq3TL6lJW9XC0gHivlpGgKnUskJW+TawhI00se9BUmOpnaXUHiC9qDXysrfSJ+fXqH2Y8gLDcwT9lgkdz4TU3qLlMLHC2gXfIdIpQgUVfuhRPriJCsfgF0r4kvFk2JG7hIORsxtJYITFeJhKhRLP4P8/gOkxLEfczsgIElRGloajV4erD2S7Api1ekkDZi34/H7UA9C89toEPP0bXktI8J0D0u/1MfAFv1exn1Wd3DLZZX1LmFzA9gwpgbQB//on2Crw7QkRREzpVfbHoIQwiVBHCQl8IaYiALBcx+Nz+UDtuHuw52apsF+OZhsqX721SgV+gnLuJ3MNISIm4W0rnAtErP+z0Oa3rmxOd9zj5FNFL8AVaQmU1yk+05HHQE4XRhscMdJEJXHLzhnCHeAgUSNrVMQWGnV2cvj6P4l3hkG9heBgMF/PnL9+ihOCtbRCcSO1Uq1mYmxJgG9WKI6wEfVI6sFnz9qoQaL2 VF1PQJy7 QMV8KMM6mFTl+8k7yztUjqBHltFQmXCAo0VbhnebLufkewKYrsVF7lQ602RqYxigZjpEO/LX9BP2KY5GEpbnHU+0G4jHVLxtu2dTCLM9MiDkbe22MiExY8IjD8TJbJblqoGDTk5dVuif1doxyTQqxkbHEGfZjEcdFRVCOh7/mjAAtxzFR8O1Zje2O3cQTULsfu5bdfU3enUkIBZmySOEIseMMN0Q8bER615vdDG6+/Dd7JI7mV7g3WYC1xgtikrXcxJcyKozcVtF4oMJ748oQ7h1igL9eqjxMfKOplg1UMI/F6aVLsyAvhy6LIvAe3nFSeO2ZCrvGpSmLXwKGKSfo1Ld1qeZ040Xw92It1xuqTEROK78cmcz8nU7jmrOiJeoDex1UQgyr/ATDVjFvAm4YmtOeyvAjW+B+EBqNPc2ZdHDhF88ygxxkHS2m9dkz9Ck2dqfQQ2P3j91mnhNJSxizcOWVkvRbwIO8ERZasV82Sex0WL0HNMxc2Zk90/sg1mDlJgxdg3DhyO42AVm5n6Rr33idUg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ever since commit b756a3b5e7ea ("mm: device exclusive memory access") we can return with a device-exclusive entry from page_vma_mapped_walk(). page_idle_clear_pte_refs_one() is not prepared for that, so let's teach it what to do with these PFN swap PTEs. Note that device-private entries are so far not applicable on that path, as page_idle_get_folio() filters out non-lru folios. Should we just skip PFN swap PTEs completely? Possible, but it seems straight forward to just handle them correctly. Note that we could currently only run into this case with device-exclusive entries on THPs. We still adjust the mapcount on conversion to device-exclusive; this makes the rmap walk abort early for small folios, because we'll always have !folio_mapped() with a single device-exclusive entry. We'll adjust the mapcount logic once all page_vma_mapped_walk() users can properly handle device-exclusive entries. Fixes: b756a3b5e7ea ("mm: device exclusive memory access") Signed-off-by: David Hildenbrand Reviewed-by: SeongJae Park --- mm/page_idle.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/page_idle.c b/mm/page_idle.c index 947c7c7a37289..408aaf29a3ea6 100644 --- a/mm/page_idle.c +++ b/mm/page_idle.c @@ -62,9 +62,14 @@ static bool page_idle_clear_pte_refs_one(struct folio *folio, /* * For PTE-mapped THP, one sub page is referenced, * the whole THP is referenced. + * + * PFN swap PTEs, such as device-exclusive ones, that + * actually map pages are "old" from a CPU perspective. + * The MMU notifier takes care of any device aspects. */ - if (ptep_clear_young_notify(vma, addr, pvmw.pte)) - referenced = true; + if (likely(pte_present(ptep_get(pvmw.pte)))) + referenced |= ptep_test_and_clear_young(vma, addr, pvmw.pte); + referenced |= mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE); } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { if (pmdp_clear_young_notify(vma, addr, pvmw.pmd)) referenced = true;