From patchwork Mon Mar 3 16:30:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13999200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32C81C282CD for ; Mon, 3 Mar 2025 16:31:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 40D6428000E; Mon, 3 Mar 2025 11:31:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 34765280003; Mon, 3 Mar 2025 11:31:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1996428000D; Mon, 3 Mar 2025 11:31:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EE3AC280003 for ; Mon, 3 Mar 2025 11:31:01 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7B624120C5F for ; Mon, 3 Mar 2025 16:31:01 +0000 (UTC) X-FDA: 83180779122.21.9B13AA6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf26.hostedemail.com (Postfix) with ESMTP id 726F8140023 for ; Mon, 3 Mar 2025 16:30:58 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AYQ3OBYV; spf=pass (imf26.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741019458; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dM3d7FaE0s/3W03pV+cOCeas1Rcyk2uf0DwTQW3h1Ys=; b=WBpU31nvMtiBQdkyhpEX0ThnbPU8zwtIPFDmAx9STxW1rfXz1O41PbELNqg6Ye6cS0VHur ug4YxMgLRyE0dXdJCnVq2EQXdudC9t4EZvUmKaAdhdJX/nNuU7BDJWBetf43QOJy7qhsv7 KnYVPguhRX/tN31VcmwcS95M2Pv7hwg= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AYQ3OBYV; spf=pass (imf26.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741019458; a=rsa-sha256; cv=none; b=Q3Mdo9P+7IOSGzMtdQO3W1lHJUnnCoFjFOQu7y+1Hmz1X/XM5dyidu5DJpc3lN9UPeunTT jv+vndsj82k/ON+CPqxIjDenb7dOz4MrkpFYqAmnvDcJy8AobL8psbpPfxsYgZy7Dv+8no Q8wXRX+Hc1eEICaksvio++NTdX9TCJI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741019457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dM3d7FaE0s/3W03pV+cOCeas1Rcyk2uf0DwTQW3h1Ys=; b=AYQ3OBYVp3u4GgUydN25A1a75DADqy0p5DM3eo0rwFcmjo2Bj1XbLplpEFCIBaiZfTl9No ASYp8p0qzqYloQFBPPiQjP5K9iGGw95iTD65BGTKTfH5P3zzbHzuRt+QiSFdocNjO0xoDt T9QsA/Yz0sHlUqczrxKDbNpTgwzkSgg= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-139-OGf2ypZIMvKIt_8wgT5yPw-1; Mon, 03 Mar 2025 11:30:55 -0500 X-MC-Unique: OGf2ypZIMvKIt_8wgT5yPw-1 X-Mimecast-MFC-AGG-ID: OGf2ypZIMvKIt_8wgT5yPw_1741019454 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-390f7db84faso1754494f8f.2 for ; Mon, 03 Mar 2025 08:30:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741019454; x=1741624254; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dM3d7FaE0s/3W03pV+cOCeas1Rcyk2uf0DwTQW3h1Ys=; b=k2EIx5FuBfqkGdWDbk9v3AxlatUySq7iZVf4muC2rKnWX1yx+y8GcfKoVd9JtoDb3D Uk09nkWwcUxQY0IRwdq4xFO4UAqAOdxgB5muKNv+zQkWpYdQwjhKYyuPHx8dH7j0fZK2 OwsWRJU0WKNXoWDHxYtZ7DCARrRi3HuVmnJSnEaRTXl0PxkKhKFQr+1SRPxwT1ttrpCg iqJuF5bdaopppN88lHY52BUKerSuQdRT5S17Ji6xvBjbW3R/KnHBPseHp9zzbf7Cmsyy NSAQWqvQso6BxCVpMUW0XztDwFI2CTTqg1Q5+BWKFkAGykuZjV0arzeoPJSE/tjBYmO7 cjUw== X-Forwarded-Encrypted: i=1; AJvYcCVc3rM3I5CfbL8wX4S+A0SIlUgJLAib960l0vkO7ZSKEyyGe8+P/BrfFbC4SPA6DvBnsVAvRD4pNQ==@kvack.org X-Gm-Message-State: AOJu0YzQxsGBhmfeQqVd/qC4g3bAYl1jY/3BFtowULKsZl372UUUHoMQ ZptLUmVswXKlADrTE6nlQc1Nz7weV7PrJiOqEng3CuN8iXYpCchu0hk4COsDvHQpef3CgwxdUFp f9Jb8QjRVkCJWK/wnyLxv17g6ff1XT5zKqowrOkpJYyvg49V5X11LCVB+x+k= X-Gm-Gg: ASbGncsbeYbcUxQn1JlGq7uTx+/W3ZfM7dZCxQ4hKn45eenhNIN7e9dwOAqsfClR9Th M73ZLSyKnzoHI5a8CCtmlEVOUy6ek/17n1oQEtdDLvoQOSmeG9YEieuHCKCvsmdXVGZmvxp5kvJ zs7n6yQYUYGdx9V1EiDQUDPVmwWByxCxdLnssYaaj++umDVwmXpjNuQnMrGIVQf92FcyTLVXgKr z59C5whlEnCaEZvXNiTct3MYvaIkbFPMe3nX0JRD90xhEgYBVNRKDyLnphTqd1T5NyR0EfmoBAa vvsxtbRJYbW0D0H7YNMmdlQdLgqQMXg67xnGirYkGNbGBc0bgqCKCTI1NsYaM7SqS3BaiSJrfx2 c X-Received: by 2002:a05:6000:4c6:b0:38d:c44a:a64b with SMTP id ffacd0b85a97d-390eca2589amr11095192f8f.48.1741019453856; Mon, 03 Mar 2025 08:30:53 -0800 (PST) X-Google-Smtp-Source: AGHT+IFDds/9qj55eCL2sJuyoEBZpLXmN9fljVQo3ZoynXY2Ft4dglS/KTUjLVmC7la2kWf5hOqFtQ== X-Received: by 2002:a05:6000:4c6:b0:38d:c44a:a64b with SMTP id ffacd0b85a97d-390eca2589amr11095139f8f.48.1741019453402; Mon, 03 Mar 2025 08:30:53 -0800 (PST) Received: from localhost (p200300cbc7349600af274326a2162bfb.dip0.t-ipconnect.de. [2003:cb:c734:9600:af27:4326:a216:2bfb]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-43bbbff18b3sm56885465e9.24.2025.03.03.08.30.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 03 Mar 2025 08:30:53 -0800 (PST) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Tejun Heo , Zefan Li , Johannes Weiner , =?utf-8?q?Michal_Koutn=C3=BD?= , Jonathan Corbet , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Muchun Song , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn Subject: [PATCH v3 16/20] fs/proc/page: remove per-page mapcount dependency for /proc/kpagecount (CONFIG_NO_PAGE_MAPCOUNT) Date: Mon, 3 Mar 2025 17:30:09 +0100 Message-ID: <20250303163014.1128035-17-david@redhat.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250303163014.1128035-1-david@redhat.com> References: <20250303163014.1128035-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 7J8scf22mgA_CAbzyWUQ6D8mtbYpWZeQmD6TOJDmNms_1741019454 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-Rspam-User: X-Stat-Signature: 7uga4idtqoz4medoo4bokdws8cgibq5s X-Rspamd-Queue-Id: 726F8140023 X-Rspamd-Server: rspam07 X-HE-Tag: 1741019458-572382 X-HE-Meta: U2FsdGVkX18Z0qp41hfVwMmZF07FM12fdiMkJHlYxFIdNaWUr3hSWk5HkM2pwpcFefwC9fMnTgxbCAPS6R4xv7LT9pCXbAmd168zp6p9p+J5xh0G0lcP8vGsgdD+/vQmXzqYvucwuEcSdjyulzLUqzBAMXvbkm7t8xoV0CiLXIjft+iw66vT8ExUlVqUOGNViRh1OE+tBccZSoRShGXa0v77fwR+JelH8Cgn1TBUBA8P+u4xIsRNc14RpAaaLDHCdf53HOIjE1vA8HL0QD5lKYmpSa30+jGHuAHAUtLrJNyJz1iKGB8WEQ7gS/tI2C98y6k/TQy6GXA5NS1634Ua2ueDAkdq5s+vW86DhHmWCvtYee33aVl8F8gCCo8S5JcyUfh90N1m3clV9ypVvJF4hG1jB1L2mKilB6bIWn1F3qu78lc/PXEq9TI0KlVIwHHh8LiUNcJ7qMU+YVHflqKHPfXYC7Oql7u64K5c5FWcnEkFvYqIq8hmbyNWg4DV6Lg3uFvaeXlh/KuARLPmPX3z+tLjfcUN+bFm6mxiBBMWsT4vNf99tSIpsNP4I8Lf73uJKmX5VwNMn671slup9rJg9uxrgaVxf3lD6bjm34Dyzag7II0WuxW/hV6vcFOyfDXb85C8BhZWgZWIAAzhDbDvbiBphymYGZD6h56uwpcJqhKYNerNsMRRkowsXn10V2jsqdJt7ztXz+9hnG9KAA8B3vF29JNdRP14npKbNFjZR+Qvp7y9riWzkMwHCNqNJp4KiLq5GUTGYfCUWtBnPQuD2u0IyCZF6N5F7IDJRFhswl9ExEB3JRLlUcAZzQGgJrC7TowdoTTlizikTCamHx6RKjx9300iLW3uPgdx4ReO2xw4qfzOKDf7Ts31vXiPxyZGQ57CWcOFB+/oLMIQWkvjoE0kJvVWnFzCf+zM3vTpHd02BnxATF5vexGo4I+pJ411DHKB+8NaIfzK1JJYFxB FCQ1BYLK 0DjE5zJwOjc13H/GvQ4/9M+Ya5g4X++32+6ELyYXRs4TvNbz1rsnJJ+oDClS9abTpd5MNPXVcSvIyhzMT69KCGH5WAmAnfFv9WzsbW8tXDzWfH8bsjRbRPWpCQqqBcXoBML5aL8bdoSibZrY0R0FxbGeeuBwr7Y7IHDH5up1wPYfsC13uR3OyYGcqgQWX/yGLz7ncyQAVXkyaStfP2DWE8+WQ2lcMRNSGQKCKMRafQkK/gKhA1ilJh4i/GgkNARaxFlpQlKFkqEQewudMTX8SZ1n55EkyYYkN5bJkIjEwBipAMcVoLzqnOyrNc3N7SeENSsbYHbO/Me40731tnC0Fse4KXcDUAWkicJ1sMVgt9FwwXGOq1jQHpT0WeL1vts74JW+PU+5eKOSW1QWi4qQBnxjak8S3SqpqymsCzi01jq5kM9tSteIzOnE6kxPfL7aXyv4sQshK2qLeaTY3wNEL6owbjVdEkSE6EUMSGqd9Oz3kw4oRB/FcnSAJl8PccgWIANIqd0q6gReIqJVCUBnvVmQlMg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's implement an alternative when per-page mapcounts in large folios are no longer maintained -- soon with CONFIG_NO_PAGE_MAPCOUNT. For large folios, we'll return the per-page average mapcount within the folio, whereby we round to the closest integer when calculating the average: however, we'll always return at least 1 if the folio is mapped. So assuming a folio with 512 pages, the average would be: * 0 if not pages are mapped * 1 if there are 1 .. 767 per-page mappings * 2 if there are 767 .. 1279 per-page mappings ... For hugetlb folios and for large folios that are fully mapped into all address spaces, there is no change. We'll make use of this helper in other context next. As an alternative, we could simply return 0 for non-hugetlb large folios, or disable this legacy interface with CONFIG_NO_PAGE_MAPCOUNT. But the information exposed by this interface can still be valuable, and frequently we deal with fully-mapped large folios where the average corresponds to the actual page mapcount. So we'll leave it like this for now and document the new behavior. Note: this interface is likely not very relevant for performance. If ever required, we could try doing a rather expensive rmap walk to collect precisely how often this folio page is mapped. Signed-off-by: David Hildenbrand --- Documentation/admin-guide/mm/pagemap.rst | 7 ++++- fs/proc/internal.h | 35 ++++++++++++++++++++++++ fs/proc/page.c | 11 ++++++-- 3 files changed, 49 insertions(+), 4 deletions(-) diff --git a/Documentation/admin-guide/mm/pagemap.rst b/Documentation/admin-guide/mm/pagemap.rst index a297e824f9900..d6647daca9122 100644 --- a/Documentation/admin-guide/mm/pagemap.rst +++ b/Documentation/admin-guide/mm/pagemap.rst @@ -43,7 +43,12 @@ There are four components to pagemap: skip over unmapped regions. * ``/proc/kpagecount``. This file contains a 64-bit count of the number of - times each page is mapped, indexed by PFN. + times each page is mapped, indexed by PFN. Some kernel configurations do + not track the precise number of times a page part of a larger allocation + (e.g., THP) is mapped. In these configurations, the average number of + mappings per page in this larger allocation is returned instead. However, + if any page of the large allocation is mapped, the returned value will + be at least 1. The page-types tool in the tools/mm directory can be used to query the number of times a page is mapped. diff --git a/fs/proc/internal.h b/fs/proc/internal.h index 1695509370b88..96ea58e843114 100644 --- a/fs/proc/internal.h +++ b/fs/proc/internal.h @@ -174,6 +174,41 @@ static inline int folio_precise_page_mapcount(struct folio *folio, return mapcount; } +/** + * folio_average_page_mapcount() - Average number of mappings per page in this + * folio + * @folio: The folio. + * + * The average number of user page table entries that reference each page in + * this folio as tracked via the RMAP: either referenced directly (PTE) or + * as part of a larger area that covers this page (e.g., PMD). + * + * The average is calculated by rounding to the nearest integer; however, + * to avoid duplicated code in current callers, the average is at least + * 1 if any page of the folio is mapped. + * + * Returns: The average number of mappings per page in this folio. + */ +static inline int folio_average_page_mapcount(struct folio *folio) +{ + int mapcount, entire_mapcount, avg; + + if (!folio_test_large(folio)) + return atomic_read(&folio->_mapcount) + 1; + + mapcount = folio_large_mapcount(folio); + if (unlikely(mapcount <= 0)) + return 0; + entire_mapcount = folio_entire_mapcount(folio); + if (mapcount <= entire_mapcount) + return entire_mapcount; + mapcount -= entire_mapcount; + + /* Round to closest integer ... */ + avg = ((unsigned int)mapcount + folio_large_nr_pages(folio) / 2) >> folio_large_order(folio); + /* ... but return at least 1. */ + return max_t(int, avg + entire_mapcount, 1); +} /* * array.c */ diff --git a/fs/proc/page.c b/fs/proc/page.c index a55f5acefa974..23fc771100ae5 100644 --- a/fs/proc/page.c +++ b/fs/proc/page.c @@ -67,9 +67,14 @@ static ssize_t kpagecount_read(struct file *file, char __user *buf, * memmaps that were actually initialized. */ page = pfn_to_online_page(pfn); - if (page) - mapcount = folio_precise_page_mapcount(page_folio(page), - page); + if (page) { + struct folio *folio = page_folio(page); + + if (IS_ENABLED(CONFIG_PAGE_MAPCOUNT)) + mapcount = folio_precise_page_mapcount(folio, page); + else + mapcount = folio_average_page_mapcount(folio); + } if (put_user(mapcount, out)) { ret = -EFAULT;