From patchwork Mon Mar 3 16:29:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13999174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B53EC282D1 for ; Mon, 3 Mar 2025 16:30:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9573280002; Mon, 3 Mar 2025 11:30:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A44E76B0085; Mon, 3 Mar 2025 11:30:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BF94280002; Mon, 3 Mar 2025 11:30:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 651CA6B0083 for ; Mon, 3 Mar 2025 11:30:33 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 26077B6421 for ; Mon, 3 Mar 2025 16:30:33 +0000 (UTC) X-FDA: 83180777946.04.2C7BBDA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 23373180026 for ; Mon, 3 Mar 2025 16:30:29 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MOLaBRvc; spf=pass (imf16.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741019430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XXsaZOoTqmrj90mJbBg0yoAKUyLdOWV9hsabR6NX2f0=; b=CSYNZJyKTH3E5NUQpVn3hBnZvt5+UGssItBJryFQ6/nDXKN+x787V3+UbRGc29B2UXSKmD vBsY9nHcTgVH1xE35LJnWIcnridVpT3MFujXpnHzldv/r0GndR0cjD+sIljRGUOrETAE4P sGgYuUasZEaawhPjGfXNO/5c+IEHC8k= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MOLaBRvc; spf=pass (imf16.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741019430; a=rsa-sha256; cv=none; b=XXOR93Ilxd9vGX1OLRjJgik2EDShb1knENkGbBTRKJuRyNK/taTcek64juVrWAgItxuZXT exPpydMpGl5wh9i0+8YFdud6HY2bVIAZPrM0InWsCfyAsIBZY/DXxwLQP32CDyweUhhF8Q A6CHv9BU0VdIzm0UI49q+fnEtiNG9C0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741019429; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XXsaZOoTqmrj90mJbBg0yoAKUyLdOWV9hsabR6NX2f0=; b=MOLaBRvc1xAcPzsA7FnKy9qyl7O7OqrwuClmVNiLO39sgNTIxQNU0fKBOehWbIKaWTQq3/ Wf+LsUFP3qBbv4Rw8SngnBlZrBh9PNtx/10YDLrJvbjA3/lzb/jM864Y+oPIU0L1tvD57N Q1UY4KeMK8U7RFWhFN1f2HTPvrJ8TxY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-134-VlWHn8jtPeCrSqd6LD1Wiw-1; Mon, 03 Mar 2025 11:30:27 -0500 X-MC-Unique: VlWHn8jtPeCrSqd6LD1Wiw-1 X-Mimecast-MFC-AGG-ID: VlWHn8jtPeCrSqd6LD1Wiw_1741019426 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-43ba50406fcso27671685e9.3 for ; Mon, 03 Mar 2025 08:30:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741019426; x=1741624226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XXsaZOoTqmrj90mJbBg0yoAKUyLdOWV9hsabR6NX2f0=; b=AmJxeUkFOyXTFcntv2qfWGFQYYEErCIRvXP+9WXxF+rFlv9NUXZA100SvIg8KQVCHV cv+WkUQjlcetJIdBZrF2RMsRgr0OdG3L4NdRR4kxbIMZ3YDjJB+uuVZrKT93KI/l+GmY TP7rekmRRtXm7Chr5+q2H9R4yAoB9DDJYqxIq6EgtkKamFDyXjBRZJm3iK4hcgTfave7 nCj+UKewmnJR0Jv39yw1m1tNJStLJwK7ML2cYn78PV+zjm8fHDGeU0L1D5TUZJ/Yn8gF +SqVbwYRY6MLTdi+3oDQZeU/p2AQJuGPvnziPXh56iapO3Dl4tnOITzyVadjauf8Iyms T6ZA== X-Forwarded-Encrypted: i=1; AJvYcCUGqGp5fEsEG4wvzYsURY9SgipwUm7TuYPkt87bL1W2PHytyRRwcwmgYryC5hCRs9LFy3r/7kiahw==@kvack.org X-Gm-Message-State: AOJu0YxmIVGxlAyzxD4S7ZnMIZlQVWe6xygqsrqG3lgms7mI4Dn57kkI u/7LqAfV/7gWSuehvrtvpJUHJtgAD2c4O6MsqhNIRW6Wu+BLoQBDGhrBKK9hl5JHmltA7Pqz7l/ VA06myHc2X0MxxTAQxDukd19CXezuw7tC/KdV4L66x/WDhRjs X-Gm-Gg: ASbGncsXrfFdh10It2ds/sIyJefSnYS+vPowpxzUQySEyWgPLz9gpegW3LYJe/4K+fx 1wMNdvJR+pvRAyQZgp8jZ1rq4AYVPfTJa/ucjxxeUB5skub/wXf45bAUmAWHJb5D7GDZDbtlRrK nK+UvYF0Yyk+9hlTGY3UB+Jk9jLuEvPiSKwNYxSqZnuh1Ajal7LaO8loCOpI7SqRWwhQAjRgdUG yoFnLkR+IwC03fmILjqye21+h3heZndN8nqoHzVhD92kwRIjyFEXGIIH9wkpOjzfMd05jejWkNb 3MNvZ0JV0UgaP/K6M4iNSYf8E1SR6WpdmnxMBc1GZkFGmp6tMR3qyTboj03djOuUb9Gcyhv4Oo7 j X-Received: by 2002:a05:6000:1565:b0:391:9b2:f49a with SMTP id ffacd0b85a97d-39109b2f905mr4268668f8f.55.1741019425852; Mon, 03 Mar 2025 08:30:25 -0800 (PST) X-Google-Smtp-Source: AGHT+IHyH4J1EAqaHR0tUbY9OuS6FmG3s0lngE2eSOblLtE+C7Z78Ry//BFuo4023zOsYhDT+DJ8lw== X-Received: by 2002:a05:6000:1565:b0:391:9b2:f49a with SMTP id ffacd0b85a97d-39109b2f905mr4268630f8f.55.1741019425450; Mon, 03 Mar 2025 08:30:25 -0800 (PST) Received: from localhost (p200300cbc7349600af274326a2162bfb.dip0.t-ipconnect.de. [2003:cb:c734:9600:af27:4326:a216:2bfb]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-390e47a7b88sm14820945f8f.40.2025.03.03.08.30.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 03 Mar 2025 08:30:25 -0800 (PST) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, David Hildenbrand , Andrew Morton , "Matthew Wilcox (Oracle)" , Tejun Heo , Zefan Li , Johannes Weiner , =?utf-8?q?Michal_Koutn=C3=BD?= , Jonathan Corbet , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Muchun Song , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn Subject: [PATCH v3 04/20] mm: move hugetlb specific things in folio to page[3] Date: Mon, 3 Mar 2025 17:29:57 +0100 Message-ID: <20250303163014.1128035-5-david@redhat.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250303163014.1128035-1-david@redhat.com> References: <20250303163014.1128035-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: gXUGd8ivnOvWAoMRel7VnXfekuvZyOJ7gC2mh-A2114_1741019426 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 23373180026 X-Stat-Signature: 5fn3bauu5sq8g163wq4bcpyabzrog8k8 X-HE-Tag: 1741019429-748821 X-HE-Meta: U2FsdGVkX19DhZ/I2qyMxdDOYJEN2rrdPmxyoGAOsUYhtXeWrRbO5hMVoOGY378qsgR5+kRl27UK8omwZgIoSeOrYwK4TRjDUp9th8oRJ1QTfacpI7CpXA8hqOoJDBXU/+1O9z74DpytcY/wGMLkRlvSx3NX+7vwBHm4ugUgVQMtI2uTJz4ZXXKmKoEEZtjUhG9UWi3gVKXMsTvopaavPlaKnQSezfWsXOBQcI+JEfGQF3n2HW+iV8ECX83cW6Qrs85YFQbyBFzPNkd6fMmE4WDZ26NkWXOJgQBt2Mkl0h5l5NPousGJhbASljGU3xlzQ0InIu4wKyXRT+uizGChHd8dRMBMRXAriRhplq2NxMWxymXsjm3nHDi6Isy5+iEzgV6yriLnyzeCD404lw66LooC/7XTJdGhUOM8FrXMpHie7ZE3psrnW3O0ROwHys4ha8GBnlZtZldthzMCVeY3gMxJ5kQrcFgLpwHpQqUutT1kwbF9yws5faZkHG1GpDi9D2kjuZ+UZzhpddkz8YsBELzOc5DeFRB7kWeNOt6zXhKmvEewn7DjQGQ1j/yo4VObYxpjB/TswzGDAnbGGgghNOT8rO0hb5/lzw8UUN8B5iL/BmeUiK+qNyFZUzTxlIXRGObl5SB0ycpBsBXpYdTL1OcygOjuTBiS5ILL5rZ+337TYASsI+XdfFxKb8GaxHgETc8k9B2wcN3tAPFZvxK64pbYuo/9bqjTw6B2L2BdOG8pNrkqsd3EicQcDbF7ZcP2zHKnNTaiIQqVtlflFxluHKmvpKk4nBvE09NFWDmoX8iif80ILP85gR0k4+5IEHOzqSOj75nCz2pSaTsFTVeF2O6g5hb2q4NJyYaa5H9mQ5n3Effk7jIyZ9EsDEgqRUlw+GvZ7zKruyVWwL/jDegN06kF3WgeEytwkGOs8KVnHQw89nNTHqx9WqAOoUE/i7Adtj6cVNwugFCyGiGUcKe TRqKLiwX BDR8FVsVqdFn8/tsEnNfZWJwxM3v3zgVSc6fyF3jy4IPB9/u6a/MB4wAbTnbTUes3wqxGv6YPC5vRERFEIghLEPm074ir8ACe+B2K7GpAQEhzzKhIv6VjPfv0GsLluV7O41mpu6hPXHXeVMnxzCRI8gkWsQ7M0CGFBuCu3sHKS2JoaOpj3IqBb7Z8nPFKyCf6YE4+ltLTpCFRLnivFNW8QKC1UbvpOW+WO/icN8PAAofo/HBDRYervwMauT4jksoT89EtVAMPWcwIEeCEbiyP05eaRMZV2J+yON5SYZLbb02HmPDIvVJGYEGdM6rFr2H1+o4gaGr7v0Zs2fiJ5LgI6QKlEJCzEj9e5oWYhqSZXh5/qokU5poWV/1Xbq7akqjDFivjmKhRzQXn1nAbB56MoIHNOciqEuG+qFAM+dJZsrCGua7REnOOOa65TQMET+jOnEq85c6jtkPGIVnwSuFhb1y5yDXRprrvy/RY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's just move the hugetlb specific stuff to a separate page, and stop letting it overlay other fields for now. This frees up some space in page[2], which we will use on 32bit to free up some space in page[1]. While we could move these things to page[3] instead, it's cleaner to just move the hugetlb specific things out of the way and pack the core-folio stuff as tight as possible. ... and we can minimize the work required in dump_folio. We can now avoid re-initializing &folio->_deferred_list in hugetlb code. Hopefully dynamically allocating "strut folio" in the future will further clean this up. Signed-off-by: David Hildenbrand --- include/linux/mm_types.h | 27 +++++++++++++++++---------- mm/hugetlb.c | 1 - mm/page_alloc.c | 5 +++++ 3 files changed, 22 insertions(+), 11 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index e81be20bbabc6..1d9c68c551d42 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -405,20 +405,23 @@ struct folio { unsigned long _flags_2; unsigned long _head_2; /* public: */ - void *_hugetlb_subpool; - void *_hugetlb_cgroup; - void *_hugetlb_cgroup_rsvd; - void *_hugetlb_hwpoison; + struct list_head _deferred_list; /* private: the union with struct page is transitional */ }; + struct page __page_2; + }; + union { struct { - unsigned long _flags_2a; - unsigned long _head_2a; + unsigned long _flags_3; + unsigned long _head_3; /* public: */ - struct list_head _deferred_list; + void *_hugetlb_subpool; + void *_hugetlb_cgroup; + void *_hugetlb_cgroup_rsvd; + void *_hugetlb_hwpoison; /* private: the union with struct page is transitional */ }; - struct page __page_2; + struct page __page_3; }; }; @@ -455,8 +458,12 @@ FOLIO_MATCH(_refcount, _refcount_1); offsetof(struct page, pg) + 2 * sizeof(struct page)) FOLIO_MATCH(flags, _flags_2); FOLIO_MATCH(compound_head, _head_2); -FOLIO_MATCH(flags, _flags_2a); -FOLIO_MATCH(compound_head, _head_2a); +#undef FOLIO_MATCH +#define FOLIO_MATCH(pg, fl) \ + static_assert(offsetof(struct folio, fl) == \ + offsetof(struct page, pg) + 3 * sizeof(struct page)) +FOLIO_MATCH(flags, _flags_3); +FOLIO_MATCH(compound_head, _head_3); #undef FOLIO_MATCH /** diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 70a012af4a8d2..c15723c8d5e7f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1634,7 +1634,6 @@ static void __update_and_free_hugetlb_folio(struct hstate *h, folio_ref_unfreeze(folio, 1); - INIT_LIST_HEAD(&folio->_deferred_list); hugetlb_free_folio(folio); } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ae0f2a2e87369..2fc03cb13e49d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -975,6 +975,11 @@ static int free_tail_page_prepare(struct page *head_page, struct page *page) goto out; } break; + case 3: + /* the third tail page: hugetlb specifics overlap ->mappings */ + if (IS_ENABLED(CONFIG_HUGETLB_PAGE)) + break; + fallthrough; default: if (page->mapping != TAIL_MAPPING) { bad_page(page, "corrupted mapping in tail page");