From patchwork Mon Apr 7 23:42:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 14041999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2954BC36010 for ; Mon, 7 Apr 2025 23:42:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EEAF36B0010; Mon, 7 Apr 2025 19:42:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC0AC6B0012; Mon, 7 Apr 2025 19:42:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 988AD6B0023; Mon, 7 Apr 2025 19:42:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 688636B0010 for ; Mon, 7 Apr 2025 19:42:30 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 668141CCF62 for ; Mon, 7 Apr 2025 23:42:31 +0000 (UTC) X-FDA: 83308874502.11.ABAD016 Received: from mail-yw1-f173.google.com (mail-yw1-f173.google.com [209.85.128.173]) by imf22.hostedemail.com (Postfix) with ESMTP id 9BCB6C0003 for ; Mon, 7 Apr 2025 23:42:29 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=lAPnccGz; spf=pass (imf22.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.173 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744069349; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VDaYNr62oZV/U6TDpPRjwWYHhp0Fz9Nf+NOXmppHJcs=; b=PxFBi0aSE5+7Qo7kR2RfanYXb+chVpncpW2eX7XL1SFIsG+T4+2fhB1ed0TG88jJqj54XO UNzhNfz06kfhnteQMpJpPm8DXOO1cBkuyhrAcNrXmqxxSEvEpA2Xe71+WiMbyp27waluma owgtcpjYXn98yVNZ+klAqj61wKpjm6A= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=lAPnccGz; spf=pass (imf22.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.173 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744069349; a=rsa-sha256; cv=none; b=sggfpnqwK0mIQyAlu4rOmnwT9A44T6wGM9S50mRXuul62pvrlAYgsW6IlZMYXBPmB+imY/ 0iGNzdmQGmsCmCZLoIn0keyjLKZpu233cK2QDLmtZ4cnDHi1yAU6+I+ncxFod7gdXximOc iwm/hGaycXmoNWf/pUSQxgurDQk8eyo= Received: by mail-yw1-f173.google.com with SMTP id 00721157ae682-6ff07872097so45461167b3.3 for ; Mon, 07 Apr 2025 16:42:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744069348; x=1744674148; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VDaYNr62oZV/U6TDpPRjwWYHhp0Fz9Nf+NOXmppHJcs=; b=lAPnccGzN33RiOLaP5ZtSsbKsW/fXmculdy/9N8JT2UAaGwU9BDEKW8Y3Y2amDs029 1xJGZN4jAutm6nTh+u4EoAoePghfnBF6irDeDyAgmQnXV7XZP2ljAc6xVgZT5qArhjT8 pNYCRdRCS61JtTuykNjuETAGz+4AiAOVS+XxDXhXUGss3AcEGPfjfekiR4QlidCb5cBx nm0itpdWolJ42W1PebbU9ALjesAdyBmRoP9tk1UWtlwSF9yP1L5gKRoRIRnCAhnJpXsL nYJqafoM4dJB0z1wLTtk9UkAzoEgZEXwRpYcIltV7NxtmVn7yRJuH8YAxWpiF3K/UYyr RwJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744069348; x=1744674148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VDaYNr62oZV/U6TDpPRjwWYHhp0Fz9Nf+NOXmppHJcs=; b=LBCEgxABDUCHOjfi5E/kDyrO91Z6GydNOu0qVT2D4Ad8zNVl9OBB9EaKfgtWPK50Ca xjQmKD7vkH7u/ducn4SFgOZ1nJ2HIBZP8/4CAfJOfxAjI78lINdrCiix86K/BjkTtVSo WRc4zKH/VW7HEFTQxhol5k3xu/kmcvkEq4M/j5njIdNfaGbiHN/nd70hO6RxOmHg/3ic V5lT7mY/SO1dP382wnGPMWOvhxYYK54Up76Xipcl+y5fQYD4Z+jpbOC1QODl8KHbPSWc mm+RPQJnGbxE49pmk/JK63yBFPWA6bzmBDizIF1Re1gIO5QYrVEb4zjuWJURb5VqPXSu KrWg== X-Gm-Message-State: AOJu0Yxx563BDmEsE7GWUSrM9wGppmAKooyprL/8tgtuvMHkh2AOxlK+ 1zhdCovoubiyedsmii0IbVRZOcKES0xqopha8qs8p1bpyawy42EZu7/Awg== X-Gm-Gg: ASbGnctdUU2zNhK1s2o3akjBJWa+K1vRreExTI0p5k9K+iuze2+/m6Ya1qDfeSbVljS 5SLOqVknwZWAHJfB6zm1JcZEHUorSWPAAWKWYhjmeJUOebEl01qAGp90dop/5wf/tQM4q/0ck+q x06Hme1q8mSdoe6Zt76mj5sunDC7P//Y5fpHYJ8tHepNf8mti2798dlPFiLhwdEepK7OJmq1W29 eyaORHMXV1zuFFprlR9EHlqBsH9sEnH2QV8J7VsjT4Oe7o9RNULkJyK28ht0pA3TqRPAkt/hpHN tW9LW0Q/6thHWTDnwHfct6UOwlqZAB6UGV4= X-Google-Smtp-Source: AGHT+IHmrBK25BV1qDKlvbY5RSNbeCthvTW9ZJSLIvxYr/6iVq0MDu+RnyoEtmVX9dC9D27ahdmdJA== X-Received: by 2002:a05:690c:6089:b0:6ff:1c8d:fbfc with SMTP id 00721157ae682-703e16250b7mr266617397b3.31.1744069348608; Mon, 07 Apr 2025 16:42:28 -0700 (PDT) Received: from localhost ([2a03:2880:25ff:4::]) by smtp.gmail.com with ESMTPSA id 00721157ae682-703d1fa73e9sm27925297b3.106.2025.04.07.16.42.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Apr 2025 16:42:28 -0700 (PDT) From: Nhat Pham To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, hughd@google.com, yosry.ahmed@linux.dev, mhocko@kernel.org, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, len.brown@intel.com, chengming.zhou@linux.dev, kasong@tencent.com, chrisl@kernel.org, huang.ying.caritas@gmail.com, ryan.roberts@arm.com, viro@zeniv.linux.org.uk, baohua@kernel.org, osalvador@suse.de, lorenzo.stoakes@oracle.com, christophe.leroy@csgroup.eu, pavel@kernel.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-pm@vger.kernel.org Subject: [RFC PATCH 07/14] swap: implement the swap_cgroup API using virtual swap Date: Mon, 7 Apr 2025 16:42:08 -0700 Message-ID: <20250407234223.1059191-8-nphamcs@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250407234223.1059191-1-nphamcs@gmail.com> References: <20250407234223.1059191-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9BCB6C0003 X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: eq6suk3t6ezh9c4hy53emffbo584yh73 X-HE-Tag: 1744069349-585791 X-HE-Meta: U2FsdGVkX19wa/i4Pf/QZZVGmlgzKBEqGPX1uHuoNCMZSvCK7fqDcXJIXMOLupM1a1Bcs0QeJmLJioUha7tTJBfUBq/jFLbNf9KRRkAgj7P7pWcSXWCT5Qv6WG+hmAxykvfwTyNtasPktOsf09n/ePn3VNFNd/7AxLIJGi7lgYMHZvmfba3inaf4A/uAPoek9qcloY6i9e2ciP54DtVNPPhCyqT+k/HtucNATTRoPHGsaOJQLtsTNDbF7WE897VeZEHpshAX+bN2VIn+ZNFR2Q25p6FAS2ijLvGHGwvespZypVs9Ht6/Fh8OFbiLCZ3mNlOHxgjlbPq4oj2a8fPQiYdf/u31MeETikBlsTJz0NIaYwnaoBmpV2aydhAyCNNObtKp5zwwj+Rq15wFoGsipVnBGwHeg/F45wASKVIWfwUvKdL5t7mhs8LRsUcjd89Ee+AB3WWidCwszsDv5Rfn+dijUmbOOXr6pyHIkwtL7tkK4Z0TOak1D7FfdhughOIvYdKU2d0wr48KyCn4RfqXjVi3umHF1sR7CAkOZ3ATkUoZHs3J1hbiSou3zW1CBHAfvP7ZcMRGQBZ8LWb3yRSvk4tcAzRGyog29DEpklVND2A/q1qn/4OxBh4g0WZh3Xd8dUn707vUqB28kZHkV1QrSLSTesZx3+Z0t2mNYP9d5Hvtw7j/xkERv7TTCcb8i4Ksw2XUQhSpE6IgiMsMnvyrl7sHunXwcW1momgJrILHbXec68scExHeug4XkzY8Z0+jsaxq7k/3DIWNWuh0kTArocTK6BLd4DlQFHwchoDCnAJBBTUYxME0wPkiYIzokiM/73wjiBN64u1aJ1zTBhMh4XUGv5sU9kctgFbTthoRQgB9dpBSZMufnF3N5N7dMSFFL6rBCzJLCvn8VYt4KsG4n1bzekJqtf0eefqDvT3dnV9h1w3B+A7h9iETQDMPYk3sr52dpGvdQZm4AyFzoEn XOSWV6KF u7dxoa/WSjzmr9bAjRUKn3yNVom83mYgYjwEXL8KdwSs61rJPQ/P0XUyER9gsmpp2xVWg5C0cmRG1RJbC8CXM3PvLwcWhquyzX3a9V+z8kR8rI7IsXisRCQSoRqbixNZapjduvJOr6hHAUMZGfTEvZzMOK4S6s26QX40WuPKjib5iR01wY+YOUxJjaRwOkWC5m2kEvtd79r6mh+XBeDTP8h2XyLUEVfi76AU6kJ+Y7UCUPBWggjDyc8+oPJc86D5LoWofwQqYwZjMBxGWDQZ0yZ98H32TAqMvr5eV501N9LLDAXTvuPx+g8N1pjt4csN8kkC3Qq5aYu/skkufG0nhkl3VRB/i8hOCdHcqLiRf6znrZih+N4L+bIIkDBDqBlvbrQyQyrf+U+DzpXMojiRlCEyXpjwab5qxpeppy+0vnX/8WYqxoCnMayWeJCFySdYa+IEvwLdu9BVZqREVpN+RgZHkbxZBU8k1feRu4G1rYfIc5WA4WjYZc+pTXTHRNBMqYwi+agZgMRFwYuN+3s3RKwWq9hnW41hrZDmJ0pXnYOd0Nis= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Once we decouple a swap entry from its backing store via the virtual swap, we can no longer statically allocate an array to store the swap entries' cgroup information. Move it to the swap descriptor. Signed-off-by: Nhat Pham --- mm/Makefile | 2 ++ mm/vswap.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 73 insertions(+), 1 deletion(-) diff --git a/mm/Makefile b/mm/Makefile index b7216c714fa1..35f2f282c8da 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -101,8 +101,10 @@ obj-$(CONFIG_PAGE_COUNTER) += page_counter.o obj-$(CONFIG_MEMCG_V1) += memcontrol-v1.o obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o ifdef CONFIG_SWAP +ifndef CONFIG_VIRTUAL_SWAP obj-$(CONFIG_MEMCG) += swap_cgroup.o endif +endif obj-$(CONFIG_CGROUP_HUGETLB) += hugetlb_cgroup.o obj-$(CONFIG_GUP_TEST) += gup_test.o obj-$(CONFIG_DMAPOOL_TEST) += dmapool_test.o diff --git a/mm/vswap.c b/mm/vswap.c index 23a05c3393d8..3792fa7f766b 100644 --- a/mm/vswap.c +++ b/mm/vswap.c @@ -27,10 +27,14 @@ * * @slot: The handle to the physical swap slot backing this page. * @rcu: The RCU head to free the descriptor with an RCU grace period. + * @memcgid: The memcg id of the owning memcg, if any. */ struct swp_desc { swp_slot_t slot; struct rcu_head rcu; +#ifdef CONFIG_MEMCG + atomic_t memcgid; +#endif }; /* Virtual swap space - swp_entry_t -> struct swp_desc */ @@ -122,8 +126,10 @@ static swp_entry_t vswap_alloc(int nr) return (swp_entry_t){0}; } - for (i = 0; i < nr; i++) + for (i = 0; i < nr; i++) { descs[i]->slot.val = 0; + atomic_set(&descs[i]->memcgid, 0); + } xa_lock(&vswap_map); if (nr == 1) { @@ -352,6 +358,70 @@ swp_entry_t swp_slot_to_swp_entry(swp_slot_t slot) return entry ? (swp_entry_t){xa_to_value(entry)} : (swp_entry_t){0}; } +#ifdef CONFIG_MEMCG +static unsigned short vswap_cgroup_record(swp_entry_t entry, + unsigned short memcgid, unsigned int nr_ents) +{ + struct swp_desc *desc; + unsigned short oldid, iter = 0; + + XA_STATE(xas, &vswap_map, entry.val); + + rcu_read_lock(); + xas_for_each(&xas, desc, entry.val + nr_ents - 1) { + if (xas_retry(&xas, desc)) + continue; + + oldid = atomic_xchg(&desc->memcgid, memcgid); + if (!iter) + iter = oldid; + VM_WARN_ON(iter != oldid); + } + rcu_read_unlock(); + + return oldid; +} + +void swap_cgroup_record(struct folio *folio, unsigned short memcgid, + swp_entry_t entry) +{ + unsigned short oldid = + vswap_cgroup_record(entry, memcgid, folio_nr_pages(folio)); + + VM_WARN_ON(oldid); +} + +unsigned short swap_cgroup_clear(swp_entry_t entry, unsigned int nr_ents) +{ + return vswap_cgroup_record(entry, 0, nr_ents); +} + +unsigned short lookup_swap_cgroup_id(swp_entry_t entry) +{ + struct swp_desc *desc; + unsigned short ret; + + /* + * Note that the virtual swap slot can be freed under us, for instance in + * the invocation of mem_cgroup_swapin_charge_folio. We need to wrap the + * entire lookup in RCU read-side critical section, and double check the + * existence of the swap descriptor. + */ + rcu_read_lock(); + desc = xa_load(&vswap_map, entry.val); + ret = desc ? atomic_read(&desc->memcgid) : 0; + rcu_read_unlock(); + return ret; +} + +int swap_cgroup_swapon(int type, unsigned long max_pages) +{ + return 0; +} + +void swap_cgroup_swapoff(int type) {} +#endif /* CONFIG_MEMCG */ + int vswap_init(void) { swp_desc_cache = KMEM_CACHE(swp_desc, 0);