From patchwork Fri Sep 29 18:00:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 13404692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7927CE728CF for ; Fri, 29 Sep 2023 18:01:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5903F8D00FA; Fri, 29 Sep 2023 14:01:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 540978D002B; Fri, 29 Sep 2023 14:01:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BAB18D00FA; Fri, 29 Sep 2023 14:01:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 291298D002B for ; Fri, 29 Sep 2023 14:01:28 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E9FFD4052F for ; Fri, 29 Sep 2023 18:01:27 +0000 (UTC) X-FDA: 81290402214.05.BDD49D4 Received: from out-203.mta0.migadu.com (out-203.mta0.migadu.com [91.218.175.203]) by imf26.hostedemail.com (Postfix) with ESMTP id 0838C14002B for ; Fri, 29 Sep 2023 18:01:25 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ImdhAu3A; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf26.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.203 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696010486; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QxcU5o8Zg0M6jb420o2zIxJRSsKU6LzkAHBX3GqSFms=; b=JVS5bABBE3aHtTu7dD8rHfVF0PitqjnEwVfVzvSWCMur77A2+MXkjUg0wC0BoUxiYrInWA vwXfJsxG36TRmkrV33bcnzAH8LLYDI2bmdAahHCCKhJ9ephKMDrPhlIXCkCExBnYdJ1dUH zPPJIv84gM6Q/GCc3RLUGDoINgc1WnE= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ImdhAu3A; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf26.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.203 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696010486; a=rsa-sha256; cv=none; b=g2IAhmEk69mfx5dzlgqrbhi0NUa/l+oDGJo26BLAlUppRh+C3WsyBLvMbjG/gd2uwjPaIz DL0UOnK9InVN00Ktx67z0rVqcCRDAxW81Fx0oCj57xjI+2tvS26ifCY8p8Olr5HJCXMCEK gs7aEqBJQYMMWAeeSHgcyjUxpu4wpvk= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1696010482; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QxcU5o8Zg0M6jb420o2zIxJRSsKU6LzkAHBX3GqSFms=; b=ImdhAu3ALKSdHrgwpMCUhyBJ4W47XhCUnKHwtUs8l/RzPj6hc/NfPFnlbT6TZr/umtTkJh xhHHggDB8z5vJqFJ1wexSJW0oPmubHVLru/4OppYheK+YhIChrW1Atc7eG+tLhXmZpg8Xt CFlnH7b6YjSVLnA1ud3LguQ/xHfr6LM= From: Roman Gushchin To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Dennis Zhou , Andrew Morton , David Rientjes , Vlastimil Babka , Roman Gushchin Subject: [PATCH v1 3/5] mm: kmem: make memcg keep a reference to the original objcg Date: Fri, 29 Sep 2023 11:00:53 -0700 Message-ID: <20230929180056.1122002-4-roman.gushchin@linux.dev> In-Reply-To: <20230929180056.1122002-1-roman.gushchin@linux.dev> References: <20230929180056.1122002-1-roman.gushchin@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 0838C14002B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: qfu433n58sf9owzfn58m9uatduqdgedq X-HE-Tag: 1696010485-568737 X-HE-Meta: U2FsdGVkX19FyLhOWk9o2KpF7wjytTMCAoJemdJ5Sca1ICNY//WemcRo35C5obRHsm3kRQZODBZ73TohcENv/XWiiGcmJqcTJuFuJdA7X2ZubCG6DRl6K6+25ibXW4xOVgDRizXxvHc71Q2AyUapp9OmVnWzyvK4loP1/SyGh5OQLMChKyq2Eochg3xY3rEPMA+KbyW7lL1yt+TK3W1olpwD8mDbZmSun22JB/j7TF1a5bgTlQxGbRanw84iT6lUmRRv+9zB/fQy2jSu4XLj2YWmICDMs0C/3npciiSnzosCQTvdwFc7sQiB5FUdAegHZFDcnBoJNZ1A7WSwXlSobqknvhdFAwLtO2sbdkzmDend0HKMqYO01Hw6IgiEYEYoWCNrRd0TfvYLoX8/J/sBKrAvDCntw8J86Ht/RCYay1hvQupMmZBQWoXDiiWBIpOuVXCk0vn65rt524jg4MyDaze0aG7myT9/52jqF445EmtuFJhb4xu5jrc3fqTf7eeNqzz0oSWuhZCMG+i/9ROBe969MrFs5Wsk1n4MUNQN0XW8q/74Zqdxvwc5Xuwgg7jJc+OXjbeevjdr39X6dCUoscki7frzL3qbtpmLCK/kZqhzi0BUnIWabcpoLyiV6zAWRbQuz0WvIKDYyrB10DokmatjX8UZXFfYzKNjTsv3+r9GTzlZFsiuD5xx0i7bwkvdIjEW+uWm/W7+aOMm6XIXUAyJq7dn9Ys+tEV6L4GITHo3NBV0lGDaX6/77AGBAOawr8hEA/QLtJsbDcvscpqDkpufB/lG6+2yukFX9WSdDm4Yw7A29ec7VSLxQrtOUJiUSypoRvIMOxsvwZVpG4PqZAjMRQrs9NPs2Zc1vf1FHMJG1mta9az8AZSLAP5MkM/PULrp1XPhDPNzIkEpyiMY0MsDPClXpRZIvxW9BpiNXIRbhVrSb6nL/zXoKur0IruVkSDY86a2lsedsfJ8Zxg AQ5lsta2 wr642IhM0NHvCPX7YXQx+HIwNqR9DnBGfmv7tsdkON+xNvQS2bpY/Gr/sqmmyhYzMAOU37AMLdgx04OQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Keep a reference to the original objcg object for the entire life of a memcg structure. This allows to simplify the synchronization on the kernel memory allocation paths: pinning a (live) memcg will also pin the corresponding objcg. The memory overhead of this change is minimal because object cgroups usually outlive their corresponding memory cgroups even without this change, so it's only an additional pointer per memcg. Signed-off-by: Roman Gushchin (Cruise) --- include/linux/memcontrol.h | 8 +++++++- mm/memcontrol.c | 5 +++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 1c1ebb269ac1..e59dea9d8666 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -299,7 +299,13 @@ struct mem_cgroup { #ifdef CONFIG_MEMCG_KMEM int kmemcg_id; - struct obj_cgroup __rcu *objcg; + /* + * memcg->objcg is wiped out as a part of the objcg repaprenting + * process. memcg->orig_objcg preserves a pointer (and a reference) + * to the original objcg until the end of live of memcg. + */ + struct obj_cgroup __rcu *objcg; + struct obj_cgroup *orig_objcg; /* list of inherited objcgs, protected by objcg_lock */ struct list_head objcg_list; #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ec28f9cfc2f0..e9890f6e4da7 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3803,6 +3803,8 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) objcg->memcg = memcg; rcu_assign_pointer(memcg->objcg, objcg); + obj_cgroup_get(objcg); + memcg->orig_objcg = objcg; static_branch_enable(&memcg_kmem_online_key); @@ -5297,6 +5299,9 @@ static void __mem_cgroup_free(struct mem_cgroup *memcg) { int node; + if (memcg->orig_objcg) + obj_cgroup_put(memcg->orig_objcg); + for_each_node(node) free_mem_cgroup_per_node_info(memcg, node); kfree(memcg->vmstats);