From patchwork Thu Jul 20 07:08:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13319865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 892DEEB64DD for ; Thu, 20 Jul 2023 07:08:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 23C942800BC; Thu, 20 Jul 2023 03:08:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C79728004C; Thu, 20 Jul 2023 03:08:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 018202800BC; Thu, 20 Jul 2023 03:08:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DF1C528004C for ; Thu, 20 Jul 2023 03:08:32 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id AD5E7160188 for ; Thu, 20 Jul 2023 07:08:32 +0000 (UTC) X-FDA: 81031112064.12.1393554 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf17.hostedemail.com (Postfix) with ESMTP id EE7DE40002 for ; Thu, 20 Jul 2023 07:08:30 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=PyMiTiRs; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3bd24ZAoKCDgsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3bd24ZAoKCDgsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689836911; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P4XNYMJYgxf4Y2ysW943aMZDD9DgQegnQSW9aAxie4w=; b=BeiTQ8SQkYUBH3KClBabLzGH9aaEwHDj0dSCiQxB3pD8FTT1LFfhnm46DvYpTfIU7MQ2fC 9JomcbCkehd08385pHWQps+gV7Ei6bNVk9vPG+QvsUM7YMeCS5Rcel+LEtBmwZAOK1HEoQ BZYMsEzROuEwmQ23QaCbT++fzerYyDs= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=PyMiTiRs; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3bd24ZAoKCDgsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3bd24ZAoKCDgsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689836911; a=rsa-sha256; cv=none; b=yCw5WywLe9ucXVSqTHkQo3YtQwGZV0WZ8nkMtvISoptiKksAOqzC3h9lJEfNrgOv5CM4ei vIq11HyM+QtlCr+0BitwEsXqzmWicU3tcxbuli/dC+GrOBBVfmsKKqKCv4ytbUdt0mmJdx KaIs0J4w/vJgpZbGbTk5JA16Fc0SbT8= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-55c7bb27977so397067a12.0 for ; Thu, 20 Jul 2023 00:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689836910; x=1692428910; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=P4XNYMJYgxf4Y2ysW943aMZDD9DgQegnQSW9aAxie4w=; b=PyMiTiRslPlYxeAzNGyf+s2RMn2Y5D+io4F49/M2XRwJrS8TwV/LhLqLj0BM+kLdbs SJdh95igG91J+W+YW3QhUBoovRNlfDdy0rtENHO9Q0tdlMq6Mi6nZDp8wPmseQG3GEz7 yTNtdLVRNpjOoL79dp3VDPbMRHNV99200JwxoCivq9oH22XjolvYivx1g2ceZNC2Pn7s Da/CFyreh5mYpyR/lWRRdvMG1hkWdPWRhELE3hNZjAEKxkN7vhsHgeLNCSqpeMvOeAHq lRZ2ErdDGOl8n4urJ5Mk+4WBu+P+b25Mk8sxetPQU+cK2DCaPQPxX5kv/nJSdD/CPPwf W7UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689836910; x=1692428910; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P4XNYMJYgxf4Y2ysW943aMZDD9DgQegnQSW9aAxie4w=; b=aNWjO2xe2GtmYqaN0mxqA/8G44A7ox5aLf4RxPZtVNJjg4A1KewZonm8j8Oj89e6YQ X0JzZP/kF/VrkyAEv7ehGLuQqEaelcfHEwp1Jgcn0TDaUyK1E/y7LEGE5wbFD8hLhmAM v0uS63EN+LFcIB4ZXXljsRKtu91pC5mowZ6bReOjaCYYIDYu98UdlpfAvJJA3v3q2RhY L3DErwGJBRPlKB+AOo0T+CcK/EbjMl8Z6SOzNYmbVjODcMifBelth0wS7I2dU9mFvaPn JlRtHv7P5/WpP149Vq6e6X7LWqcU9pN3F65DH2P6UhBnzha9emm/95jC1o8d++nDQWxw tFIA== X-Gm-Message-State: ABy/qLZKSBGE3B3wVTBhwAswPCMZEPn/sH4uslY55JOQSvlZvQM+ibg0 yua9/KJu/u6n3MpE0nHKdVPH43m32Oihu3/b X-Google-Smtp-Source: APBJJlEGR67b7I5wM02HN8lsWXpPeepUUbVvmNMwkaq7frNphjV2zKpBSorV/PgnNk3Mo7NS/0jW1PWGUeUjmfjP X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:3e8f:0:b0:563:3b08:f869 with SMTP id l137-20020a633e8f000000b005633b08f869mr22818pga.2.1689836909753; Thu, 20 Jul 2023 00:08:29 -0700 (PDT) Date: Thu, 20 Jul 2023 07:08:18 +0000 In-Reply-To: <20230720070825.992023-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230720070825.992023-1-yosryahmed@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230720070825.992023-2-yosryahmed@google.com> Subject: [RFC PATCH 1/8] memcg: refactor updating memcg->moving_account From: Yosry Ahmed To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt Cc: Muchun Song , "Matthew Wilcox (Oracle)" , Tejun Heo , Zefan Li , Yu Zhao , Luis Chamberlain , Kees Cook , Iurii Zaikin , "T.J. Mercier" , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: EE7DE40002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: qf19eh9wacrcu5e4i5tz15fp19h5o71t X-HE-Tag: 1689836910-38944 X-HE-Meta: U2FsdGVkX18qy/ONYdVDai92C0kdKngMhVuROLySVndHtHPkSCNIjeMEeNJj6rMeS6DxDh42Co8ckLCXDdiskkxSR4pDwy7OiSTakdhYdXc8+AgI02QFXuIe5k6sk9R033gg4Kq/vrMdLm3lNt7FLVcOqjG5pi/G4HdrGzMr2C4Cu4NTGFqqaqefNYfW73N/cT3VgScrxsCKPF0MaGOwB6ciKWyV5SbfqpZ4Iu4S/7VzCZQ9B+dTPZDIeAtbXXeBmWW8QvVsh1LuUitZiwcY5wVEscE7wBWAFQ+3tFP5UIZwvzy/yGuLy3Zv5/YwfNHrHVT1mPFEn7nJ3oVqq1DOAIogptTwyoXdlrv4iBcWDZ0iRSgKiHOs5d+rn4niJMKcmw0fZS0cjTw5lwvSPp6yM33EZkgAxCncj9V4Yy96QjIZfOmgpHmucFwpBzxtOr8QI9Fl0HZ+4NT6zJlY4GeIFoZHRBoWVIlJH1SJ0GeBjYcaXlYdvZTv7sXrmRrJzimvCrMlnP/l8iH/N2MJzkVfhWQsGqkoNaXHkY0JC4b9cLvrevpWOtMR0XcD/B/KkT/qmkJQhGCyTK44LkH3T4ca5BPdV//n/V1Ls+05yQuM3G0a1Cewf8hkz/OEhgMpcuUagSf47HuVbVHtG/G8XvRYXDYVgtPIGSbXnX9zr73A/ARymEOwF1XddHPHwBSv9hsZWE2XqH8TFTXat0VwylSUcHXDaAxq+teX+hViZ8s/WKSTODys+y1J+Gh5T7DYQUiBjp4cM7tXkTuCK6TExDXASpZtuej5zsANv4Wi16/8NjxfrGmtbmIdegVVV77hhRwq1VcCacubQAtjLbi45m8Hsrz1GAl8agyEKRWSFEIsRb1aTAyvUhNSIOXga1suWIpXkXj3Mhy0kc/3JJu6QvcqNXyqHDjR1QnIXdkxW60o+inT9pT28cbDMS/eH9t7RDBxSHPfm+UFKVwHmg3zC78 ZKl16J8G pDiggzZ5v59GyD6KBV+hMgusQq7gb3/jgQremsPHwfCtWZGcDjacPd0iiP8CTEUNxab5WMoogAJZ+ZlnwvjU2oqE/4otC4jLXCfmW6oQogaUcG8yEs14IH96tlWQcZAYs4pKY/e3aUeJlKMPkvSmU85lXLhy+5Sk7r3UkNJk8PkZk9+SYHS/3iPmZNiWPSWT+imldvUFp1A8GRjfzQ26irmhwt19fqhdQ6uThB9oRf73ogDxuNwoFy0KpZ//zcgg/prIca5zxbnmtyJtrwofLbIG5286esRFQWhfTocVh5+n6NxC2Mujnu5YYGyzboKXzVmxF6q52d0M6s4JlgLMZKCA+qNh66S1KyGIrQyp2xuMPn4zDCKZKUWXgKEEOh/nYmbzZguco+yvCM9qNfp4NTb7iWdpCHkKTZ7FdJcPe6Is3IhLVqH7rt14KJ0Z9D7mvZ0h41yP3YB8aFzWKpZ7H1b8ZyofYI+QuVvTgkEbySLO4eYTNrGE8O8w2C9ugE7ivjRWK6mzN7armRNF9dfCimkMnb9CMV+WSv0j54MCDxFC1sFWgOhQoYXso3521Guqq+r9a4vKeyYbP4165KqY0BAdjxpalEqzgnwMaxYLRrQm+o9DHVG/h/MMiRrpHCdhPp+foCBNwXuJv5J/njQL0y34RFHXZfcJe0hhIQs7EbXB8R8Z8OUaxiZL6EQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: memcg->moving_account is used to signal that a memcg move is taking place, so that folio_memcg_lock() would start acquiring the per-memcg move lock instead of just initiating an rcu read section. Refactor incrementing and decrementing memcg->moving_account, together with rcu synchornization and the elaborate comment into helpers, to allow for reuse by incoming patches. Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e8ca4bdcb03c..ffdb848f4003 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6305,16 +6305,26 @@ static const struct mm_walk_ops charge_walk_ops = { .pmd_entry = mem_cgroup_move_charge_pte_range, }; -static void mem_cgroup_move_charge(void) +static void mem_cgroup_start_move_charge(struct mem_cgroup *memcg) { - lru_add_drain_all(); /* * Signal folio_memcg_lock() to take the memcg's move_lock * while we're moving its pages to another memcg. Then wait * for already started RCU-only updates to finish. */ - atomic_inc(&mc.from->moving_account); + atomic_inc(&memcg->moving_account); synchronize_rcu(); +} + +static void mem_cgroup_end_move_charge(struct mem_cgroup *memcg) +{ + atomic_dec(&memcg->moving_account); +} + +static void mem_cgroup_move_charge(void) +{ + lru_add_drain_all(); + mem_cgroup_start_move_charge(mc.from); retry: if (unlikely(!mmap_read_trylock(mc.mm))) { /* @@ -6334,7 +6344,7 @@ static void mem_cgroup_move_charge(void) */ walk_page_range(mc.mm, 0, ULONG_MAX, &charge_walk_ops, NULL); mmap_read_unlock(mc.mm); - atomic_dec(&mc.from->moving_account); + mem_cgroup_end_move_charge(mc.from); } static void mem_cgroup_move_task(void)