From patchwork Sat Feb 26 20:41:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 12761452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE626C433F5 for ; Sat, 26 Feb 2022 20:42:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C8718D0003; Sat, 26 Feb 2022 15:41:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ED3EE8D0005; Sat, 26 Feb 2022 15:41:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB72A8D0002; Sat, 26 Feb 2022 15:41:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0086.hostedemail.com [216.40.44.86]) by kanga.kvack.org (Postfix) with ESMTP id 912E48D0008 for ; Sat, 26 Feb 2022 15:41:53 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 227E2181CA09B for ; Sat, 26 Feb 2022 20:41:53 +0000 (UTC) X-FDA: 79186102506.29.87E89AC Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf20.hostedemail.com (Postfix) with ESMTP id 7FD671C0006 for ; Sat, 26 Feb 2022 20:41:52 +0000 (UTC) From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1645908111; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ve4oxATZGbwpLn9yS3gNeW25czwPhD7pWlWAGFyb+pk=; b=1bbLabG79sUK8TM/MWVQjlXrms8r108UojZ+z/5ltrQgXxLFHmBGjFgjLtFnMLjqsGUs68 Q/WLVjW8qWEVuYjkKVfxivCOA1rVzhst1soFfeefujAaghuS6LHwwT2CRaybw3D3dqOfwP F81KzPdY87dYsME3qUxVi8O9507dnyYtXNXcRQUWuSVOYyTWTP4mEhrurpZgyq5eAJHJDC lgcSvluZjnVDWoDewPsZt+pd4EzTH6qN+W4veKMmrZEiFaK0H87NxGvEhMIQWjH9/tftLc uqKg7wxn9zsSVtBeW0ih5xwGybPA3uiUVBueNJqWdXGO9UjV4dVfV/RemraaKg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1645908111; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ve4oxATZGbwpLn9yS3gNeW25czwPhD7pWlWAGFyb+pk=; b=nZU3LbdBMl3zmc30iXmeSoZFk98tNGWWbI32mBLGxBQ6pfib+AfI9+0l5jUzpnh44x+pdQ NhXBGZN/6wm84HAA== To: cgroups@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , Johannes Weiner , Michal Hocko , =?utf-8?q?Michal_Koutn=C3=BD?= , Peter Zijlstra , Thomas Gleixner , Vladimir Davydov , Waiman Long , Sebastian Andrzej Siewior , Shakeel Butt , Roman Gushchin , Michal Hocko Subject: [PATCH v5 4/6] mm/memcg: Opencode the inner part of obj_cgroup_uncharge_pages() in drain_obj_stock() Date: Sat, 26 Feb 2022 21:41:42 +0100 Message-Id: <20220226204144.1008339-5-bigeasy@linutronix.de> In-Reply-To: <20220226204144.1008339-1-bigeasy@linutronix.de> References: <20220226204144.1008339-1-bigeasy@linutronix.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7FD671C0006 X-Stat-Signature: hsixp96uehekob7rf61rpkyew3wnnusd X-Rspam-User: Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=1bbLabG7; dkim=pass header.d=linutronix.de header.s=2020e header.b=nZU3LbdB; spf=pass (imf20.hostedemail.com: domain of bigeasy@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=bigeasy@linutronix.de; dmarc=pass (policy=none) header.from=linutronix.de X-Rspamd-Server: rspam07 X-HE-Tag: 1645908112-986062 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Provide the inner part of refill_stock() as __refill_stock() without disabling interrupts. This eases the integration of local_lock_t where recursive locking must be avoided. Open code obj_cgroup_uncharge_pages() in drain_obj_stock() and use __refill_stock(). The caller of drain_obj_stock() already disables interrupts. [bigeasy: Patch body around Johannes' diff ] Signed-off-by: Johannes Weiner Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Shakeel Butt Reviewed-by: Roman Gushchin Acked-by: Michal Hocko --- mm/memcontrol.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 238ea77aade5d..4d049b4691afd 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2251,12 +2251,9 @@ static void drain_local_stock(struct work_struct *dummy) * Cache charges(val) to local per_cpu area. * This will be consumed by consume_stock() function, later. */ -static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) +static void __refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) { struct memcg_stock_pcp *stock; - unsigned long flags; - - local_irq_save(flags); stock = this_cpu_ptr(&memcg_stock); if (stock->cached != memcg) { /* reset if necessary */ @@ -2268,7 +2265,14 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) if (stock->nr_pages > MEMCG_CHARGE_BATCH) drain_stock(stock); +} +static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) +{ + unsigned long flags; + + local_irq_save(flags); + __refill_stock(memcg, nr_pages); local_irq_restore(flags); } @@ -3185,8 +3189,16 @@ static void drain_obj_stock(struct memcg_stock_pcp *stock) unsigned int nr_pages = stock->nr_bytes >> PAGE_SHIFT; unsigned int nr_bytes = stock->nr_bytes & (PAGE_SIZE - 1); - if (nr_pages) - obj_cgroup_uncharge_pages(old, nr_pages); + if (nr_pages) { + struct mem_cgroup *memcg; + + memcg = get_mem_cgroup_from_objcg(old); + + memcg_account_kmem(memcg, -nr_pages); + __refill_stock(memcg, nr_pages); + + css_put(&memcg->css); + } /* * The leftover is flushed to the centralized per-memcg value.