From patchwork Fri Oct 1 19:09:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 12531629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4A34C433FE for ; Fri, 1 Oct 2021 19:10:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4FEBB6137A for ; Fri, 1 Oct 2021 19:10:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4FEBB6137A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id E29D294013F; Fri, 1 Oct 2021 15:10:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD8E4940121; Fri, 1 Oct 2021 15:10:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC71994013F; Fri, 1 Oct 2021 15:10:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id BD3D8940121 for ; Fri, 1 Oct 2021 15:10:21 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 84B128249980 for ; Fri, 1 Oct 2021 19:10:21 +0000 (UTC) X-FDA: 78648809442.04.D99386C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 3E809F000345 for ; Fri, 1 Oct 2021 19:10:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633115420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=eTKxhjw7Yo5S8XqVnMv4HlnJDXI4fKHSCti6PPjnd+Y=; b=KTY1RDQbAmdVwupYMjlcf8uuZfB1GHGtsiSS74AAK7Swv9LMN1reuenj1UXhpvcfvufVoa D80HIqVuYn6lwAxQSdSbQxKJEAFMwaNdZtKO1vnqP5dj7/RRUSFvFrLY4pJH2Jv1Es7Lz3 gptLzLtsntACh6HuSdiCWWIsKkMsQA8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-466-iuOMRinSNDSYTE6u2ziHIg-1; Fri, 01 Oct 2021 15:10:19 -0400 X-MC-Unique: iuOMRinSNDSYTE6u2ziHIg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8AFC78145E5; Fri, 1 Oct 2021 19:10:17 +0000 (UTC) Received: from llong.com (unknown [10.22.33.73]) by smtp.corp.redhat.com (Postfix) with ESMTP id 55E2F60BD8; Fri, 1 Oct 2021 19:10:16 +0000 (UTC) From: Waiman Long To: Johannes Weiner , Michal Hocko , Vladimir Davydov , Andrew Morton , Vlastimil Babka , Roman Gushchin Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Waiman Long Subject: [PATCH 3/3] mm, memcg: Ensure valid memcg from objcg within a RCU critical section Date: Fri, 1 Oct 2021 15:09:38 -0400 Message-Id: <20211001190938.14050-4-longman@redhat.com> In-Reply-To: <20211001190938.14050-1-longman@redhat.com> References: <20211001190938.14050-1-longman@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3E809F000345 X-Stat-Signature: k35ey6kp78nywsc15qf3zz1jpe49bm1o Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KTY1RDQb; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf16.hostedemail.com: domain of longman@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=longman@redhat.com X-HE-Tag: 1633115421-585682 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To ensure that a to-be-offlined memcg fetched from objcg remains valid (has non-zero reference count) within a RCU critical section, a synchronize_rcu() call is inserted at the end of memcg_offline_kmem(). With that change, we no longer need to use css_tryget() in get_mem_cgroup_from_objcg() as the final css_put() in css_killed_work_fn() would not have been called yet. The obj_cgroup_uncharge_pages() function is simplifed to perform the whole uncharge operation within a RCU critical section saving a css_get()/css_put() pair. Signed-off-by: Waiman Long --- mm/memcontrol.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 8177f253a127..1dbb37d96e49 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2769,10 +2769,8 @@ static struct mem_cgroup *get_mem_cgroup_from_objcg(struct obj_cgroup *objcg) struct mem_cgroup *memcg; rcu_read_lock(); -retry: memcg = obj_cgroup_memcg(objcg); - if (unlikely(!css_tryget(&memcg->css))) - goto retry; + css_get(&memcg->css); rcu_read_unlock(); return memcg; @@ -2947,13 +2945,14 @@ static void obj_cgroup_uncharge_pages(struct obj_cgroup *objcg, { struct mem_cgroup *memcg; - memcg = get_mem_cgroup_from_objcg(objcg); + rcu_read_lock(); + memcg = obj_cgroup_memcg(objcg); if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) page_counter_uncharge(&memcg->kmem, nr_pages); refill_stock(memcg, nr_pages); - css_put(&memcg->css); + rcu_read_unlock(); } /* @@ -3672,6 +3671,13 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg) memcg_drain_all_list_lrus(kmemcg_id, parent); memcg_free_cache_id(kmemcg_id); + + /* + * To ensure that a to-be-offlined memcg fetched from objcg remains + * valid within a RCU critical section, we need to wait here until + * the a grace period has elapsed. + */ + synchronize_rcu(); } #else static int memcg_online_kmem(struct mem_cgroup *memcg)