From patchwork Fri Jun 8 20:35:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 10455225 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DD25060159 for ; Fri, 8 Jun 2018 20:35:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D82572927F for ; Fri, 8 Jun 2018 20:35:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC45A295F2; Fri, 8 Jun 2018 20:35:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F67A2927F for ; Fri, 8 Jun 2018 20:35:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BEDDA6B0003; Fri, 8 Jun 2018 16:35:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B9E6E6B0005; Fri, 8 Jun 2018 16:35:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB61F6B0006; Fri, 8 Jun 2018 16:35:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f71.google.com (mail-pl0-f71.google.com [209.85.160.71]) by kanga.kvack.org (Postfix) with ESMTP id 6B7FF6B0003 for ; Fri, 8 Jun 2018 16:35:57 -0400 (EDT) Received: by mail-pl0-f71.google.com with SMTP id x32-v6so7894385pld.16 for ; Fri, 08 Jun 2018 13:35:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2ppRF4EUqUk+t+LzOigvvm1S6WDO9mSilo5DEd81uAk=; b=TLJQbalg2BkxV8Aiayh56U3UY+G7jrh3LVJsunDGA6z9g3EMtrzSv7hyPwjGgtE80a x9cdxlHvW9uLfB6a8dUZ/6L1hlqc+vhGEzfYMFc3cHJkZ8gcuWWrZekFVwHmwhs05Fl6 /ZWi/xn9IkGhqY93UqQU0GAmEtGwrAHXRLmbbDOFM4+v+9CxmB1MA0fA7pdNqBg/08aM vPYaLOEM+65RGkfyrRbHSpbcqvl5f2ZJNv+pETJtA88ORG/DKeNqZ5imOzrzB9AzZzu6 U10up2gVCMlvGkFTkfkwJYPagx43QoPXl4cjZNWqVDEbyelJ3OIIhD5NiBhWepaRB5mU StTA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of akpm@linux-foundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Gm-Message-State: APt69E1+jl7b/tJTZAmgLvI2H4I3neRCQgxguwNWkUWwGUvmE0CIZ2s0 bmlcYeievIatOKDkvW0osuXS0UGItSon42x0jLgyrYYuj6TRk4vrZH4OYF30oljJUFM1Ur+h3W9 qYB+FSpyULpssayU2PWd/3eJ8mUoQcz/1wGXhUEXonRRbMA1payoG16oD0AbxREThIQ== X-Received: by 2002:a63:6dc5:: with SMTP id i188-v6mr6438455pgc.391.1528490157083; Fri, 08 Jun 2018 13:35:57 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKMV4RsTVcpzW4zD51sRp3uxWrI64j4Oupv351b3iS0fl+PiJETtnGRWaUQJ33Viv8ZmfRy X-Received: by 2002:a63:6dc5:: with SMTP id i188-v6mr6438373pgc.391.1528490154511; Fri, 08 Jun 2018 13:35:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528490154; cv=none; d=google.com; s=arc-20160816; b=JN3sYDVbevQp+q6/jYYrxxpXRxEZ/wQSGqqW5fdDk8KahlfC5Rr4Hue9HQv67ixP5d wtQa7Ww9pfeq4g1jdmS1hcAk9I9k4CT6qxdFppdcmt/Pwl5BeIWRXps8kc5EYPsOcUFa PA/hUm+Qo+gz//Z/v9bO4/KYUE43YHqRwV3Fw7O6k74zE01Zq+HoMEeaUXfMyX9v3jXg amkcMeH5GKOoba80t379P2OKFZoaoN15FUIxfOEMieZMjpyWLdykCwRQSrPLSSY3YlWP 0Q1FSJoqD7gbK11XnTC/pUA4FiO6j/HjjOmms/VJWT5V7Ji4oyxXq++HO0G6nAkGP6Wu fslg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=2ppRF4EUqUk+t+LzOigvvm1S6WDO9mSilo5DEd81uAk=; b=xot5ThsTa5lrgHx9D8NYpIm6Ccsl5BivpVjOBslME/b/q9luoHBjvwES6Y9fnSoYXL UhZkJA2g0ioYfxdo1ykKtVrgcEnBBDXcELc0wfLRSH6eXozvvajyJKZ06otxaza/m8pw 4lRZAJr38VJzip2a+Gi8U2spzJSmEPO411tKZCuGAnGYcbEusnRuRjF3gdIavkj5Kj8k XA8t6cVKWjDa3ej1wUwKtdKsdMZwvWGICC954oHd7N7Zi1EaxlCkH8HDE8GjHniskfb8 VmVQT4goVttjuyuZe/iBVxfA/jXvoh4A4eR6fXJI3qRjhECO0OXHFRhmyqiCOvHrX9CV 3OaQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of akpm@linux-foundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTPS id j6-v6si17470786pll.54.2018.06.08.13.35.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Jun 2018 13:35:54 -0700 (PDT) Received-SPF: pass (google.com: domain of akpm@linux-foundation.org designates 140.211.169.12 as permitted sender) client-ip=140.211.169.12; Authentication-Results: mx.google.com; spf=pass (google.com: domain of akpm@linux-foundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org Received: from akpm3.svl.corp.google.com (unknown [104.133.9.92]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 49E3DA7B; Fri, 8 Jun 2018 20:35:53 +0000 (UTC) Date: Fri, 8 Jun 2018 13:35:52 -0700 From: Andrew Morton To: Shakeel Butt Cc: Michal Hocko , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Greg Thelen , Johannes Weiner , Vladimir Davydov , Tejun Heo , Linux MM , Cgroups , LKML Subject: Re: [PATCH v3] mm: fix race between kmem_cache destroy, create and deactivate Message-Id: <20180608133552.d9839eb08f6b98ec4d0ad783@linux-foundation.org> In-Reply-To: <20180530001204.183758-1-shakeelb@google.com> References: <20180530001204.183758-1-shakeelb@google.com> X-Mailer: Sylpheed 3.6.0 (GTK+ 2.24.31; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On Tue, 29 May 2018 17:12:04 -0700 Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: > general protection fault: 0000 [#1] SMP PTI > CPU: 1 PID: 1721 Comm: kworker/14:1 Not tainted 4.17.0-smp > ... > Workqueue: memcg_kmem_cache kmemcg_deactivate_workfn > RIP: 0010:has_cpu_slab > ... > Call Trace: > ? on_each_cpu_cond > __kmem_cache_shrink > kmemcg_cache_deact_after_rcu > kmemcg_deactivate_workfn > process_one_work > worker_thread > kthread > ret_from_fork+0x35/0x40 > > To fix this race, on root kmem cache destruction, mark the cache as > dying and flush the workqueue used for memcg kmem cache creation and > deactivation. We have a distinct lack of reviewers and testers on this one. Please. Vladimir, v3 replaced the refcounting with workqueue flushing, as you suggested... From: Shakeel Butt Subject: mm: fix race between kmem_cache destroy, create and deactivate The memcg kmem cache creation and deactivation (SLUB only) is asynchronous. If a root kmem cache is destroyed whose memcg cache is in the process of creation or deactivation, the kernel may crash. Example of one such crash: general protection fault: 0000 [#1] SMP PTI CPU: 1 PID: 1721 Comm: kworker/14:1 Not tainted 4.17.0-smp ... Workqueue: memcg_kmem_cache kmemcg_deactivate_workfn RIP: 0010:has_cpu_slab ... Call Trace: ? on_each_cpu_cond __kmem_cache_shrink kmemcg_cache_deact_after_rcu kmemcg_deactivate_workfn process_one_work worker_thread kthread ret_from_fork+0x35/0x40 To fix this race, on root kmem cache destruction, mark the cache as dying and flush the workqueue used for memcg kmem cache creation and deactivation. [shakeelb@google.com: add more documentation, rename fields for readability] Link: http://lkml.kernel.org/r/20180522201336.196994-1-shakeelb@google.com [akpm@linux-foundation.org: fix build, per Shakeel] [shakeelb@google.com: v3. Instead of refcount, flush the workqueue] Link: http://lkml.kernel.org/r/20180530001204.183758-1-shakeelb@google.com Link: http://lkml.kernel.org/r/20180521174116.171846-1-shakeelb@google.com Signed-off-by: Shakeel Butt Cc: Michal Hocko Cc: Greg Thelen Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Johannes Weiner Cc: Vladimir Davydov Cc: Tejun Heo Signed-off-by: Andrew Morton --- include/linux/slab.h | 1 + mm/slab_common.c | 21 ++++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff -puN mm/slab.h~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate mm/slab.h diff -puN mm/slub.c~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate mm/slub.c diff -puN include/linux/slab_def.h~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate include/linux/slab_def.h diff -puN include/linux/slab.h~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate include/linux/slab.h --- a/include/linux/slab.h~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate +++ a/include/linux/slab.h @@ -600,6 +600,7 @@ struct memcg_cache_params { struct memcg_cache_array __rcu *memcg_caches; struct list_head __root_caches_node; struct list_head children; + bool dying; }; struct { struct mem_cgroup *memcg; diff -puN include/linux/slub_def.h~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate include/linux/slub_def.h diff -puN mm/memcontrol.c~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate mm/memcontrol.c diff -puN mm/slab.c~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate mm/slab.c diff -puN mm/slab_common.c~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate mm/slab_common.c --- a/mm/slab_common.c~mm-fix-race-between-kmem_cache-destroy-create-and-deactivate +++ a/mm/slab_common.c @@ -136,6 +136,7 @@ void slab_init_memcg_params(struct kmem_ s->memcg_params.root_cache = NULL; RCU_INIT_POINTER(s->memcg_params.memcg_caches, NULL); INIT_LIST_HEAD(&s->memcg_params.children); + s->memcg_params.dying = false; } static int init_memcg_params(struct kmem_cache *s, @@ -608,7 +609,7 @@ void memcg_create_kmem_cache(struct mem_ * The memory cgroup could have been offlined while the cache * creation work was pending. */ - if (memcg->kmem_state != KMEM_ONLINE) + if (memcg->kmem_state != KMEM_ONLINE || root_cache->memcg_params.dying) goto out_unlock; idx = memcg_cache_id(memcg); @@ -712,6 +713,9 @@ void slab_deactivate_memcg_cache_rcu_sch WARN_ON_ONCE(s->memcg_params.deact_fn)) return; + if (s->memcg_params.root_cache->memcg_params.dying) + return; + /* pin memcg so that @s doesn't get destroyed in the middle */ css_get(&s->memcg_params.memcg->css); @@ -823,11 +827,24 @@ static int shutdown_memcg_caches(struct return -EBUSY; return 0; } + +static void flush_memcg_workqueue(struct kmem_cache *s) +{ + mutex_lock(&slab_mutex); + s->memcg_params.dying = true; + mutex_unlock(&slab_mutex); + + flush_workqueue(memcg_kmem_cache_wq); +} #else static inline int shutdown_memcg_caches(struct kmem_cache *s) { return 0; } + +static inline void flush_memcg_workqueue(struct kmem_cache *s) +{ +} #endif /* CONFIG_MEMCG && !CONFIG_SLOB */ void slab_kmem_cache_release(struct kmem_cache *s) @@ -845,6 +862,8 @@ void kmem_cache_destroy(struct kmem_cach if (unlikely(!s)) return; + flush_memcg_workqueue(s); + get_online_cpus(); get_online_mems();