From patchwork Wed May 30 00:12:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 10437431 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BC0F7601D3 for ; Wed, 30 May 2018 00:12:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C9A22882C for ; Wed, 30 May 2018 00:12:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8F07C28831; Wed, 30 May 2018 00:12:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 16C0F2882C for ; Wed, 30 May 2018 00:12:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1F866B0003; Tue, 29 May 2018 20:12:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DA6796B0005; Tue, 29 May 2018 20:12:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C487B6B0007; Tue, 29 May 2018 20:12:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f71.google.com (mail-pl0-f71.google.com [209.85.160.71]) by kanga.kvack.org (Postfix) with ESMTP id 7D79A6B0003 for ; Tue, 29 May 2018 20:12:13 -0400 (EDT) Received: by mail-pl0-f71.google.com with SMTP id 89-v6so10180532plb.18 for ; Tue, 29 May 2018 17:12:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id; bh=2OJnmQ7l7NGFZP9ipCIfC7mNh4H09z1pqCsCgocmqlg=; b=MhwZTLxs0KqJ4nZXq+sP//bHzocvst+BQKOPWj7C93abH9VqUkXs90wx80V5vxLxCx grl9G/ZgKYfidpbEpKOcA0jGp6ebWgzvnKFE/v8xLejfA9Hkjit8mTBW+1Wi9SISNhLN O8AOF9WCBVWuLS5MKuEk67JufUl+8zXIeE8904AvDC24z0/5FI5pzHLhSXH9z+MWupEb 4vM7Qktei/3irKB4CJBCi9K0unO9wE234WD8ub3EAl45vHte9wwMmqAfgyfgJltun9tx 2I1Ff0NUIF4KzBa2Um7PevcVexBIECuJGTXvh/WNrarUDm8MQyhF0lF3jOwjJzWorqkd KQtA== X-Gm-Message-State: ALKqPwdbhISZlt8vVW3uKeQP3MStfEhzptOYmIR2Qhq5Z1v+HwhbKlEy LI6L0n1RURWszWx6mnDpMsik1rgPG8Xnv818L2IUR1aVVxch139V6blPFGn+KskiRL0hKxwSkqq Pe2f3/EvYiNltdw3Ehg3hT52Z52CISyogb+jpGk/cbFnRwqngfyjzjXjviiMwFMe9OADi7p7dvH b9FBO1Ttbx2rgUgCwzMXuWuiofpdAaZJepAGATqnXXhBcy9H/a8lxK3Q1lilJYdp5t9RM0hHOL1 x9lwR8TDvYmf1glHY5JI5FAwao6vSuEVQ4NkvoH/uEg3RfdMqyoDRP+rk1br++Wqrp/9ePyuTPZ C9pY1RVriGRbSau8fQ8WRSpHkKpcF8Vmjm4jYWXMbSdZQrv37YPUk351uxOtgOFAr+fR9QtXWdT c X-Received: by 2002:a17:902:380c:: with SMTP id l12-v6mr533717plc.19.1527639133154; Tue, 29 May 2018 17:12:13 -0700 (PDT) X-Received: by 2002:a17:902:380c:: with SMTP id l12-v6mr533638plc.19.1527639131809; Tue, 29 May 2018 17:12:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527639131; cv=none; d=google.com; s=arc-20160816; b=J2o12SA5oXS8gN31oT12P4j2AcsHj4JP4GKyT5pw9Sj99w211e0YnwYeyQWwSDeC2e 8iTWiZt+KkJLEi1v2tNnpR6XmtTZ85GLHZ7ytxzPH8nnrsyIS7cUI1a7YjYmiYAMS9/9 Dip7HOpqIK9L/2QC7ijiINczN/xGyNAzirI/y25FH7t1p285TZ6m6Aq6nrlkoH5IKw5/ TlpGkuBjHnfZZro69hU46m1eY/0sg/tDEA8m09B/9LETIBxyQPUbshyi0Y44o5dd4v3s 6eNcR1BleGLMI92PFxlTSO9laCjhTd2IGDWK4DVSX6k6m3zXyIWjG59UmAmnKhc+b8e3 tvoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=2OJnmQ7l7NGFZP9ipCIfC7mNh4H09z1pqCsCgocmqlg=; b=Tc3vzOLqNTselA8nRQLA/sWijYNDMWFvcty70d8IZssI7EOMYoUXzaIrsGJillH/Px +I0oxexZ3f0vn+94RxKXMeo03zChOWY0SaABEYj87lhQuppY3Ra/DGKQukxqOfu1e06L juYB3zeziAuAfO9Cc5rKVD/3yI5+v6pV+g9lG9qyIWzVVcg6Vhwu5PNRu9G81vpZK+6B /yC1Ww9ZEGQf7/RZ+iuqTLPUniSEXdqQLTpyJB7ZXooiTbqWD2G8pcF5N8RkXbncRFFQ k/KU3tRArIRnukL4OShG68lTwor/loOkPb1OUWaBwLBEqmipYxeX8bURG2CwFxHitnNz omLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sGtyIGYK; spf=pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r9-v6sor13221317pfh.53.2018.05.29.17.12.11 for (Google Transport Security); Tue, 29 May 2018 17:12:11 -0700 (PDT) Received-SPF: pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sGtyIGYK; spf=pass (google.com: domain of shakeelb@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=shakeelb@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=2OJnmQ7l7NGFZP9ipCIfC7mNh4H09z1pqCsCgocmqlg=; b=sGtyIGYK4ifx7d1x9COXRmNyEoP2iG9FUajUa3T0KuY2mASWiq5x3XrKRS2BjMiTN8 QgAm6Szq+2t895V94c9pfSbYTAIITezaFw4NC/o9wmI6uK6nyMK1tSB7WG5dXomzIbyf jRUdED+tSv65W/o/Guz/dkn2VdpB6T7wHSMLScmP4acScM2HkfVL+sJUKgLSxFgKsKJF Pdqamd4v7fuGzE2C22qLYRFO61Tdt5CfFOqlFtLMtUbN0P/+N/rnd/R+/NtPUg24PQb2 iVd+kMCuKFjjzo/Svn/s4C/Wk1kvgpIYywoJoCsyonvcJMdPyAV+EgirbxsnS+HOYldA GZFA== X-Google-Smtp-Source: ADUXVKJdoVfbFqHJtR/NfpcDsu6SYqAIxVrCJexmhsyr7KlwqRGy3pGvp6p05Ojq4JDGGMcRBcVHNQ== X-Received: by 2002:a62:9e88:: with SMTP id f8-v6mr512253pfk.206.1527639131138; Tue, 29 May 2018 17:12:11 -0700 (PDT) Received: from shakeelb.mtv.corp.google.com ([2620:15c:2cb:201:3a5f:3a4f:fa44:6b63]) by smtp.gmail.com with ESMTPSA id r79-v6sm13295844pfe.115.2018.05.29.17.12.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 May 2018 17:12:09 -0700 (PDT) From: Shakeel Butt To: Michal Hocko , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Greg Thelen , Johannes Weiner , Vladimir Davydov , Tejun Heo Cc: Linux MM , Cgroups , LKML , Shakeel Butt Subject: [PATCH v3] mm: fix race between kmem_cache destroy, create and deactivate Date: Tue, 29 May 2018 17:12:04 -0700 Message-Id: <20180530001204.183758-1-shakeelb@google.com> X-Mailer: git-send-email 2.17.0.921.gf22659ad46-goog X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The memcg kmem cache creation and deactivation (SLUB only) is asynchronous. If a root kmem cache is destroyed whose memcg cache is in the process of creation or deactivation, the kernel may crash. Example of one such crash: general protection fault: 0000 [#1] SMP PTI CPU: 1 PID: 1721 Comm: kworker/14:1 Not tainted 4.17.0-smp ... Workqueue: memcg_kmem_cache kmemcg_deactivate_workfn RIP: 0010:has_cpu_slab ... Call Trace: ? on_each_cpu_cond __kmem_cache_shrink kmemcg_cache_deact_after_rcu kmemcg_deactivate_workfn process_one_work worker_thread kthread ret_from_fork+0x35/0x40 To fix this race, on root kmem cache destruction, mark the cache as dying and flush the workqueue used for memcg kmem cache creation and deactivation. Signed-off-by: Shakeel Butt --- Changelog since v2: - Instead of refcount, flush the workqueue Changelog since v1: - Added more documentation to the code - Renamed fields to be more readable --- include/linux/slab.h | 1 + mm/slab_common.c | 21 ++++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 9ebe659bd4a5..71c5467d99c1 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -658,6 +658,7 @@ struct memcg_cache_params { struct memcg_cache_array __rcu *memcg_caches; struct list_head __root_caches_node; struct list_head children; + bool dying; }; struct { struct mem_cgroup *memcg; diff --git a/mm/slab_common.c b/mm/slab_common.c index b0dd9db1eb2f..a3496375f5ea 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -136,6 +136,7 @@ void slab_init_memcg_params(struct kmem_cache *s) s->memcg_params.root_cache = NULL; RCU_INIT_POINTER(s->memcg_params.memcg_caches, NULL); INIT_LIST_HEAD(&s->memcg_params.children); + s->memcg_params.dying = false; } static int init_memcg_params(struct kmem_cache *s, @@ -608,7 +609,7 @@ void memcg_create_kmem_cache(struct mem_cgroup *memcg, * The memory cgroup could have been offlined while the cache * creation work was pending. */ - if (memcg->kmem_state != KMEM_ONLINE) + if (memcg->kmem_state != KMEM_ONLINE || root_cache->memcg_params.dying) goto out_unlock; idx = memcg_cache_id(memcg); @@ -712,6 +713,9 @@ void slab_deactivate_memcg_cache_rcu_sched(struct kmem_cache *s, WARN_ON_ONCE(s->memcg_params.deact_fn)) return; + if (s->memcg_params.root_cache->memcg_params.dying) + return; + /* pin memcg so that @s doesn't get destroyed in the middle */ css_get(&s->memcg_params.memcg->css); @@ -823,11 +827,24 @@ static int shutdown_memcg_caches(struct kmem_cache *s) return -EBUSY; return 0; } + +static void flush_memcg_workqueue(struct kmem_cache *s) +{ + mutex_lock(&slab_mutex); + s->memcg_params.dying = true; + mutex_unlock(&slab_mutex); + + flush_workqueue(memcg_kmem_cache_wq); +} #else static inline int shutdown_memcg_caches(struct kmem_cache *s) { return 0; } + +static inline void flush_memcg_workqueue(struct kmem_cache *s) +{ +} #endif /* CONFIG_MEMCG && !CONFIG_SLOB */ void slab_kmem_cache_release(struct kmem_cache *s) @@ -845,6 +862,8 @@ void kmem_cache_destroy(struct kmem_cache *s) if (unlikely(!s)) return; + flush_memcg_workqueue(s); + get_online_cpus(); get_online_mems();