From patchwork Fri Nov 27 04:14:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 11935027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 058BEC2D0E4 for ; Fri, 27 Nov 2020 04:14:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 415DE207D8 for ; Fri, 27 Nov 2020 04:14:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="hBMWXQGd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 415DE207D8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=fb.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 588716B006E; Thu, 26 Nov 2020 23:14:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 512636B0070; Thu, 26 Nov 2020 23:14:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 400BD6B0071; Thu, 26 Nov 2020 23:14:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id 2663A6B006E for ; Thu, 26 Nov 2020 23:14:57 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DF836363C for ; Fri, 27 Nov 2020 04:14:56 +0000 (UTC) X-FDA: 77528882592.21.run61_2a0844827385 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id BE33C180442D4 for ; Fri, 27 Nov 2020 04:14:56 +0000 (UTC) X-HE-Tag: run61_2a0844827385 X-Filterd-Recvd-Size: 6032 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Nov 2020 04:14:55 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.42/8.16.0.42) with SMTP id 0AR4EtXL022341 for ; Thu, 26 Nov 2020 20:14:55 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=facebook; bh=nd8TtOAa9xHhX+Sd5SU6lTBEa7NEXDyESd69KLboYpI=; b=hBMWXQGdxqjQKsOn3eNr13b0n8cvY2Z/Ozmr4SI5UEfqXCJ3sMbj1EGPgmEP9+d915DX lHiAmCWTFBdET9G4iqNL6HtqfkdyC3bcfNSQvIyY/mX4vFpUP94MuW6bHfhHxgLtCKBF EYtxF4X9Onn5V7aYPu8pIVmO4BPNW6uWySk= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0089730.ppops.net with ESMTP id 352ghthq6p-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 26 Nov 2020 20:14:55 -0800 Received: from intmgw001.41.prn1.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1979.3; Thu, 26 Nov 2020 20:14:26 -0800 Received: by devvm3388.prn0.facebook.com (Postfix, from userid 111017) id 1FDC9176D279; Thu, 26 Nov 2020 20:14:13 -0800 (PST) From: Roman Gushchin To: Andrew Morton , CC: Shakeel Butt , Johannes Weiner , Michal Hocko , , , Roman Gushchin , Subject: [PATCH] mm: memcg/slab: fix obj_cgroup_charge() return value handling Date: Thu, 26 Nov 2020 20:14:05 -0800 Message-ID: <20201127041405.3459198-1-guro@fb.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312,18.0.737 definitions=2020-11-27_01:2020-11-26,2020-11-27 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 suspectscore=2 clxscore=1015 mlxlogscore=746 malwarescore=0 mlxscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 priorityscore=1501 spamscore=0 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2011270023 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Commit 10befea91b61 ("mm: memcg/slab: use a single set of kmem_caches for all allocations") introduced a regression into the handling of the obj_cgroup_charge() return value. If a non-zero value is returned (indicating of exceeding one of memory.max limits), the allocation should fail, instead of falling back to non-accounted mode. To make the code more readable, move memcg_slab_pre_alloc_hook() and memcg_slab_post_alloc_hook() calling conditions into bodies of these hooks. Fixes: 10befea91b61 ("mm: memcg/slab: use a single set of kmem_caches for all allocations") Signed-off-by: Roman Gushchin Cc: stable@vger.kernel.org Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt --- mm/slab.h | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index 59aeb0d9f11b..5dc89d8fb05e 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -257,22 +257,32 @@ static inline size_t obj_full_size(struct kmem_cache *s) return s->size + sizeof(struct obj_cgroup *); } -static inline struct obj_cgroup *memcg_slab_pre_alloc_hook(struct kmem_cache *s, - size_t objects, - gfp_t flags) +/* + * Returns true if the allocation should fail. + */ +static inline bool memcg_slab_pre_alloc_hook(struct kmem_cache *s, + struct obj_cgroup **objcgp, + size_t objects, gfp_t flags) { struct obj_cgroup *objcg; + if (!memcg_kmem_enabled()) + return false; + + if (!(flags & __GFP_ACCOUNT) && !(s->flags & SLAB_ACCOUNT)) + return false; + objcg = get_obj_cgroup_from_current(); if (!objcg) - return NULL; + return false; if (obj_cgroup_charge(objcg, flags, objects * obj_full_size(s))) { obj_cgroup_put(objcg); - return NULL; + return true; } - return objcg; + *objcgp = objcg; + return false; } static inline void mod_objcg_state(struct obj_cgroup *objcg, @@ -298,7 +308,7 @@ static inline void memcg_slab_post_alloc_hook(struct kmem_cache *s, unsigned long off; size_t i; - if (!objcg) + if (!memcg_kmem_enabled() || !objcg) return; flags &= ~__GFP_ACCOUNT; @@ -382,11 +392,11 @@ static inline void memcg_free_page_obj_cgroups(struct page *page) { } -static inline struct obj_cgroup *memcg_slab_pre_alloc_hook(struct kmem_cache *s, - size_t objects, - gfp_t flags) +static inline bool memcg_slab_pre_alloc_hook(struct kmem_cache *s, + struct obj_cgroup **objcgp, + size_t objects, gfp_t flags) { - return NULL; + return false; } static inline void memcg_slab_post_alloc_hook(struct kmem_cache *s, @@ -494,9 +504,8 @@ static inline struct kmem_cache *slab_pre_alloc_hook(struct kmem_cache *s, if (should_failslab(s, flags)) return NULL; - if (memcg_kmem_enabled() && - ((flags & __GFP_ACCOUNT) || (s->flags & SLAB_ACCOUNT))) - *objcgp = memcg_slab_pre_alloc_hook(s, size, flags); + if (memcg_slab_pre_alloc_hook(s, objcgp, size, flags)) + return NULL; return s; } @@ -515,8 +524,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, s->flags, flags); } - if (memcg_kmem_enabled()) - memcg_slab_post_alloc_hook(s, objcg, flags, size, p); + memcg_slab_post_alloc_hook(s, objcg, flags, size, p); } #ifndef CONFIG_SLOB