From patchwork Wed Dec 2 18:27:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9795C71155 for ; Wed, 2 Dec 2020 18:27:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4411222248 for ; Wed, 2 Dec 2020 18:27:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4411222248 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C337D6B005C; Wed, 2 Dec 2020 13:27:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE39E6B005D; Wed, 2 Dec 2020 13:27:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AADE86B0068; Wed, 2 Dec 2020 13:27:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id 96CED6B005C for ; Wed, 2 Dec 2020 13:27:47 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5E2458249980 for ; Wed, 2 Dec 2020 18:27:47 +0000 (UTC) X-FDA: 77549175774.06.slave37_2f01697273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 3E86E100484B4 for ; Wed, 2 Dec 2020 18:27:47 +0000 (UTC) X-HE-Tag: slave37_2f01697273b5 X-Filterd-Recvd-Size: 4444 Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:27:46 +0000 (UTC) Received: by mail-pj1-f66.google.com with SMTP id r9so1488087pjl.5 for ; Wed, 02 Dec 2020 10:27:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3zNax7Y+1Ikzzmhg7iyW9MLSCw7jdgQnq4TmPmAJA28=; b=lqgtJVieKYGe00RIy86iIPDzKFYzexsfSaJr1YoR0txN85LXyOfkwRqTWB4Yi4HbdH nW1fn6UWKm75o2mYOsdPMdaVFWvTVzYmb3IHzsTFljeNu2jYbGsayg5p/0JjFIKUy7fi EcVJnKMcNIxe8nbRgx7UFLKUrp5JpvY6Ut1jHyBgbJv0ywwGLMkwB3LRH1YWUgazYloO FyEMdudReYKkvu8AvWCzlu/nf0Y6nkMhXK3doSrKwApihvi4aJpDpKf9Y8GzIlX70gTU HrWUPyuhdv2b7Xk0Cq6TUZ/dng83s4OMzmHCh3za7c3rpSqI13eT9uaq9BIQzuKJTWHf Bs7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3zNax7Y+1Ikzzmhg7iyW9MLSCw7jdgQnq4TmPmAJA28=; b=Q4WFlxs4ydnSDAbD28WwQFwxc61rZveVfc5Jc1J6OtrPqAIxPmF3zhjZRdz+Lp5T6V p5MkW5uCzZxxOKZ7OckcgYdHp/h/yY6Fs69+nWb9Zxbs0jaff/CxOpAR+QCoR7r1unxA 59TAXwsZ1048crLGlFBoie2NS9CSWL64140g+0y5ZAfZrrmILrnqACjwCli1Ynhb08eR sIy8Smsy1jn7oWMzyfYx0F78xHs7Ii1R+aF9jx2pvNHrscvbOiPuFrzn/X2viQW1hZkb r11sDlmY/dsZGPSTJ+4q2MSYF8v+LT8xdbhmH2BCjNHOs8ue0vR9mmE7X3CORf6x/MMa 8crA== X-Gm-Message-State: AOAM531CVgJ7vADhxiOmAHsmHGFCepLjYJu8xgihW8sb9QRQzBNrvaCG 6N9UOQyHB/J4Cn+5bHxrkt0= X-Google-Smtp-Source: ABdhPJz1U890O+WgeLnLZND4JEg1gfgDFo2nW9DjhiZUqUmGdG+/Pm+qglOPbdvkQ2Toj/Szkz158w== X-Received: by 2002:a17:902:b691:b029:d8:ebc8:385e with SMTP id c17-20020a170902b691b02900d8ebc8385emr3855010pls.48.1606933665893; Wed, 02 Dec 2020 10:27:45 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:44 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/9] mm: vmscan: simplify nr_deferred update code Date: Wed, 2 Dec 2020 10:27:17 -0800 Message-Id: <20201202182725.265020-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently if (next_deferred - scanned) = 0, the code would just read the current nr_deferred otherwise add the delta back. Both needs atomic operation anyway, it seems there is not too much gain by distinguishing the two cases, so just add the delta back even though the delta is 0. This would simply the code for the following patches too. Signed-off-by: Yang Shi --- mm/vmscan.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 7b4e31eac2cf..7d6186a07daf 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -528,14 +528,10 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, next_deferred = 0; /* * move the unused scan count back into the shrinker in a - * manner that handles concurrent updates. If we exhausted the - * scan, there is no need to do an update. + * manner that handles concurrent updates. */ - if (next_deferred > 0) - new_nr = atomic_long_add_return(next_deferred, - &shrinker->nr_deferred[nid]); - else - new_nr = atomic_long_read(&shrinker->nr_deferred[nid]); + new_nr = atomic_long_add_return(next_deferred, + &shrinker->nr_deferred[nid]); trace_mm_shrink_slab_end(shrinker, nid, freed, nr, new_nr, total_scan); return freed; From patchwork Wed Dec 2 18:27:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07D60C71155 for ; Wed, 2 Dec 2020 18:27:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5F06E22248 for ; Wed, 2 Dec 2020 18:27:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F06E22248 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A889F6B005D; Wed, 2 Dec 2020 13:27:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A0F936B0068; Wed, 2 Dec 2020 13:27:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 925AE6B006C; Wed, 2 Dec 2020 13:27:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id 7A30B6B005D for ; Wed, 2 Dec 2020 13:27:50 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 4150D349B for ; Wed, 2 Dec 2020 18:27:50 +0000 (UTC) X-FDA: 77549175900.06.car52_4400f44273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 06F8F10048171 for ; Wed, 2 Dec 2020 18:27:49 +0000 (UTC) X-HE-Tag: car52_4400f44273b5 X-Filterd-Recvd-Size: 4149 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:27:49 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id f1so1121890plt.12 for ; Wed, 02 Dec 2020 10:27:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zvp2Z7uKK5PPjiIE9UzxqFK/LcqQo3ZGRbQdLWMTrqs=; b=vQtKypllp/ZZ0qsyFl+L1bdVT8aToTf0+8odd+XSapcKWxc5LUWAr4l3jkmZ4vhQOW KN7rORAMQob3HKpAq9Bo/WslPYsUTtUae9QaKYOJffWRQqfbndIMK3TpSU35pwR5iTD4 TsnDUEvpS8m5tQkWYi2rdtAsknL5ro5v4cAqgmX72mN/HT819mvMJnK8e0mh6JNV6Maj 2NsjjVMtDssxfgCYKk3P2Eqp9uvihVqFSXIBowtndIoItqCBuTdOMlxlD1s57LlUvnCI qRKvscIQ16bRuTecy7wqTbfkB45/nvW14IB3Y5Iv2MzVJdc7BX/EYEctNRmDkqjzVpMx +8tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zvp2Z7uKK5PPjiIE9UzxqFK/LcqQo3ZGRbQdLWMTrqs=; b=q6QWDtIiwrc+8/s+dPPrBix5cTFZCIIun7MCRAT/GbOoAzJ43+tOxxtQUsoTeXn/i0 oyAK038RLAKuzTIym/H9vbGcOPTDJcEdoKBludwDeiZi7I5n1/u2vECI7lOtD7ro+SAA y5OIvdHNWwUl+HHJPM5DHfKcO4rAAQYxMzTynoNqXoZOuO3f1P6//XIx1I2GkAqFwBkx 7rV0ZC6TO17yVOm75Op1QCdQ2bwMoQJKlCzoixJitRRF06+y+ZzMVOItWk8ZSSFC4sqc X2t+ZwvKNW10HHBlZcpPwr+Kl44ZFiHohufgSN+Vm72Jv94RCkLy0kKHuISLCtIDkhdu fgig== X-Gm-Message-State: AOAM5335wHenz1cJuv0/BcFGH7pktby53DM03U09kVvZ96KdhLyDEhRv C7IR8pO5BXzK/Q0bktVvA82ltWzcbs/12g== X-Google-Smtp-Source: ABdhPJwbRBqa8Oz+BOZml8FzyUoa1O4bjuenGRS20w3xqULuzoR4981HLWGbv0U06BXH5Qa8LXKx0g== X-Received: by 2002:a17:90a:19d5:: with SMTP id 21mr1047317pjj.187.1606933668614; Wed, 02 Dec 2020 10:27:48 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:47 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/9] mm: vmscan: use nid from shrink_control for tracepoint Date: Wed, 2 Dec 2020 10:27:18 -0800 Message-Id: <20201202182725.265020-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The tracepoint's nid should show what node the shrink happens on, the start tracepoint uses nid from shrinkctl, but the nid might be set to 0 before end tracepoint if the shrinker is not NUMA aware, so the traceing log may show the shrink happens on one node but end up on the other node. It seems confusing. And the following patch will remove using nid directly in do_shrink_slab(), this patch also helps cleanup the code. Signed-off-by: Yang Shi --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 7d6186a07daf..457ce04eebf2 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -533,7 +533,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, new_nr = atomic_long_add_return(next_deferred, &shrinker->nr_deferred[nid]); - trace_mm_shrink_slab_end(shrinker, nid, freed, nr, new_nr, total_scan); + trace_mm_shrink_slab_end(shrinker, shrinkctl->nid, freed, nr, new_nr, total_scan); return freed; } From patchwork Wed Dec 2 18:27:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F208BC64E7C for ; Wed, 2 Dec 2020 18:27:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6312A22248 for ; Wed, 2 Dec 2020 18:27:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6312A22248 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AAD706B0068; Wed, 2 Dec 2020 13:27:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A860D6B006C; Wed, 2 Dec 2020 13:27:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94F456B006E; Wed, 2 Dec 2020 13:27:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0141.hostedemail.com [216.40.44.141]) by kanga.kvack.org (Postfix) with ESMTP id 7BB516B0068 for ; Wed, 2 Dec 2020 13:27:52 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 45958349B for ; Wed, 2 Dec 2020 18:27:52 +0000 (UTC) X-FDA: 77549175984.20.game57_2f0470c273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 21C83180C07AB for ; Wed, 2 Dec 2020 18:27:52 +0000 (UTC) X-HE-Tag: game57_2f0470c273b5 X-Filterd-Recvd-Size: 5161 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:27:51 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id b12so1445040pjl.0 for ; Wed, 02 Dec 2020 10:27:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L0EYVG9/JHZSwQKcdm/uLO/GzIIIcVeao0a1qNmlzMo=; b=GXcmtb/78TmgRjL90+9AKiKDg4mnBqNkTFZGUjNrkQyhPFWnxWHSVxs0DTYbJk6Phx TazQkZ9fOF7szRFViPIaI0ROE8eFgpvx9II8n3p38lZSVEjQ3dRqbbIMY3MVykotTJsl BNShLtVE/G84sNhuNL1LeF627T1FpxJ6SOdiwmWLT3yS5yTqHbSnYuid/xR5f77I54nR W0iyAUkHH4fT4u/nrsyR9xbgpZQiPCdF2WmENoSClFzMTLy0fdn0/lSSBI23C2/JJrw7 XLS1iN+WD0VnI0t67kf8JimWA2DEpqR+PKa9DNv1EyRWPwGphhTjcTHMHeI/cBf2652r xDnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L0EYVG9/JHZSwQKcdm/uLO/GzIIIcVeao0a1qNmlzMo=; b=HvYEqc2RS0zzzUCY73Nik6kz61QbenOHpXIYocrsowPj1wxcbUMlp4mcsMbu2NvQxS ce+hw2suwjulRKyw0vjQY8DbYAp3fODv49o8kEbKHXWWl8017Gew4lSmFmJZhMDFFF3q UTebJ92kMVYaqV0E5fORty702PCFDaLKFGqwc2A/J8VrAV4P7sZQlpcNoXAX1CUT96eS w4JNSrJqO2yVMy5vfpR/Yvxz+4rMxEg95pBcK1rvk9hMbcVw3jxJLORoMcOG+YJZmlEv wJUOtqSkHbByxkfQ2ErYzCnRWj0fmYS0lwNUhHTyCMLQMgGLUpHcwe0lL8L1y1UTZlYP ksrg== X-Gm-Message-State: AOAM530UijNKTZOKO6glz14Gwbsjw9UHMPhbr2Jamp5J69yg2Cf4Z3I2 2Pzc1GlveRHprcW5UbQcu/s= X-Google-Smtp-Source: ABdhPJz6Mh4mDqhuFxz84ELSR6dVfwNcpfWdx8Zo/0l4bcZNoiNkq6y2HGDqfzHOf/F2Rv9sAuVCLQ== X-Received: by 2002:a17:90a:4817:: with SMTP id a23mr1102425pjh.16.1606933670937; Wed, 02 Dec 2020 10:27:50 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:49 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/9] mm: memcontrol: rename memcg_shrinker_map_mutex to memcg_shrinker_mutex Date: Wed, 2 Dec 2020 10:27:19 -0800 Message-Id: <20201202182725.265020-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The following patch will add memcg_shrinker_deferred which could be protected by the same mutex, rename it to a more common name. Signed-off-by: Yang Shi --- mm/memcontrol.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 29459a6ce1c7..19e41684c96b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -395,7 +395,7 @@ EXPORT_SYMBOL(memcg_kmem_enabled_key); #endif static int memcg_shrinker_map_size; -static DEFINE_MUTEX(memcg_shrinker_map_mutex); +static DEFINE_MUTEX(memcg_shrinker_mutex); static void memcg_free_shrinker_map_rcu(struct rcu_head *head) { @@ -408,7 +408,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, struct memcg_shrinker_map *new, *old; int nid; - lockdep_assert_held(&memcg_shrinker_map_mutex); + lockdep_assert_held(&memcg_shrinker_mutex); for_each_node(nid) { old = rcu_dereference_protected( @@ -458,7 +458,7 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg) if (mem_cgroup_is_root(memcg)) return 0; - mutex_lock(&memcg_shrinker_map_mutex); + mutex_lock(&memcg_shrinker_mutex); size = memcg_shrinker_map_size; for_each_node(nid) { map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, nid); @@ -469,7 +469,7 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg) } rcu_assign_pointer(memcg->nodeinfo[nid]->shrinker_map, map); } - mutex_unlock(&memcg_shrinker_map_mutex); + mutex_unlock(&memcg_shrinker_mutex); return ret; } @@ -484,7 +484,7 @@ int memcg_expand_shrinker_maps(int new_id) if (size <= old_size) return 0; - mutex_lock(&memcg_shrinker_map_mutex); + mutex_lock(&memcg_shrinker_mutex); if (!root_mem_cgroup) goto unlock; @@ -500,7 +500,7 @@ int memcg_expand_shrinker_maps(int new_id) unlock: if (!ret) memcg_shrinker_map_size = size; - mutex_unlock(&memcg_shrinker_map_mutex); + mutex_unlock(&memcg_shrinker_mutex); return ret; } From patchwork Wed Dec 2 18:27:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CCC1C71155 for ; Wed, 2 Dec 2020 18:27:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CF6B622249 for ; Wed, 2 Dec 2020 18:27:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF6B622249 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 52FD06B006C; Wed, 2 Dec 2020 13:27:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E4A66B006E; Wed, 2 Dec 2020 13:27:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D0596B0070; Wed, 2 Dec 2020 13:27:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0095.hostedemail.com [216.40.44.95]) by kanga.kvack.org (Postfix) with ESMTP id 2735B6B006C for ; Wed, 2 Dec 2020 13:27:55 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DBE538249980 for ; Wed, 2 Dec 2020 18:27:54 +0000 (UTC) X-FDA: 77549176068.26.stew49_0c0374b273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id B3A651804B676 for ; Wed, 2 Dec 2020 18:27:54 +0000 (UTC) X-HE-Tag: stew49_0c0374b273b5 X-Filterd-Recvd-Size: 5783 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:27:54 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id b23so1579767pls.11 for ; Wed, 02 Dec 2020 10:27:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mkEYq1AbhtqAqcYcbg4MnCPD7vjrV9yb9ptbuyDsX7s=; b=e9dwHcNdu7EFBl/w9TTMJ9je2f+u7YOLF0oqHhpkacwkv0z6eoHdHbR5bBVysiUy7S k/eyxfXWwsW8tq1Rw7LSGswb7sz3iEbgs4Dzuc64wQHTZsT5dB7+wYk5p245+y054Pbx s1Uz0/gZ2OnYVxsYq95GVM5/9ZonmhfVYVWtvTU8zc6L3SL4ARxblDsZ79r1pneesFaJ ARnO7Jep0MtbYcsov1Aen2ynYSROnCY6OJZc5n91JATpvPaQ+xxFGwsJh1LARsxG27GM JMNYmZ5fMAkOQ8hNjZj0QT0gJTCH+CctT9rZ446s538NXr1uKWBwZ8QONdKGJxqjrGVq siYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mkEYq1AbhtqAqcYcbg4MnCPD7vjrV9yb9ptbuyDsX7s=; b=V7ar+Cu7BtQTfofLNb6cPX3TrUmUGbXiAcS4XMNIP+u7UJkPccL2raa9ywup7C0ghl qwPntmofao9YqvHu2bzhowoEanoUuu2SAwvjCc1FbzdPDJQKzFS89XTXl5+/pmdsDyvz H3v3h/7NRvZNwSeb9WJfAe4L1HTUgwMNsV641aD07zlY6yMIdCnfo4SPhoDC9o6LxYW8 BTELUD0f2Uixt8yiJJAknRv3mGFktIrsYaDodw6tGoE4kp7oaXZixTSRXnU5N6q6Wx7V g5s8Ij8Yr4AfssEjFo2LorqQXFnKCEQB2p+RGyj/MAUczaXG8wkNhbRV1rCW4wr1/5mD pQrQ== X-Gm-Message-State: AOAM531RnqkMHeb20PRZIcAVDUstrblXxMTuHxZSwPLCy5OVyyrxlr+I s6Mq8Bi+Csui8DdLXHcr2QY= X-Google-Smtp-Source: ABdhPJzPNtHrC4FduKm4RHeTLfsFp82wDr46oGUu8j76eW7YY+ermZiSR0NsIBwV4iSHp3D3Ek/b0Q== X-Received: by 2002:a17:902:7297:b029:da:861e:eae1 with SMTP id d23-20020a1709027297b02900da861eeae1mr3726068pll.8.1606933673363; Wed, 02 Dec 2020 10:27:53 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:52 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/9] mm: vmscan: use a new flag to indicate shrinker is registered Date: Wed, 2 Dec 2020 10:27:20 -0800 Message-Id: <20201202182725.265020-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently registered shrinker is indicated by non-NULL shrinker->nr_deferred. This approach is fine with nr_deferred atthe shrinker level, but the following patches will move MEMCG_AWARE shrinkers' nr_deferred to memcg level, so their shrinker->nr_deferred would always be NULL. This would prevent the shrinkers from unregistering correctly. Introduce a new "state" field to indicate if shrinker is registered or not. We could use the highest bit of flags, but it may be a little bit complicated to extract that bit and the flags is accessed frequently by vmscan (every time shrinker is called). So add a new field in "struct shrinker", we may waster a little bit memory, but it should be very few since there should be not too many registered shrinkers on a normal system. Signed-off-by: Yang Shi --- include/linux/shrinker.h | 4 ++++ mm/vmscan.c | 13 +++++++++---- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 0f80123650e2..0bb5be88e41d 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -35,6 +35,9 @@ struct shrink_control { #define SHRINK_STOP (~0UL) #define SHRINK_EMPTY (~0UL - 1) + +#define SHRINKER_REGISTERED 0x1 + /* * A callback you can register to apply pressure to ageable caches. * @@ -66,6 +69,7 @@ struct shrinker { long batch; /* reclaim batch size, 0 = default */ int seeks; /* seeks to recreate an obj */ unsigned flags; + unsigned state; /* These are for internal use */ struct list_head list; diff --git a/mm/vmscan.c b/mm/vmscan.c index 457ce04eebf2..0d628299e55c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -378,6 +378,7 @@ void register_shrinker_prepared(struct shrinker *shrinker) if (shrinker->flags & SHRINKER_MEMCG_AWARE) idr_replace(&shrinker_idr, shrinker, shrinker->id); #endif + shrinker->state |= SHRINKER_REGISTERED; up_write(&shrinker_rwsem); } @@ -397,13 +398,17 @@ EXPORT_SYMBOL(register_shrinker); */ void unregister_shrinker(struct shrinker *shrinker) { - if (!shrinker->nr_deferred) - return; - if (shrinker->flags & SHRINKER_MEMCG_AWARE) - unregister_memcg_shrinker(shrinker); down_write(&shrinker_rwsem); + if (!shrinker->state) { + up_write(&shrinker_rwsem); + return; + } list_del(&shrinker->list); + shrinker->state &= ~SHRINKER_REGISTERED; up_write(&shrinker_rwsem); + + if (shrinker->flags & SHRINKER_MEMCG_AWARE) + unregister_memcg_shrinker(shrinker); kfree(shrinker->nr_deferred); shrinker->nr_deferred = NULL; } From patchwork Wed Dec 2 18:27:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE702C71155 for ; Wed, 2 Dec 2020 18:27:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 46AFC22248 for ; Wed, 2 Dec 2020 18:27:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 46AFC22248 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C5ACB6B006E; Wed, 2 Dec 2020 13:27:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C0DE66B0070; Wed, 2 Dec 2020 13:27:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B20186B0071; Wed, 2 Dec 2020 13:27:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 994C36B006E for ; Wed, 2 Dec 2020 13:27:57 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 63E90180AD807 for ; Wed, 2 Dec 2020 18:27:57 +0000 (UTC) X-FDA: 77549176194.05.coach66_2f12176273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 4923E18030159 for ; Wed, 2 Dec 2020 18:27:57 +0000 (UTC) X-HE-Tag: coach66_2f12176273b5 X-Filterd-Recvd-Size: 12141 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:27:56 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id s2so1586039plr.9 for ; Wed, 02 Dec 2020 10:27:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+now6GXkLhkQRGDQHjpHa4FbHWB432OuUSJYQ5tg+pI=; b=D0aFlfvHsuZ2kjGJz4ZhdA39FJT6b7FReaIjJfGRKgFd9J9LzyVlhmm5r0Fq5smXv9 FljoCZsjWDbRPay8zpMA9Yoe2LGFFURSihESWgCjGClGnsHOV4CSLKHV1ObdJY32zrJ1 BDgFATRngM0fIMZQYN5zk8MJy/5hpRlojQrxoluCuQdLDzvyj7+VKP1zzVNpVAxpG11O dSAXZx+sjog/MN2LhcFYbsEPoTBSCK50NbLYCNMpUnegxIjEyc076mfKXdkWf1m8eaSp 8Y6yuBqtmbyPSxX2Y45GiFznUxGB0K41dtbeg+DOqohZX+OXMDKy9nZMVwXJjf9/D7pE NpYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+now6GXkLhkQRGDQHjpHa4FbHWB432OuUSJYQ5tg+pI=; b=kbzbKmNEOx28uaVOI9qvLgApDEqUWyfk1bli3Vnx/jryS/58hbvol8Vk2tgJ4+ZpAM QltsmrzQHxEplObuzvxINVxpf8HkJGmUwq02g8Cx/3GxLSw6yyzcQsuS/WOx3FpqTC4r 4avkod25d4sVZ67cd7NL3rvaPcqLaM0Z3/8C9woLrJMaW2sHRp/6IXUxIDs3WC/WRxn1 oe2vZDKZDWF5T45ObPoIu98FEg270gatla13f5Rg4154OCIouIStKWqD/PHokI6V5mRo P6MjwmofCTw04CE23sDjAvsnx0VnKp2u2TzcV42KMX9NprLlRWlOZByH8OT/Mb1ulOnV SshQ== X-Gm-Message-State: AOAM5328pO8glf//DVs/IrUu0tW7zyRkHFQq1GNSUq6s46wDhviDn/wM fW8RiXfufLuFOHcn2WO9pNk= X-Google-Smtp-Source: ABdhPJy5Y6eaGT8FaEdvLzURm/UqsiizIu/LAUICpZ5RukCtFGdPwPLHXZHQBi5PokaLDCLslQ3IuQ== X-Received: by 2002:a17:902:d34a:b029:da:861e:ecd8 with SMTP id l10-20020a170902d34ab02900da861eecd8mr3632408plk.45.1606933675682; Wed, 02 Dec 2020 10:27:55 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:54 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/9] mm: memcontrol: add per memcg shrinker nr_deferred Date: Wed, 2 Dec 2020 10:27:21 -0800 Message-Id: <20201202182725.265020-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently the number of deferred objects are per shrinker, but some slabs, for example, vfs inode/dentry cache are per memcg, this would result in poor isolation among memcgs. The deferred objects typically are generated by __GFP_NOFS allocations, one memcg with excessive __GFP_NOFS allocations may blow up deferred objects, then other innocent memcgs may suffer from over shrink, excessive reclaim latency, etc. For example, two workloads run in memcgA and memcgB respectively, workload in B is vfs heavy workload. Workload in A generates excessive deferred objects, then B's vfs cache might be hit heavily (drop half of caches) by B's limit reclaim or global reclaim. We observed this hit in our production environment which was running vfs heavy workload shown as the below tracing log: <...>-409454 [016] .... 28286961.747146: mm_shrink_slab_start: super_cache_scan+0x0/0x1a0 ffff9a83046f3458: nid: 1 objects to shrink 3641681686040 gfp_flags GFP_HIGHUSER_MOVABLE|__GFP_ZERO pgs_scanned 1 lru_pgs 15721 cache items 246404277 delta 31345 total_scan 123202138 <...>-409454 [022] .... 28287105.928018: mm_shrink_slab_end: super_cache_scan+0x0/0x1a0 ffff9a83046f3458: nid: 1 unused scan count 3641681686040 new scan count 3641798379189 total_scan 602 last shrinker return val 123186855 The vfs cache and page cache ration was 10:1 on this machine, and half of caches were dropped. This also resulted in significant amount of page caches were dropped due to inodes eviction. Make nr_deferred per memcg for memcg aware shrinkers would solve the unfairness and bring better isolation. When memcg is not enabled (!CONFIG_MEMCG or memcg disabled), the shrinker's nr_deferred would be used. And non memcg aware shrinkers use shrinker's nr_deferred all the time. Signed-off-by: Yang Shi --- include/linux/memcontrol.h | 9 +++ mm/memcontrol.c | 112 ++++++++++++++++++++++++++++++++++++- mm/vmscan.c | 4 ++ 3 files changed, 123 insertions(+), 2 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 922a7f600465..1b343b268359 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -92,6 +92,13 @@ struct lruvec_stat { long count[NR_VM_NODE_STAT_ITEMS]; }; + +/* Shrinker::id indexed nr_deferred of memcg-aware shrinkers. */ +struct memcg_shrinker_deferred { + struct rcu_head rcu; + atomic_long_t nr_deferred[]; +}; + /* * Bitmap of shrinker::id corresponding to memcg-aware shrinkers, * which have elements charged to this memcg. @@ -119,6 +126,7 @@ struct mem_cgroup_per_node { struct mem_cgroup_reclaim_iter iter; struct memcg_shrinker_map __rcu *shrinker_map; + struct memcg_shrinker_deferred __rcu *shrinker_deferred; struct rb_node tree_node; /* RB tree node */ unsigned long usage_in_excess;/* Set to the value by which */ @@ -1489,6 +1497,7 @@ static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) } extern int memcg_expand_shrinker_maps(int new_id); +extern int memcg_expand_shrinker_deferred(int new_id); extern void memcg_set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 19e41684c96b..d3d5c88db179 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -395,6 +395,8 @@ EXPORT_SYMBOL(memcg_kmem_enabled_key); #endif static int memcg_shrinker_map_size; +static int memcg_shrinker_deferred_size; + static DEFINE_MUTEX(memcg_shrinker_mutex); static void memcg_free_shrinker_map_rcu(struct rcu_head *head) @@ -402,6 +404,11 @@ static void memcg_free_shrinker_map_rcu(struct rcu_head *head) kvfree(container_of(head, struct memcg_shrinker_map, rcu)); } +static void memcg_free_shrinker_deferred_rcu(struct rcu_head *head) +{ + kvfree(container_of(head, struct memcg_shrinker_deferred, rcu)); +} + static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, int size, int old_size) { @@ -432,6 +439,36 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, return 0; } +static int memcg_expand_one_shrinker_deferred(struct mem_cgroup *memcg, + int size, int old_size) +{ + struct memcg_shrinker_deferred *new, *old; + int nid; + + lockdep_assert_held(&memcg_shrinker_mutex); + + for_each_node(nid) { + old = rcu_dereference_protected( + mem_cgroup_nodeinfo(memcg, nid)->shrinker_deferred, true); + /* Not yet online memcg */ + if (!old) + return 0; + + new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, nid); + if (!new) + return -ENOMEM; + + /* Copy all old values, and clear all new ones */ + memcpy((void *)new->nr_deferred, (void *)old->nr_deferred, old_size); + memset((void *)new->nr_deferred + old_size, 0, size - old_size); + + rcu_assign_pointer(memcg->nodeinfo[nid]->shrinker_deferred, new); + call_rcu(&old->rcu, memcg_free_shrinker_deferred_rcu); + } + + return 0; +} + static void memcg_free_shrinker_maps(struct mem_cgroup *memcg) { struct mem_cgroup_per_node *pn; @@ -450,6 +487,21 @@ static void memcg_free_shrinker_maps(struct mem_cgroup *memcg) } } +static void memcg_free_shrinker_deferred(struct mem_cgroup *memcg) +{ + struct mem_cgroup_per_node *pn; + struct memcg_shrinker_deferred *deferred; + int nid; + + for_each_node(nid) { + pn = mem_cgroup_nodeinfo(memcg, nid); + deferred = rcu_dereference_protected(pn->shrinker_deferred, true); + if (deferred) + kvfree(deferred); + rcu_assign_pointer(pn->shrinker_deferred, NULL); + } +} + static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg) { struct memcg_shrinker_map *map; @@ -474,6 +526,27 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg) return ret; } +static int memcg_alloc_shrinker_deferred(struct mem_cgroup *memcg) +{ + struct memcg_shrinker_deferred *deferred; + int nid, size, ret = 0; + + mutex_lock(&memcg_shrinker_mutex); + size = memcg_shrinker_deferred_size; + for_each_node(nid) { + deferred = kvzalloc_node(sizeof(*deferred) + size, GFP_KERNEL, nid); + if (!deferred) { + memcg_free_shrinker_deferred(memcg); + ret = -ENOMEM; + break; + } + rcu_assign_pointer(memcg->nodeinfo[nid]->shrinker_deferred, deferred); + } + mutex_unlock(&memcg_shrinker_mutex); + + return ret; +} + int memcg_expand_shrinker_maps(int new_id) { int size, old_size, ret = 0; @@ -504,6 +577,34 @@ int memcg_expand_shrinker_maps(int new_id) return ret; } +int memcg_expand_shrinker_deferred(int new_id) +{ + int size, old_size, ret = 0; + struct mem_cgroup *memcg; + + size = (new_id + 1) * sizeof(atomic_long_t); + old_size = memcg_shrinker_deferred_size; + if (size <= old_size) + return 0; + + mutex_lock(&memcg_shrinker_mutex); + if (!root_mem_cgroup) + goto unlock; + + for_each_mem_cgroup(memcg) { + ret = memcg_expand_one_shrinker_deferred(memcg, size, old_size); + if (ret) { + mem_cgroup_iter_break(NULL, memcg); + goto unlock; + } + } +unlock: + if (!ret) + memcg_shrinker_deferred_size = size; + mutex_unlock(&memcg_shrinker_mutex); + return ret; +} + void memcg_set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) { if (shrinker_id >= 0 && memcg && !mem_cgroup_is_root(memcg)) { @@ -5400,8 +5501,8 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css) struct mem_cgroup *memcg = mem_cgroup_from_css(css); /* - * A memcg must be visible for memcg_expand_shrinker_maps() - * by the time the maps are allocated. So, we allocate maps + * A memcg must be visible for memcg_expand_shrinker_{maps|deferred}() + * by the time the maps are allocated. So, we allocate maps and deferred * here, when for_each_mem_cgroup() can't skip it. */ if (memcg_alloc_shrinker_maps(memcg)) { @@ -5409,6 +5510,12 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css) return -ENOMEM; } + if (memcg_alloc_shrinker_deferred(memcg)) { + memcg_free_shrinker_maps(memcg); + mem_cgroup_id_remove(memcg); + return -ENOMEM; + } + /* Online state pins memcg ID, memcg ID pins CSS */ refcount_set(&memcg->id.ref, 1); css_get(css); @@ -5469,6 +5576,7 @@ static void mem_cgroup_css_free(struct cgroup_subsys_state *css) cancel_work_sync(&memcg->high_work); mem_cgroup_remove_from_trees(memcg); memcg_free_shrinker_maps(memcg); + memcg_free_shrinker_deferred(memcg); memcg_free_kmem(memcg); mem_cgroup_free(memcg); } diff --git a/mm/vmscan.c b/mm/vmscan.c index 0d628299e55c..cba0bc8d4661 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -219,6 +219,10 @@ static int prealloc_memcg_shrinker(struct shrinker *shrinker) goto unlock; } + if (memcg_expand_shrinker_deferred(id)) { + idr_remove(&shrinker_idr, id); + goto unlock; + } shrinker_nr_max = id + 1; } shrinker->id = id; From patchwork Wed Dec 2 18:27:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23F7CC64E7C for ; Wed, 2 Dec 2020 18:28:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9ED2B22248 for ; Wed, 2 Dec 2020 18:28:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9ED2B22248 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 24D846B0070; Wed, 2 Dec 2020 13:28:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D3196B0071; Wed, 2 Dec 2020 13:28:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C0236B0072; Wed, 2 Dec 2020 13:28:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0229.hostedemail.com [216.40.44.229]) by kanga.kvack.org (Postfix) with ESMTP id E2AC86B0070 for ; Wed, 2 Dec 2020 13:27:59 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A4313180AD802 for ; Wed, 2 Dec 2020 18:27:59 +0000 (UTC) X-FDA: 77549176278.14.step30_440b7a0273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 8107818229837 for ; Wed, 2 Dec 2020 18:27:59 +0000 (UTC) X-HE-Tag: step30_440b7a0273b5 X-Filterd-Recvd-Size: 7708 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:27:58 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id v3so1570500plz.13 for ; Wed, 02 Dec 2020 10:27:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WRhj+7Yw6nzO31JxEIwcT1NZoXG/VTPVzLY9DATZGMw=; b=udkLiOrTpnntm/BoSKN6xy1KfnG22UeN3YaATFMhEgkjws17uqY62Eq50VzYN0qqLZ 1otEnXdRn8Aya6N7TZ0BMj8yUS3YSZqyLJR2xJ3l4MqIjf8dBnPFOa/YqAV5YmXXM0UX eykp67hoXd8/9AivpVGnM9rx1n4IbTRz/I7LLboJvfwfn29qbOuP14dHyzdv+axkXDHf O/ni5EkpulwQNRn1IavIIZzquyKh8t50anN7MtewYS2e/Vm3BEJv/87JfcdIRYGoEabb sIX1bvczm3IG2omxpJFuFcrJkSN6OmdmXZWCJfnUYrWs4y3FeIMOthgbnydJK+iRTXCz jYew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WRhj+7Yw6nzO31JxEIwcT1NZoXG/VTPVzLY9DATZGMw=; b=pNRbzP4aZIRBZDHG/APyrrIyg4kQdn+oc3dB+jmV3w4y40Q41TfdHEbFY7r43iApKD 6IEWcsxQ7vLdYkq6n7zGXld/Lm2uzHJgmeGri5ENU42saOpGsaQbYVCEbS9vqOi0x3J+ Lp8biuIgXWu+gXiRAYBDkRD7WwE9fD0bsAQ8V3SF01VMW3qqx98pLANUL4ws9wzAIm1l MiUobkF8XUTJckvLgX40Wz6qwg7H4JmhTUQW9yJl203+OASwb3uFd4pgjWcV8gsmrnVr DzKqoW+Uw5+v7MP747t5W/nnY11EJI2eIbK6UeZboP6VHaGyCNWv+UeH3Qm/+no+GdQ7 6ZvQ== X-Gm-Message-State: AOAM533ZUEB/GKO+/NaX/zkzvIIfRNjLhoX8sGCT8mcTvr3btCquQoEg GUVsshUToGXOWM32uWU8Pa0= X-Google-Smtp-Source: ABdhPJyZuE3BsVEDtG861FaWW1Yoxny4ZeCG0JCGZCytbEtkzuSlK9Bur5LHBr+r//ymjguIfWrd7g== X-Received: by 2002:a17:902:7606:b029:da:246c:5bd8 with SMTP id k6-20020a1709027606b02900da246c5bd8mr3825003pll.27.1606933678126; Wed, 02 Dec 2020 10:27:58 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:56 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/9] mm: vmscan: use per memcg nr_deferred of shrinker Date: Wed, 2 Dec 2020 10:27:22 -0800 Message-Id: <20201202182725.265020-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use per memcg's nr_deferred for memcg aware shrinkers. The shrinker's nr_deferred will be used in the following cases: 1. Non memcg aware shrinkers 2. !CONFIG_MEMCG 3. memcg is disabled by boot parameter Signed-off-by: Yang Shi --- mm/vmscan.c | 88 +++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 82 insertions(+), 6 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index cba0bc8d4661..d569fdcaba79 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -203,6 +203,12 @@ static DECLARE_RWSEM(shrinker_rwsem); static DEFINE_IDR(shrinker_idr); static int shrinker_nr_max; +static inline bool is_deferred_memcg_aware(struct shrinker *shrinker) +{ + return (shrinker->flags & SHRINKER_MEMCG_AWARE) && + !mem_cgroup_disabled(); +} + static int prealloc_memcg_shrinker(struct shrinker *shrinker) { int id, ret = -ENOMEM; @@ -271,7 +277,58 @@ static bool writeback_throttling_sane(struct scan_control *sc) #endif return false; } + +static inline long count_nr_deferred(struct shrinker *shrinker, + struct shrink_control *sc) +{ + bool per_memcg_deferred = is_deferred_memcg_aware(shrinker) && sc->memcg; + struct memcg_shrinker_deferred *deferred; + struct mem_cgroup *memcg = sc->memcg; + int nid = sc->nid; + int id = shrinker->id; + long nr; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + if (per_memcg_deferred) { + deferred = rcu_dereference_protected(memcg->nodeinfo[nid]->shrinker_deferred, + true); + nr = atomic_long_xchg(&deferred->nr_deferred[id], 0); + } else + nr = atomic_long_xchg(&shrinker->nr_deferred[nid], 0); + + return nr; +} + +static inline long set_nr_deferred(long nr, struct shrinker *shrinker, + struct shrink_control *sc) +{ + bool per_memcg_deferred = is_deferred_memcg_aware(shrinker) && sc->memcg; + struct memcg_shrinker_deferred *deferred; + struct mem_cgroup *memcg = sc->memcg; + int nid = sc->nid; + int id = shrinker->id; + long new_nr; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + if (per_memcg_deferred) { + deferred = rcu_dereference_protected(memcg->nodeinfo[nid]->shrinker_deferred, + true); + new_nr = atomic_long_add_return(nr, &deferred->nr_deferred[id]); + } else + new_nr = atomic_long_add_return(nr, &shrinker->nr_deferred[nid]); + + return new_nr; +} #else +static inline bool is_deferred_memcg_aware(struct shrinker *shrinker) +{ + return false; +} + static int prealloc_memcg_shrinker(struct shrinker *shrinker) { return 0; @@ -290,6 +347,29 @@ static bool writeback_throttling_sane(struct scan_control *sc) { return true; } + +static inline long count_nr_deferred(struct shrinker *shrinker, + struct shrink_control *sc) +{ + int nid = sc->nid; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + return atomic_long_xchg(&shrinker->nr_deferred[nid], 0); +} + +static inline long set_nr_deferred(long nr, struct shrinker *shrinker, + struct shrink_control *sc) +{ + int nid = sc->nid; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + return atomic_long_add_return(nr, + &shrinker->nr_deferred[nid]); +} #endif /* @@ -429,13 +509,10 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, long freeable; long nr; long new_nr; - int nid = shrinkctl->nid; long batch_size = shrinker->batch ? shrinker->batch : SHRINK_BATCH; long scanned = 0, next_deferred; - if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) - nid = 0; freeable = shrinker->count_objects(shrinker, shrinkctl); if (freeable == 0 || freeable == SHRINK_EMPTY) @@ -446,7 +523,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, * and zero it so that other concurrent shrinker invocations * don't also do this scanning work. */ - nr = atomic_long_xchg(&shrinker->nr_deferred[nid], 0); + nr = count_nr_deferred(shrinker, shrinkctl); total_scan = nr; if (shrinker->seeks) { @@ -539,8 +616,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, * move the unused scan count back into the shrinker in a * manner that handles concurrent updates. */ - new_nr = atomic_long_add_return(next_deferred, - &shrinker->nr_deferred[nid]); + new_nr = set_nr_deferred(next_deferred, shrinker, shrinkctl); trace_mm_shrink_slab_end(shrinker, shrinkctl->nid, freed, nr, new_nr, total_scan); return freed; From patchwork Wed Dec 2 18:27:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D8DAC71155 for ; Wed, 2 Dec 2020 18:28:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DD74022249 for ; Wed, 2 Dec 2020 18:28:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD74022249 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 46F6C6B0071; Wed, 2 Dec 2020 13:28:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CD426B0072; Wed, 2 Dec 2020 13:28:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26E386B0073; Wed, 2 Dec 2020 13:28:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id 0F7886B0071 for ; Wed, 2 Dec 2020 13:28:02 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BEF4E1EFD for ; Wed, 2 Dec 2020 18:28:01 +0000 (UTC) X-FDA: 77549176362.20.party71_4c0ba14273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id A3CE1180C07AB for ; Wed, 2 Dec 2020 18:28:01 +0000 (UTC) X-HE-Tag: party71_4c0ba14273b5 X-Filterd-Recvd-Size: 4804 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:28:01 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id o7so1438296pjj.2 for ; Wed, 02 Dec 2020 10:28:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nkAj+kfHLpRV/jrsXiaGZp9Y0zC5JrRi0VSe75y5/ok=; b=mFFs1yr0iQTTA94bdtdkPFC5iIeeiXvamHuI6dQajmLDW8uGAAoqBp0Lco3wkWfx9X XgJ97GBMEc2SMDjQPCj+a5M07Gh39FRvnOmmvwjmF1+esdeCTOYDHpJ9B1vdKIIVhiVL cxZ2ENAG7T2UE3wDBTh1PYjNJRdmB1+Y/KukMvEL/zwL7J/XVwCcZaZBJC+zR5/xVc52 ELrSZNt13NL5slziBmuUUaYzyR3dBL0T55R1IcembCzKHT+z/H8uz5Swh1XykgYqXNxo H840U2OiFmjZnLoc2uPad4qAR41nfo846N8h8Fd5P2oP2YpqAVUTqJuvqR2XhfmcJ5VI CvnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nkAj+kfHLpRV/jrsXiaGZp9Y0zC5JrRi0VSe75y5/ok=; b=BZYN2ZSk4LNSvCSsPSk3Wh420YJAIX64jxr9luqGKxee/x+pAHLD4zwyrgXdtWjAvR OdGwmTdx17qcj2PCNMmZy524EZrPtyKDQz/muCRnGidR9KgY8lzwYd8aHY+eV2lotvvo VcAP9NJ1ECgvOfbvYJK+9+0VaPTlynFr+sfw+KAizZD8I+sII3DbNSBkaPyKverM0YdL ler7kXBNJ8jqPv3gFM2xztz0+PkAMxGyLsRmiZCTpJ0M53tfEQxMTp3KJKF1y3bA48hw v+Ph1fVI3or9OuPfxVzg7gysI6yWGd9gwRyhS0fJW8Ur6QN9CD8FrqETILA3kpsm1s3P iwXA== X-Gm-Message-State: AOAM531tWQKsoSPCzmpRgH7dEvKmuA268cpRRCSHIdA+i1b6DGCHwWjx hmaOnT7C/jqORh1NS/Befvg= X-Google-Smtp-Source: ABdhPJysUUGeVaOYuNGBSl2GIw5j+IQLduO6JMDdjjacV2utahIR7URRPrC/8JjUx772SiJV7yEvWw== X-Received: by 2002:a17:90b:1945:: with SMTP id nk5mr1107509pjb.30.1606933680300; Wed, 02 Dec 2020 10:28:00 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:27:59 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/9] mm: vmscan: don't need allocate shrinker->nr_deferred for memcg aware shrinkers Date: Wed, 2 Dec 2020 10:27:23 -0800 Message-Id: <20201202182725.265020-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now nr_deferred is available on per memcg level for memcg aware shrinkers, so don't need allocate shrinker->nr_deferred for such shrinkers anymore. Signed-off-by: Yang Shi --- mm/vmscan.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index d569fdcaba79..5fd57060fafd 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -420,7 +420,15 @@ unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone */ int prealloc_shrinker(struct shrinker *shrinker) { - unsigned int size = sizeof(*shrinker->nr_deferred); + unsigned int size; + + if (is_deferred_memcg_aware(shrinker)) { + if (prealloc_memcg_shrinker(shrinker)) + return -ENOMEM; + return 0; + } + + size = sizeof(*shrinker->nr_deferred); if (shrinker->flags & SHRINKER_NUMA_AWARE) size *= nr_node_ids; @@ -429,26 +437,18 @@ int prealloc_shrinker(struct shrinker *shrinker) if (!shrinker->nr_deferred) return -ENOMEM; - if (shrinker->flags & SHRINKER_MEMCG_AWARE) { - if (prealloc_memcg_shrinker(shrinker)) - goto free_deferred; - } - return 0; - -free_deferred: - kfree(shrinker->nr_deferred); - shrinker->nr_deferred = NULL; - return -ENOMEM; } void free_prealloced_shrinker(struct shrinker *shrinker) { - if (!shrinker->nr_deferred) + if (is_deferred_memcg_aware(shrinker)) { + unregister_memcg_shrinker(shrinker); return; + } - if (shrinker->flags & SHRINKER_MEMCG_AWARE) - unregister_memcg_shrinker(shrinker); + if (!shrinker->nr_deferred) + return; kfree(shrinker->nr_deferred); shrinker->nr_deferred = NULL; From patchwork Wed Dec 2 18:27:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D753C71156 for ; Wed, 2 Dec 2020 18:28:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2279222249 for ; Wed, 2 Dec 2020 18:28:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2279222249 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8DA1D6B0072; Wed, 2 Dec 2020 13:28:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 83B226B0073; Wed, 2 Dec 2020 13:28:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72A4F6B0074; Wed, 2 Dec 2020 13:28:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 57FBD6B0072 for ; Wed, 2 Dec 2020 13:28:04 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 16C7C181AEF1F for ; Wed, 2 Dec 2020 18:28:04 +0000 (UTC) X-FDA: 77549176488.30.range23_560cede273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id EF23B180B3C85 for ; Wed, 2 Dec 2020 18:28:03 +0000 (UTC) X-HE-Tag: range23_560cede273b5 X-Filterd-Recvd-Size: 5828 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:28:03 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id q22so1759841pfk.12 for ; Wed, 02 Dec 2020 10:28:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vKQ+xPb0r9nNI6kfI9ZBDBE9gitSqzBXebPL/8fpVzE=; b=nj/7BqYlZwg8znD81/vfXzs6ArF+M8qXSuJ5a9WuI9e/hfRhm1v4/gYPWalwc/TlM4 6F/rSomsJXUQ4EpDQYFpSAh82V/WqdjDxQ2e8nvwVXN7wBTBOk61H5J76pHR1CCCBP6D nlYjFzW2cVvnSbb1SgCjZYUhc6Buvlkzq0ysPluyZJKMDazAYg5pvKm6HEKabirJdpd0 uznrcMTWGebypvAmrpfEszv8FySwjX9gEMsLuqnOlTc4nluRGe74wVn2CsrAkjtWn0q7 H6fB3X1eUY8eDTWJO1Byk1fTG7f99wZrqKamabm9OrNTJe1U0dkxa51GjjCm53Yu8RsL hh8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vKQ+xPb0r9nNI6kfI9ZBDBE9gitSqzBXebPL/8fpVzE=; b=eBMWSEZq4+Z+01FOlwh9JCKe9xXnSJjEBRMf5ZGfE9GBgcy1bocrfOUHAYjGHCidzD tMI3Uz36eYgQ2FkQ648s6pz6xj2mb1RxNw94JWNLAzKZjyknxHntJtmoIF8JX83MYagQ kXjf8mZK0j2lv44Uyzed6TWp4jrpHcPVt2YYESmvyAxAdujO7SW9Hp5tHnArieQe2j0I VTfOpoCte5kOQ2h5RVM8ssU1zKRPyXvcyS2Pp8KOGA+RasVx2qULtz3b2Cfeb91YAhDM ZLPSb79wWsx1Dehf8HL7HCTQigusjjrXCjyRLuugxeD/Jwxd6dY854dD8Ko3UH4GPjRi 9GNg== X-Gm-Message-State: AOAM531BFLxNmTudFZULm7ZD34yKRUv0ahwEGGvrYsPXR2yGQH4oIO8H mK6FqWlo03Ql8+8o+OzGFyg= X-Google-Smtp-Source: ABdhPJxQKoIFdMyU/1Ay7uLq3AbzqAGodA79ySyV5eVU7XWUWSccqPYCWM43OGq8/UFqVQgC2Ouo/Q== X-Received: by 2002:a05:6a00:134d:b029:18b:2cde:d747 with SMTP id k13-20020a056a00134db029018b2cded747mr4069247pfu.60.1606933682661; Wed, 02 Dec 2020 10:28:02 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.28.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:28:01 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 8/9] mm: memcontrol: reparent nr_deferred when memcg offline Date: Wed, 2 Dec 2020 10:27:24 -0800 Message-Id: <20201202182725.265020-9-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now shrinker's nr_deferred is per memcg for memcg aware shrinkers, add to parent's corresponding nr_deferred when memcg offline. Signed-off-by: Yang Shi --- include/linux/shrinker.h | 4 ++++ mm/memcontrol.c | 24 ++++++++++++++++++++++++ mm/vmscan.c | 2 +- 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 0bb5be88e41d..8f095e424799 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -82,6 +82,10 @@ struct shrinker { }; #define DEFAULT_SEEKS 2 /* A good number if you don't know better. */ +#ifdef CONFIG_MEMCG +extern int shrinker_nr_max; +#endif + /* Flags */ #define SHRINKER_NUMA_AWARE (1 << 0) #define SHRINKER_MEMCG_AWARE (1 << 1) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d3d5c88db179..df128cab900f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -59,6 +59,7 @@ #include #include #include +#include #include "internal.h" #include #include @@ -619,6 +620,28 @@ void memcg_set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) } } +static void memcg_reparent_shrinker_deferred(struct mem_cgroup *memcg) +{ + int i, nid; + long nr; + struct mem_cgroup *parent; + struct memcg_shrinker_deferred *child_deferred, *parent_deferred; + + parent = parent_mem_cgroup(memcg); + if (!parent) + parent = root_mem_cgroup; + + for_each_node(nid) { + child_deferred = memcg->nodeinfo[nid]->shrinker_deferred; + parent_deferred = parent->nodeinfo[nid]->shrinker_deferred; + for (i = 0; i < shrinker_nr_max; i ++) { + nr = atomic_long_read(&child_deferred->nr_deferred[i]); + atomic_long_add(nr, + &parent_deferred->nr_deferred[i]); + } + } +} + /** * mem_cgroup_css_from_page - css of the memcg associated with a page * @page: page of interest @@ -5543,6 +5566,7 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) page_counter_set_low(&memcg->memory, 0); memcg_offline_kmem(memcg); + memcg_reparent_shrinker_deferred(memcg); wb_memcg_offline(memcg); drain_all_stock(memcg); diff --git a/mm/vmscan.c b/mm/vmscan.c index 5fd57060fafd..7dc8075c371b 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -201,7 +201,7 @@ static DECLARE_RWSEM(shrinker_rwsem); #define SHRINKER_REGISTERING ((struct shrinker *)~0UL) static DEFINE_IDR(shrinker_idr); -static int shrinker_nr_max; +int shrinker_nr_max; static inline bool is_deferred_memcg_aware(struct shrinker *shrinker) { From patchwork Wed Dec 2 18:27:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11946709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54DD8C71155 for ; Wed, 2 Dec 2020 18:28:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D18222224B for ; Wed, 2 Dec 2020 18:28:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D18222224B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 09CD96B0073; Wed, 2 Dec 2020 13:28:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 04BF06B0074; Wed, 2 Dec 2020 13:28:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E55BB6B0075; Wed, 2 Dec 2020 13:28:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id CECD46B0073 for ; Wed, 2 Dec 2020 13:28:06 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9690E181AEF1F for ; Wed, 2 Dec 2020 18:28:06 +0000 (UTC) X-FDA: 77549176572.12.news39_5e10dbf273b5 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 7481818013510 for ; Wed, 2 Dec 2020 18:28:06 +0000 (UTC) X-HE-Tag: news39_5e10dbf273b5 X-Filterd-Recvd-Size: 5740 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Wed, 2 Dec 2020 18:28:05 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id o4so1620321pgj.0 for ; Wed, 02 Dec 2020 10:28:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/KThC6++Y6o6leiTwROJ96GvddZx8+czWeSmb5amZVQ=; b=EMi/tuOVxp5UeIBbQHuc0MvOjXaJL4xMfzAPKC7vAdSGUX+AukgNU6lbSCEl1pa40s CzGFvy2f9sxECBStwdH2cBnDzBzlMji6d0TkKGY36wzDoRVCBRuf5Irax63BIyEYZsV1 hd+v6o02bEmjxBhOs0e07+ABx6uC99D3Xg3bhAUFvSZcPWzJzjpxVs7Pg9F6XbO+PP/Y 8KbvW7Sbe06Jvw/Ig0FqkoPmY8jWSszcCz+ff02L+L9d4Q9R6UKL+fhF6h0Uv+TyVPDj q6dqCTWgosHvZbrEXyjzSVAS2jtMKMCtFGukFVkeyUr8DKuEOAbBBwsLwLHWlKpfihhn pEOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/KThC6++Y6o6leiTwROJ96GvddZx8+czWeSmb5amZVQ=; b=Rf3WxVvxXY8WxGD9LwJlewjMm+H2JYN/6oqT/sUwEAH+8iKwqlImKXyYmCFNOKdxYc ajbCw3wAgTg5yFc9KZ+ifDaOqHrZDq4KVFUURM/gJjKwdyq3hd7l14gCglNLYFY1TW4J hi1t3+1UZFHmmrT3rv6BSuA0KrGKi3l0I1c5xVNrv7xibJlwgkKgvt+92ljThOtFfOHF a6fYwZHgwtChLba6BOCUulMaVqxkuWo89uOcoFyk5J+4pFXjPxW4BmJq5ElyYK0pnCDj qTs+yd97tr7FcM4FOijLt5r19sfXyY5Y/8Vgp1vUFJEe7s8VIkc30ul8NQICs/e+ZJLZ /S2w== X-Gm-Message-State: AOAM5323jRnBvDyejk9q5PRSpW9Tb1iAp+R8IYr01SBlN1PCtlUKR+Jv vRUe10ooMM9TQ+1HNAsa5uw= X-Google-Smtp-Source: ABdhPJwastASUnMXqQBRmmSHtYhnA9Bcu2/dYOOI2LdtM0gtv3ElGxk1WiZ9iMjXMoN8OXJgmnux8A== X-Received: by 2002:a63:d45:: with SMTP id 5mr1078522pgn.0.1606933685165; Wed, 02 Dec 2020 10:28:05 -0800 (PST) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id c6sm396906pgl.38.2020.12.02.10.28.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 10:28:04 -0800 (PST) From: Yang Shi To: guro@fb.com, ktkhai@virtuozzo.com, shakeelb@google.com, david@fromorbit.com, hannes@cmpxchg.org, mhocko@suse.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 9/9] mm: vmscan: shrink deferred objects proportional to priority Date: Wed, 2 Dec 2020 10:27:25 -0800 Message-Id: <20201202182725.265020-10-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201202182725.265020-1-shy828301@gmail.com> References: <20201202182725.265020-1-shy828301@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The number of deferred objects might get windup to an absurd number, and it results in clamp of slab objects. It is undesirable for sustaining workingset. So shrink deferred objects proportional to priority and cap nr_deferred to twice of cache items. Signed-off-by: Yang Shi --- mm/vmscan.c | 40 +++++----------------------------------- 1 file changed, 5 insertions(+), 35 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 7dc8075c371b..9d2a6485e982 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -525,7 +525,6 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, */ nr = count_nr_deferred(shrinker, shrinkctl); - total_scan = nr; if (shrinker->seeks) { delta = freeable >> priority; delta *= 4; @@ -539,37 +538,9 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, delta = freeable / 2; } + total_scan = nr >> priority; total_scan += delta; - if (total_scan < 0) { - pr_err("shrink_slab: %pS negative objects to delete nr=%ld\n", - shrinker->scan_objects, total_scan); - total_scan = freeable; - next_deferred = nr; - } else - next_deferred = total_scan; - - /* - * We need to avoid excessive windup on filesystem shrinkers - * due to large numbers of GFP_NOFS allocations causing the - * shrinkers to return -1 all the time. This results in a large - * nr being built up so when a shrink that can do some work - * comes along it empties the entire cache due to nr >>> - * freeable. This is bad for sustaining a working set in - * memory. - * - * Hence only allow the shrinker to scan the entire cache when - * a large delta change is calculated directly. - */ - if (delta < freeable / 4) - total_scan = min(total_scan, freeable / 2); - - /* - * Avoid risking looping forever due to too large nr value: - * never try to free more than twice the estimate number of - * freeable entries. - */ - if (total_scan > freeable * 2) - total_scan = freeable * 2; + total_scan = min(total_scan, (2 * freeable)); trace_mm_shrink_slab_start(shrinker, shrinkctl, nr, freeable, delta, total_scan, priority); @@ -608,10 +579,9 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, cond_resched(); } - if (next_deferred >= scanned) - next_deferred -= scanned; - else - next_deferred = 0; + next_deferred = max_t(long, (nr - scanned), 0) + total_scan; + next_deferred = min(next_deferred, (2 * freeable)); + /* * move the unused scan count back into the shrinker in a * manner that handles concurrent updates.