From patchwork Fri Sep 17 03:48:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12500967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DDE9C433F5 for ; Fri, 17 Sep 2021 03:53:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E95C7610C7 for ; Fri, 17 Sep 2021 03:53:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E95C7610C7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 946576B0073; Thu, 16 Sep 2021 23:53:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F627900002; Thu, 16 Sep 2021 23:53:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C0196B0075; Thu, 16 Sep 2021 23:53:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 711636B0073 for ; Thu, 16 Sep 2021 23:53:28 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 35CE839B3B for ; Fri, 17 Sep 2021 03:53:28 +0000 (UTC) X-FDA: 78595695696.38.4115A52 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf21.hostedemail.com (Postfix) with ESMTP id D9361D0306F7 for ; Fri, 17 Sep 2021 03:53:27 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id n18so5277875plp.7 for ; Thu, 16 Sep 2021 20:53:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OTxY68Ufdm8UNqRMLWBsmc/KZ1S77UCqKKx7OF0ikR0=; b=LQ9nA+W3wSxOR7xZ474CGNgBcGrd1BIpSguaUrKsFyLhTaoslBzbnnnlCDs4Mt5udu hoecHN/AqwbODexKb4BBBgI5z6TeSrowqKADJ+IMHGNn4MW+Dmy3SQKo7P5BHUQRiiEB gdMuhZS8DTqgKUcKS4vGn0Qn3kwVVtYggZLbegSpqZy6BoE2O59+yskSJn3nYtKz/CvX /Nq0SKyUq9BpYWX0FDsU5DMoLPmfZYqBPpp4hEaQxikU57l++QpAavXkmLs+Q5f2w2bh 1g2pOb7J5TE9cmZ2OhGLgOS8ym3LnnUXwBdkEYIgWpxM8D3BWSl7YyDnLUUQ4jcvDFII hnQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OTxY68Ufdm8UNqRMLWBsmc/KZ1S77UCqKKx7OF0ikR0=; b=RHiBaCAwsSNwj5NvGnvNndIDCSBrcx9iPC7hlgCJoH2ZC8E+Bx5ygC3UjxvQZ0FuUW wc5zIEbPm05iWD3f17dwnU9H/DjeHIkA7T7r/LztlsX/jiwgHZuzpYtCabRJKSA3jFN9 gS0bqooRzA7eOQ0SuCgouOKjc6HjY9XjSm21yAFwQ2u3tlNdCqnMYCy6oEfiybmRjje2 3W2WhMSZMbLxWHR223QKNAf8qWB7+9Y6xeiht2zlN/Lj2jTM4XF3CYL/bJWDyqoHhpgl 2oNNbMqUn2OAEKLRnBHEOlb+5wu3T3v5pj6usQdnyvJRnWdULbBLLW6wrdjp9DoBFh6C ai2w== X-Gm-Message-State: AOAM533WX/4mLHBl8WGr8l2bQjeCmLc3U9iqp1+IYlkdp3RRhlFjnLu5 zI8dBvddUfT82iyM3jwRRdSx7g== X-Google-Smtp-Source: ABdhPJzyLYofu0O3pVXPfpDvq5aDxDGFtaZHYS+cZ3rJOmnnqAcKP4GWumAMa+/z8lSNcXSk65Vjbw== X-Received: by 2002:a17:90a:bf82:: with SMTP id d2mr10003439pjs.201.1631850806876; Thu, 16 Sep 2021 20:53:26 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.226]) by smtp.gmail.com with ESMTPSA id g12sm8997704pja.28.2021.09.16.20.53.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Sep 2021 20:53:26 -0700 (PDT) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, mhocko@suse.com, song.bao.hua@hisilicon.com, david@redhat.com, chenhuang5@huawei.com, bodeddub@amazon.com, corbet@lwn.net, willy@infradead.org Cc: duanxiongchun@bytedance.com, fam.zheng@bytedance.com, smuchun@gmail.com, zhengqi.arch@bytedance.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH RESEND v2 2/4] mm: hugetlb: replace hugetlb_free_vmemmap_enabled with a static_key Date: Fri, 17 Sep 2021 11:48:13 +0800 Message-Id: <20210917034815.80264-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210917034815.80264-1-songmuchun@bytedance.com> References: <20210917034815.80264-1-songmuchun@bytedance.com> MIME-Version: 1.0 Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=LQ9nA+W3; spf=pass (imf21.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D9361D0306F7 X-Stat-Signature: t3ypmq945izxiaq1nnd7h9zn7htyxt8m X-HE-Tag: 1631850807-173615 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page_head_if_fake() is used throughout memory management and the conditional check requires checking a global variable, although the overhead of this check may be small, it increases when the memory cache comes under pressure. Also, the global variable will not be modified after system boot, so it is very appropriate to use static key machanism. Signed-off-by: Muchun Song --- include/linux/hugetlb.h | 6 +++++- include/linux/page-flags.h | 6 ++++-- mm/hugetlb_vmemmap.c | 10 +++++----- 3 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index f7ca1a3870ea..ee3ddf3d12cf 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1057,7 +1057,11 @@ static inline void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr #endif /* CONFIG_HUGETLB_PAGE */ #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP -extern bool hugetlb_free_vmemmap_enabled; +DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + hugetlb_free_vmemmap_enabled_key); +#define hugetlb_free_vmemmap_enabled \ + static_key_enabled(&hugetlb_free_vmemmap_enabled_key) + #else #define hugetlb_free_vmemmap_enabled false #endif diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 7b1a918ebd43..d68d2cf30d76 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -185,7 +185,8 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP -extern bool hugetlb_free_vmemmap_enabled; +DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + hugetlb_free_vmemmap_enabled_key); /* * If the feature of freeing some vmemmap pages associated with each HugeTLB @@ -204,7 +205,8 @@ extern bool hugetlb_free_vmemmap_enabled; */ static __always_inline const struct page *page_head_if_fake(const struct page *page) { - if (!hugetlb_free_vmemmap_enabled) + if (!static_branch_maybe(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + &hugetlb_free_vmemmap_enabled_key)) return page; /* diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 527bcaa44a48..5b80129c684c 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -188,9 +188,9 @@ #define RESERVE_VMEMMAP_NR 1U #define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) -bool hugetlb_free_vmemmap_enabled __read_mostly = - IS_ENABLED(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON); -EXPORT_SYMBOL(hugetlb_free_vmemmap_enabled); +DEFINE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, + hugetlb_free_vmemmap_enabled_key); +EXPORT_SYMBOL(hugetlb_free_vmemmap_enabled_key); static int __init early_hugetlb_free_vmemmap_param(char *buf) { @@ -204,9 +204,9 @@ static int __init early_hugetlb_free_vmemmap_param(char *buf) return -EINVAL; if (!strcmp(buf, "on")) - hugetlb_free_vmemmap_enabled = true; + static_branch_enable(&hugetlb_free_vmemmap_enabled_key); else if (!strcmp(buf, "off")) - hugetlb_free_vmemmap_enabled = false; + static_branch_disable(&hugetlb_free_vmemmap_enabled_key); else return -EINVAL;