From patchwork Fri Aug 26 23:06:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 12956682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9CA7C0502A for ; Fri, 26 Aug 2022 23:06:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F18D940008; Fri, 26 Aug 2022 19:06:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 479FA940007; Fri, 26 Aug 2022 19:06:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F3FB940008; Fri, 26 Aug 2022 19:06:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1925D940007 for ; Fri, 26 Aug 2022 19:06:51 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E719A14082A for ; Fri, 26 Aug 2022 23:06:50 +0000 (UTC) X-FDA: 79843280580.23.929755E Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf16.hostedemail.com (Postfix) with ESMTP id 81C80180020 for ; Fri, 26 Aug 2022 23:06:50 +0000 (UTC) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-33dc390f26cso46956127b3.9 for ; Fri, 26 Aug 2022 16:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc; bh=AErA6r+S0T2+MIHy6k4+EgHRpVoazsJhQoBwdMNs6OQ=; b=rQ2yqilamgWFxD5OxWxgwRd3ZenVlKBZDKCtoZMBtHyXDycAdbTNWiEQHDPPndoANt LwY9Mg1z2RiY3386JkyJR7WT7DpwsI4P+BZcka6KIByvHSOta6ou1A6gWtDUdzLPCOPF SI6m5D9rOnOrAbDWL+zxlYC2x2uWNZYc8i3xwOZxKGmqUCcX6CYOwHjp2hxp3Nae2Kwu GuWdgKVb0raU3DhmOwE4lzqZLulzNQU+XeaQgCIGpTQfXnNluBS5aclifoklvZHElqFX vWBKb4w1xMXEYteERS9hJrB4eVnct4iR/Ggw7iQH7IxzAdMPnsLOgzxzouCOWlboIvTv 1o1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc; bh=AErA6r+S0T2+MIHy6k4+EgHRpVoazsJhQoBwdMNs6OQ=; b=YdIAF4Dp3sk0vOcoXcYsL8Y0vqXIXwLR/kAo9IejjxJ2XZAIeMHE1PsoSRAZyuwz7t v6vUcAhHGeY0EB56fjmg6xTh3v9fgrmRwVTwPk6Rpnu9FxIhvVm+ks/rwz7xm5KKJmU9 RjFDSuo+BNXrKUrfbl7UUesARSRxkH0LSnbmIP5xXaXfCh/Qsqc0xxqCXprtYD+iXVFz raRxobf9B/Pi6aflUYPb2A9oL0VK4j2X9ZHqbfjuUd/HH1U1XZI7rJXlvhvd73YgBmgo dACe9EyvRvtE7ksvcbbHPwqkRvFBgnkkjRQd6nsyaIYlRcE7EVgAgsScUYe01BbBdixp Km1g== X-Gm-Message-State: ACgBeo2CttvVFM7+tuHqdw2yKCgllwGvtSnhNyPWBSQmtahIHm7EYRhD sioiXaoLKLd5lwEo5WL331KsIkaa2Xv1ww== X-Google-Smtp-Source: AA6agR6q5EJP/GkNg/PNBka+3A3ihvPAbli2Ztt0HU+f7NUyjnHeTeAKy9bBjR/3gpFsR9Z44fIBCV6ivqlklQ== X-Received: from shakeelb.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:262e]) (user=shakeelb job=sendgmr) by 2002:a25:2493:0:b0:695:839:3a5a with SMTP id k141-20020a252493000000b0069508393a5amr1776831ybk.468.1661555208691; Fri, 26 Aug 2022 16:06:48 -0700 (PDT) Date: Fri, 26 Aug 2022 23:06:42 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.37.2.672.g94769d06f0-goog Message-ID: <20220826230642.566725-1-shakeelb@google.com> Subject: [PATCH v2] mm: deduplicate cacheline padding code From: Shakeel Butt To: Michal Hocko , Feng Tang Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Shakeel Butt ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661555210; a=rsa-sha256; cv=none; b=U+/ibDgFajjqNSqENluETcdqlQYbkBviAthG9GBVOy10Dy8qLjS1kew1iINBpyb0GgAfr3 cNa3hF8zxpyzrTeQ9/NfxsGHWSlgMuupYYLikN7pMjJNXgNW+t61SGPYe4zk0GJLaGlvnU +us7ja8sB24e0SvzrdY49bEpWbih6nU= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rQ2yqila; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3CFIJYwgKCO8jYRbVVcSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--shakeelb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3CFIJYwgKCO8jYRbVVcSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--shakeelb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661555210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=AErA6r+S0T2+MIHy6k4+EgHRpVoazsJhQoBwdMNs6OQ=; b=3jbogGHsQ35+3vSuvu4dXUMYKjndvLYPdVm+UfIbCCm9uPocVKm7yfThCOG0fjM7gOd5VB MUiYuxQdCHRdsgvyGuuJv5ifysugw/oUK/wcXuP1jOKe3xj7nDQGE1R1LS2MKwpK5i6ZzS qFbQjGdggihrhxY/jt4F3wIIzboc938= X-Rspamd-Queue-Id: 81C80180020 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rQ2yqila; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3CFIJYwgKCO8jYRbVVcSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--shakeelb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3CFIJYwgKCO8jYRbVVcSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--shakeelb.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam01 X-Stat-Signature: 1pr38iictpjxwmsxzdmqwbkf71r8m4ed X-HE-Tag: 1661555210-427929 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are three users (mmzone.h, memcontrol.h, page_counter.h) using similar code for forcing cacheline padding between fields of different structures. Dedup that code. Signed-off-by: Shakeel Butt Suggested-by: Feng Tang Acked-by: Michal Hocko Reviewed-by: Feng Tang --- Changes since v1: - add semicolons in mmzone.h include/linux/cache.h | 13 +++++++++++++ include/linux/memcontrol.h | 13 ++----------- include/linux/mmzone.h | 24 +++++------------------- include/linux/page_counter.h | 13 ++----------- 4 files changed, 22 insertions(+), 41 deletions(-) diff --git a/include/linux/cache.h b/include/linux/cache.h index d742c57eaee5..5da1bbd96154 100644 --- a/include/linux/cache.h +++ b/include/linux/cache.h @@ -85,4 +85,17 @@ #define cache_line_size() L1_CACHE_BYTES #endif +/* + * Helper to add padding within a struct to ensure data fall into separate + * cachelines. + */ +#if defined(CONFIG_SMP) +struct cacheline_padding { + char x[0]; +} ____cacheline_internodealigned_in_smp; +#define CACHELINE_PADDING(name) struct cacheline_padding name +#else +#define CACHELINE_PADDING(name) +#endif + #endif /* __LINUX_CACHE_H */ diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 344022f102c2..60545e4a1c03 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -185,15 +185,6 @@ struct mem_cgroup_thresholds { struct mem_cgroup_threshold_ary *spare; }; -#if defined(CONFIG_SMP) -struct memcg_padding { - char x[0]; -} ____cacheline_internodealigned_in_smp; -#define MEMCG_PADDING(name) struct memcg_padding name -#else -#define MEMCG_PADDING(name) -#endif - /* * Remember four most recent foreign writebacks with dirty pages in this * cgroup. Inode sharing is expected to be uncommon and, even if we miss @@ -304,7 +295,7 @@ struct mem_cgroup { spinlock_t move_lock; unsigned long move_lock_flags; - MEMCG_PADDING(_pad1_); + CACHELINE_PADDING(_pad1_); /* memory.stat */ struct memcg_vmstats vmstats; @@ -326,7 +317,7 @@ struct mem_cgroup { struct list_head objcg_list; #endif - MEMCG_PADDING(_pad2_); + CACHELINE_PADDING(_pad2_); /* * set > 0 if pages under this cgroup are moving to other cgroup. diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 14919b2fb5f5..b30f5f0dd13f 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -121,20 +121,6 @@ static inline bool free_area_empty(struct free_area *area, int migratetype) struct pglist_data; -/* - * Add a wild amount of padding here to ensure data fall into separate - * cachelines. There are very few zone structures in the machine, so space - * consumption is not a concern here. - */ -#if defined(CONFIG_SMP) -struct zone_padding { - char x[0]; -} ____cacheline_internodealigned_in_smp; -#define ZONE_PADDING(name) struct zone_padding name; -#else -#define ZONE_PADDING(name) -#endif - #ifdef CONFIG_NUMA enum numa_stat_item { NUMA_HIT, /* allocated in intended node */ @@ -837,7 +823,7 @@ struct zone { int initialized; /* Write-intensive fields used from the page allocator */ - ZONE_PADDING(_pad1_) + CACHELINE_PADDING(_pad1_); /* free areas of different sizes */ struct free_area free_area[MAX_ORDER]; @@ -849,7 +835,7 @@ struct zone { spinlock_t lock; /* Write-intensive fields used by compaction and vmstats. */ - ZONE_PADDING(_pad2_) + CACHELINE_PADDING(_pad2_); /* * When free pages are below this point, additional steps are taken @@ -886,7 +872,7 @@ struct zone { bool contiguous; - ZONE_PADDING(_pad3_) + CACHELINE_PADDING(_pad3_); /* Zone statistics */ atomic_long_t vm_stat[NR_VM_ZONE_STAT_ITEMS]; atomic_long_t vm_numa_event[NR_VM_NUMA_EVENT_ITEMS]; @@ -1194,7 +1180,7 @@ typedef struct pglist_data { #endif /* CONFIG_NUMA */ /* Write-intensive fields used by page reclaim */ - ZONE_PADDING(_pad1_) + CACHELINE_PADDING(_pad1_); #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT /* @@ -1239,7 +1225,7 @@ typedef struct pglist_data { struct lru_gen_mm_walk mm_walk; #endif - ZONE_PADDING(_pad2_) + CACHELINE_PADDING(_pad2_); /* Per-node vmstats */ struct per_cpu_nodestat __percpu *per_cpu_nodestats; diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h index 78a1c934e416..c141ea9a95ef 100644 --- a/include/linux/page_counter.h +++ b/include/linux/page_counter.h @@ -7,22 +7,13 @@ #include #include -#if defined(CONFIG_SMP) -struct pc_padding { - char x[0]; -} ____cacheline_internodealigned_in_smp; -#define PC_PADDING(name) struct pc_padding name -#else -#define PC_PADDING(name) -#endif - struct page_counter { /* * Make sure 'usage' does not share cacheline with any other field. The * memcg->memory.usage is a hot member of struct mem_cgroup. */ atomic_long_t usage; - PC_PADDING(_pad1_); + CACHELINE_PADDING(_pad1_); /* effective memory.min and memory.min usage tracking */ unsigned long emin; @@ -38,7 +29,7 @@ struct page_counter { unsigned long failcnt; /* Keep all the read most fields in a separete cacheline. */ - PC_PADDING(_pad2_); + CACHELINE_PADDING(_pad2_); unsigned long min; unsigned long low;