From patchwork Wed Oct 28 01:41:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wonhyuk Yang X-Patchwork-Id: 11862301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 235C861C for ; Wed, 28 Oct 2020 01:42:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C3A6F22258 for ; Wed, 28 Oct 2020 01:42:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J1HeKq9c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C3A6F22258 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 786726B005C; Tue, 27 Oct 2020 21:42:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 710046B005D; Tue, 27 Oct 2020 21:42:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FF2C6B0062; Tue, 27 Oct 2020 21:42:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id 2C4706B005C for ; Tue, 27 Oct 2020 21:42:33 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B4EA28249980 for ; Wed, 28 Oct 2020 01:42:32 +0000 (UTC) X-FDA: 77419634544.13.ink08_5f0cc7c27280 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 94CAD18140B60 for ; Wed, 28 Oct 2020 01:42:32 +0000 (UTC) X-Spam-Summary: 1,0,0,2a5912a5a2f01c2e,d41d8cd98f00b204,vvghjk1234@gmail.com,,RULES_HIT:41:69:355:379:541:560:800:960:973:988:989:1260:1345:1437:1534:1542:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3165:3354:3865:3866:3867:3868:3870:3871:3874:4321:5007:6261:6653:7514:7576:7903:9413:10004:11026:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12683:12895:13141:13161:13229:13230:14181:14394:14687:14721:21080:21324:21444:21451:21611:21627:21666:30034:30054,0,RBL:209.85.214.194:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100;04yrwncfecxj1c8qwgkceggh7bzacyciyjghs9p7w4ccc4a71wsjeot5qbh839e.9ueafxeqzjbo5zkdn5ja5pfe3gum9zay9oot395i6n1qejytn91fs9fejqd9iem.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:69,LUA_SUMMARY:none X-HE-Tag: ink08_5f0cc7c27280 X-Filterd-Recvd-Size: 4886 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Wed, 28 Oct 2020 01:42:32 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id b19so1712412pld.0 for ; Tue, 27 Oct 2020 18:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=AZcSyo5g/nEswTTuCvZSdfCYIDYbgJRtMj8/IIpSBag=; b=J1HeKq9cAGBCEK/0sJqmzZcoYWBA/AtRucxK9ZkByM1ib6ECdFs7q+oAIGG/XZQjxX ITyLa20MVBg+ln7mEwWXRq0MV+9UJHkki7IuVz2ujNx1tOTMCxOZG+3qPR0vq2mff8Yf TZu2+EjVUIEu5UBUkRwT0Ekp0FmOFxIiwmeP74s2iDqraHwsJy2CkoFmYRcFdFUgo5qJ V6lXfRzG28FLR/HbPv6Aj1SgIUDimlfuanzQGamQxjR/tx4M9/a8xnYCHsx7zJuk9UhH v/s2+ZRW6B78kD9KX+g6DHryl068R9lrScZMEkuk1bf2wPPv2YGcrmumnxiKy8tNYFwg D+Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=AZcSyo5g/nEswTTuCvZSdfCYIDYbgJRtMj8/IIpSBag=; b=h1722PtEkE5UF3lP+MQ9rZ9B81C9kvW+KPeLKfov57J53LTbzrmDzkLudUUE7PBm3r i9WES0dqDI2VjnI2UkMaW68yYTxTTlBiUGjCkBXWSzsKPNjZyLL8Muz+ize6ulS4Xun6 6RB1uwoIPevdoHUNUUVfcJ1wQNjo+lNVN1fOkP9tyP99FpJr3LNzyknBnMQzoSDUzONz C8XPEhRUDMfvUp0ExyY0wyoQD9pbMRc4SPym9mO1lfK1rmr0xyGQieWSBCjrFJi0xf33 YrptjMbMrg2mRfpGs28s5CQal3rApdIAbsKjYFBq6j6eXuA0dJAQgUlvOcrFtrck0VLg fn0g== X-Gm-Message-State: AOAM5335QMs4TTgUwz56pTV41WToE8EZOaso9GiYI5C+LgGkVorYCbGN t33MjwXIklvkY2aM6QMoN5S725uMTLUjXg== X-Google-Smtp-Source: ABdhPJxoDHR5WmpgVXQyJMJmEo0Gru9Af44fxAdrFxwbeOhwTE1kmPYaPX1G6q2aV/lO7h7YrJzWHg== X-Received: by 2002:a17:90a:65cc:: with SMTP id i12mr4500977pjs.193.1603849350852; Tue, 27 Oct 2020 18:42:30 -0700 (PDT) Received: from DESKTOP-P2JGRFE.localdomain ([1.234.114.36]) by smtp.gmail.com with ESMTPSA id x7sm3609569pfr.61.2020.10.27.18.42.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Oct 2020 18:42:30 -0700 (PDT) From: Wonhuyk Yang To: linux-mm@kvack.org Cc: Wonhyuk Yang Subject: [RESEND PATCH] percpu: Reduce the number of cpu distance comparisions Date: Wed, 28 Oct 2020 10:41:22 +0900 Message-Id: <20201028014122.26187-1-vvghjk1234@gmail.com> X-Mailer: git-send-email 2.17.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wonhyuk Yang To build group_map[] and group_cnt[], we find out which group CPUs belong to by comparing the distance of the cpu. However, this includes cases where comparisons are not required. This patch uses a bitmap to record CPUs that is not classified in the group. CPUs that we know which group they belong to should be cleared from the bitmap. In result, we can reduce the number of unnecessary comparisons. Signed-off-by: Wonhyuk Yang --- mm/percpu.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index 66a93f096394..d19ca484eee4 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -2669,6 +2669,7 @@ static struct pcpu_alloc_info * __init pcpu_build_alloc_info( { static int group_map[NR_CPUS] __initdata; static int group_cnt[NR_CPUS] __initdata; + static struct cpumask mask __initdata; const size_t static_size = __per_cpu_end - __per_cpu_start; int nr_groups = 1, nr_units = 0; size_t size_sum, min_unit_size, alloc_size; @@ -2702,24 +2703,27 @@ static struct pcpu_alloc_info * __init pcpu_build_alloc_info( upa--; max_upa = upa; + cpumask_copy(&mask, cpu_possible_mask); + /* group cpus according to their proximity */ - for_each_possible_cpu(cpu) { - group = 0; - next_group: - for_each_possible_cpu(tcpu) { - if (cpu == tcpu) - break; - if (group_map[tcpu] == group && cpu_distance_fn && - (cpu_distance_fn(cpu, tcpu) > LOCAL_DISTANCE || - cpu_distance_fn(tcpu, cpu) > LOCAL_DISTANCE)) { - group++; - nr_groups = max(nr_groups, group + 1); - goto next_group; - } - } + for (group = 0; !cpumask_empty(&mask); group++) { + /* pop the group's first cpu */ + cpu = cpumask_first(&mask); group_map[cpu] = group; group_cnt[group]++; + cpumask_clear_cpu(cpu, &mask); + + for_each_cpu(tcpu, &mask) { + if (!cpu_distance_fn || + (cpu_distance_fn(cpu, tcpu) == LOCAL_DISTANCE && + cpu_distance_fn(tcpu, cpu) == LOCAL_DISTANCE)) { + group_map[tcpu] = group; + group_cnt[group]++; + cpumask_clear_cpu(tcpu, &mask); + } + } } + nr_groups = group; /* * Wasted space is caused by a ratio imbalance of upa to group_cnt.