From patchwork Thu Mar 17 06:50:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bibo mao X-Patchwork-Id: 12783593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDEA4C433F5 for ; Thu, 17 Mar 2022 06:50:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CCA88D0001; Thu, 17 Mar 2022 02:50:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0547D6B0072; Thu, 17 Mar 2022 02:50:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E85378D0001; Thu, 17 Mar 2022 02:50:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id D56D56B0071 for ; Thu, 17 Mar 2022 02:50:30 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 99867217C0 for ; Thu, 17 Mar 2022 06:50:30 +0000 (UTC) X-FDA: 79252954620.14.7DDF8E6 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by imf13.hostedemail.com (Postfix) with ESMTP id 2BAA62001D for ; Thu, 17 Mar 2022 06:50:27 +0000 (UTC) Received: from localhost.localdomain (unknown [10.2.5.185]) by mail.loongson.cn (Coremail) with SMTP id AQAAf9Axisww2jJi_bYKAA--.8164S2; Thu, 17 Mar 2022 14:50:24 +0800 (CST) From: Bibo Mao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, David Hildenbrand , Yang Shi Subject: [PATCH v3] mm/khugepaged: sched to numa node when collapse huge page Date: Thu, 17 Mar 2022 02:50:24 -0400 Message-Id: <20220317065024.2635069-1-maobibo@loongson.cn> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-CM-TRANSID: AQAAf9Axisww2jJi_bYKAA--.8164S2 X-Coremail-Antispam: 1UD129KBjvJXoW7AF1DAF4DGF18Cw13uw1xXwb_yoW8WF4fpF WDJ3yDGrWDXry8Kws2qw1DAryrtr95trWvqw13Aan7tr98Jw10ga4UZayUAFy7JFZ7GFWf JrWYvrnY9F10q3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnUUvcSsGvfC2KfnxnUUI43ZEXa7xR_UUUUUUUUU== X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Rspamd-Queue-Id: 2BAA62001D X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none X-Stat-Signature: gboh1cjwgfx5n9xxdjngxd9dfuh1hk7p X-Rspamd-Server: rspam07 X-HE-Tag: 1647499827-224583 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: collapse huge page will copy huge page from general small pages, dest node is calculated from most one of source pages, however THP daemon is not scheduled on dest node. The performance may be poor since huge page copying across nodes, also cache is not used for target node. With this patch, khugepaged daemon switches to the same numa node with huge page. It saves copying time and makes use of local cache better. With this patch, specint 2006 base performance is improved with 6% on Loongson 3C5000L platform with 32 cores and 8 numa nodes. Signed-off-by: Bibo Mao --- changelog: V2: remove node record for thp daemon V3: remove unlikely statement --- mm/khugepaged.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 131492fd1148..b3cf0885f5a2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1066,6 +1066,7 @@ static void collapse_huge_page(struct mm_struct *mm, struct vm_area_struct *vma; struct mmu_notifier_range range; gfp_t gfp; + const struct cpumask *cpumask; VM_BUG_ON(address & ~HPAGE_PMD_MASK); @@ -1079,6 +1080,13 @@ static void collapse_huge_page(struct mm_struct *mm, * that. We will recheck the vma after taking it again in write mode. */ mmap_read_unlock(mm); + + /* sched to specified node before huage page memory copy */ + if (task_node(current) != node) { + cpumask = cpumask_of_node(node); + if (!cpumask_empty(cpumask)) + set_cpus_allowed_ptr(current, cpumask); + } new_page = khugepaged_alloc_page(hpage, gfp, node); if (!new_page) { result = SCAN_ALLOC_HUGE_PAGE_FAIL;