From patchwork Fri May 27 10:06:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 12863188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B153C433F5 for ; Fri, 27 May 2022 10:07:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CBCE8D0003; Fri, 27 May 2022 06:07:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8581B8D0002; Fri, 27 May 2022 06:07:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71AEA8D0003; Fri, 27 May 2022 06:07:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 631858D0002 for ; Fri, 27 May 2022 06:07:01 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 32B5C35957 for ; Fri, 27 May 2022 10:07:01 +0000 (UTC) X-FDA: 79511094642.26.B9759D0 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf06.hostedemail.com (Postfix) with ESMTP id 774CF18003E for ; Fri, 27 May 2022 10:06:57 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id z11so4160133pjc.3 for ; Fri, 27 May 2022 03:07:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=oq3zK8XqR9nMwWZwXePmBSIgomGYIVajuwM2hdD7Hjo=; b=K/zdyJneFK1xeK2FB1J37H0n37VhRUlG64dLbnJeI/um5dho1xi/cHqIe/2fKLKEVM QIC3W9YGc31R+RuGJ/MJ960dYEsi2Rr8McFBSvovtcKWlVpn+I8IjyNJEkfYjxIs3P/l Z4uKwckiFEsrS7KAP7o6HpwX5Lg2CLkgrUHlouJeWDkLsDS9pNyN4Px4ZICdd/nahFFu LWCkC6k1gbtfU8WZL/3DMCGLeC8gXymf7bVA9dIGDnWLAh6XfnreH+moluFWWYf4CRWO zlm86sDREegQYFNsZiIRMp7uB7+N6/OgeBq1UjYxOpby0pGmNsZF8hEVVuwyJ5S+WHD+ vzTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=oq3zK8XqR9nMwWZwXePmBSIgomGYIVajuwM2hdD7Hjo=; b=lvAg9F7pbP/Tq1AjEsm26+1jPHmhOR1ks9HxEezhXA4YB2Af9xSsmYEspufi5lvKR6 /4ZdLJVwEwrt774xlrOb1H4uirChV/zL6zY5fgCUzPGuAJlP46ArBoPCeOSVN95yRfcJ urqf3rAL+3a8zx/KoHuIh247kmfXa+bp4HV2Bgj9JOoEFEacmbUQ64Fgd3o60gwVBW9s poFX8pClz4VYnf6BDy1G3BapbIVZ6WpqptNKDEPmHUpIHNqhzsKFSPj6ahUAYpGjssD9 mFSnc5q5Za8GgCrJr2MS6OlebIcnqnt6KgYpCcIwXsM73+QGIclTDP7Y8vno7AdKSj1V CLVg== X-Gm-Message-State: AOAM531hJK7s5T7PRbGYXpUtZVnzw5cXkUXwGuLCPBmfqoRSha2XFPVi Sqv6j2Qw7+nWv2+Zq0DETx8= X-Google-Smtp-Source: ABdhPJwNM81uvxUW12Ps2OhrOu0tnOnxN00PMC2hAqI2+tTt8+glgHOcyWMuMVFXvCdlRDOg5DwAmQ== X-Received: by 2002:a17:902:d549:b0:163:7f27:f748 with SMTP id z9-20020a170902d54900b001637f27f748mr6584533plf.116.1653646019672; Fri, 27 May 2022 03:06:59 -0700 (PDT) Received: from localhost.localdomain (47-72-206-164.dsl.dyn.ihug.co.nz. [47.72.206.164]) by smtp.gmail.com with ESMTPSA id z187-20020a6233c4000000b0050dc76281aesm3078104pfz.136.2022.05.27.03.06.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 May 2022 03:06:59 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, catalin.marinas@arm.com, will@kernel.org, linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, zhangshiming@oppo.com, guojian@oppo.com, hanchuanhua@oppo.com, Barry Song , "Huang, Ying" , Minchan Kim , Johannes Weiner , Hugh Dickins , Andrea Arcangeli , Anshuman Khandual , Steven Price , Yang Shi Subject: [PATCH v2] arm64: enable THP_SWAP for arm64 Date: Fri, 27 May 2022 22:06:44 +1200 Message-Id: <20220527100644.293717-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 774CF18003E X-Stat-Signature: zwzcm1yzhuunagf33qaowh9ohae61wih Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="K/zdyJne"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=21cnbao@gmail.com X-Rspam-User: X-HE-Tag: 1653646017-266439 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Barry Song THP_SWAP has been proved to improve the swap throughput significantly on x86_64 according to commit bd4c82c22c367e ("mm, THP, swap: delay splitting THP after swapped out"). As long as arm64 uses 4K page size, it is quite similar with x86_64 by having 2MB PMD THP. So we are going to get similar improvement. For other page sizes such as 16KB and 64KB, PMD might be too large. Negative side effects such as IO latency might be a problem. Thus, we can only safely enable the counterpart of X86_64. A corner case is that MTE has an assumption that only base pages can be swapped. We won't enable THP_SWP for ARM64 hardware with MTE support until MTE is re-arched. Cc: "Huang, Ying" Cc: Minchan Kim Cc: Johannes Weiner Cc: Hugh Dickins Cc: Andrea Arcangeli Cc: Anshuman Khandual Cc: Steven Price Cc: Yang Shi Signed-off-by: Barry Song --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable.h | 2 ++ include/linux/huge_mm.h | 12 ++++++++++++ mm/swap_slots.c | 2 +- 4 files changed, 16 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a4968845e67f..5306009df2dc 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -101,6 +101,7 @@ config ARM64 select ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_NO_INSTR + select ARCH_WANTS_THP_SWAP if ARM64_4K_PAGES select ARCH_HAS_UBSAN_SANITIZE_ALL select ARM_AMBA select ARM_ARCH_TIMER diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 0b6632f18364..06076139c72c 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -45,6 +45,8 @@ __flush_tlb_range(vma, addr, end, PUD_SIZE, false, 1) #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#define arch_thp_swp_supported !system_supports_mte + /* * Outside of a few very special situations (e.g. hibernation), we always * use broadcast TLB invalidation instructions, therefore a spurious page diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index de29821231c9..4ddaf6ad73ef 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -461,4 +461,16 @@ static inline int split_folio_to_list(struct folio *folio, return split_huge_page_to_list(&folio->page, list); } +/* + * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to + * limitations in the implementation like arm64 MTE can override this to + * false + */ +#ifndef arch_thp_swp_supported +static inline bool arch_thp_swp_supported(void) +{ + return true; +} +#endif + #endif /* _LINUX_HUGE_MM_H */ diff --git a/mm/swap_slots.c b/mm/swap_slots.c index 2a65a89b5b4d..10b94d64cc25 100644 --- a/mm/swap_slots.c +++ b/mm/swap_slots.c @@ -307,7 +307,7 @@ swp_entry_t folio_alloc_swap(struct folio *folio) entry.val = 0; if (folio_test_large(folio)) { - if (IS_ENABLED(CONFIG_THP_SWAP)) + if (IS_ENABLED(CONFIG_THP_SWAP) && arch_thp_swp_supported()) get_swap_pages(1, &entry, folio_nr_pages(folio)); goto out; }