From patchwork Mon Jul 31 07:48:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13333948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 677CDC04E69 for ; Mon, 31 Jul 2023 07:39:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01B3A280012; Mon, 31 Jul 2023 03:39:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0D7428001C; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8E38280012; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9EA4128001A for ; Mon, 31 Jul 2023 03:39:24 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4D01780107 for ; Mon, 31 Jul 2023 07:39:24 +0000 (UTC) X-FDA: 81071106648.17.FA86951 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf25.hostedemail.com (Postfix) with ESMTP id 0CF72A0019 for ; Mon, 31 Jul 2023 07:39:21 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690789162; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y3tb33D7Yj/stY7JznHb58iHPQ693NkVRx8ipzJMNtk=; b=q4ujG1j7nzX1PEmTyKIXltLRZRWnc9j4qdAWlSO6DMU/YfB5/3Fimn3K+QsZcZXhgusam3 zmWr8QRD6VfiLALGfMJp0p8j6n6tZfvneV3qwhV48Ix3lo91InVxASTrEhG/CWDNG9eUfn pI2v5eK/RfymW6HsyTun6ok64tUXVq4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf25.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690789162; a=rsa-sha256; cv=none; b=MAA0naQdtDzWhSD+HObCpLqWzyQIJYINW3nGNWAK/tTqRxzpdpVcDdUkQStq05pKI9DW4G aZgNmdBMm40FcuB2t9qACSENy5U4b7xK2XHHD55TixRWUVebkh4TgX4/Fqciq8+hw5hUnJ MZ/jOyoUvjrdFeBHtEykLSHEfkuZlmE= Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RDqp763FHzVjdw; Mon, 31 Jul 2023 15:37:35 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 31 Jul 2023 15:39:16 +0800 From: Kefeng Wang To: Andrew Morton , Catalin Marinas , Will Deacon , Mike Kravetz , Muchun Song , Mina Almasry , , , , , CC: , , Kefeng Wang Subject: [PATCH 4/4] arm64: tlb: set huge page size to stride for hugepage Date: Mon, 31 Jul 2023 15:48:29 +0800 Message-ID: <20230731074829.79309-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230731074829.79309-1-wangkefeng.wang@huawei.com> References: <20230731074829.79309-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0CF72A0019 X-Stat-Signature: diphinxmduncpprg5gbfn9ui63jt57qh X-Rspam-User: X-HE-Tag: 1690789161-613569 X-HE-Meta: U2FsdGVkX18o4/fJ0roERZ0w8tFOh0iXlNiKwl0tyugU4Zir/nxhQHraz5Vi62hqhpZMGt9LsB3jliByq5qd44PIWUPL8nKx0QSE4MvSSjGORzOxO94BXkz03htg9L8dXVrHH7rl7D3eG8cUGvTNbFYZ4m0luhsKY6L9f+qTeBDCfoNtja+f58V/rH2J3qb0Q45wCJzlwP75x6yONpAFPu+F8G8FuIJIivjPsKU7yzShwkWBWBlJ157/bagl0BVo3C9fC/6qVbmqkfDif1yexK/WWYx8q+d6HNfIh9IBppda42nt5uYCAfq9mRUGlDS7DhMZQ5ldP1m9GFNe/pWxarM4KJaT0vnakVbv3a8kXVEBhj6VjrMGKaifjH8xxtOG37UYl2aetPtxTvf4PB94hUzeqhEOLOCx2tYuwvWXirsnRRSaAjVK7APL26BuZGPa6E/NHPGfVZ9PMLbVbRX4mIqDbdVZce+Y6zPnV63THimk5WEjzEXvDtsifHvr5iEWTGwHfWX1c39JQZ7oyhMw4omtGGYQ4JT6tiC2g+H/eC7N7/SZLbUpzTopx9iEidgccG0ohUdJiMd8TblrokPRNSxMEL7WZhZ7rGK4kzBULqp8Tpf53sw0liO/b5jGRpiXluM8GwbHsinIKkBxRBTY924J3+flw0ynvctx5AhxYmNM2bSV6u8qHjtW/itSVjvOzwn0b7Okj/GWeqfxI2PO/RgvuxubyAKKja3sMkUx79G0jBoITOPNO6HQA/TADci+FmmTWCmTlSEQ7t8AS9W179ACoExtC5gYBdsgop1SjyxePKzy9b+pCNBW8w/KJKV6ZjrY7+OeldmrQF8NaiIvUL2eGufof7c0M56wKj9aiTpLbDGgMK1w+3CegMo/3DhGJuL+7IYpYZP2tCEFS7B0pxBH/xUBGXLjzFWt8sNESg8y3lIAwc1xDtE9pE4B/JQsCceathS9AwXaHYdgvWZ 5XF7uRKr 605PPJd5R2aTVHOxxfvJPfy8MaRTPKvLxAIQ0SBnJiQz622OMyFY8LKFc7i8pPnLCjBSbCCYEuOQ1Mj+cZNWhim/5MGDCpitNDyPqg+6JCV7lNsANtE/1OC0mespcUKtYPPkht1bbkTXIlgp9hBf80Yl+Sg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is better to use huge_page_size() for hugepage(HugeTLB) instead of PAGE_SIZE for stride, which has been done in flush_pmd/pud_tlb_range(), it could reduce the loop in __flush_tlb_range(). Signed-off-by: Kefeng Wang --- arch/arm64/include/asm/tlbflush.h | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index 412a3b9a3c25..25e35e6f8093 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -360,16 +360,17 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ish); } -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - /* - * We cannot use leaf-only invalidation here, since we may be invalidating - * table entries as part of collapsing hugepages or moving page tables. - * Set the tlb_level to 0 because we can not get enough information here. - */ - __flush_tlb_range(vma, start, end, PAGE_SIZE, false, 0); -} +/* + * We cannot use leaf-only invalidation here, since we may be invalidating + * table entries as part of collapsing hugepages or moving page tables. + * Set the tlb_level to 0 because we can not get enough information here. + */ +#define flush_tlb_range(vma, start, end) \ + __flush_tlb_range(vma, start, end, \ + ((vma)->vm_flags & VM_HUGETLB) \ + ? huge_page_size(hstate_vma(vma)) \ + : PAGE_SIZE, false, 0) + static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) {