From patchwork Mon Jul 17 13:10:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yicong Yang X-Patchwork-Id: 13315654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D738C001DC for ; Mon, 17 Jul 2023 13:12:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 215118D0003; Mon, 17 Jul 2023 09:12:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C4916B0074; Mon, 17 Jul 2023 09:12:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F323C8D0003; Mon, 17 Jul 2023 09:12:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E2CF66B0072 for ; Mon, 17 Jul 2023 09:12:17 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A1A2BA030D for ; Mon, 17 Jul 2023 13:12:17 +0000 (UTC) X-FDA: 81021142314.27.EE264D6 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf14.hostedemail.com (Postfix) with ESMTP id 919A9100026 for ; Mon, 17 Jul 2023 13:12:13 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of yangyicong@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=yangyicong@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689599534; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sUBByuZXbpyKrc9wDQ9lkzXHe8yqE1f/9cc7ANd+/4c=; b=MwrewPdASXa/soy2+D//zgb7u8EYasgfCMB+Ddy5oyeIsbk8dmfGeO8rv1NDPkWaNq64zV DM3YibVMON1gLOb/sOf5mpY9nosIwcNzliwatUdLz9m1dV6NF4qWYGH7oLeaUgfmnf404Z 0IxjtxjkHTSbXcfp30avI/Z/hC1znTc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of yangyicong@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=yangyicong@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689599534; a=rsa-sha256; cv=none; b=UU3QJIOxnl2G1iXjr4oasM8Lkw4eBObGFYmV97erD8CE7bWI4uBiroxPHALm8nPt5LsmR4 m8Hz74b0swuxIDGtrBumKBsXHs7xLfQQNT30iSct8NJqZPK1MktNT3WTPTOJVV5OCbh7Al N7SIqnrtiy5X89ozhlF5HxBeNo8Hs00= Received: from canpemm500009.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4R4Mqr0bWqzLngj; Mon, 17 Jul 2023 21:09:44 +0800 (CST) Received: from localhost.localdomain (10.50.163.32) by canpemm500009.china.huawei.com (7.192.105.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 17 Jul 2023 21:12:09 +0800 From: Yicong Yang To: , , , , , , , , , CC: , , , , , , , , , , , , , , , , , Barry Song <21cnbao@gmail.com>, , , , , Barry Song , Nadav Amit , Mel Gorman Subject: [PATCH v11 2/4] mm/tlbbatch: Rename and extend some functions Date: Mon, 17 Jul 2023 21:10:02 +0800 Message-ID: <20230717131004.12662-3-yangyicong@huawei.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20230717131004.12662-1-yangyicong@huawei.com> References: <20230717131004.12662-1-yangyicong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.163.32] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To canpemm500009.china.huawei.com (7.192.105.203) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 919A9100026 X-Rspam-User: X-Stat-Signature: gynsr45ti5j47scaqr93dhu45nn89gxf X-Rspamd-Server: rspam01 X-HE-Tag: 1689599533-436682 X-HE-Meta: U2FsdGVkX1/PvRcbX/Coz4hltxK1nfPiHPYI+R1FLt5XqT34o6rY0qpG87jlNmr1ZMT2jX7udFPaVluwiBLqzH05RHkTOdhR51UPlEFWautJe5RmpDFGt0wt7u6/NorYGIKxcDxtAKDFOGuR7uotGAPst2Vew911pVC5AhhXEK3y5OYsl12EN2jReioKnkS1j7gc5UYutKXNcf0aHlvRoKtCRcZTFAgv9C47+JEDER19/wVF5mLjyYW/lW1tXeNWb131XoUOtUzn8N1fILQB9Ji7mF5Bp36c3UBWmWsjzjspl7GwJLshC1uTlgBrdfC3a/q2KqQG3LqroUMnyUxprQNIW5NS3eh7vw2gr6gYIB/Tcfu0PJFl4EbI2X7hJb9r0dpvor6ySnkegTWqYTrm/dTNgjda2q9AB63COMX/GIRw/3QDOcoPFFO8D7r43dRUBaddEE100oJcsmHobnXgQqBry/FTFWDj6VBAW3ZmPUrNJmODCA4fbcTY5teEH0MLrSiIUeBL8lOTDNXu++KAKo9DX1cvYQhDCg2BkiddOPtfABuRfAbGjUyP5Gl41c12QzO1PxH6tI/MX+KbU9/2XVktdJKVW5PjYYQ3xBhebxVPVbXVX5q4mN/9iCF93gXxZnP7dyMlzCAfLqw9ntF8wN985lmy+2pfOX2wfTeVkwosKzHF3FmV4glYipurv5/SOi54eQ6XOVssdTH47GcuVvKHcFbRwfVZJt47DqazTsn0lXwGiqwq26kmuxZw1N3czzoDdpg5EfcXsgYszc6aUJ9aZid2it921wIoPQ9QhqlqxlA48IG9quvkFkiD9nEtC/wV1aOXKDpQge7rcYpBBceXcJafp1HmPbSA3T45S/DyD9LeFUfXFBOn9Vet6zcEs+NboFNcxfJtR0Z7gjBFgRjpLV4+0Be4uRqSeFTDKs9eJq19o8fcU+t+sYkZAAyqPw/wjA9L7RrKSdPUfyQ DSke2a2U AF1yRW71GHVz9psu3tWBZHQGWXPlPxv1pskMrZzno7J9EH5L56iNFZwP7pP0hazxQ5LGUBw5Ujm1WhrqJ8vBv6uXhiXTq8X+L4+Wte25z+wOKzixdmzVvGnuacoPEOyUuu8iODe2dd/SgZYRUpRoi0VFVnKZJoqVTV6HKNUx2iZ/dyAaZoey0FS3ROAvJ3pt+flKocivo7D8o/psHreC7v+Pm4mi3o5VtlgYSnukl57PZb38uuiIJnr38F+CxO//p3SH236VQT0bsDCAoPAkpiM/eerhzS4K+APr870sU3exj0ObagdTCe1TEQ/wRg3fjrWJXRRkzOHIrCcTUQKhbbUTi0SA89F7cKjHEURi6CX6EWzx2PZAhVGaHNuelgRFRyGOpWoTKndocf5RwP95RsdBUNSOzgTzMaP4k0xMhjw3HvRM00cYWZ3AqG+IXCBYf6ax8nRBUO5KkcroRjK+r7gRcXHYMpgipp9qW9JvjggPGhE6tHSn2Xps2SA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Barry Song This patch does some preparation works to extend batched TLB flush to arm64. Including: - Extend set_tlb_ubc_flush_pending() and arch_tlbbatch_add_mm() to accept an additional argument for address, architectures like arm64 may need this for tlbi. - Rename arch_tlbbatch_add_mm() to arch_tlbbatch_add_pending() to match its current function since we don't need to handle mm on architectures like arm64 and add_mm is not proper, add_pending will make sense to both as on x86 we're pending the TLB flush operations while on arm64 we're pending the synchronize operations. This intends no functional changes on x86. Cc: Anshuman Khandual Cc: Jonathan Corbet Cc: Nadav Amit Cc: Mel Gorman Tested-by: Yicong Yang Tested-by: Xin Hao Tested-by: Punit Agrawal Signed-off-by: Barry Song Signed-off-by: Yicong Yang Reviewed-by: Kefeng Wang Reviewed-by: Xin Hao Reviewed-by: Anshuman Khandual Reviewed-by: Catalin Marinas --- arch/x86/include/asm/tlbflush.h | 5 +++-- include/linux/mm_types_task.h | 4 ++-- mm/rmap.c | 12 +++++++----- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index cf2a1de5d388..1c7d3a36e16c 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -276,8 +276,9 @@ static inline u64 inc_mm_tlb_gen(struct mm_struct *mm) return atomic64_inc_return(&mm->context.tlb_gen); } -static inline void arch_tlbbatch_add_mm(struct arch_tlbflush_unmap_batch *batch, - struct mm_struct *mm) +static inline void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, + struct mm_struct *mm, + unsigned long uaddr) { inc_mm_tlb_gen(mm); cpumask_or(&batch->cpumask, &batch->cpumask, mm_cpumask(mm)); diff --git a/include/linux/mm_types_task.h b/include/linux/mm_types_task.h index 5414b5c6a103..aa44fff8bb9d 100644 --- a/include/linux/mm_types_task.h +++ b/include/linux/mm_types_task.h @@ -52,8 +52,8 @@ struct tlbflush_unmap_batch { #ifdef CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH /* * The arch code makes the following promise: generic code can modify a - * PTE, then call arch_tlbbatch_add_mm() (which internally provides all - * needed barriers), then call arch_tlbbatch_flush(), and the entries + * PTE, then call arch_tlbbatch_add_pending() (which internally provides + * all needed barriers), then call arch_tlbbatch_flush(), and the entries * will be flushed on all CPUs by the time that arch_tlbbatch_flush() * returns. */ diff --git a/mm/rmap.c b/mm/rmap.c index 6480e526c154..9699c6011b0e 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -642,7 +642,8 @@ void try_to_unmap_flush_dirty(void) #define TLB_FLUSH_BATCH_PENDING_LARGE \ (TLB_FLUSH_BATCH_PENDING_MASK / 2) -static void set_tlb_ubc_flush_pending(struct mm_struct *mm, pte_t pteval) +static void set_tlb_ubc_flush_pending(struct mm_struct *mm, pte_t pteval, + unsigned long uaddr) { struct tlbflush_unmap_batch *tlb_ubc = ¤t->tlb_ubc; int batch; @@ -651,7 +652,7 @@ static void set_tlb_ubc_flush_pending(struct mm_struct *mm, pte_t pteval) if (!pte_accessible(mm, pteval)) return; - arch_tlbbatch_add_mm(&tlb_ubc->arch, mm); + arch_tlbbatch_add_pending(&tlb_ubc->arch, mm, uaddr); tlb_ubc->flush_required = true; /* @@ -726,7 +727,8 @@ void flush_tlb_batched_pending(struct mm_struct *mm) } } #else -static void set_tlb_ubc_flush_pending(struct mm_struct *mm, pte_t pteval) +static void set_tlb_ubc_flush_pending(struct mm_struct *mm, pte_t pteval, + unsigned long uaddr) { } @@ -1579,7 +1581,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, */ pteval = ptep_get_and_clear(mm, address, pvmw.pte); - set_tlb_ubc_flush_pending(mm, pteval); + set_tlb_ubc_flush_pending(mm, pteval, address); } else { pteval = ptep_clear_flush(vma, address, pvmw.pte); } @@ -1962,7 +1964,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, */ pteval = ptep_get_and_clear(mm, address, pvmw.pte); - set_tlb_ubc_flush_pending(mm, pteval); + set_tlb_ubc_flush_pending(mm, pteval, address); } else { pteval = ptep_clear_flush(vma, address, pvmw.pte); }