From patchwork Mon Dec 23 02:55:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rik van Riel X-Patchwork-Id: 13918387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D69BE77188 for ; Mon, 23 Dec 2024 03:03:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B06BE6B0088; Sun, 22 Dec 2024 22:03:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB6686B0089; Sun, 22 Dec 2024 22:03:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97E2B6B008A; Sun, 22 Dec 2024 22:03:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 786AA6B0088 for ; Sun, 22 Dec 2024 22:03:39 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2B5F014154F for ; Mon, 23 Dec 2024 03:03:39 +0000 (UTC) X-FDA: 82924726794.08.E340E4F Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf06.hostedemail.com (Postfix) with ESMTP id 432C2180009 for ; Mon, 23 Dec 2024 03:03:11 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734922982; a=rsa-sha256; cv=none; b=Dx1VfZ2kylM9J7njuykwJQ9vZtTTAeiUqHuCX+yGpTbhwPkZUE3Zl3ZSX1J6b7dA2P5mpa bwV66AIFTXdOMk+OKd9SbDm5aaDUCFmavL36m0VFqWJQ8FVsyqn3fYawVZA1W4pYF8z3rQ cgTvmk3qi1fePhiKNW3Uw/BmJPSOGMc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734922982; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NgX1sHYsFGUW87qNeidZuY+X0xFEZh+rGS/R4qqusWc=; b=ylD8guTtB3YMuV9aVmqPbvhMSno2xNjzLi+rAJE/Jgruvs0vXlfnDUaNy0npJG/vmTmlT+ O4yxdp3+hy2Rmh4YyW8oi3pEVhP6qOge8YX2BNDL/VCORJVbQ0bU28TWZ2GkG+72byqFTe LIQbiBGh+FLqYUZxKISUrkFZBGoOxTw= Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1tPYdp-000000001Ih-2W79; Sun, 22 Dec 2024 21:57:57 -0500 From: Rik van Riel To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, akpm@linux-foundation.org, linux-mm@kvack.org, Rik van Riel Subject: [PATCH 07/11] x86/mm: use broadcast TLB flushing for page reclaim TLB flushing Date: Sun, 22 Dec 2024 21:55:13 -0500 Message-ID: <20241223025751.3268975-8-riel@surriel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241223025751.3268975-1-riel@surriel.com> References: <20241223025751.3268975-1-riel@surriel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 432C2180009 X-Stat-Signature: wqgam7nmb9pd47rp7m6iqkffqhsa4rmt X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1734922991-745915 X-HE-Meta: U2FsdGVkX18mSKbnKNsbtBe4SFOq94Ffr+S0skUW0kRhjZ89KYnGetNuiGB92ndjl2ZRzuhQPtPuLgFV533E+8r805/X+3yzGtn+sFL5XlQlnxSxA6inDQh+YSbH8uhpxlnBFcXW8eaxj3nkB4WFav870hqLhyFwGuFYX6nYtdeJ8LLioL32zxS8rZuMVBcohnfzjqfEzAbNmhqTdtXuqLNvnqKEjCsoTgTJYPvFAdIL4TNAZ/cIGMTUY8AOp0HxUY/KINSQ1UG5MQ96qCB9j36QLI+d18isLCpr7hvqhmwDE7cDmyi4YHX5yP8BrpC1gFC3vC87DssBSb6Kw4xm9/nEZiu+mClTeO+wV4OG5oeoFFTQSOlmS6721cM9TwhJ+PP8Vax1sg9W8uWzCCC1n4DrBzrrz+Cl0oq0oepqSruFXj1lDVEWjUsfuH5PQqHdmCeYmbVkHdeHWkSNL3CBKm8D71ZGqD8f/fhC1w0JZk/xMSf4rx/zTAkue90n2gnB2OQ7QE8GsqKc7A7v3Z/V/G6fdWCFt3FycR/YWr6YMVex6ieDhicf90gH18qhNTXIZ2eSE0SorAC/r7iWUZxYtVwvbiyyM9U9vGz9OlXp5/IzBwskkbyMmGmr+xexYILHvK7sBRylgSTHONMN1OechbdaGp/s0ShKl+k9GBvCGFEoZgo4UBfbCGnGWdwGWB3MmG6EmddwAmyxzrpxwWVIUfm4BhmoeamibrdSMp9xMadMA7xNcoxLKtBnGA5VQUY8WJKIYOg7x8bAzJqDZaKq8c8EodTv8D/oR8WXUpGHbU2H4L1QHg0wBUbE11jTZecmjQm/ikDHvsrIJewPo3fAJydumrzw8amMaPwaOwcM4XpzEZEEPpXNb+nd1UOSr2IPTFE6GBV06rC+BAFwgVJdrWlAghhaIjIKTANK+0b2hqpQzeKnsAg8ILkikPkoQkPAqLCkrV8RcRRdL+2x9SD YZ/5wtsp kBoT/xQ5U50t1zaN5u5winv7A+JTp5rS0tsfEZZEWhgCHQsMmrD5YzNunSlPdk9mWQusM0qIeeXDkbW4naH3MrGXCU+URS2mmic8fISXVwC+d3XtNFodpN8u9VbImLjnxbyfsz1n8hyRafh3NQRS6X5DIYDZGe62nb4l5BRsScFl7xmhP6Z0jMTc3n6N+Hk4j/zx18wLLKIXWcK8pnqnk4IHnNdrYG94Ji9N8xi/T3ZxqmCyIEJzEiZxmyxK7SSmDH1B2ceT3ftJRAX21R0PX66x6eeR1tSnuh3kWY4rqmt742prfpEhnVjRoLOLHh8CJRm9A X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the page reclaim code, we only track the CPU(s) where the TLB needs to be flushed, rather than all the individual mappings that may be getting invalidated. Use broadcast TLB flushing when that is available. Signed-off-by: Rik van Riel --- arch/x86/mm/tlb.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 266d5174fc7b..64f1679c37e1 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1310,8 +1310,16 @@ EXPORT_SYMBOL_GPL(__flush_tlb_all); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) { struct flush_tlb_info *info; + int cpu; + + if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) { + guard(preempt)(); + invlpgb_flush_all_nonglobals(); + tlbsync(); + return; + } - int cpu = get_cpu(); + cpu = get_cpu(); info = get_flush_tlb_info(NULL, 0, TLB_FLUSH_ALL, 0, false, TLB_GENERATION_INVALID);