From patchwork Fri Jul 8 00:30:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 12910704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 069D6C43334 for ; Fri, 8 Jul 2022 08:05:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB39D6B0071; Fri, 8 Jul 2022 04:05:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E641C6B0073; Fri, 8 Jul 2022 04:05:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2BA86B0074; Fri, 8 Jul 2022 04:05:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C36B96B0071 for ; Fri, 8 Jul 2022 04:05:32 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 8DC0E80366 for ; Fri, 8 Jul 2022 08:05:32 +0000 (UTC) X-FDA: 79663198104.24.842847F Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by imf19.hostedemail.com (Postfix) with ESMTP id 29EFF1A004B for ; Fri, 8 Jul 2022 08:05:32 +0000 (UTC) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DB4AA34AA0 for ; Fri, 8 Jul 2022 08:05:31 +0000 (UTC) X-FDA: 79663198062.17.2CBA272 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf01.hostedemail.com (Postfix) with ESMTP id 731564001D for ; Fri, 8 Jul 2022 08:05:31 +0000 (UTC) Received: by mail-pj1-f45.google.com with SMTP id j1-20020a17090aeb0100b001ef777a7befso1223079pjz.0 for ; Fri, 08 Jul 2022 01:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=KpsViEXZYHaSKdo2Y5tfQa2GHPWQpzwRjvyBNJo59WU=; b=QD+aXyufjYV5Rhg7g9mfLcGxaoTj7cmPQTrt3D9uRtWwYIH8z6Valy/m/ZlLU7GeiT 1EnfnMAEEekJP2mj88bW+jDYHsM+CJ5HSgYc03i+rLDa/K26IcLX7dDBB8n8qZWjD6Pk 1/WG8KcKtyXf2eHl6M0nDWp3YEfU+dkn1DAACfrWjXSt6kPCHsuhmG4pk6jtiRZRJFLV Or/krkaK+/Hc2bddE6KxyoiuQ5W5z3Sk/MRH1seFkab8gN0j6IHeiC4CKI2ZNb5JjwIN GiMozYnu5loweh0SxxE/VnRPyHhT+dKJRkCKtm3rSmU962lysfmEkrp88Ejw531Rluop WCKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=KpsViEXZYHaSKdo2Y5tfQa2GHPWQpzwRjvyBNJo59WU=; b=29RnKxkJdIMwjEIjLhim5VGg8U5fdANOseMyCfJFhSVmD9yH3xfWGyrwMtnDhXG1Fi km2u8Os7Nub0lN6oPk6X/Wiu53PaXr9pSNMVPgTD/pxyw3c+0iZ1fQ0ePD18wS8lFGSN Dzs6tX/iLhumR32Be38i6xjhd0VqSSTXjIUktyQkmR3CfJwIHW5wMBLJJCQN0zzL2iHL ycyjkoKEhxgj4W2rSVgijvvNFBgeZMQhm6YD8j/+Wdfa19JvF+iNqm7RckJP8CZvuKlM ymv3jt1K3rrgRT0Rci+CNjYSavFKqDZ9r/K36+oUZ3QTeBdTd3yEq+VEU9AI95sKJ8p9 k/fQ== X-Gm-Message-State: AJIora+stK7cd38Sxti+xPJrk/f9XZemQo4I1U2JWyTqTTVVdmpQaL8E abXZEmKaQIugvQn5bdnYwlw= X-Google-Smtp-Source: AGRyM1uaRRfDeP9jrOYeJppPRbXd05sAchFcz1uJ3YbkZdmE4BJ78z/Mok35DRObt1hqvwYOPFO2Pw== X-Received: by 2002:a17:902:bd43:b0:16b:953d:9e0e with SMTP id b3-20020a170902bd4300b0016b953d9e0emr2269539plx.63.1657267530175; Fri, 08 Jul 2022 01:05:30 -0700 (PDT) Received: from sc2-haas01-esx0118.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id e11-20020a6558cb000000b00408b89e4282sm27207738pgu.47.2022.07.08.01.05.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jul 2022 01:05:29 -0700 (PDT) From: Nadav Amit X-Google-Original-From: Nadav Amit To: linux-kernel@vger.kernel.org, Hugh Dickins , Thomas Gleixner Cc: Ingo Molnar , Borislav Petkov , x86@kernel.org, Linux MM , Nadav Amit , Dave Hansen , Peter Zijlstra , Andy Lutomirski Subject: [PATCH] x86/mm/tlb: ignore f->new_tlb_gen when zero Date: Thu, 7 Jul 2022 17:30:53 -0700 Message-Id: <20220708003053.158480-1-namit@vmware.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657267532; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=KpsViEXZYHaSKdo2Y5tfQa2GHPWQpzwRjvyBNJo59WU=; b=7vS42toamqnlZ6iEwmNUVmR/DsCB8JG/ypcflB5lnbOYXXlzHdGFj5qQBDYVsr9Vyl3xR0 fYxIHuoifvv1gHpCLs2XvrnivPbhzI0DZuXupFG0vEOlXBI0kGuxLhbUP1aS9xhKUubgGF oGfNIxWl5O/iCFbQynYGspEaDpGzxqM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657267532; a=rsa-sha256; cv=none; b=mfVARaj04+HRiHhC0InnROJ30aWobOicVesdUJUYagDqwoIoiLXOMnZZckNqPC+8BDUJtW BP87qzYhW2poeg8hITr9F7h/pWeI3TTtzSkyHfdV79MpBp51UYjea0XIhTpEcainumPYks T9MvGmHv0AfRPgxF8EFw002+Trr6kiM= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=QD+aXyuf; dmarc=pass (policy=none) header.from=gmail.com; spf=none (imf19.hostedemail.com: domain of MAILER-DAEMON@hostedemail.com has no SPF policy when checking 216.40.44.14) smtp.mailfrom=MAILER-DAEMON@hostedemail.com X-HE-Tag-Orig: 1657267531-228346 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=QD+aXyuf; dmarc=pass (policy=none) header.from=gmail.com; spf=none (imf19.hostedemail.com: domain of MAILER-DAEMON@hostedemail.com has no SPF policy when checking 216.40.44.14) smtp.mailfrom=MAILER-DAEMON@hostedemail.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 29EFF1A004B X-Stat-Signature: ixcbrqunuja4itd9aicdkf44e7dzx9cs X-Rspam-User: X-HE-Tag: 1657267532-187774 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nadav Amit Commit aa44284960d5 ("x86/mm/tlb: Avoid reading mm_tlb_gen when possible") introduced an optimization of skipping the flush if the TLB generation that is flushed (as provided in flush_tlb_info) was already flushed. However, arch_tlbbatch_flush() does not provide any generation in flush_tlb_info. As a result, try_to_unmap_one() would not perform any TLB flushes. Fix it by checking whether f->new_tlb_gen is nonzero. Zero value is anyhow is an invalid generation value. In addition, add the missing unlikely() and jump to get tracing right. Fixes: aa44284960d5 ("x86/mm/tlb: Avoid reading mm_tlb_gen when possible") Reported-by: Hugh Dickins Cc: Dave Hansen Cc: Peter Zijlstra (Intel) Cc: Andy Lutomirski Signed-off-by: Nadav Amit Tested-by: Hugh Dickins --- arch/x86/mm/tlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index d9314cc8b81f..d81b4084bb8a 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -771,14 +771,14 @@ static void flush_tlb_func(void *info) return; } - if (f->new_tlb_gen <= local_tlb_gen) { + if (unlikely(f->new_tlb_gen != 0 && f->new_tlb_gen <= local_tlb_gen)) { /* * The TLB is already up to date in respect to f->new_tlb_gen. * While the core might be still behind mm_tlb_gen, checking * mm_tlb_gen unnecessarily would have negative caching effects * so avoid it. */ - return; + goto done; } /*