From patchwork Wed Mar 27 04:49:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92456C47DD9 for ; Wed, 27 Mar 2024 04:50:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 115A16B008A; Wed, 27 Mar 2024 00:50:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 029C56B0092; Wed, 27 Mar 2024 00:50:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E167B6B0093; Wed, 27 Mar 2024 00:50:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D2D3A6B008A for ; Wed, 27 Mar 2024 00:50:42 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9B44FA0A5C for ; Wed, 27 Mar 2024 04:50:42 +0000 (UTC) X-FDA: 81941593524.04.16419FB Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf09.hostedemail.com (Postfix) with ESMTP id D61E6140018 for ; Wed, 27 Mar 2024 04:50:40 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=dVAjFjT8; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf09.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=shQKeKcQeNJUU/JQ5KWmpXdfmdrhkhe16Q2MMRo9oOg=; b=cN57ypTokklHcwuGZM3nsqCTnCpzAMzsVZlurkwlN0S/TyBmJvzkGfHJdF8qSZXU/awJQF qWR92r8m0hNqdC1mOy0vADwUWsGPlQ2eaHFeC3uQp9cs2+/502EZ+jLaJ1f+G89EKQzIR3 15FszjbcUZ8JpMZgLSJG2HQzLmOQXHs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=dVAjFjT8; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf09.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515040; a=rsa-sha256; cv=none; b=IuMfupNCE8AtSr094ozYfeK/k78BHw/m06iqlMwJfH6K8AqQw5/X2hA/gt+bwNGm0iG4Lx Hn1W/hZQrtw/FsH+ZhJUMlsCbRpLESPtF7HgEMuzGUYiezCwaK31Xi+VyEF+NCGJUtGirE 13OxmB865vt9M0WWakJZXc6g6cX/HMU= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6e6b729669bso5246791b3a.3 for ; Tue, 26 Mar 2024 21:50:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515039; x=1712119839; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=shQKeKcQeNJUU/JQ5KWmpXdfmdrhkhe16Q2MMRo9oOg=; b=dVAjFjT8Wn/+r6HZtPotKViosMYqoXaz/smTPCWltbiKwqpDkM3V+TdqOHZw+xfWYR ernS4AXzgViJ1rAXRR7s/OUMqIkdOgGh86ZvHVP8gO+91UTptERXD5CMQkUtgldSpEP2 MWAwbt09IBTL9rTva8PGI2F3miQ6ZtdDKxlsNlVi0+cyhBlHAnWXYR5SfcxoZK/+nnvi Zf/z1jnQMF+jrlUkMg+Ix+bBjHHCuQeHEDBvm6VeC+w3VaRq1n8xwQFqCJanCQgX5dJf E+A6enzFzbEOla2g1WI61G5Joij4rS/xZjlmfPK8+0m3t04/K5JkPBM4Yq0AiG5t5LxD CoRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515039; x=1712119839; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=shQKeKcQeNJUU/JQ5KWmpXdfmdrhkhe16Q2MMRo9oOg=; b=gOTdaWl3e5cLzs1fPGLMUuUHZ2S1RcQW2o2GcQpzRa41PH8JfRTyZJsAovCHBVHYQv 5qzBL9As02RDvXIC6/DW1XF/7LnBS79BFZCn7DHZmHkavKD0Kk3wLbB/x9CjpMaterDA JEyFK7uK6PLVDmKWXpxvRz2bhuXtA8DbowdffdZuFTtn4vACT534txvOSSnrU9hvFqNP uSKPWxfTjLwa5q9Kqmy2Ipnrd7ILq4vAMptxPtC67qMP9x0IJTTf7LVM0TxQqlj84dQ6 0+rvsbiNIDnHJ3gmuTX4LuUVhQ8P2hTzGVJnzJXTsJgbCT3u0IbKV9FJF/4WFek3WcZX +tqw== X-Forwarded-Encrypted: i=1; AJvYcCUAZR5O4h3SewPXSn819XV26tg+pUMNtidNFA1rnaEFcyw11Lx9fzMmGTd4AiR79fG74r6HJRlAosOQ9I2X/bUDslY= X-Gm-Message-State: AOJu0YwM2vq2IolbtM3lNVT9+iGrmrGls+Ifs/+XB/YN35ASQVbL0l/A Vq/oYyJ++UiqyCm22O5Z1VL1vG2SsL5FRfGXK6Ut0szlmm4dJoIIVOEtF/WMe48= X-Google-Smtp-Source: AGHT+IHpHyQ5wkbbDg1/9QOIdZT7U8XEUGKwcMibXVBUL9WTi2GIa5kveh+VvtS/wwBFRvp4M1UHpQ== X-Received: by 2002:a05:6a21:164e:b0:1a3:4979:f25e with SMTP id no14-20020a056a21164e00b001a34979f25emr128528pzb.59.1711515039607; Tue, 26 Mar 2024 21:50:39 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:38 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 01/13] riscv: Flush the instruction cache during SMP bringup Date: Tue, 26 Mar 2024 21:49:42 -0700 Message-ID: <20240327045035.368512-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D61E6140018 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 6hz4hys71gikzcjdda5ktfh5x1dkq6yk X-HE-Tag: 1711515040-768329 X-HE-Meta: U2FsdGVkX18VOmvGZCAptAVLTBxsPRaNTjGZ8SF0QCmFVUCtGriZmEmhx9+bXQ3z7F7HG/s9Y7rUtdGSAKHS91sBNZK1KNtsBY9pAjCV23YLmY+b1Ikbh8BB5OXdC9gSfhmvXSYnYUqcVhRIDuUa1hPrEc43zxbrpUwwX9C9vK2stkWL8aUh/f8/dtKiVYNTcL9OG1k77E/sqyemMfZU18iy088PCtpPuWyCwMLAlhZ+EdIjVNwcTVDN0A6SB+ZXIHGzRP16FxTkTSg2tg6DKSvFSCckx/Zt6O1JRxwwmbrCg6ijhAHg8iRvgISEO8qcLIc5437Dylgoz5q3DBQZj/GhSECSHauaYQU6D8j2Lq116lUSPwupn9QQs2m7sbpbzIejQMINNO5MiyGkBFiPfsnGPdTRhcTvGmYKdpLjmu8fTEwZGn89wtDexVnh5MjqUkERHF7sa07xxGJZXNXwW7RF4VqtSt+Y5alNs7NdBr6ukjdfWj/xHmbm7zcoimpYtCMmkv466QljDN8Y1fPjKe3MGn62HQ5aMt55ENs5shw4yv8y9CHtdJg+ceQ9SZZ2k/zIECnr2fveKnsfJIrDyXNAPiSvF6WvhJVhr34qWuoK+m/H574MULAHKfcPwq9OioTAKFGrkd7FbbGcV4Q4I61OIfod9qadv8MDT63GQ3sWEumexCRWOIvBU1AofJoURzTSA5OZy9ZUpQBgmumkPyWjf/NbcqT1ytwbhDdQQn6WH+wa/GaNOaRWJZdF8kvVpWBo9vXmqjqKGeqEPDKCno8P0XKyl3MhugWI8+ZmK+Ef/3z/pRjvwwRsgWvjVLD52QAc+ZD65glcGtSUwsfpYF9wRm10SCM+LP7r3l4NcL0nRX+eaKQKnZnmHMGZuHmQpZbd4P6H/8FVZhajfkxjJ1XeHW/OFAh8ljoG2ZW1xhByMPreO1OBdWdaH2cW4Z47peaPsPoVPS3a9UqXe/w 6ddvWF1P saIUdTK7W37my8RvksxVjN8hj1Tb48nmSKHruluA1PDTTqucO3c8uOLFFSHqBSwbZOxC1V5qZuwxLMT82WlJpEUaUFGuw0WlRPlm/c8TgDPYwLts= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000012, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instruction cache flush IPIs are sent only to CPUs in cpu_online_mask, so they will not target a CPU until it calls set_cpu_online() earlier in smp_callin(). As a result, if instruction memory is modified between the CPU coming out of reset and that point, then its instruction cache may contain stale data. Therefore, the instruction cache must be flushed after the set_cpu_online() synchronization point. Fixes: 08f051eda33b ("RISC-V: Flush I$ when making a dirty page executable") Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/kernel/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c index d41090fc3203..4b3c50da48ba 100644 --- a/arch/riscv/kernel/smpboot.c +++ b/arch/riscv/kernel/smpboot.c @@ -26,7 +26,7 @@ #include #include -#include +#include #include #include #include @@ -234,9 +234,10 @@ asmlinkage __visible void smp_callin(void) riscv_user_isa_enable(); /* - * Remote TLB flushes are ignored while the CPU is offline, so emit - * a local TLB flush right now just in case. + * Remote cache and TLB flushes are ignored while the CPU is offline, + * so flush them both right now just in case. */ + local_flush_icache_all(); local_flush_tlb_all(); complete(&cpu_running); /* From patchwork Wed Mar 27 04:49:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605546 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D25DC47DD9 for ; Wed, 27 Mar 2024 04:50:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 200386B0093; Wed, 27 Mar 2024 00:50:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 187A86B0095; Wed, 27 Mar 2024 00:50:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0517C6B0096; Wed, 27 Mar 2024 00:50:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E7A496B0093 for ; Wed, 27 Mar 2024 00:50:45 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BB8481A0546 for ; Wed, 27 Mar 2024 04:50:45 +0000 (UTC) X-FDA: 81941593650.30.D39E78E Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by imf07.hostedemail.com (Postfix) with ESMTP id D259140010 for ; Wed, 27 Mar 2024 04:50:43 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=LJeD13l6; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515043; a=rsa-sha256; cv=none; b=bk/hWrC6aKXmuFx1WJADFx0wKKuUQh1aX+U2vPpaT7/+0MjZs1heaaRypS+b8Jm7Sii8ZA 18D6s1vkpQSho3T2wNpIRo4hJwajU3KhKTefqgq/nYCoCCuWdHMjFq1EjzF/UZp/nwbGP1 sa9Vd+DF+GmrSZNPCjuQdEd116FoQ/c= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=LJeD13l6; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515043; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Tmp/TOSalTk1E9Om53lqcGFTwB3eFzfKle7hmp+7bao=; b=3lwhRQHSZ439j8fRaBzYX7chKQkNubv/WzLuImOOqSb84SiQw5F9+8W1Xkxiu9bicUyyZP 62SgKVOY3/ctBmrwR7JcKa57x6wKjQDJWSet/lbC7hILToFINGVCsmMBkXkIqS96LM7wo6 DzzIrbsoIC5qv73VT/UlXeSTwa024Is= Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-6e67d42422aso3794906a34.0 for ; Tue, 26 Mar 2024 21:50:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515043; x=1712119843; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tmp/TOSalTk1E9Om53lqcGFTwB3eFzfKle7hmp+7bao=; b=LJeD13l6RhF2OWdBw874ogDgs1M6kSSndaSou18TmG8ShX8pz7PiPsrjQjandjV/Dq swp0yjwvdpz5yT/t0m9A3ipch9QFBO36bmH3bMQ6vrbK/03bI9ndtGk3HcUt+CUGv2F2 7FFKqJCQRBDHHM/T6hGv9/mOPoVkPgbhHCOkHeH7baMa0jQdm20o3cT5qixLB3KncO6R PG0Bd4H2+08iRagRdEibGAjJKUPpi3OQCHZgRT6N1kLDhytEbGEbN9wGxtGTlWJo2azF 6+3OTjq2j+exaotoU3IlkQ1vnmih6x7czf/DANzDU7O5zUMCifOaKTwv5MLVR5XFANRO a/8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515043; x=1712119843; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tmp/TOSalTk1E9Om53lqcGFTwB3eFzfKle7hmp+7bao=; b=M2gnYum+Zw0FttnZ1nVv0aLd3OfkjLwSMqoxPir2ShYANUGDB93zeHljbcpd6CxxhN qwiUKsGzEW/nZZAs1TRqneyldJehy5bPxM3Z0O4AlJSfRlGyQkFusZs6wiZ0Gz4PGbgR d8QVbL1WrqT5YuL2U1VvNxv8BkfCw0JrGQ866qHcFaOENW93VBNGTKUwORs9A3+ocXfs cQW8gF8esTh5nhLe/QRXCaZx95lO+g7XTqJNHCpem1F49oDUCpHKcYETnlFgvLP0F7aS iVo07tehoi9AYxRMIgBnCtbArQSNsoxsio3EV00Q6jMRSU3I1aLYp/Hxl6uQAvkP4jo4 In5g== X-Forwarded-Encrypted: i=1; AJvYcCU1fwXuCyLEQCnDaP5HCPmE/DZKeFcy3VFhDPwM6K0y87kh3aH7BcpAyLFNRLMZuUMXvWuB9OwnfuDoghCQ25AhqN8= X-Gm-Message-State: AOJu0YwiVol+F0RLMbU6fhJLKo/ntPVdjmUdI8wyVRy8H4dGiZMQa9ba /sIEQVdzM9FaRHh7CwTemKImZp0MoziEb3FpsYnEeo3Ynast8RltpMPw6fbSfNM= X-Google-Smtp-Source: AGHT+IFl3z8IwkcLePPXROsXhoQYUv/uXEkbIlJCNYH90sGzXXkCB+mcXoO2kI0LDgwvd8XvzM/1fA== X-Received: by 2002:a05:6830:1bd5:b0:6e6:99da:8944 with SMTP id v21-20020a0568301bd500b006e699da8944mr3176474ota.26.1711515043004; Tue, 26 Mar 2024 21:50:43 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:42 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 02/13] riscv: Factor out page table TLB synchronization Date: Tue, 26 Mar 2024 21:49:43 -0700 Message-ID: <20240327045035.368512-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D259140010 X-Stat-Signature: uchf37qxjjqmph7rk3iiauj8xkxrtq37 X-Rspam-User: X-HE-Tag: 1711515043-385632 X-HE-Meta: U2FsdGVkX1+WfjEvmkvu81HjoK+cmZAIossAwMttkc0g73K1KABNDosusz5VIy1+rAWFLwxWc25v4R6Y6Tn+smjTNkSXjJ0dTL7N/OlQYsbKYDAhIexmGmRB44fTY1D56Hq01OzWCyriYBYPGLbtfa540xouU8fCaNX8O4c7fTp1b9KD27tkwHLmSA73PUKnNyvSsflzXDUgqrfKFgLS4z1d3DV3KTsk2Efsgl1PPMfgDVfgZy5rRMs75KrTjetNFZxWjvanEWa8sQS8QDrdpbhXuWEeNyLgigo5nmgGh/GXzj1E7HyBBLzWqQgT80KBRG6fl/BdEWGGSRupRXvBemg9I4IqQNKHdFQG/QUZd+p4EdWi4foKD0cUnzfUkHzJKHc+kgYWNjn6I2ejiPb3zNN2/GSEW+tQ9EL5UYh5nmMh01LdkX38jPahu41TdVD245ffRDs38ZOj+LqkwL+j+4EWNR/pHg8vUqxMWobxLLcYUj2u9U5r4FhxJ1Y1R48g/0QjrWZtGbMCkNqfxCuv+/D9kGFCtGHdsjZcGetg8ePPlsHb6PIAI7EtVcVUOYMB65vCvZ2IsRiboj6qyN8B4W4izJEjHvWnfVCQzPz8GfjedYmOvQpMJbENirubPrhgWtgjmE73EogklmTpZSc7EhmQwPT/qgIPWahI67FYys8B46YCdAOgVbhQgaOh9ZrHUddzNnGzRlJMhVNabW4Z/9ammDrlT6cEirM4AgduLp5yzxu8+AARabB1XekePTXSNVeBAcGObmAZdZbtXEx/8AkbUTXJ/emqo4J1gFsZr06iOgOd5Jtxu459HSIV2m++MjDW+f/tI6FOANvub+d5J4f4Ni9IyrqpTCEKs/rqZtoNUcgNnvHbTxGMY3vWOdG8FJV64YTqM+Tv+aV+8YtPw822TThNdBtGYFEhYlQS5ommthlJeIFK16AH0lXYKnaTQRP4fuq+xhnA62yOe34 RoBghqGM 1dF7tHSIs9m0CXcvsYCsuymvbHKvpDh3SmfFHaKBJdECOOlTzbea8WwStx17QZg5qDTCt5Fk6yxuDsu8y8nsBveaUFAknoVTH97xA7kP/6adgKB6o59j5hSgHxz3GWnT4cxoP/3ge7uECdEt67FXHlAk37KBHt31+oESJ6mSDVOwa3f2iXqJn5+Ikt7E9PA26cDqUZv2O3/+DQBx9Mpp7m3/AIsmPaNM1kOxbu90HvIcC5rMJecVqDPLcHK9ulzFOzJTH4AZCqL3A1QLf8IhKQZJ0x4PZ4jjaR6N41wtiJKexsr7oemx097npFvfiKMy0WWbMqqAmDkVQdlCbt5lo56KHWngBvrFJCaiX4I0yUSFOG3BBfKTOKTKMXnpvnGa2utSTOLGTvFSIAqQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The logic is the same for all page table levels. See commit 69be3fb111e7 ("riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU"). Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v6: - Move riscv_tlb_remove_ptdesc() definition to fix 32-bit build Changes in v5: - New patch for v5 arch/riscv/include/asm/pgalloc.h | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index deaf971253a2..b34587da8882 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -15,6 +15,14 @@ #define __HAVE_ARCH_PUD_FREE #include +static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + if (riscv_use_ipi_for_rfence()) + tlb_remove_page_ptdesc(tlb, pt); + else + tlb_remove_ptdesc(tlb, pt); +} + static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte) { @@ -102,10 +110,7 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, struct ptdesc *ptdesc = virt_to_ptdesc(pud); pagetable_pud_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } } @@ -139,12 +144,8 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, unsigned long addr) { - if (pgtable_l5_enabled) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); - else - tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); - } + if (pgtable_l5_enabled) + riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -176,10 +177,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, struct ptdesc *ptdesc = virt_to_ptdesc(pmd); pagetable_pmd_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* __PAGETABLE_PMD_FOLDED */ @@ -190,10 +188,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, struct ptdesc *ptdesc = page_ptdesc(pte); pagetable_pte_dtor(ptdesc); - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, ptdesc); - else - tlb_remove_ptdesc(tlb, ptdesc); + riscv_tlb_remove_ptdesc(tlb, ptdesc); } #endif /* CONFIG_MMU */ From patchwork Wed Mar 27 04:49:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0C3FC54E67 for ; Wed, 27 Mar 2024 04:50:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D8EA6B0096; Wed, 27 Mar 2024 00:50:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 341766B0098; Wed, 27 Mar 2024 00:50:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13EF16B0099; Wed, 27 Mar 2024 00:50:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E98826B0096 for ; Wed, 27 Mar 2024 00:50:47 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 86FD81203EA for ; Wed, 27 Mar 2024 04:50:47 +0000 (UTC) X-FDA: 81941593734.06.5D70BC6 Received: from mail-yw1-f178.google.com (mail-yw1-f178.google.com [209.85.128.178]) by imf13.hostedemail.com (Postfix) with ESMTP id BCD4D20016 for ; Wed, 27 Mar 2024 04:50:45 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=b0OiK0IU; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.128.178 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515045; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G6C+MSH7tzUf74TudLETMF5DBzv2rpp8bzfwEKTodUs=; b=M/Pc+XE+IPjb+OX+/axr1YCpj1L0IQEWQJxVOLNXqBqmwmMcNStrcKfCWUJMbRg2/L/qmw EBHv7uprcnTjQ6u9YpPD6UcTAt+QDuxJ1zaYzkwLZEFgO41ZI1rUG+N0aqY/a6RX6aD3oV ZHR/Ssw8zewYVAl53LRJ/+ZhCvTi5PE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515045; a=rsa-sha256; cv=none; b=M6MH/6l84DAJHnILfdjSSqr0Xfa51WQl0/JoLUaxnI8i0hHvUiFgs9n2vRdcKIKuOma7gs 6zDtNVaxLr3dUXky6Ohp62GCOiTBML7unVwBupg5PPCH+JlUYl814yg58NoifHivdrYGVN h/hhhdGTXjRAt2iKAnVdGS/jMKm0zf4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=b0OiK0IU; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.128.178 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-60a0599f631so55694577b3.2 for ; Tue, 26 Mar 2024 21:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515045; x=1712119845; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G6C+MSH7tzUf74TudLETMF5DBzv2rpp8bzfwEKTodUs=; b=b0OiK0IUkDnk1hOdJYJMFLf8hIpzIMxO2lR+qoBiB0JyWRnhFIKSN9bw4cN7iotOy9 emRhm9M9h1EimXMmDTpWGtelor732AbfYHZlm23AGx95+Jp/nPnWMOQt9DlzWg2QZIbT vB7BUmHBqCEE8fVoAMeACd09V1k5a36lv5Kb+5gk4eJoIcXH45ftZP0ZXeYTB+o83wqq MSe5jXiVA4RsxY4ZbYCXesCCiWwphNmz0ArRMLmc2A7LqDsnWwPTzc+KZy9scQ942ifg Nzx45GHcTolLR49TG6G+UWp3fPBTSVhfzygBiK7DFnNh6wAnc32NJ61GQU5s8kb75W64 KTKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515045; x=1712119845; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G6C+MSH7tzUf74TudLETMF5DBzv2rpp8bzfwEKTodUs=; b=qBFKNbr+EL40w7dXalgIK9ydOjDGpW/zV4I8ythoYBeXzKViqhJsi3JPZ/3uNsQJMp zNtnTdbFdGpOwQVBkIeVEkVEnvHC6d7ihy0w/V1fmdsAg+W9ON9m307dOyjm+auB5dMW Y49H8p0Sy284ZDnNAwk1evIrw1KS3NtjbKOfSSkmKiAnSF3V4ebj7xFu2Z+OoVLo4Xeu NEnaJiW+EhT2CSCljQeYtG/OOqzxaYg5IlmxD1j4jaeFgAbwYHvJAYDG4PFd88g/hZ9N QSmVLEMbBwcN7dDHIr7NNBEF5Oszb5CzddJePU9FlESQrDoZkIXEp7Uv/MP+XJq71ELx Oo/w== X-Forwarded-Encrypted: i=1; AJvYcCWPZo9n75uExnP9M73MXJQgH2eodmXlxahd4BqJZuSMOCRZ8gkeeoDo/GN4BtjKc8wsuzbs7eLAzryzxXlko9ntiDs= X-Gm-Message-State: AOJu0YyiDJYJO31ABLu99MZD2tfvTN181OSLsxXXgE2ZL1X+3ywFfJXM +vRAPiuyRD9SXqEbyIf31Uhh9jMH2lpZK0GBR09RfesI/L9WGwnMNRzWCEwQcB0= X-Google-Smtp-Source: AGHT+IH+jSoiPAnTh406w99fZLlqeG+exqQi31XclYwr8IOVXBgedOOFyirc6G8eGQn9SYpg/+yAIQ== X-Received: by 2002:a81:8084:0:b0:611:9be5:4e17 with SMTP id q126-20020a818084000000b006119be54e17mr18863ywf.51.1711515044780; Tue, 26 Mar 2024 21:50:44 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:43 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland , Anup Patel Subject: [PATCH v6 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Date: Tue, 26 Mar 2024 21:49:44 -0700 Message-ID: <20240327045035.368512-4-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Stat-Signature: oynn5prf3fghfuyyjbqdxmf3qyg1w451 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: BCD4D20016 X-Rspam-User: X-HE-Tag: 1711515045-5767 X-HE-Meta: U2FsdGVkX18kWpHsCNqD6zsAOGACFp2qmhGxRLU5O3w38ZCFVqxlczGTg9RBd1Tdvf5tg0YRfJSiw+I6iEbaMTkCUf9dA6HfNyXqcDf0O1TC9myJRyUCPjf8XugwndMM/qSZltfQy4pKYMlO/f3H++mwj2NeTbRVAcG7uhIQd4aPrV+Xtt1XW1y7D4SCacMDH43z5nOJpC/eSz7a8o60JYNtYEaAJQcSfCgD8iZIroa7XMy55tCybo3RTLAc8sWCU7DcMTgFCyTWsK1vGpTFQEqpBrK2KqBIBgcQSkzb84TzALPykfzwv7l1qeArvvj4XBZ9Bg816XDqCywEZsd6g7n+wU2EMfGAZfRQyDN1XwTnBisnUq8Sc1+wwZ/d+VNT05KsJVJH1J4ZaV1kXcM57+TqmVWfuwBXGLewbKRKsXPuWec9N8IanCoIA454GyCZg4ey5e8hzjiDQ4dz2ccCija6DBWLDOZTPbjlzUD4B334yjoOUHJQUFkZNIfXEX51atgetXqJzepdW9/lxZaY5TfAAfg/x0RWkQKoDCURkwsesCIm+ZCO8eXkj+3tEONedZF+0gO+Ld/skxsMlA3uAvoMEkuSHriJpaDmKuEjcpUvym3by0fKvNtTr3B2FWEEdx2Q0WF12nOTpby922KRbbosSoFTs4TjabHywB1Wjm2SdhmNr+QR+F4PuHglf3l6vbTSX5eqleCAk3DQAc+GqjTi/em7oLI7W9zKW4iQr3MuD5msBE/K+0enB5Os+JWi1IanwtZaQeGeuW1yaLGzAxLbNEdMbz19yArn5XQk6udvGzCDVQvi67G/vwDqW3Z+yez/rqtWwZDPL68DUsbBIg5v6lSUR+ln+tu3fa8kYWhAgiGJQwUlEiQWfp49YolDfFXN/fjoFbCgL2j5YQOW8yJlu3BMNzmlLodWMBsGxXa566N6saCmC4KiSBhlZF+KnY3oZbA9b2YteMDxLXN A6mEGtpL yRJ2xt59r+ZtIrc9WUnsqo+gfTGD6OfnA9hJYLybkyMm7c0Zwu7amqwzfrpicY1ZCAnzWxG77FZZxEYJ6Go2yCmeJRoTdHF8Vh9wjHbSqRcEv8KwkRqwrF66cwhJ1vGwh83WTHgkzorPvFTM8aIZGIa9lPFYdR0nExNOoYkEZWYatlFDFIQ8MyH0tQKpp492/Gt3b/OneFRKlclvYfpD4zYC1kPWrmcVIrUzjpOYkdOjb6qSZT92FK5Z3WMDKI9ppo28NsKqRkHR1spLLT3NFH1CbHLykJW6xYiR9JQEwdO20L607Wi7gmGtha65lZUHV0Dgd9rpkNKvhx07ENOgo3BSh2KBE0MXdBQBFEu8Z3WRYXwOTc01KzwArGZDtt4bnGgQMQnSQc8gxwQ3LWB2L6Ui+/H5//CpGvAqOnig3Pa8rRyRGhta0m/xtEsTGLX7KigdO+OV1J83yFE8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: An IPI backend is always required in an SMP configuration, but an SBI implementation is not. For example, SBI will be unavailable when the kernel runs in M mode. For this reason, consider IPI delivery of cache and TLB flushes to be the base case, and any other implementation (such as the SBI remote fence extension) to be an optimization. Generally, if IPIs can be delivered without firmware assistance, they are assumed to be faster than SBI calls due to the SBI context switch overhead. However, when SBI is used as the IPI backend, then the context switch cost must be paid anyway, and performing the cache/TLB flush directly in the SBI implementation is more efficient than injecting an interrupt to S-mode. This is the only existing scenario where riscv_ipi_set_virq_range() is called with use_for_rfence set to false. sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only calls riscv_ipi_set_virq_range() when no other IPI device is available. This allows moving the static key and dropping the use_for_rfence parameter. This decouples the static key from the irqchip driver probe order. Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is enabled. Optherwise, IPIs must be used. Add a fallback definition of riscv_use_sbi_for_rfence() which handles this case and removes the need to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. Reviewed-by: Anup Patel Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v6: - Clarify the commit message for patch 3 based on ML discussion Changes in v5: - Also switch to riscv_use_sbi_for_rfence() in asm/pgalloc.h Changes in v4: - New patch for v4 arch/riscv/include/asm/pgalloc.h | 7 ++++--- arch/riscv/include/asm/sbi.h | 4 ++++ arch/riscv/include/asm/smp.h | 15 ++------------- arch/riscv/kernel/sbi-ipi.c | 11 ++++++++++- arch/riscv/kernel/smp.c | 11 +---------- arch/riscv/mm/cacheflush.c | 5 ++--- arch/riscv/mm/tlbflush.c | 31 ++++++++++++++----------------- drivers/clocksource/timer-clint.c | 2 +- 8 files changed, 38 insertions(+), 48 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index b34587da8882..f52264304f77 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -8,6 +8,7 @@ #define _ASM_RISCV_PGALLOC_H #include +#include #include #ifdef CONFIG_MMU @@ -17,10 +18,10 @@ static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) { - if (riscv_use_ipi_for_rfence()) - tlb_remove_page_ptdesc(tlb, pt); - else + if (riscv_use_sbi_for_rfence()) tlb_remove_ptdesc(tlb, pt); + else + tlb_remove_page_ptdesc(tlb, pt); } static inline void pmd_populate_kernel(struct mm_struct *mm, diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 6e68f8dff76b..ea84392ca9d7 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -375,8 +375,12 @@ unsigned long riscv_cached_marchid(unsigned int cpu_id); unsigned long riscv_cached_mimpid(unsigned int cpu_id); #if IS_ENABLED(CONFIG_SMP) && IS_ENABLED(CONFIG_RISCV_SBI) +DECLARE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +#define riscv_use_sbi_for_rfence() \ + static_branch_unlikely(&riscv_sbi_for_rfence) void sbi_ipi_init(void); #else +static inline bool riscv_use_sbi_for_rfence(void) { return false; } static inline void sbi_ipi_init(void) { } #endif diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index 0d555847cde6..7ac80e9f2288 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -49,12 +49,7 @@ void riscv_ipi_disable(void); bool riscv_ipi_have_virq_range(void); /* Set the IPI interrupt numbers for arch (called by irqchip drivers) */ -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence); - -/* Check if we can use IPIs for remote FENCEs */ -DECLARE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -#define riscv_use_ipi_for_rfence() \ - static_branch_unlikely(&riscv_ipi_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr); /* Check other CPUs stop or not */ bool smp_crash_stop_failed(void); @@ -104,16 +99,10 @@ static inline bool riscv_ipi_have_virq_range(void) return false; } -static inline void riscv_ipi_set_virq_range(int virq, int nr, - bool use_for_rfence) +static inline void riscv_ipi_set_virq_range(int virq, int nr) { } -static inline bool riscv_use_ipi_for_rfence(void) -{ - return false; -} - #endif /* CONFIG_SMP */ #if defined(CONFIG_HOTPLUG_CPU) && (CONFIG_SMP) diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c index a4559695ce62..1026e22955cc 100644 --- a/arch/riscv/kernel/sbi-ipi.c +++ b/arch/riscv/kernel/sbi-ipi.c @@ -13,6 +13,9 @@ #include #include +DEFINE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +EXPORT_SYMBOL_GPL(riscv_sbi_for_rfence); + static int sbi_ipi_virq; static void sbi_ipi_handle(struct irq_desc *desc) @@ -72,6 +75,12 @@ void __init sbi_ipi_init(void) "irqchip/sbi-ipi:starting", sbi_ipi_starting_cpu, NULL); - riscv_ipi_set_virq_range(virq, BITS_PER_BYTE, false); + riscv_ipi_set_virq_range(virq, BITS_PER_BYTE); pr_info("providing IPIs using SBI IPI extension\n"); + + /* + * Use the SBI remote fence extension to avoid + * the extra context switch needed to handle IPIs. + */ + static_branch_enable(&riscv_sbi_for_rfence); } diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 45dd4035416e..8e6eb64459af 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -171,10 +171,7 @@ bool riscv_ipi_have_virq_range(void) return (ipi_virq_base) ? true : false; } -DEFINE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -EXPORT_SYMBOL_GPL(riscv_ipi_for_rfence); - -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr) { int i, err; @@ -197,12 +194,6 @@ void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) /* Enabled IPIs for boot CPU immediately */ riscv_ipi_enable(); - - /* Update RFENCE static key */ - if (use_for_rfence) - static_branch_enable(&riscv_ipi_for_rfence); - else - static_branch_disable(&riscv_ipi_for_rfence); } static const char * const ipi_names[] = { diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index bc61ee5975e4..d76fc73e594b 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,7 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (IS_ENABLED(CONFIG_RISCV_SBI) && !riscv_use_ipi_for_rfence()) + if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); @@ -69,8 +69,7 @@ void flush_icache_mm(struct mm_struct *mm, bool local) * with flush_icache_deferred(). */ smp_mb(); - } else if (IS_ENABLED(CONFIG_RISCV_SBI) && - !riscv_use_ipi_for_rfence()) { + } else if (riscv_use_sbi_for_rfence()) { sbi_remote_fence_i(&others); } else { on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 893566e004b7..0435605b07d0 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -79,10 +79,10 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_ipi_for_rfence()) - on_each_cpu(__ipi_flush_tlb_all, NULL, 1); - else + if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); + else + on_each_cpu(__ipi_flush_tlb_all, NULL, 1); } struct flush_tlb_range_data { @@ -103,7 +103,6 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - struct flush_tlb_range_data ftd; bool broadcast; if (cpumask_empty(cmask)) @@ -119,20 +118,18 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, broadcast = true; } - if (broadcast) { - if (riscv_use_ipi_for_rfence()) { - ftd.asid = asid; - ftd.start = start; - ftd.size = size; - ftd.stride = stride; - on_each_cpu_mask(cmask, - __ipi_flush_tlb_range_asid, - &ftd, 1); - } else - sbi_remote_sfence_vma_asid(cmask, - start, size, asid); - } else { + if (!broadcast) { local_flush_tlb_range_asid(start, size, stride, asid); + } else if (riscv_use_sbi_for_rfence()) { + sbi_remote_sfence_vma_asid(cmask, start, size, asid); + } else { + struct flush_tlb_range_data ftd; + + ftd.asid = asid; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } if (cmask != cpu_online_mask) diff --git a/drivers/clocksource/timer-clint.c b/drivers/clocksource/timer-clint.c index 09fd292eb83d..0bdd9d7ec545 100644 --- a/drivers/clocksource/timer-clint.c +++ b/drivers/clocksource/timer-clint.c @@ -251,7 +251,7 @@ static int __init clint_timer_init_dt(struct device_node *np) } irq_set_chained_handler(clint_ipi_irq, clint_ipi_interrupt); - riscv_ipi_set_virq_range(rc, BITS_PER_BYTE, true); + riscv_ipi_set_virq_range(rc, BITS_PER_BYTE); clint_clear_ipi(); #endif From patchwork Wed Mar 27 04:49:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34A1CCD11DD for ; Wed, 27 Mar 2024 04:50:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD5316B0098; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C94A76B0099; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB1396B009B; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 93B636B0098 for ; Wed, 27 Mar 2024 00:50:49 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5F2DE1606B4 for ; Wed, 27 Mar 2024 04:50:49 +0000 (UTC) X-FDA: 81941593818.07.6E5F18E Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf29.hostedemail.com (Postfix) with ESMTP id 862C612000B for ; Wed, 27 Mar 2024 04:50:47 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="J1l3/XvX"; spf=pass (imf29.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515047; a=rsa-sha256; cv=none; b=62uoAoUE8ACbTuIUyCCHDtdxgRRFhdyDhdMp/Dap8d92mbnSNLP7Al+uH50mn5cg15nIca Vr2LTkvsl9KkF+1u0tXjQFFg2SzJzg0Ls45aN2tz2AkfsKrCyJ383ROFc8WR1t6WSGlaZp 9yoQc8qNBtIrwgvT7qNKlxGHAWIbyrA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="J1l3/XvX"; spf=pass (imf29.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2fwmOl0CxtBfQxlaR9NQyd7IGBnsfsK+MLDyKM1aSvU=; b=KpnH72bMpJklXOaVwWWiYeurXCtZeH+uFSU0oZOr54nFB3Ecs/PoB4KJ0usZ1GFxNkgEJo ZkvyeArkjaQ+cbGlr+3KYaj5h8J3s/g305Ov2BejUMOw/gT1l8PTKfEUkLPnvb5qTzYYCG hDvtjetpAiIYuPXAHSE8SMpUqXFJalw= Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6e6ee9e3cffso363724b3a.1 for ; Tue, 26 Mar 2024 21:50:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515046; x=1712119846; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2fwmOl0CxtBfQxlaR9NQyd7IGBnsfsK+MLDyKM1aSvU=; b=J1l3/XvXFYxCwlDbvmmJQ8kENx29feo+qSrpnp4nFuY2Cy3Ft4B1UGIl/mS6+N2v4r eF5BWYyxKqomnFqRRqvIyLDnbz8NOBqcKVOCntuBVEhMIHyI+JtwOi9X6SIr5lHXtusv zLS7F90aFfu+JhEbUbu5HX6Z1JNxWetdD6Ulc8WqxGY/dgUH429a4bv3C41ZmflK4dmh ME5q8OlR0loM/+yUjXAEoPNSOqdBoTs0qnPZQR74hmtP45LDONm7ncYc7hxEnGdODzo7 qVGsKVKCRLGZOEFFC2TzQUh0ytmOjczY9QT9P1N93lQDEhugAwtHdBDU2Lk0yvi54yWJ +oaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515046; x=1712119846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2fwmOl0CxtBfQxlaR9NQyd7IGBnsfsK+MLDyKM1aSvU=; b=hv/SPOYEGwfyJGQpPKS0b23cXk+pL3WZPfyf1pCpSr3w6DvbglWYuXlVOcHKnQk+ym W4/ile7LHQGe9klDBnRxFl+E/iFSmo8YtRc7U9595xexG/EbJA7hS1+oZc4HKX4i/G11 klc00B5eJ9upQH+dyoXxKnr+IBr6q0psJyLIIXxed4hj3DyR3BM/nGjxE6Y/G+5EFEz6 1N0ZkzTn8WOPBHSXZhJRV3J2x4uoWIzh9gWXIOwHfDT0WY13afBlznLx92peWpg3TS96 AIGi4ftRrl3tqKwCA8CkSXeY0KiuL72Lw3s9dT9ZgZCxNddygCtbQ8KM++UAj67Hu1mW ikeg== X-Forwarded-Encrypted: i=1; AJvYcCX5gIFYemDMmzMNblcBYdFoIYiejB5AmpScfls+d+tifTpDd6AC0VDn6PbwLatQ88oJJFCoE0eOSnUxxfR6n4vB3Oc= X-Gm-Message-State: AOJu0YwsjrHE0AnGIjoKY+HtWXOrsvvNhkwghmsFwZeYtHqGxp2LmiIs DCAykWTAIIU1O+SVgIHW29+NbybeQbtAsPj1m0Ox+Tg08sy7Qq0bBY3TyQ4RdfY= X-Google-Smtp-Source: AGHT+IGHAmpkwghveoQCxqBYVJp1RVR15SSZ2ypXvD8LXUp94lm0FkTd5l29fFm2kWdFZwCld1auyw== X-Received: by 2002:a05:6a20:3d11:b0:1a3:6a11:4898 with SMTP id y17-20020a056a203d1100b001a36a114898mr4924839pzi.3.1711515046438; Tue, 26 Mar 2024 21:50:46 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:45 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 04/13] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Tue, 26 Mar 2024 21:49:45 -0700 Message-ID: <20240327045035.368512-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 862C612000B X-Stat-Signature: bkpbbx86943nb8bssc7gducjdkkp5dgm X-Rspam-User: X-HE-Tag: 1711515047-875262 X-HE-Meta: U2FsdGVkX182nEs2mz7G8j3Gf7TJm4A+rv0cuviL4/oKjoyBsSlv1ewNnMC/SEtV2XWz+aEKkNZRnuW/U9sW6M/xibkqZQWUIEHpXOCZoEnz5mTSRY6a0Z33Pe4UZlgfHRcEGV+JoeKx1epKZgEmyRFchAk3TpE3cKZDRMeyp83EuS7E2htvuKQrrcPByIL3FjtLG5V7WZwCfBMeY88WMV4euQxzdq1kG//IEjfLNKQjnq+7FHqxVr2/m8Q/Q2bCA0bNuHXzef9ubfAhclIAgxop9E9Fa02J8+nQVqLpXICxX89yJRre0hq4VrRam4X6VU7+oGgG3sn/RU+go7/vsV2NOZYFZiK46fo4SnBUVCnh2gUjpZhTI8ee0D6Zqo5S0bgyQuQZQhEKb1Xj8/o+9AGLBJUZeXg6j3azmOu+gyV65qM8B8cmvEFzZMlck78CudZO3QQ2pDlcRNeJJJy4hkz8xJmT3RYyKhTEX/rh16roHh20zNOAg83w8T37lln0DCERjhkt8JSnRINvR7TqjTilpzQFXy71CuhO4iMo87JXGHffNXKERTEbQVVjnZGMyn5tr9gVlt4ZhUAxlGIG3SUrMdGtQwv9ozcdRlyEF7fieSrY+b1XL9DMtoSEqYLpjFyt8MmoVIeiIXp9m02PMNAnD+eYjiSVMoq8M40rXsiGMNk0358879dQ4nkHv6BHT9IOYmis+Eft61unSPOu2NrGAIlr/0QMwfmxF0aCkDRx/Mej8ED1C2Mrr9/+wqvpxgVcagosdOlPMERTjudsTtskGnaF4pj2wiCym4OlkmMOMIvjg2lnd++LZtANMIFZKt+wIpEjuPZY8GEuRYl9QplvPYFMC+8sPm4yifE/k2fQHv2azYv6G8iOG+jEm5RaE1lbfBngjJPv3WO5pOZ1ZKr8kwq3ZxLVDQQ0UrzvOnmDrhfJr5Fqv9kIfxBF1yQnc5TMPZNqud6nFOjGAnb H19RGHBL Pc2riEbHbgDLletcj1iNqvwoqf5Y2VPCvS6WWqHqzqIqkiG+ACMsgDMne0THZQG0p70SBbf00jC0RMKx2QRVeRVoYm2wCW/hFsGPQvZnsr/dMmX8ttkAYnOy8JboSqioHc4O1EncEkcQvKLdMPfJzt61hKighLFqvz/W9867xnyx8T6rmB8fc1W9iBMXFK9xZcqZ3eyZEdsmCzqfWfd79pqif999dSeWDVhP2cHAWBJ5WENVWpqDXlsmy4iOLey4arWVoiZKmsS2PYDz51vDr0oGIGyS2Sdosc2f5+j8IyR3JnOls0PF4K7ok4nRmXn/VviXp6P5SNmNo+q4eMvhye9g0SSJ2DfWawMsPYTx3MKDHI2Bj46Kyw7H2/OSU9Nby5AcrZTA59cv2m2XcTldkbY8q9yLIOGVBLOh718qrUcA4m5M1UCfAYS/Pl7MyA46lZYU81snHSUrYHyo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0435605b07d0..da821315d43e 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -103,22 +103,15 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, unsigned long start, unsigned long size, unsigned long stride) { - bool broadcast; + unsigned int cpu; if (cpumask_empty(cmask)) return; - if (cmask != cpu_online_mask) { - unsigned int cpuid; + cpu = get_cpu(); - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - } else { - broadcast = true; - } - - if (!broadcast) { + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -132,8 +125,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (cmask != cpu_online_mask) - put_cpu(); + put_cpu(); } static inline unsigned long get_mm_asid(struct mm_struct *mm) From patchwork Wed Mar 27 04:49:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A917EC54E67 for ; Wed, 27 Mar 2024 04:50:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 422406B0099; Wed, 27 Mar 2024 00:50:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D3E26B009B; Wed, 27 Mar 2024 00:50:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24C176B009C; Wed, 27 Mar 2024 00:50:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0FD0A6B0099 for ; Wed, 27 Mar 2024 00:50:51 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E1F3CA05F4 for ; Wed, 27 Mar 2024 04:50:50 +0000 (UTC) X-FDA: 81941593860.20.DD32CC7 Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by imf17.hostedemail.com (Postfix) with ESMTP id 20C874000B for ; Wed, 27 Mar 2024 04:50:48 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Q+LmYxwD; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515049; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8r90DmKfTzo12+b9oHF/xBbsXVRSBOTytxxA308aMe8=; b=3iva4nVJZbw9w0SjEiLYlp4Xg+wxqbK/pDLoaPsUqFdJcWs9Vm6WX+YVHT1FdOyZU6mi7t JaEHME/elxeoqjMghQeTzVuPQ9T+gywRjetqmqz7gtcy2PF8vyxOCyUPdOvaEer2J8O+WF 58oAg8UY1dqaYuTPnA7UUAdGmio5/r4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Q+LmYxwD; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515049; a=rsa-sha256; cv=none; b=edaffm77R7ob0xlqT6UZUpO6qoZ3bl+J5tCXPQfxEv5/FEgWRiwbQGTCLw/InAsFgHvL09 I5A/j4VEK2U/XPsVejB4J/BSIW4kvxAu6c+8SW1KEpC9cGJq+kdtduJqjNiUCV/1lsvgPC AZP6WCtbilj0nrquacfDsfpLvIHVkEY= Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-6e6ce0702c3so1683045a34.0 for ; Tue, 26 Mar 2024 21:50:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515048; x=1712119848; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8r90DmKfTzo12+b9oHF/xBbsXVRSBOTytxxA308aMe8=; b=Q+LmYxwD+v4odx40f3QLjMK50TBatVjdUDOk21Bvm98rojkUw2ECpDDslsJ+ARYjIW 5d/yFE0Y347FvEm0RXW6maQpydSPauSQkNQcrZs/jHQ8qREDgQ/ybDaMEH5tD/6fDB3G 2Vdn/v6rN8b9+6h6yc4wE4c0u8/6eqI/Wh+hVa1Cnj/I/m3PaBY03osqk16KSsJ2pmNJ S5yX1KiSfvojwd33hZZNpjF5c5W7VIysR5NxgOA5cfHmAfILktAAbq6BkgKtZVP8I/yv XxNe5soKSmYW3uWP4t/TkE78cFp6m+pmtsX1YRQjxaSHrQEcf1ZibSzpzSBC32mAg1wB mPhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515048; x=1712119848; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8r90DmKfTzo12+b9oHF/xBbsXVRSBOTytxxA308aMe8=; b=PWPhPMiDiRJs6A6iienjZ1VYASEZSFfboliZo7nBUUTchDwIXLPbYO6Yow3DUC1Tpr Dn1FCkhY7fHlYSl6R/YfBavmS49sj4eGfujMRpIH+O5d/SqEy16YngkX300+N/Lpx6zj Qx6jK1eej1UDEK3aYRc6vOIQWkFd4wJvPGygBrzlF3nyaokwLCGVYFKsf93Q/qlG9qVy nAAFUJ4hcb/ljSxUlNF3PB2cww6TEZ7jZS+O50sRVVGip9mrMnjb3Qt7N6o2W+qqI94L Rps3V2G1CA1JuVir3cm+N1LJnj6xGyEOahk5818o3tu61r1OhkQ0DQwXjEGV1oozYT3O bLVg== X-Forwarded-Encrypted: i=1; AJvYcCWlIKQhC5CewYEPIOUDwNdosu3L92MnLe+k0taJ8ESqJXHtjsANScVBoRHCkl4XwX0MX/1EI7kggF+hST/AaXyu1fw= X-Gm-Message-State: AOJu0YwMji1Qx8RIxiWnlyxGQ2iHdWzuAWoRZmbKk4P3RjrnzBY0gduU Mi54qgaOMlI1naKzHDL5AN7JW9Y91n+YLStF+ADvv/8sJt6OJaiYu+EoClhxTmdWoXqWUt1CxZp U X-Google-Smtp-Source: AGHT+IE/gtW/a2EQvN+qhLpDTcQ1EZrF+9VNRUlZn60iPj1iarim6P+gGyjDapgmLmO07m/XYQ3rgg== X-Received: by 2002:a05:6830:1e72:b0:6e6:7db1:b444 with SMTP id m18-20020a0568301e7200b006e67db1b444mr154078otr.36.1711515048291; Tue, 26 Mar 2024 21:50:48 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:47 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 05/13] riscv: Only send remote fences when some other CPU is online Date: Tue, 26 Mar 2024 21:49:46 -0700 Message-ID: <20240327045035.368512-6-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 20C874000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: mhaq58zeicfkz687wdic4io5pxgpncfk X-HE-Tag: 1711515048-537783 X-HE-Meta: U2FsdGVkX18nwSRaTzMpAUS7XQFE7NeSnlnRg62zHnFpF0xec6TrALx2pl94yd2Slhd/wy2+oPPy7TOQ/2iP4+4ngLedyAOaoQKlDxdxEwKbt1t0EIdgECgojUAu8uWQJ9Nm0mTYGEu5iieSWowcl+DGiJbzVfWuA8QuCNbXDDR7A2yM9zNf7AEg5W7iGsccgnvr9yKQMl8UGd556bA5wzzRWfT08/rK25jeRLPdgtIIh9cHLewY8fm5JxlCvfXeEC9QadOGytKU3dQ3ItIJRay410hWEk+QvenA1e2HUhvWycpnQNB/zmO1DHAhZWCutz6tJjhNQVF/HdT8uTmlgnA2GyAFmJ/zmwM+hnsyfxzrBQX7d1HvsRAnDpGt3HbfKvqrPzlxfMeyiMK3QjdUWyoWzJNAgXKE97s/HAsxZ/Qv7EqarjJb1v4d9+l5xwTfPktQjd8eozxNZUuXKldmPA5METgVnweMBPHIPob7ONtVo9g8f/76sblE2rk9np57w2Tw3XZ+EcWw272DblwJvPe4HXYoRXx54kgOZDsh3M3VXBowggq9dop8HdZVOA7RSW3nQz+zF764sBQ4O07XOTdgVC7OjkQs0TD2Op2ltYJHSa9c4zpp6WW6zEhb/Y21wU25bI6Mwr+zH4pgrg2JTuv8gPxX0ytznJ+IjXGL4rvnDN/r6X7IIPhedKLR7o2u0mILSXTo2PydhVB3DUM8i0wpRFwfvKmXzIu3ZsUeipUZVfBdmLjohKxjhbPaoqaYmHTtabRSnQ7T3KFgHnkx5v8+KHlyJhTsENnrPUnMBvofW5vsXvRPEG2w6vlJtvT8SCJCEkWtO0ikojQ41gx7B6vvuLdkjIugwJzzGD7XXMEG0OStYod9LE+qkIMMTs0GDKEYOrHZdhgjA3Jn7CqhDgewNjIF8I10yUZIzc0uRnLD6ETTQKfpxJfHkqsdjgPFtzQG+UzmHTJjv7Rs1Ba 0TEXwwxH qpg/PZ+kPiu6yidqRfpECgmlrcwzg7hp7inTXdPDVX6RmQSW98nJQ8T+Sv5ajSQP9L4igr4S/PWTDv3A/EOjsEobo7/OaOPbxXN8z3MaQllkS4+zFZ6DC0t1XPMG9dguQS5J4PYFm6gTS8n91bpkJp/sUFi3q4B0YdenKxn7PDgzpGfukUVKr4mKUgZCanwf/LV4V5TKxDEKsXZWVF1IGcTkD6gCkTruLbp8LmyupLBXfYB1LkN35BJ7/M7UDP0afFsmrEVlHoh6qA0i/kVhZH3QNrffKpyxEELWv218J9SuAf3XRaEhwBJBSAqxavxWpgrbUwUjOia14blSi8hZVCJUp+yxjXMf0gQH/BpCaJx2f/eQZUFu/mNedAwhFxD5Pa0dg6NbKdmOeDck= X-Bogosity: Ham, tests=bogofilter, spamicity=0.060444, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If no other CPU is online, a local cache or TLB flush is sufficient. These checks can be constant-folded when SMP is disabled. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- (no changes since v4) Changes in v4: - New patch for v4 arch/riscv/mm/cacheflush.c | 4 +++- arch/riscv/mm/tlbflush.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index d76fc73e594b..f5be1fec8191 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,9 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + return; + else if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index da821315d43e..0901aa47b58f 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -79,7 +79,9 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + local_flush_tlb_all(); + else if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); else on_each_cpu(__ipi_flush_tlb_all, NULL, 1); From patchwork Wed Mar 27 04:49:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06D65C54E67 for ; Wed, 27 Mar 2024 04:50:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BDFE6B009C; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66D7C6B009D; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BFCB6B009E; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 383F46B009C for ; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CE2FD16060C for ; Wed, 27 Mar 2024 04:50:53 +0000 (UTC) X-FDA: 81941593986.02.C3551C0 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf30.hostedemail.com (Postfix) with ESMTP id 25B7E80005 for ; Wed, 27 Mar 2024 04:50:51 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=ZB122B02; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515052; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7nL2m+hLarkGQvecSEd0KAq0QtYwqUItfZBnVAx3yeI=; b=iiloRkcMwZmohdgqeOltTEACr5lxgXEFqInB9kP5HoAIQ30Hnb5TwWYKEgwOjWrUDBpkco Kue8nLoVW3ekxj5vMj81ppDq83gjRj9y5O7MSmkCH9UO3BPmHwKMHOyJ775uAHqQX7h3of pXvrOmuhP3moqUy2LsQ2bIUyZCrHwqM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=ZB122B02; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515052; a=rsa-sha256; cv=none; b=7FisQIn6ydnGtPeXszQOUJeFd+hXBHvA7SlwkEg5WbekoFL/7Me+w74jUv2Zez4h4X/Mer t1qms5a74guBx347TF8lHF8RFUPVk7L3dUizm2bDUSLgQbNTZ2P56k/q/XxfjfOF3ENJu4 QjMjXU55FCFdMkCHhGNIv9tztbjB0WQ= Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6e73e8bdea2so5252903b3a.0 for ; Tue, 26 Mar 2024 21:50:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515051; x=1712119851; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7nL2m+hLarkGQvecSEd0KAq0QtYwqUItfZBnVAx3yeI=; b=ZB122B029vx2sBz/7sPoTpodARK0beMX8+hdhsPb/r2mSX9V2lGcAuv3Ia7ZZ4yveU 9eXEllJ12jlmgjQt7yVL8lfp+gEQBAOVEaNt+2hbJ38EaRI4s89XsIIwMv+82KwGTpGs mC1aedY7EZLvPZEXho2sAwpEOa3gbge8+CHwDoXJKKzp4KPajw78QSofJuLn3XaTL4X1 w9RZpvrx3IDRzCDS7RJYjqo7edVfGJHklN9t7+k/rlNeqtkDuqMujQvWQrNBrxUCo4Fc Dgg3uwQZuRQdHlaYaL8kKVlVScxX18yRjQvZBoTzPDB3woRjDNeG9cJQH0aRm0GVjcLp /V4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515051; x=1712119851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7nL2m+hLarkGQvecSEd0KAq0QtYwqUItfZBnVAx3yeI=; b=FyXat8ZuSibtIfu9I7QEbNCQW7imFVHQtTmWf66hZvmxNZJpnoyi3FeOMCkRXSvgxR xxtwyTt5SEHlCSzByA742AdQrU9PO2NcrUrSGFbUCsDabxo3beYh3PLIsy36q7y0sLOx aTOHlOVH43lmEh64QLrFjS9gCZr+aMjZBUhSv4Ez6pqEOq4yxPTjYq7qKraeLwGOijwp a+wVwX7af4zfOthDuHCHyJP5I1mAo3Zy2HnwD/EmLzQOPgJy4qbdyiP3L/QpHFeD4M0L uXw9ElbIPR1v/CMMarNLWVniCN0r8dJ91dVeURO6moSI340wZpaoa8ZktSlfdV1bKeFI MV1A== X-Forwarded-Encrypted: i=1; AJvYcCU4ShjZacedMthgM7h0pi338Fy4KfwG+CbCzeZboqfy6rZ100xttb86+bexG86k+2lytlBrpnQlfrvNgvDzlV/dpYA= X-Gm-Message-State: AOJu0Yw3Nwp2jn3BuNO3KqBbPELJ/4epfWp0Gr9bWpAn2uWzVrr8dqeG 9D47O9SKIepiTi7zDQ97yXqQ42ac9W+T7gGdpvsbpyYivsNW5ZeRQVSopZOBpuE= X-Google-Smtp-Source: AGHT+IGyz4xXUqNq5aXCdK4lWIBgBxDZJLK6nH/Xm7w+Te1ce++GEK9gbqC/aCbf24OJ2Eh82pVCjA== X-Received: by 2002:a05:6a00:13a2:b0:6ea:8e89:7eca with SMTP id t34-20020a056a0013a200b006ea8e897ecamr1940335pfg.32.1711515050263; Tue, 26 Mar 2024 21:50:50 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:49 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 06/13] riscv: mm: Combine the SMP and UP TLB flush code Date: Tue, 26 Mar 2024 21:49:47 -0700 Message-ID: <20240327045035.368512-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 25B7E80005 X-Rspam-User: X-Stat-Signature: 39ebw6ferybewehc73dydu6pw1h7y4sh X-Rspamd-Server: rspam01 X-HE-Tag: 1711515051-519388 X-HE-Meta: U2FsdGVkX1+JuI8VT4///PjqzFp8gzykZa0g81HX4aOCyi69X1kLmu2hYmY6yZ/o1Vr/cF+r1nmx2xazAYFqt4fGBeMjg0pGiyHcliRv1BsH7YMV/O58JQZ7jFgnu2vkUnBPbFzpWnH1R4dGNjwE54pVamrkImx1NHQUik2pMgWya0h9VzSuLyhwT77mfD9X+GkVo1CIMSpsCnymOdyOyuX0t2AAblw7F6PdPmJFlyDG4TVz2GEDMud0QpS/dU5yyaE7RRx4ZxvjPdQTK6Z9ETz1trnK9ONqJR7DgdOkDk8dtJLX57mSSD183jPd+F8/Eunbj6SYCeXRvLaq4b1qvkllnjWUAuBJneD/8V1Ynqx2vz0qoSq1VbrTQWDXzu6iaDKVGL+2M6DK73DyrcPy7kQiCt47tk5EKFYk5+jDj9xn/Tupel3ccil70E146FHqPPo4xGpC22VRcNEcXecIviXlOa64rgrcZHkwrXXPAbJYNnZosUUGno/8FjESyYswCMawC31zc10B/27nfVOcdcMuUfRLVQlLC54El4rx5fOby1/XKDlUABsgyl5zgIs8MDSVT2RgllqNLANUJlvxlwQxMkF/uA1gV6TyaEdQdtfWm/h52svHDA/ZfpWPDH1kHBlwG/0syF+u6N8axnxR7PdltMQ77mTZmnqmvRdAp0t22cMeuQ4S5aoB5S0w5RTZgSF8xnj0VrOh2FCQDTiFq2zSXIljl92E0FFeN4M8znjmL3mea/m8BYA4yF9mjI24Sh9woCQ5oAn3kwhgMwCUtIYp3CZ2gaiW3HMZSUGRqCvTUytWjDEM/X2Aw6tpMTAEQKH3bCl1hEkufe4G3dk83cyfK1uOwiY0OtIxbTNpqVKNtN8zCyk7AigDL++PBtEI/fU2hQMIYZqm+ZW1J19x8anrEt2QYIs8bRZx9NE14ZGZPO/Alry+xA/rJaFX/z4GmOEgjik3cUxdnSWYW4b Wxln6PSV uPLP4ZIiPC+Vu9H7WmRYv7XhLpx7K+hbT2QmvdqmfwqposViXMGySCvKEGIu8HYy2LBeQM/V3xYiz8tYQSeSUuX8WY8Hn+ESneEO0ivqywJ19yGSBTrUkH0XfKQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland Reviewed-by: Yunhui Cui --- (no changes since v4) Changes in v4: - Merge the two copies of __flush_tlb_range() and rely on the compiler to optimize out the broadcast path (both clang and gcc do this) - Merge the two copies of flush_tlb_all() and rely on constant folding Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/Kconfig | 2 +- arch/riscv/include/asm/tlbflush.h | 31 +++---------------------------- arch/riscv/mm/Makefile | 5 +---- 3 files changed, 5 insertions(+), 33 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56..442532819a44 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -60,7 +60,7 @@ config RISCV select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_USES_CFI_TRAPS if CFI_CLANG - select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if SMP && MMU + select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH if MMU select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4112cc8d1d69..4f86424b1ba5 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,12 +27,7 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, @@ -54,28 +49,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); - -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#define local_flush_tlb_kernel_range(start, end) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 2c869f8026a8..cbe4d775ef56 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,14 +13,11 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o pgtable.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o From patchwork Wed Mar 27 04:49:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4003CD1288 for ; Wed, 27 Mar 2024 04:50:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A2AC6B009D; Wed, 27 Mar 2024 00:50:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0263E6B009E; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE5476B009F; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C9BD26B009D for ; Wed, 27 Mar 2024 00:50:54 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A177FA0673 for ; Wed, 27 Mar 2024 04:50:54 +0000 (UTC) X-FDA: 81941594028.24.6E915C7 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) by imf14.hostedemail.com (Postfix) with ESMTP id CEB3A100007 for ; Wed, 27 Mar 2024 04:50:52 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="Yz1JOL4/"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf14.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515052; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kHzKH2DTKf6Vc5BlA0YNBJTz56l+llAyGgYGMd65KTs=; b=cFSXAU+iAVthXsHCayLO090EjwZ1uxBBQvfPpjVDd1TCHqdxVdkv8S12a5MwEdQMNNTt6O elVoCYNk5XbVPT/uyNz0VI5xEE9+28eIDojCHXLq54EXEz09sNG+DEYRIWo1RJ0s3NQTtw zbuE7F3KJNXp+TZcl0b5Y8JN1bGeMdw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="Yz1JOL4/"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf14.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.49 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515052; a=rsa-sha256; cv=none; b=hFvTKtiMbP1T6YjL/Cn0wFdUWQbOfGM2X+CpbygcVINTY8qiPdTO77as25t7+6+/NB3mj1 MCah09cvD864qyIR0U/mkL/TmN9Q5ZXRVH4mx7Ow5/xqAmYXKjP9gmlbbPeaAeZng2lEqC VFp3o80ELQbx9If8Rz/nFJxjgWKHmeI= Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-6e6f72b63e2so16253a34.3 for ; Tue, 26 Mar 2024 21:50:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515052; x=1712119852; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kHzKH2DTKf6Vc5BlA0YNBJTz56l+llAyGgYGMd65KTs=; b=Yz1JOL4/UCYmV6S4j5D6e1yCSp3yxcxQS25Aw571p8emF9JChB2k/3fp37ZzJbfp4Y zIlTNUmCi7s/rq8mgXe2o9TZuwLraXPCrwwmjg80mqtPOqLZNtu4twLmDHZrQiNctddV FknlfUhNaLgmFNY0RvxV+Q4sNuLfMYlC71SMUnX5JN34Xl/zITaHG5ByBAoAmE3tzkBO aJBtfT0FXHUFxKxlcJJyK0s5V0shm8RnhScCEyNpz6wZEMBDhaoHKkYVgSg9182/W34I OwQRTWWUdY9Z1FJN78yVoqBOOSoKMDG/v+f0VFg6Q5EUHN9OXnkajri+J68ju31hhIKj TQBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515052; x=1712119852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kHzKH2DTKf6Vc5BlA0YNBJTz56l+llAyGgYGMd65KTs=; b=CgbM+49j+fKXbTVrNMYacZ7v1BIC0dwQF+FtCuRwLX2t4JrU2s24rRL7ShQ20IV4a3 LKf9kFox+U+UfWXF1vMcqfxGLlX4s8jydVl8S44Z+BO40/PCltHh7sBEh+UNiAA4fSIS xumBvRfJ72OQbBoKuYxSxDuC/1J0QkNIEK42iFpkRDG55TikOKDsA83TpxDhOUR4sTmL cgMnoPxFviUZDElG9NxARPG5vDzIj0uXXb4fyTd42mZ6qGH5LxlJlsR4w1rSTvk4CJF7 09+EeF7wS+PAmAL/IWe0VmapxTdvLAYN3DU0vQY58ug5sOOVBB9B1wlmbrImwbjP67dS Hufg== X-Forwarded-Encrypted: i=1; AJvYcCUhD9hdwIU1sHtAK+PuLTrdO7Kmpw5ynnM1PUTx+gR1yl7fx3FHXSx+FIMsHLjHYclhHiidudduT34+1iqk7L/Zj1M= X-Gm-Message-State: AOJu0Yx+ZpnOHMiSqXCWaNh0emXwlDrgp5dV/osiIxj4zZyt9Y8wFrV+ ptYf3nqmjbnegBc0q94rodzmEiS+iaE+meotSlAIgfpFfnGwBXwcA991Ztd3FWo= X-Google-Smtp-Source: AGHT+IFNZ9hbtEq1CmgyYTIu4vUTxuEiR33JD4yc+xyNKXepJuE48avCWWfj7WZgftYdmFVpAMAO4A== X-Received: by 2002:a9d:6e0c:0:b0:6e6:b2ed:f12a with SMTP id e12-20020a9d6e0c000000b006e6b2edf12amr1764968otr.30.1711515052040; Tue, 26 Mar 2024 21:50:52 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:51 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 07/13] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 26 Mar 2024 21:49:48 -0700 Message-ID: <20240327045035.368512-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: hcdy1dtw4juh41rf9jwend5z6r6g573t X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CEB3A100007 X-HE-Tag: 1711515052-114508 X-HE-Meta: U2FsdGVkX1/lCN/AdSOYLEeT6ue7/BWhgTx7QpNHF11aBQjdEvJ+u98YPLEcS9WRJfoTDgTFjeYU6B2wZs3r7qqpoYrGgY3WC7fUnLh3N4vct7EwmwlpG6rvvcRlJnqe3ut8AjN9s+SW1D76Uv9/LQ8AWpHg2b0tKq8p6rIniTm2KJI/LFuvuinBn10cLYftvlkfT1Eh/0PWWs9cCPw9YFwBcSa6gLlOdMDeiqB2vHILK1OgnnpMR/2yKHkK0/56iVyM/oW9nec26mNBmPK5+IK0igCzFyVH2UCinU7BI5V7pfUgh9IXsyWpx/msuJwoTmfHSEujbrzuCvjklMssHFMbUs/DOJq60wJoEf2BINlpADdWyOV4d/4Ca7Iccubbc/wwRM2nnX+eu5yrm17jdSWul2ecwN+sR/GkVzAKDlHcoJT8JvFYzvRpSqZtfhsBV60oKWnsTokuIqU+ZVuXS1lVBWbe1yW3rConD3a+BkLhPr2zAhqWQ0+NTQEZvI5br93wtVRrzLmFdvAap0dztUyPU5XtF3aPealNG1HoMx9JI12AFDyqq0+zzdhLewfM//e2QnyymrmYhsn3ztGhZKKxzisW11F1u+3xHFRc1bNa3131q0NuyryBttDq+PB6DKpIrqObsrgJm21aM8uuvHACbrNZ3HPT8aJ71uCbFyiQQ/8LxKv6c5LrMNnF+1gWw0WkEN/bSKIc4o/TlSO+eqb8ejvDsq1XfUpivb5b9Weu3HHtDn7YYjmritsalBz6svn0tDsB3D9RzzS2FavbpuWXzDzAdADVT7fSY2MunJpIi9cI+nYDmtoTGn/ZU9LsrGKcvWvLQ172iI/oGzhkeMWbE1/X5AAj8Q9NvwrIXSVTUjQR3WtskRfs54K44g0af6oYlvbu44+e88hkiNlq7rz6LKVngpQW/nY61cbo6SJawxlnu2cqHRb4Xs+3RmQ3keQKAlQOgryl7HadRcV gmVvaAuB Q1BQiXKS4yC/uEG+IItzbemObNOU8xKtzChWYbeOD9cf1c7mjomHigJq75/Yg9aE6uT6NhKrs6eZBTeSQyCWi6d29pj8gu8/qV+/L0XWxtWJZHFWwC5MGk39CIIFpjsTnPIYwgXR4eEIwMC4UPuTecexlKjq6+WiEzaqCELSw4SIfXl6x1oqzJ3uzczdkdeRKkbAGQhT5bT1FQJJU32e66lK28lW10vANcbk0kJnnFJQskcGDSNCcp9HqUikEI7z5/7gJPlJOrviLSril8K4CGP7hcIWapHTlwzkKNhcS2pX6oRQ9hUhSxwO3i9MC9LSPa4P/Tgp1VhiQdnkejP/hqKO5I0E0h11cUN3mLTcm1bhYBbu19izJXkQ13Uy3fjW+SO9LzNF4Yo6zKdU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 1f2dbfb8a8bf..35ce26899960 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -43,11 +43,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 4f86424b1ba5..463b615d7728 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 0901aa47b58f..ad7bdcfcc219 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. From patchwork Wed Mar 27 04:49:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BCC2C47DD9 for ; Wed, 27 Mar 2024 04:51:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE7426B0085; Wed, 27 Mar 2024 00:50:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D48F16B009F; Wed, 27 Mar 2024 00:50:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC2E76B00A0; Wed, 27 Mar 2024 00:50:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A78236B0085 for ; Wed, 27 Mar 2024 00:50:56 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7AABC1403D6 for ; Wed, 27 Mar 2024 04:50:56 +0000 (UTC) X-FDA: 81941594112.03.EBBC88D Received: from mail-ot1-f50.google.com (mail-ot1-f50.google.com [209.85.210.50]) by imf30.hostedemail.com (Postfix) with ESMTP id A558D8000C for ; Wed, 27 Mar 2024 04:50:54 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=crFgZsC+; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.50 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515054; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4DMxbhiunBS0lKLURYVBJG3nyObzwUO8ZrXza9ckh2c=; b=eI7ViTvFtsAnpscsjwBxnjX+I7duxpV+d9C0iTIIUjvjHTvaDRKUPc9VdgFyIUKJkd9zrA k4k4Geo8qCGzktT3OmYnuZswDYRzfIJJ9zaMDEbGT9co2d/M/FyDf7sVBkVyEjtBHVOSr8 6z64OhavXC70hbBkdyvUMV6NWJuHa6c= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=crFgZsC+; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.50 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515054; a=rsa-sha256; cv=none; b=zMOTGla02jC2a97wXLf/41WA6/IZJR0lJpPWflwwZLEezQotyMBKk/M7nH98cPAS9VqCCV WWfCB5nikzw+qVK92w1rHJ4Io1ZFNKJc5ScAS+l9POKXc5YahuCWUhXuSAYda9GargWIPZ Ijj0HrZiSY2gcAfx/C+LdrsXNBjikO8= Received: by mail-ot1-f50.google.com with SMTP id 46e09a7af769-6e6a9e2e2b6so3481379a34.1 for ; Tue, 26 Mar 2024 21:50:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515054; x=1712119854; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4DMxbhiunBS0lKLURYVBJG3nyObzwUO8ZrXza9ckh2c=; b=crFgZsC+PAacXnM5PlRp0C3h3LW/mzW66S2XEFUpmVzKP5sCmt7VOr0zSNJ5L0Bd0J N824yiZSMUlxzVHyf817bFV/n6/zKXKLtGMHao8wSh9z8M2SXudp7PiBEf5aCBVqds1d No4Ku0+XSgq5ur8A1WoIw24XQTxE7G+tYWX8qi2m73zcM8llpf+M72w26qQl4oo5pTl4 2ZekdgRANexpCM5kSQ1kagrv+CaA7/+3HpBrW+o586n/bgeQNdtuxKOY/3bmYCUarBJg 32msRh4Zmk6Mt33eYeziOpWZzjLxdRjgsHdo7ou9QvfJNgIxGJn5FnIGyuur2yOBxGQ9 d8NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515054; x=1712119854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4DMxbhiunBS0lKLURYVBJG3nyObzwUO8ZrXza9ckh2c=; b=CkE1VoxQXV9KBls5g0daugzxY8TvI798XJua2FIOrbc/N7j08XlZuvMIk05H8WzJu6 erF6tcEYs8ug4lcqIWmCsAy11k+aKzQu6VIPYHdwGGy/Xppfp6t4RGzPdN6mUN6LBAZ6 fTuoIAS9xHFxE6FOOqSVifbOHSeHm9LNCpTVr6yMNjPyeF9UU8G7DtCeJdvfUSenaRR1 8Cnb4p3nFrGsXg95cE757IuIpZoURU2sN0Kc/KChCPlKc4erRr8cCet0srhtgzhxTQ0A wdWEP8+Ph3H+VhETBVIvRpOC9ybdYTw+feuvr8bWeMy5uFsrUnv4TlVVHoFvdhsKhqy0 z50w== X-Forwarded-Encrypted: i=1; AJvYcCWE/qIrSC5qD0K/v9YE9fsV5mj5G00EiCecnRT2UvWkLk5HFcCj2bVt7dHflaYZebiqwpRynlP7MLNSd2JXhZ/tlSg= X-Gm-Message-State: AOJu0Yww0tpZJMscPW/4frWdRcT+SVpKjW0AAir99o2UvHA/gZdBg01B nm3JYz8kRboztOlDRCp7n4B9U2wzVNVAofdVSIWwEEAL6aUWFwlWyPuX+jVCKko= X-Google-Smtp-Source: AGHT+IEKVJUxYPHGmGQ7jZ/KC0+y46rPbmkW2DQ4LqVeHQ3jmUBdRhZe5PVjdoDThc5iPwQM0jPRUg== X-Received: by 2002:a9d:61d8:0:b0:6e6:9af5:752b with SMTP id h24-20020a9d61d8000000b006e69af5752bmr1730488otk.0.1711515053696; Tue, 26 Mar 2024 21:50:53 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:52 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 08/13] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Date: Tue, 26 Mar 2024 21:49:49 -0700 Message-ID: <20240327045035.368512-9-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A558D8000C X-Rspam-User: X-Stat-Signature: 6zrjtrtwrg1tjmn3cyi8a8qetpauzugu X-Rspamd-Server: rspam01 X-HE-Tag: 1711515054-214761 X-HE-Meta: U2FsdGVkX1/VsqhpdzCMLAW0avzfmXXpBNZWm9p2TDNtxmdiBwbeHbDhaRec2bT9Y6xXjvANLV38z8JPZUBJ1hsFRjfOa1Bkwls9i+xMbzdE9eyNIc3qU9tQCSoVWirtcRfPfKjO6/cFk8+WpaMdvqQVvut75ZinxJMGusIgWVuqcJm0WegPUlT9FmYqQmjHSqbka8soXY1J1wGwhVv5ZaBB73DRQwfN36Zle9hEcnCdJ0cOKCDBl6H1tffGkjuAph1+hSINhX6t5Aqlp2fYNJlUhl2P3NgC+gIHLNObKr2/VKO2VUU6oATRT+tUSBlyzcjlZLBOlnEYJpjGwodIRwyWzmKVIVRT50BXE60wn+hKL/sqIT7y6KvnM/n4/1kqIZYsCuuRWflYhUBVBGJ9yl6SYhUfwAxIse2mmXrcxQk3j7Hu+biNmbtOfAsdFqxQ8dqIBen6iCMOcnqZZFk5qfbXvEGGrlZo1iU4VijHllRF7AQlK3bTHhhQp04ttfXS4Fmri2xlRQiare1rmzwTL3pSABiDwHC4VI5c/C3WcFzCv+fub/nk3XfE1GWqrIbOnngLNm9fPemoCai8vCHP2wCpCzZqhPja8XwBDon2VDY/5zwQSn9gEQeriCRSMuXQWApUT3jisRqWlaNqJaw0hujssxghXlojh9H5UDsZarEAvKMA5gDnEsfYPJdmBAo/tNVFwzAo/RxFqSgPBe/Nu5g1SDt+s6qTg37LaMhuYlUaqGJoC2quuCQ/q37s+lw7GMsk/kD2zxRj12Badcr2WJ1HOS1X8HIwDN34svGxk5/PnMMS3IqNRFxhthoZZ8SSDoFbegR8dX1J3Toj5Ad8bi1OYP6ireNz1b7KXM7pq5cS7t8V0qT8+lON1eJax6ChItl/gQ+8JaDqEohLm+k9GTM7DVHUQkDWWWEXJ4Cl7w7RJwN67M+Pz+W3RXrn5RnFMY4rh71iQmvAqEBaJKw QIpIBw0l 4ftSrZgikJwqqwFGJytr1kMHfM+NXU8lp0tvyX+zxcOyrShN96jINfWYDb+R0b/qoJF9Jhi+cGiEO0FVfZJ2Hb1LDTD5X2d40CUFqq/4XnNIIaxVrQxhffXG8PVoIoa1g6rXuZZBrEyKBdfc7Tfpfk3HdZYDI5uaXEuqdywSedpUFfkOaArYEpJW2gGje/DUTuwL6OnzK9FoqkKpJRgN5WTThdp1v4HJ9WYmYmr1f5Ng/yqN/KvJRF5aTqjDFsAUqs7RVqy3zhqo+k38l7eN3RowJRZKiLqBHceUxEEvhKkLAacgB3fvmZ57isZDAEZbNfGFOoIb4g+B0O7l7DjcwR7Pj1JxMkxQdZ0PP8iY12s5jvkhJvSPKjc/813g3Hn3bjUgtRr35nJ/4rKY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.001892, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Implementations affected by SiFive errata CIP-1200 have a bug which forces the kernel to always use the global variant of the sfence.vma instruction. When affected by this errata, do not attempt to flush a range of addresses; each iteration of the loop would actually flush the whole TLB instead. Instead, minimize the overall number of sfence.vma instructions. Signed-off-by: Samuel Holland Reviewed-by: Yunhui Cui --- Changes in v6: - Clarify the commit message for patch 8 based on ML discussion Changes in v4: - Only set tlb_flush_all_threshold when CONFIG_MMU=y. Changes in v3: - New patch for v3 arch/riscv/errata/sifive/errata.c | 5 +++++ arch/riscv/include/asm/tlbflush.h | 2 ++ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..716cfedad3a2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -42,6 +42,11 @@ static bool errata_cip_1200_check_func(unsigned long arch_id, unsigned long imp return false; if ((impid & 0xffffff) > 0x200630 || impid == 0x1200626) return false; + +#ifdef CONFIG_MMU + tlb_flush_all_threshold = 0; +#endif + return true; } diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 463b615d7728..8e329721375b 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -66,6 +66,8 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, unsigned long uaddr); void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); + +extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index ad7bdcfcc219..18af7b5053af 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -11,7 +11,7 @@ * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. */ -static unsigned long tlb_flush_all_threshold __read_mostly = 64; +unsigned long tlb_flush_all_threshold __read_mostly = 64; static void local_flush_tlb_range_threshold_asid(unsigned long start, unsigned long size, From patchwork Wed Mar 27 04:49:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0501EC54E67 for ; Wed, 27 Mar 2024 04:51:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC8366B00A1; Wed, 27 Mar 2024 00:50:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B5B9E6B00A2; Wed, 27 Mar 2024 00:50:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EEA46B00A3; Wed, 27 Mar 2024 00:50:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8B7036B00A1 for ; Wed, 27 Mar 2024 00:50:58 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6172316060C for ; Wed, 27 Mar 2024 04:50:58 +0000 (UTC) X-FDA: 81941594196.13.28303E5 Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) by imf21.hostedemail.com (Postfix) with ESMTP id AF7891C0005 for ; Wed, 27 Mar 2024 04:50:56 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=jr7zVYI5; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf21.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515056; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2OOp6ZEyIwMU5OnG6xTEWcirae/pY2nPqfU9WqSousc=; b=5g0vo07QAkWxdj0NSZSWYFWowRPG5p4tYiQH15f76kNw5qdWA6lWQ1aT18ElqUn8AO7k7s WQ/GskhyUi+nO/szxi1woo1U8y8JXMt42frphmPuzFHtmx57x/hcsY6KGBK1DdkvA7Ddt9 EgmXy5FLcDhwopJWtGTOZetH4p7tHhY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=jr7zVYI5; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf21.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.45 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515056; a=rsa-sha256; cv=none; b=QZEkW1Nt4MzCUbqPLC0MekQB08MV2TEJYwMnIvdl6qcJK/88NMzL9E+lUqWYv+JuQD/fus in50yHH9uSJ4Hvapto4//4l4bonh1TNxt8j4e8n9qTxoXTFXdY6Cxf5gMknOM6MOXaUIMR zu3lMPsLCzIia2d4gDBhcB5ztJ31c30= Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-5a5362ceb7bso1591026eaf.1 for ; Tue, 26 Mar 2024 21:50:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515056; x=1712119856; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2OOp6ZEyIwMU5OnG6xTEWcirae/pY2nPqfU9WqSousc=; b=jr7zVYI5HWjs9uj/Fa3gE0Ldk1d5KOiB1ApFwyZ0snzq3GAJn6Z75DMiGuy9zio4FY k5JYU1mFRD6afLn2bhArFRHw5E9gp3R8ZbnN2DdLKwxvaeapJ1XT1FvXYcYd0F7jvbZm 5IBwJyT9PzB/pxGzevjdDOzkECzfCWi8SeQ6q3phkL7NiLFov4AJWt2tT6JelBlcILhT tsmXVtqyCKAOFsFJnUKcazNeM0CsF/fn71Ap1BsksUeEwB3wwckgZWcSqmBSW8czF+Wo GzR74obTxJCG9NzNPk/5hVRiG9kojjJ170F3eg4y4vMF4WfBV78A9/khBzstQsDJY5PX lfVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515056; x=1712119856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2OOp6ZEyIwMU5OnG6xTEWcirae/pY2nPqfU9WqSousc=; b=jVuHjk0tXFxbWW7F0f5ADZ7boqdfeQjHeiNa5lKSkYuRCTh11StGvbe0a3fRdnusJL SR0/rfu3Rjf4k/NINaihXbQdOkhT5Jv8DtPs1uGKhWaJ7sHyEeTZU8efuFHSKKp+nQay +JMKMDa1+l54PC0VnhWmQkvAm6hRqn1NEnWNTh4vawIGQKsKxaTI2c+qtZY+MAx2wuuL V2lZH4lrj9NcDCHcL550yAasdfOA3xcm3l2Kd+rrm7NOlOhT8/wyvYWfmtmCJGWxfVSh wLlqqeZrXlaxbmwrA1ebGM8tRq5jQqqRkr/MRwaL0GdYZJB5pci2RFrg13rJuvYz+zM6 k6fQ== X-Forwarded-Encrypted: i=1; AJvYcCX3x7YNcs0WkdglkH9TtPT/snfGwrjFhA0JbmfTWpCyxXJC4NOcajH65jsu3afZKPqWgA5sXWuXFgouZfuVvfxWgyA= X-Gm-Message-State: AOJu0YyxiApoceTUBdVmJV/4BNm1NPimWl4sFTOy+0ryUmCgnMVJ10gK Z6/XEXxnnRUj0ZsPyPxRdYlbL3yZoIhglC8Mty09l/NCA5K8tgjsneSnZQIxv/E= X-Google-Smtp-Source: AGHT+IH2eQoyDSJi6zqBjW5QAslrBjHf9N1FFSb9F+mmhA++Fk/xLCVtnw/qNQbwZOYB8ND1BkvynQ== X-Received: by 2002:a05:6358:7584:b0:17c:1e6c:14c0 with SMTP id x4-20020a056358758400b0017c1e6c14c0mr311341rwf.16.1711515055737; Tue, 26 Mar 2024 21:50:55 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:54 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 09/13] riscv: mm: Introduce cntx2asid/cntx2version helper macros Date: Tue, 26 Mar 2024 21:49:50 -0700 Message-ID: <20240327045035.368512-10-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: AF7891C0005 X-Stat-Signature: uzeaprabyicz61eqor4emq3gb3tokcgz X-Rspam-User: X-HE-Tag: 1711515056-543734 X-HE-Meta: U2FsdGVkX1+wjtdyzthjAtbzNtmHNDI6ZbPK55EHuBPU5LF+/nYN/dTmO1IRwKw+CTxkkRcA0FInKm80a6a51xB1F7Kv8VcMHeyjdtYZK9uwVNytJPqux4NqUxVLMsMdL7C28YbTLx7vvEQx9NapeGX2Df7T0bEC0epu2tMcgTT4ZUOFVmVbpNAaU4Y4EwKmfHCoFVvWvlWGhAdcvOcfo7FYmj9Hu6gXTvbthLboB1sQoOYXLto8wY/j/wQ7KgYwdMR7Y//tW9nexcg6P9042FJVIhBy4opQmMyWIzqLlzFbBU9MUcwHZF8k4YeJSzCOv7gs7JKjeCi02o5i6/xMvgXNDCrpeX1oEMTrv0l4HMMQQf/0HdZdeUSIt4tfLOgKxFeUh7rDN+H2b8bGOnJvJjXh1QlMD+7zttMLpc7nIDFdgdxx88tvqG30jrGi8+M54WuCofHaeZpU5WK0VV3SyZ6AbQHNMUpmUEbl9dGhoMkXtXPg/GYVlFYtnb5d2l7tQ6Wc8WItDjDTlan29t5eZz+ICwagpI7WXc+4nVaR1KY70ntvUsK07T0xxB3DxhQZJdYUAt6el+POI/ebZMGfS+go2jCCOJplrPZXlVkeguGR97G03Mz7JZMIPk3NJjnC1y+4usfbvb4cRPWmT3kadr9Y1ctgp2aTd0IbiWch8gNgUeKSw5XoX4bR2+JLZG1uZvBRfwXDyecc7dhrI+v5YC8QCr2QxDoVA+bzR+E36Q6a44ibQ0xQSLITfMEyO31HstAok/7284PRiC/afuf/E6kzbFB/r/rzo2MEOe++njeKto7UN4cuBBA5MHzU4D+6XP6vPlEysfqGy6pCi+f1KvMaf/mU6MowiUbx3MNr9FuK14wdHYw77qbG8jHr7ydum1TXmmCo8vyAIGX187xBv5ye2wn1c7fPd4MR41KMqUoQNosQ3mzPMtVukUIRdnJTMLDUMCubSlz+wU31AC3 ap5gExl4 3fwTDUvm87naQV6KxezfJo1B9f5h5deMSVKOVoFDd7YC/MDDza9Gt4UEqvYISSNfC8JVB4r9yjHPYW1lkHPWJtr5+7YYm1X4Y5W0j+HQ6xoG41Q4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When using the ASID allocator, the MM context ID contains two values: the ASID in the lower bits, and the allocator version number in the remaining bits. Use macros to make this separation more obvious. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 3 +++ arch/riscv/mm/context.c | 12 ++++++------ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 355504b37f8e..a550fbf770be 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,6 +26,9 @@ typedef struct { #endif } mm_context_t; +#define cntx2asid(cntx) ((cntx) & asid_mask) +#define cntx2version(cntx) ((cntx) & ~asid_mask) + void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index ba8eb3944687..b562b3c44487 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -81,7 +81,7 @@ static void __flush_context(void) if (cntx == 0) cntx = per_cpu(reserved_context, i); - __set_bit(cntx & asid_mask, context_asid_map); + __set_bit(cntx2asid(cntx), context_asid_map); per_cpu(reserved_context, i) = cntx; } @@ -102,7 +102,7 @@ static unsigned long __new_context(struct mm_struct *mm) lockdep_assert_held(&context_lock); if (cntx != 0) { - unsigned long newcntx = ver | (cntx & asid_mask); + unsigned long newcntx = ver | cntx2asid(cntx); /* * If our current CONTEXT was active during a rollover, we @@ -115,7 +115,7 @@ static unsigned long __new_context(struct mm_struct *mm) * We had a valid CONTEXT in a previous life, so try to * re-use it if possible. */ - if (!__test_and_set_bit(cntx & asid_mask, context_asid_map)) + if (!__test_and_set_bit(cntx2asid(cntx), context_asid_map)) return newcntx; } @@ -168,7 +168,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) */ old_active_cntx = atomic_long_read(&per_cpu(active_context, cpu)); if (old_active_cntx && - ((cntx & ~asid_mask) == atomic_long_read(¤t_version)) && + (cntx2version(cntx) == atomic_long_read(¤t_version)) && atomic_long_cmpxchg_relaxed(&per_cpu(active_context, cpu), old_active_cntx, cntx)) goto switch_mm_fast; @@ -177,7 +177,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) /* Check that our ASID belongs to the current_version. */ cntx = atomic_long_read(&mm->context.id); - if ((cntx & ~asid_mask) != atomic_long_read(¤t_version)) { + if (cntx2version(cntx) != atomic_long_read(¤t_version)) { cntx = __new_context(mm); atomic_long_set(&mm->context.id, cntx); } @@ -191,7 +191,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) switch_mm_fast: csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | - ((cntx & asid_mask) << SATP_ASID_SHIFT) | + (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); if (need_flush_tlb) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 18af7b5053af..35266dd9a9a2 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -110,7 +110,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, static inline unsigned long get_mm_asid(struct mm_struct *mm) { return static_branch_unlikely(&use_asid_allocator) ? - atomic_long_read(&mm->context.id) & asid_mask : FLUSH_TLB_NO_ASID; + cntx2asid(atomic_long_read(&mm->context.id)) : FLUSH_TLB_NO_ASID; } void flush_tlb_mm(struct mm_struct *mm) From patchwork Wed Mar 27 04:49:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B63EDC54E67 for ; Wed, 27 Mar 2024 04:51:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC4936B00A2; Wed, 27 Mar 2024 00:51:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4D0B6B00A3; Wed, 27 Mar 2024 00:51:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 902E56B00A4; Wed, 27 Mar 2024 00:51:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7D2C56B00A2 for ; Wed, 27 Mar 2024 00:51:00 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3ADB5A03B9 for ; Wed, 27 Mar 2024 04:51:00 +0000 (UTC) X-FDA: 81941594280.16.08EC3B5 Received: from mail-oo1-f54.google.com (mail-oo1-f54.google.com [209.85.161.54]) by imf08.hostedemail.com (Postfix) with ESMTP id 5783E160004 for ; Wed, 27 Mar 2024 04:50:58 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Mhwl5sPi; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf08.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=b1TH14cie+n3bC799DliJVlnqSVvrz8KednnRbZEoC1A7hWIpyWuMGGGfqMG/AQrwOB3/p NtjXWSYdAMS2o+LKnYizQRC3/1NccJ0EymvX+O33bDqXYRI1j4cMFD8PLgj6Z6Hgisp1iA M87QLQX7z0zi/YFxKkVR17WCtg5alug= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Mhwl5sPi; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf08.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515058; a=rsa-sha256; cv=none; b=Rp4HgDCo6pnX7bVf5QkP46ckxiHyipti71u6IDHsEedjlSMvmFa5TbtUHhC/kPwAOlBRhg QOwd2ek6fG0NnMgJKC9iqXGSDpME2xDkmABnJjYafhUKtBS4mddJXJgz6sPPt6sg5pyPNI a3jY6XWHHmYHCrSrTtJJe0tPkWhnca0= Received: by mail-oo1-f54.google.com with SMTP id 006d021491bc7-5a4859178f1so3414916eaf.0 for ; Tue, 26 Mar 2024 21:50:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515057; x=1712119857; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=Mhwl5sPiFAlAaqcwHLsIWL8i5cFJe0STLg+RHwUa7OOKTRaQNdfS4ZEGZ057G8Q9R8 GRpMWWfuH6zi+JdZPQQPAweZVCuGsQ6DFCK6+PfqL6u+3D3QnYlH6JLwU5IUmXgvl2Ri FchmTWAiAVW7KuCzMs5+Ea89gEJ0EFCDLSgvryt1HrMY+JHOgRfML2xyhQ7yn0qCYZoF weqk/2yaAdTSvgX/bPyEjrMz8rpk+R2ThSrVbyfdh3ioa0VFuPBIYbtZO+/X08YY+AaL xf0MxIqDFokbp7HMP60jXwHxMOkwyyTZ9syR26+6+zaFdzm7zQmrHg7zj1s+cigo8v+w JZZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515057; x=1712119857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A+Ju3+5NsQ7zqLi78mVdBh82QtKzTZZzrfFOA+rFDFs=; b=Q/HEJWbSYOmlG5BMOTO4rLgnLXzDmVzukydXjAWvkW/pNMaYu89fQKttpQFX0vKCUI /a2PBeE4eYQGSp1E2FQTNzrAbNBtwuzzAPC/8GlyiSVkKJYrzAQUbePoLjBfxGIlQgGD azZqBpIyNL3TJV0RLXX7N4fbu1QRAPTRkFhchfCdqzF/SC+/C92AI1YanqPttdROFBtk btXaTgZhvLtXJg5iseaXcR4QertdIIH63AuuznFq99Kc0Mk9i+FG8kw8Rg4GvcEYvDqL nRAzKIvMUtKGpdYLqjMqmZWjhN8Td8fWOkbcNsKW0np1waa4Eg3DyhlBCkGpkaQ8/l6L CQPw== X-Forwarded-Encrypted: i=1; AJvYcCX7rBIUvwqF2HyfMrHCx+yISDnyo/PVxHKUhsqSpJyRZBjzGkC7LzR4krKrxRleS+oPAJ5hV/w2/+WR76mKnMutngk= X-Gm-Message-State: AOJu0YxfxlOUkg3N++yJ38ZI+YinEvsWSJJbmtNmkk9/3/2brfH2VNfc gWLNwkUJKQ8XtUCyDVYU09HGaNw1quGF8MXzKWtkZD0BbJQYcpwOzJleEv4Bqto= X-Google-Smtp-Source: AGHT+IE4clbeICIX6LaCSNcjSNKRNzkyOLoTskHKcKR+t8ek8fOhgjRruwacqNmtd1HR0FedGGJk+g== X-Received: by 2002:a05:6358:6397:b0:17e:c8d9:328 with SMTP id k23-20020a056358639700b0017ec8d90328mr5244771rwh.11.1711515057489; Tue, 26 Mar 2024 21:50:57 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:56 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 10/13] riscv: mm: Use a fixed layout for the MM context ID Date: Tue, 26 Mar 2024 21:49:51 -0700 Message-ID: <20240327045035.368512-11-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 5783E160004 X-Stat-Signature: oo1gj9igrjada4afpartrt6npkqjxbcp X-Rspam-User: X-HE-Tag: 1711515058-682732 X-HE-Meta: U2FsdGVkX1/J3SHfa2OB0ZIS6F6f92Msxpa7uXcZykN7c3gayViHgRDKpmtKCmyZVyLXVLV9XVf+Dyc472xvYDaflxaR9Avy9aX4zSC6e99DcpgBYPgqfSV0DejPGkVBKAxv5gkrYmtZwHwRhgoqhLSg1sjed7htfUSefscRAUuzcNGxJ9s2ENMVcL6T15Mkecqb1amzGDGEXskJ8nsZcPhFwePo8T4otGs8H4vEdU0QLQBsy8vugB8EbhWduLXjMXuyu6L/0VTXmB48GYyuhLVaevC0ex5P++w0F5VzeTH+Kh3qp5r1AfZ59RicrR0TgJy7i74m9cvHW0aQ/AsWWFyMWv8UTcJZxm4+LyzwDbkYof9mg5sdc16kmeIkMHd7Gdlvm1OvGEmZKUQH43/823R3yyLGSyX/wWYdTcFjAyRj5pcw6CDDqDIWx9A+ku1DWFAZeh96bBz0UdkbBkMixSPQz1w2pu/YkBK+kCizU/uvqjjyVTylkveU+BnwWkT+NpwNMv/nM46/stHbJTJc3kcqu7aNjARZSKIj8BKIpwgHI35UbZGZ5r58SDluU0Qxh6JVZvRG/O8Jnk82ufrPkNWYaAOHRP0eBjiP8SY3ykaIUagfQbzchQQMXgepthx+c57LoJeptHw/S4rSIcMNE5fsgkD6Bhu3tK8wh1RKytinlLA/TrD+tSsMh4Et7uvdivBgOERDMpO4oXYMkzw89DZrHwfA48cV7yqx1NshiLkpOCdsgHhMV2qj375ZH1AvHZaJfv9dawNHVT0ewRxPLxYcOgy01IGWDi/+EUrtwObQmSQKfyCpmjCywz7DAj9vMMGb8p2QY4JJ3DRScyR4j4Rsv9OlA+I+cV0gcDozXxbC3qooqz8zkZafnNXulp2QC64fTPacKZaLDqU6hWwh6MOdyXM984vCUqQpPQeo5BnPZW5PGrIcgSj0j668rfCnTrz5qHr7O/Rreqle6nY 8iP7Skvp IIipDh663rZ7rwn0z/9pEk2BKCjow8f/ejaauWw7oCU+1ISkvVhzLcF9AitUufPG1siCLYL7CD8eivIrkujJcc/4iw0yPrer3rmMUkeEFpd6ePLjylap2AxZYr/fx0q3xGvzZ1htAM6USiYRct3wpXuvhzWXv4aPx7Xaq7O/RGtB/v6ufKmcWyI8QU7tzlh1o/yCwhQJ/3qMq9rmuRYFd+oKTjOXJMm45EvBBqySFRdfRuxS7h2B5zDWDJstrXKA4XPulvDe5fvMDZKHQjRA3Z12cR/hiZ9zPc+pjgsX4cp/dK40nAzKi5fHfO/DPUwvYJcm8Zo5XIlkIFdy1c4nEw/YQqjK/xVTOS8I5ygzLMMEONt9ueLOORgq0DWbvDCIYJZcoxYlQwqtKdih+ZsekDYXYlKsoqo0c2PsdAIJkQfHx7oSw2OFzh60CXtefn13+rxeIbWKZcK8og9k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the size of the ASID field in the MM context ID dynamically depends on the number of hardware-supported ASID bits. This requires reading a global variable to extract either field from the context ID. Instead, allocate the maximum possible number of bits to the ASID field, so the layout of the context ID is known at compile-time. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/include/asm/mmu.h | 4 ++-- arch/riscv/include/asm/tlbflush.h | 2 -- arch/riscv/mm/context.c | 6 ++---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index a550fbf770be..dc0273f7905f 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,8 +26,8 @@ typedef struct { #endif } mm_context_t; -#define cntx2asid(cntx) ((cntx) & asid_mask) -#define cntx2version(cntx) ((cntx) & ~asid_mask) +#define cntx2asid(cntx) ((cntx) & SATP_ASID_MASK) +#define cntx2version(cntx) ((cntx) & ~SATP_ASID_MASK) void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8e329721375b..72e559934952 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -15,8 +15,6 @@ #define FLUSH_TLB_NO_ASID ((unsigned long)-1) #ifdef CONFIG_MMU -extern unsigned long asid_mask; - static inline void local_flush_tlb_all(void) { __asm__ __volatile__ ("sfence.vma" : : : "memory"); diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index b562b3c44487..5315af06cd4d 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -22,7 +22,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long asid_bits; static unsigned long num_asids; -unsigned long asid_mask; static atomic_long_t current_version; @@ -128,7 +127,7 @@ static unsigned long __new_context(struct mm_struct *mm) goto set_asid; /* We're out of ASIDs, so increment current_version */ - ver = atomic_long_add_return_relaxed(num_asids, ¤t_version); + ver = atomic_long_add_return_relaxed(BIT(SATP_ASID_BITS), ¤t_version); /* Flush everything */ __flush_context(); @@ -247,7 +246,6 @@ static int __init asids_init(void) /* Pre-compute ASID details */ if (asid_bits) { num_asids = 1 << asid_bits; - asid_mask = num_asids - 1; } /* @@ -255,7 +253,7 @@ static int __init asids_init(void) * at-least twice more than CPUs */ if (num_asids > (2 * num_possible_cpus())) { - atomic_long_set(¤t_version, num_asids); + atomic_long_set(¤t_version, BIT(SATP_ASID_BITS)); context_asid_map = bitmap_zalloc(num_asids, GFP_KERNEL); if (!context_asid_map) From patchwork Wed Mar 27 04:49:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65309CD11DD for ; Wed, 27 Mar 2024 04:51:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B3F76B00A3; Wed, 27 Mar 2024 00:51:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 166AC6B00A4; Wed, 27 Mar 2024 00:51:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F20166B00A5; Wed, 27 Mar 2024 00:51:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D5A616B00A3 for ; Wed, 27 Mar 2024 00:51:01 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B76B91604BF for ; Wed, 27 Mar 2024 04:51:01 +0000 (UTC) X-FDA: 81941594322.02.CF0EE66 Received: from mail-oo1-f53.google.com (mail-oo1-f53.google.com [209.85.161.53]) by imf26.hostedemail.com (Postfix) with ESMTP id 059D0140011 for ; Wed, 27 Mar 2024 04:50:59 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="V/E56ar9"; spf=pass (imf26.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.53 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515060; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=IZSn08tZn/nqXYbHB48w2c6c9UyaI7punNATcqUDuyeBP66WmtGrOLpc27pndO47lUp0bP k6u935odLlXLNimVAGf/9rojwy0A8LL/Ci5Tep56cxsvqaVMoyJygtjpIrNLKJ1W+EGxzb 1YxdRtI7Y/P03p6cRXGP19mX+GWEPic= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515060; a=rsa-sha256; cv=none; b=ioEgNYtQYjdR1Wxle9lKjJCDyswIu6Tw4+RLhX0K4l6lqpqD26fIS+7jknXkF3cFv55qra uPFUDT/cmeAuzQ00PyXbyRAlMjGa+uqdw9Jifx9mqpKG+kXU4Mm+ebELZ06EWet71Cd7y9 drVkDm6s90O9yRSuCBHOMuA/qqAwTDY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="V/E56ar9"; spf=pass (imf26.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.53 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-oo1-f53.google.com with SMTP id 006d021491bc7-5a4f608432bso3402264eaf.2 for ; Tue, 26 Mar 2024 21:50:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515059; x=1712119859; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=V/E56ar98qUUlViAtm8vQI3s/LcuVvYoAqwzyDlV31Ykb+nykT0zGbW6plsNWh00C+ MfGpu4dZfHhj7+19ghRkAIIK/JLY16ILE9aF+YEO1oXQ3eQwGk/hiUTPqmk/+VDeZwoq evRDeFXNnEpVDm5mQ8D5r/I79wAIkEqXdwExz87k9+eFyYDog0c382mR5Jp4CdJCo4gX Do6om8Kneq3ITL2HxinLDORIVeg4NYDNS5y6Ql6FkNc88GbVPwmFfqNjSNKApSA+uJ87 5Tsd/cA5kVbB1YHhrigpJx6n/e93hoRfiCktjP+mRXWpPxADagW+lgvrVcV6jFaxWV6i GRTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515059; x=1712119859; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4wFLc5gcnBs285NRnicKVmmQVV53EnA/2rANcUM2uCA=; b=E10/ZaqwewNl7mfB53qRfXj1jUWOSFJY0u9cfT5DuxBFJPyNioB+VrUq52tSgXDDAT ymoDXN6gZeZF/OxkySBD88p3MAtuk9D3Y0EaB2RVjAzlxNacJH8FDk2eocsyUF13/KCf U/h3tE52d77EiphZNm/1/ZXZwiCIiXVxm7YzGg62RkITrUDaeRv9OvCqOqwlux1j+PFP ykup073jYx0ujlfTaiQCfKIJvc56qIPY7eurzmdxRqIAnx5NdpAiekLckw6z37YUCBG1 aB/nzVvd/DXLl6el9mDacj5lVc5XQg9G9eqbcZSpQsh1uOSIbP7YxmpqRwF9Lvf7WnjY eLcw== X-Forwarded-Encrypted: i=1; AJvYcCV383Po3ZDi+B3egVjeMULClPVG3MbjiIDzG+W9nCgoEcQrS94RpDgOHoFOlDTMdOgjMav0RtBMBIDisSAO0BBNbKk= X-Gm-Message-State: AOJu0YyS2UqgHJgfhUJUCZvBXlE8CAGSm9IhldVb1v88TDN3RIZmSsUo DTsSa6cm9B7jb4uAicS0EKY9APu8bbaLLjA6JeFuo9uSmZA5F2cOK3UflxFC52M= X-Google-Smtp-Source: AGHT+IFmC6K+A8yipt+L5BT3VYhAZMjFopo36khv4UzVVH+BHr/EbytiKBlOS5fSbYkXOITYtqGpvA== X-Received: by 2002:a05:6358:320c:b0:17e:bb52:927f with SMTP id a12-20020a056358320c00b0017ebb52927fmr1922247rwe.1.1711515059146; Tue, 26 Mar 2024 21:50:59 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:50:58 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 11/13] riscv: mm: Make asid_bits a local variable Date: Tue, 26 Mar 2024 21:49:52 -0700 Message-ID: <20240327045035.368512-12-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 059D0140011 X-Rspam-User: X-Stat-Signature: jigw3y9y1nbryzt8en9codtixbtiume6 X-Rspamd-Server: rspam03 X-HE-Tag: 1711515059-731061 X-HE-Meta: U2FsdGVkX199j1I343cMA7WGxv1nf33gRNLuiS4amJAkYUw7SqMsOkUQ7ZJdOzGJKNZHyOMgYjbm/qaouVmBVh0/arJxjwstpu3lZ0KzRPoAVzj8qnIZ/0SGoEkvMYlnzYavzDK7ueoUO9rXvIxBozoASsZWEDjF8untQcDTyGWUs4jX5Cc35CMNxAd0PfuqAwXmuh5JSzbER5oaEsmcuCfB/IeJVlsDbxZA/qH72srArPA6zn/UcCGDTE6fWtfc/icFgiccmYY++Mk0fw/lvOoktk4Ave/iXwxiN1K+J9IdieB2UeQmWBRamlLxcOCV0KeYd21nzPniKOyOnYKawGaXQASEHPgtt00iCPOJoBDRJVEUFGVXuw3wWRe3YIC5ejVCd+iGEzoXkVA57IHC7a9fv6yHBZbusj65jt1QhYrzzipxgIljf1aBnWXXgwzF9kpcFn3Gj/OHNxipZEpKkYCk9UuJeqdRIPAsteYNUBmJph72ME9VrUq1vtI6RUHAW2OVt0HHjzorFc7MewrSeimTZxOBzT3G5PJmbGnKeI1ky+thoD0DSOOBMBJA6es5OXAxsK58Vp8NZjk/H5WG73PZPPjtAaCB1cddqP+kRTogkvI4IO901NZr+3HOWEIvHU0vQLx92+78usZNVL+1Pse70WfqE9sXSJ1zcGt5XliFkMck9z9CfHCIrW9ScpGWNYtt/vgvXQspt+3/0YfD1gV2wXzFLiSPhleL9LzNZa4RSoO9gbaT9XHTJ1kQFf/pQmgwVgPdVp6Yt9+MUliYa12ZOJXHi9NdxXpK31bWh6RBppdqgAuIj1kLOpZxNRXuXinA47FaUi6HH7ER5EU7s/VDKQUI4KGc3hhcQ8xYPbhJcTBU1/y5eXSMYheZ523jU8j+RJZkdJt6VhIhQwr94GRtnZk7jpsFudDXmEMo5lzDBymoGaETwMbLMbgPkAIaKBQiY2vUcHnyEb/yt5m eXA3e5ba zaumxMhoOfMksWrFiX2K5IhDVMhjLheHTgfHzS4lWxGXGZm1fZlX+ExBX6Zs7hARMFErg5YdgiNdZkiQu3viPMhgQM+Vcbie09+R5eaa2WzKZwpHlWHbwbYTKy4Jkt9M3noJ6EFcfYtJtCrJiGNE5qBO2gR44fmk1HEi+kQG9gR9reloyLk2TmiQ5kJlqKZIBNG9JQGl6k3kFEo/FlWV9xmPZLv3KhZl6Pk1MMMYqC+ldof93diYYmcLqSuRg35k+5nG2srsE+Dg4CaCAEMdDfR1BsOH8pXeK0oY4e6rfDIeTEaak2dBFBLHzNWVJ0gb9bndo1LfYPj7VPJKbCUPnvX04ib4u6VkSO8z2+SD9gjakG2lmHne2/jSxPn0zCVcukIX7JpH/WFfOYZDFw+jsfnypP6IUZmDpofxW4SeSUvUK96qV+GvFitwDJW0EDkGs57PvO0wpTukmlacVfAOnW+hWI3qxXdZToMIzXu1vQQEdA2H5N+VhmUO1Rw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000134, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This variable is only used inside asids_init(). Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 5315af06cd4d..0bf6d0070a14 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -20,7 +20,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); -static unsigned long asid_bits; static unsigned long num_asids; static atomic_long_t current_version; @@ -226,7 +225,7 @@ static inline void set_mm(struct mm_struct *prev, static int __init asids_init(void) { - unsigned long old; + unsigned long asid_bits, old; /* Figure-out number of ASID bits in HW */ old = csr_read(CSR_SATP); From patchwork Wed Mar 27 04:49:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56721CD1287 for ; Wed, 27 Mar 2024 04:51:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 953E16B0087; Wed, 27 Mar 2024 00:51:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B6256B00A4; Wed, 27 Mar 2024 00:51:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70D1D6B00A5; Wed, 27 Mar 2024 00:51:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5E2F86B0087 for ; Wed, 27 Mar 2024 00:51:04 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2BB9AC0522 for ; Wed, 27 Mar 2024 04:51:04 +0000 (UTC) X-FDA: 81941594448.02.5DF5926 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf17.hostedemail.com (Postfix) with ESMTP id 4F7C440004 for ; Wed, 27 Mar 2024 04:51:02 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=J0lHBFYk; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515062; a=rsa-sha256; cv=none; b=q80kLBBKpGaZREUSQAqGbpXKoo7p1k7zlkpKqpeaUOACN7pSYmLjDIJfh/vaqeFU+kO9/g sIP8iwGBrrnRGarn/JEy9/QY36MFsPxOcIBmNJi47wb5vnRBpJNiCM3jMckOaJ+nkcxQnp kb8cghIE3GYNe3ilsUooEQnoy410aWs= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=J0lHBFYk; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515062; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=y4+yMeNMv+SKatmOY+cSW7uYezHcT6XehLlzoOMZSfSwXUEmtOdH7o+WcTw3ZaWy7qUm30 9ZjrHxjvE5sYNiLSIKzd6YbHdfc2ecjxpKtMAyQ+tXMRnYy3GItopavQMge7uqFVSarNWi gozK4xKlveubNq97+5c0LEWod5zFE2c= Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6e6ee9e3cffso363816b3a.1 for ; Tue, 26 Mar 2024 21:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515061; x=1712119861; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=J0lHBFYkL1lhlgA7B593N45jwgsw2M6jhvq62k2Nfnyal8ywVjNRB1lIf685Ev2wa+ X3QR3+h0ytTeNHSvBQUX1ITyVu5R94P+0wkueDM4wdhT0iGUVG+Q2lXfALqB9aGM53Fd LSzzuoDnPiR0y29/6cjhxed5BplnsLq4+Kx8V/ifWbRGJ0VDcCFFin6/I3AlUN+nsfbk NM9FIyYh9KepKm3cxMuCfL++uMhJ298ocFlVzFZMUlu8k7n85GpH46OUQMJ3ckfyCtbC XzFop5lTdD/ByZqqIKEt8VlX1f5D80jXYSqkFAPyZNSe4FiCm0YIyEkpFMTjVHzm3/Pm hpgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515061; x=1712119861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hwpkv2OuO7YHUplKH98oFVfOa+CFbclxXNu0ysmduyU=; b=pW9ttMaJoWg1SNBXF6nGnLqyp1vbkWM7JOeUIVR7jiZRNRpnfmhAnEaJSMBiDiVp5h jP1t962briidBd3hmBIp95om3G87ukBSOL1/s8JGFETH/YwrVcZkWFgZ3j7Q/QpxaBRX odbwN1Zt4bT2OIEsPhJWA6X55h8vWrafwIbhdB/C6bAmRrYDY4Ojist9t9TMT8aXRU45 TnhiLOShNYzT3lXv9c91ZKI6yjJeOCWaj/l1bUdCFH002G1pHFopTVxADuxQDfjfDP6k NN+mYNik6vnhdeL6aciaH5iqH13ugkBcp1DogS0pm4HGYdux1bExdttdjC55/jBwsYFw 9i7A== X-Forwarded-Encrypted: i=1; AJvYcCUN/izBS49+J6AUIuZ7nt58bdCAtYzH/KCUoc8ZN7OVj1jJjN/sdvh1BwMYL05/NAxivGK96meVXHJdAMCjlLgayAg= X-Gm-Message-State: AOJu0YzL5ipSg5Vk/2ETsor5RPazaKliizoSGWXZVlE/CCxV8BmKpNmL 5eFqM/Q8c666safeDlQ5UvQb7ubaKDRGUTOsKOarGWlUuL2I4l/ysrJrDab2KkM= X-Google-Smtp-Source: AGHT+IFIatp35KNpgDUDmJycAxTP3nvYaAVXQURqJcQtx3/AD7qiy3AeB8dChiHf0CcNcubYBhb1CQ== X-Received: by 2002:a05:6a20:7345:b0:1a1:67c0:c751 with SMTP id v5-20020a056a20734500b001a167c0c751mr5157753pzc.17.1711515061281; Tue, 26 Mar 2024 21:51:01 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.50.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:51:00 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 12/13] riscv: mm: Preserve global TLB entries when switching contexts Date: Tue, 26 Mar 2024 21:49:53 -0700 Message-ID: <20240327045035.368512-13-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4F7C440004 X-Stat-Signature: mmn4pb9ww7muaiek3z1ahocc63s8uixh X-HE-Tag: 1711515062-191856 X-HE-Meta: U2FsdGVkX19DCgPlfj4yapVSHMKrOdxN2E+yolOl0GhWwp8ZLzMSP15fJ/bfso+asblKwAHqgay4Jm81+kXAABhrXRDXedfKpssY4Y10drjEBxXOIEKiJsYzCtrXaQkPrNmFOru9j5J5UF7wmfbKO3c9xf7fvEybkG3Xw82YuDF5OcJ1zyD0lnVTclbS+Y5kwdPQQTNiZgU9t8nTFm7nSTASDgDSrwJ9OpGpo5mmH/1sQscyRinjvwAViZy2za0AdQYXBr9vXFUJdH19VhK9MJyyg5JpMDfv/IZHVYJbDB86Snyz0nWNHhKKQTsi06HX48bf2qeb4HeKXoeZpPDcrQs7GmvOWANf8R5v7PjFSV6SXYJA8IBnqb8Fy2WFzeeLGZfD97oo8jpRFr6MrTkCtetTtaFCVkg9EpcKS0p6olFcl3hKbwjoqTKUllAMHEGAYAw5t7Xmh8zbb8Dq7YqZK4Plbi4Dp7xolDUIVXxS13a/fcEREd8YR+LBW38cVbQskxlzFNLSFmE0XWULCMiDmhAoNFIkdkfxn2lP4Z7FuZ+FSz6CUzGGGUpfeCRVFCcxbfm+0F7G8/GIOJtpu1oyvHgw+TIsND99Zij+fqJcy7kQqk2M0o6j3tHHvlfmeqIwP0I/jNfuK6mQ5qEdyL5IvNffCG+gV+yFi+BrLcAZmjW4oATmd7id+LIQZ3w/DoTaun7PFFy0+punjAkIR28H7S7XseVlqSszMW07opu0pVAiZ08xTbv8lPKxJAiRAfhGfPgJWEExGt2FiP5JQ0lgZ8N8KZFjSOkRtmmdsbmpCzL2fEdUdtI1ofVODIE0kKYKZHQCjvuECaZAHXcC8Pq7Yy47LCBAg492WKJXwarqrjCcWo+RQsajOr4VwjUSA47FfQBYAxp3uTwvSwnBEcmUNTAY2lUsq2kpRn7NP91O0jTLzaGLEa/KJ7zncc/Cljop4vcGLXj2rZYkx+ZloFb rUYCjYYq PNE4o8RMUbx+h8pIYIAJBgKwpZZKMMunFI/Ovsa4pX+XctR+oFG678OnlFxjjTL+bZwtxfPsXojmz3bcnX8nIiq8F/59ZVspjpqW+ULBMYUxdLbe3T0AkVp6RolK67fYoWGhgai0VaM3dg4cHRt7wCtuNNy5z7JfcQbgtcHM3yxhs3C8a3C/q+WfGwzKIVIEwX0i7Kybgh8zL3FEYd4yHZ99r7ppc50vZ/PKDNBk9mPwYOSq6U6YU/UlhOXl1fn6LyzBDhjjc0tCqkrFhYjEOvPcoDBtZvqIIrldQA1JayVpIhWej2S/mBj51EBlN/MO/mdLl0PG6cidGyFLRyviqkameAUGHIo+6+PUuxY6cRJAWNfUARS+K1sHCv1ypQZ91r8MeFEoHu+KWzg4+lhS/wyL3siC/2/xM4znz4vymXttWDWEx7qC//HhJLekSFqGWSQzKP82b+FLRTPA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the CPU does not support multiple ASIDs, all MM contexts use ASID 0. In this case, it is still beneficial to flush the TLB by ASID, as the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. This optimization is recommended by the RISC-V privileged specification: If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. It is not possible to apply this optimization when using the ASID allocator, because that code must flush the TLB for all ASIDs at once when incrementing the version number. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v1) arch/riscv/mm/context.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 0bf6d0070a14..60cb0b82240e 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -200,7 +200,7 @@ static void set_mm_noasid(struct mm_struct *mm) { /* Switch the page table and blindly nuke entire local TLB */ csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | satp_mode); - local_flush_tlb_all(); + local_flush_tlb_all_asid(0); } static inline void set_mm(struct mm_struct *prev, From patchwork Wed Mar 27 04:49:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13605557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 760DEC54E67 for ; Wed, 27 Mar 2024 04:51:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C89D76B00A6; Wed, 27 Mar 2024 00:51:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C10B46B00A7; Wed, 27 Mar 2024 00:51:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8A0C6B00A8; Wed, 27 Mar 2024 00:51:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9287C6B00A6 for ; Wed, 27 Mar 2024 00:51:06 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 61F2B120639 for ; Wed, 27 Mar 2024 04:51:06 +0000 (UTC) X-FDA: 81941594532.18.B1C1949 Received: from mail-oo1-f46.google.com (mail-oo1-f46.google.com [209.85.161.46]) by imf05.hostedemail.com (Postfix) with ESMTP id 695F5100014 for ; Wed, 27 Mar 2024 04:51:04 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=fmD3YSJ3; spf=pass (imf05.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.46 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711515064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DStjBcAn1SHrL/HpDMugRz0dC8r4hdt1hrlyujsEs2s=; b=6tooMdkRfwdocNeMjvvnDJhQehOXTbffi8DvYB9xiT0JMKFCWd3a+57/Wy/Zo9BD5dcpMb XtePzh7TpuCmrJjQenN43eY51kl2C1WUwRJKaoDXlih8b+2dKqGXZc75WYm/SLMDRZdkh+ IPGltFUvXmISiKB0Z6+6YK+RAiJWkHk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711515064; a=rsa-sha256; cv=none; b=wv0EA+jt0dVOlgW4dUKziD1TAc8FOR71QBXDtZmqL/FwMKwA0Qy4f9vLbdPBi8dDfa/zUp Ms8nFMF3u5kVPKMDtrDPN//8SYvNnz+A6ll8FXf1O5ZUZUlHQ7rNuGkmgAJSVsRoh9I9ue ps7L1lfQaFzUx765nyDyiqtPs4pyULY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=fmD3YSJ3; spf=pass (imf05.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.161.46 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-oo1-f46.google.com with SMTP id 006d021491bc7-5a53d5aaaa4so1681047eaf.2 for ; Tue, 26 Mar 2024 21:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711515063; x=1712119863; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DStjBcAn1SHrL/HpDMugRz0dC8r4hdt1hrlyujsEs2s=; b=fmD3YSJ3lU0jJPFiDanbprmiR/8i15zQij9VLxUuGsArNxmWD4gIJv5U+KhPheRKrB f52FBvUzGsQ6pu217XweB7W7Ug2OmtqaFW/RpY57AyCbk1cGXP41HlEXC4Qd9sffwhYA beuYXRoqn1dyxrwrx7CNu4X0U1FgBevTK+p5mwN0J82b2nXG6DHHMH3Zmkh2hB2hKJRw OlkrxuFmdBQ2IP5uLd2tY8wKsHspY77ZhlRLR/nWhpogCSSwjWPWdE4F6QDBh0SKwt+H 4/f43mJOQmtuVP21VPtvidzLQRtkr1SwCSfj75Ogg0Zol4Ppd9yYVSZHSW9Wsz4urXmH O2YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515063; x=1712119863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DStjBcAn1SHrL/HpDMugRz0dC8r4hdt1hrlyujsEs2s=; b=mfO0CtbMKT8NfZ2SoBZ6BmRlQf7qIORQnVNTKjRjXkdII9FfpUtPa+IswNUnGPH9I1 cNfVZjNVNo7pOkUavuawCNL06Z35T1Ih5eRaf4Ds24RYBW8ZIhfi11WUCS6PJxYMOSZb lV1anKH5IU0NXSwGzQNzOW1BkGDM+N0Yr54XIGLOta7VQCLauoSiHxD2+sB46qg+cKNK YIfJb5eVzKGJNHOE/L9XId1rgPekm/ZtFgvkFQS1TCqa5EcpDJbpfBsnndhvIxJAME3T QK2LFuk+mfKGuDfbLtw9PmoYiKO54VIkJT3KgemYOy6FiugmqY+YNyFvLTV5ZAo4hENx bQ3A== X-Forwarded-Encrypted: i=1; AJvYcCW8FYN95OS274pKHEMDe30omaiXjx3bQ0kOwmtsSpBd1kQN01HjQ4og3BleZxjnRerJaWjDUB0jQMjxp1zlLL0zaDA= X-Gm-Message-State: AOJu0Yz9w5EJ2Tgrgh4Hp+QYIo6eNkdpxr5yHOpWx7HiHnqfMGPmEG7k qyn6YKg4mJG6+8+ao6zZcq21ROBfRABT4GlyaiRWGm8v8+uqPAo+XewxqacPiqQ= X-Google-Smtp-Source: AGHT+IElUFan01DMTcKq7mpLuQEXv/M6ZjDhhp1pW7v7cPBwp502ELftTLKlJV+xiDuinHxlNWw0zA== X-Received: by 2002:a05:6358:520d:b0:17f:1d34:6253 with SMTP id b13-20020a056358520d00b0017f1d346253mr268628rwa.15.1711515063614; Tue, 26 Mar 2024 21:51:03 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id e31-20020a63501f000000b005e4666261besm8351500pgb.50.2024.03.26.21.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 21:51:02 -0700 (PDT) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Jisheng Zhang , Yunhui Cui , Samuel Holland Subject: [PATCH v6 13/13] riscv: mm: Always use an ASID to flush mm contexts Date: Tue, 26 Mar 2024 21:49:54 -0700 Message-ID: <20240327045035.368512-14-samuel.holland@sifive.com> X-Mailer: git-send-email 2.43.1 In-Reply-To: <20240327045035.368512-1-samuel.holland@sifive.com> References: <20240327045035.368512-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Stat-Signature: fngknyfdx9dnky61d459njiismcmhw6u X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 695F5100014 X-Rspam-User: X-HE-Tag: 1711515064-494082 X-HE-Meta: U2FsdGVkX1/UZsqCyEjy+gI7GEGhfShB1wHpzuAP1DmY5Rq9InNwKsVVvaWcTaLiMWNmIiV45MjgvlgCDcJ7Vli63ScO/TFP/0K9HPx79IKoMJtApGm24qoP5OAWcDHHbKVw/ye85oKqd2UMsHaaRHq7PjoRgLxtxFGx429ruVMtoeYyRXQ7cNLkOzX9GnSteoAppl8ZDDCv/37ckWCc9C5qIOiPUqRKtvIX0dCxG5Kis3suzUPNM3Mwb7ktcLcyJVZEIk8YwXQTHJC28pQweNiJSmyb641hneLicdpcfEwZsIxSz0f6WgtMiOWRT3CW+EKnZBtPmbRetyHLLV2zf+gSMhi6u+k9KYdi3Tur9fFwt2Im7QoKdCc2D9qbuu39il/E7wBdVQ6gYd31QfiwYdJIxJgIEvYygleVFjKcqNxt95A16+zXVjPKfvuYfzAID/vX/rcQQ3bCvc8QnDxU9DyiVCeW9rrwzOndhN6zDjGXBDTlhRc7LWE4f7hOZspcsLSgzWRkwIeHTO2H8kLkX3FXYvjb/E0uV59wv4uwqFlO9ufD/sbKHar9lZReKurpUgAiUhn766kkIBhZsIKTvheceN2GJWwgkO+nZ7x6EtnYy6tYm3WjZqcaLMCA8C3sZunILQEFuMZYX127SNXKn48oStAr70RANNT8cOfDjOBZbrEXp4CRpFb55i7bbcgnsiizKRIXCyTWlv67jUfXT6Gg77G4zpzpvCWdlvQL8hTmsXDGXimYQ1PXxskdqxhl6yXGdhFDB72Ij64TSs/uXDj8RL0MTutlApXR4oscu3ECmuzfbNF/CyuKQKgxT1KVkhcmeN1ygaMj80KkB9/EC1bZnKp524JBFkvCuL8y59EF0+UQwifEUvNAwHpVwrC7ikhDjfPu5N7UUwLn/ma6qWrBQA1rwN3fuL3NdH9v4jWxbJQYApagkJB0GZFhb+gVW7R2rxvG1PY3dNPxPPI BAwwIUpN tWfvXa3S7+DeEQf7R2PT8/RVmsLciZ/fYNbVrqUEo+S2ISM1sDxE9wqVJJm73snXNXvz/+tussbmQZL9b3Su37nt0U56jsPdWqp5fDpve7tKxRFc/Gu9BfJzTA6sTjFs7Ic1S3uJtuX5mUfFy+fl2AsP/twfrIPKlFVa5a85vU3emuMBFZNfUtvx9KUZZlMcIgpcO68qMoWZ6qslVAE1wmBIATwnIRYy9r1viDXtk48M0Ds238uBtoZOxumuozzrtGslP4zolAZ7PZ+xC6WdkOI3bhkQ1MnMp+ScLLtm7R0vr0gfsJ3EkGcK9/ybmXSyQqufGxvLz5jqR1gBIO0/MVoD7tUsXcfJ0cVjo3uIHjoJb6ryUBIwKRV7W6GOgzzB3nsFR3Wm1AqZCFc+USGR9xGXmVRmBeRN/wVm6PbLcUWedLacmWrXFLwf2Q72waug5soGHfcMeSMoESXc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000391, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Even if multiple ASIDs are not supported, using the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. So it is always more efficient to use the single-ASID code path. Reviewed-by: Alexandre Ghiti Signed-off-by: Samuel Holland --- (no changes since v5) Changes in v5: - Leave use_asid_allocator declared in asm/mmu_context.h Changes in v4: - There is now only one copy of __flush_tlb_range() Changes in v2: - Update both copies of __flush_tlb_range() arch/riscv/mm/tlbflush.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 35266dd9a9a2..44e7ed4e194f 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -109,8 +109,7 @@ static void __flush_tlb_range(struct cpumask *cmask, unsigned long asid, static inline unsigned long get_mm_asid(struct mm_struct *mm) { - return static_branch_unlikely(&use_asid_allocator) ? - cntx2asid(atomic_long_read(&mm->context.id)) : FLUSH_TLB_NO_ASID; + return cntx2asid(atomic_long_read(&mm->context.id)); } void flush_tlb_mm(struct mm_struct *mm)