From patchwork Tue Jan 2 22:00:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73047C47079 for ; Tue, 2 Jan 2024 22:01:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 634E66B02A4; Tue, 2 Jan 2024 17:01:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 596DF6B02A6; Tue, 2 Jan 2024 17:01:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 410A76B02A7; Tue, 2 Jan 2024 17:01:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 309A46B02A4 for ; Tue, 2 Jan 2024 17:01:41 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 03A3D401A8 for ; Tue, 2 Jan 2024 22:01:40 +0000 (UTC) X-FDA: 81635743602.19.BFD029F Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf17.hostedemail.com (Postfix) with ESMTP id 08ECE40006 for ; Tue, 2 Jan 2024 22:01:38 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=UOhooY0e; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232899; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Sx1CrYCYqcy0I3lbTucfKtzktUZzOavzb19InQrqEBM=; b=0nWYQ8F29fbp+4bKKScDYKX0dOk57d2HCVnGZ1EWM33GqmuR/bYNsHrjUZWB5IWR3IR5ZD iUB+cqe9xVzStItrITaTwvg5zb8tpugv+MvuslkB94bOcYKbPwfRN25TcuvAvOLhpE0Iq9 PXxdKavfOTBP5PA9+wfusCJ2TswZLNk= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=UOhooY0e; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf17.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232899; a=rsa-sha256; cv=none; b=5EhZLArDE1gY4zlpuFVETrR8gxOJsd5DoyIwxdqr82kYDFHLRQwnhiOuWtmNtxh/yw0lrT iMtkYCieNET/jLCtIm5o+yS3qHi5AxbX8Rp9NKvmyRa6AxMXtrH3WzryHKGZw4pd3k8Maq DuqnI3D/v/uhn3LlL8K0ZNZ/ID0YQ/g= Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-28c2b8d6f2aso3491156a91.2 for ; Tue, 02 Jan 2024 14:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232898; x=1704837698; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sx1CrYCYqcy0I3lbTucfKtzktUZzOavzb19InQrqEBM=; b=UOhooY0eiRN29m8oXxlqS3fmSs3o2HMk5mzNkyeLrZSf1MuLk1SnGO0J044CylnWji 5abFXOtpGNXflNB0oSG2OLdYIVUmhIKF268fh9b5hTxgKA1gs/bZLfbcobiy7gtNWN4g l8Qmw0OnptyzyRsWTtxIMYM/ssJMDQ+X2kuv0rzGI+YO2nvZiPBAchLWb6ddKCgO9H8f +KKsIp2rPwZ8843C7X+9QEBfUzdv3s95ljqqp9NnHXpMSQIpF2jjym8dfcZlwyEoF2wI mo58zlRD61KstNsFSVYwGnTa+GX3WFQ0kJ7GFPK/J6qTQdR0Ty/s/nTjhVkGnB6rCOYW 34OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232898; x=1704837698; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sx1CrYCYqcy0I3lbTucfKtzktUZzOavzb19InQrqEBM=; b=WG8KjDJRkQmgiVeFV749tzcZRAKTSIVV0B/tg+FJTFXTenG9lHCf672JOWS47vXOZs 7Bl5qgF5Slwy7wVW1zeAi3OA9CKohUUItP3cRlp7y+ImSbxaB9p/wbpkIuEwEDEnMcVw v6WptiqIBUL7J5JpWDx442Cd6BTlZ1ySPdOqQxuzv8/53ohNY3499GVFSia7sY+qDxXL 5aieho43iz+yz6NtUxIxaYvsXA8P8jzBw1W318FZI69fd6ZlOkOu4VbPUhUHKjsTKh/i AZxKqM0tO5RlEGdWhBbsi4HhAdu/QS486p1FgPi+eof7Jeck+pgPk/7F5LYd1xWkwgZS u3Vw== X-Gm-Message-State: AOJu0YySZAMgWal2xs8/9c7K1km3zb7zdboCp+Vu26ibrAZRyIYguDrf T4JAQQ74Cq7cUDI7ZEIRfzxmC2vUJDLkXhQ66fRgf3y/ha4= X-Google-Smtp-Source: AGHT+IHSNOWtd16BNBpr13f63JilpUsY7HXImYHsivjSF6nbDu6T9MOgsg44oEC59WRv4BX5AKGF0w== X-Received: by 2002:a17:90a:728e:b0:28c:a9d0:33ff with SMTP id e14-20020a17090a728e00b0028ca9d033ffmr2151486pjg.62.1704232897945; Tue, 02 Jan 2024 14:01:37 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:37 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 01/12] riscv: Flush the instruction cache during SMP bringup Date: Tue, 2 Jan 2024 14:00:38 -0800 Message-ID: <20240102220134.3229156-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 08ECE40006 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 3536c6czmcx4mefoc95fda9dgtjxzk7o X-HE-Tag: 1704232898-343895 X-HE-Meta: U2FsdGVkX18hwkljrPH8KRPaHbrddMCBHHlO5S9KHTtdsdjh4B4xMW5YYhHKrig1/djS5GTd1ZWEdtAB4NnAsQx8RqiJCgUA+m/NKJlmxv676SrDnpIhAzpivtD3TCF88ym5m5yK+y317QMJXDeI3thQYBxEepoxMn+JIkTB/Oy5s8tVqW5f1a5mALjAWSEq2Y82ySf/PW10QA266fa3HZkxRKAkIvw4KGOIpBdIm00DpV8c7/bROMTnNpKuYbXZr6w808ySqqd7dlIdmMg65zQuVYz4eRQwERboEx+L43PCLKoQgkzvHToUMrXYUC+UJIGXlzD+BFSvNf/WumJNvH+x15/IV6gYLsFjSM53lEqGWGWD5Y8Szv3uagmWpk1xNDlGwuT6rNhok/sZPZDmVIZIGQEN9q4nX80YkkliDQ0F6vhb7XHN9qKv7l4Rjp8BVGWDTSjnQzC5OYBM9+MpkTY+jBILy3nzyv3wcBb1Kz7LsgYqHgoEwK3Wpw3yWp02QPMwL68GJZklwCORMs0MybigVuOFnhqBBwhjKbvBgLyGK8fMAxZyj+xDG3Xm4cfx9EJIMwMwiEQ4cA+hEF53tc6h1wtSK1exKn/0nVOxW17PwU7J//+pKOwldfXJinvolyukIgFbIjKTwhf9lq2kGXC0fCEhz+BRkwq3oVhNSZDJRDRMUNrC6WCCThtxaGHUUcjAYX0tUlPcHpIAw2RqQgOprs7du12aK2z5kwHwVAAUn80NLIxQ30h2p/khovyq6Zd7iRmODyMh/bKV+60oHUq9EaLwpIXQcsNANt/Spaz6VS9eNuPDACJPdKnJI7gmkZ/XnZKrTyQk/m6fzL05oVn5h5XRhT+Nq3KLW6+U3z7xxwKO4bcbntj0uou0Oz5Njkw6NFl7YtmczmQ275gt8NMmkDYvi+997gX2ukZHr1Igf2sQAW37e7umgGIMIva/b7nrL4QYo5EpDAh0glu dp/dbveL Wokiwpq3ll7hlcFNsmKQCj/DgJMX+u2nG4iYouTY0dhe8ulm5j4i3AwMep6ngAzZkP+UD1ZsP0MvAuIGKIJeyS8mSuzZtdfZKsY7/YyEUocEpRg/zKH2zRQ3OknXegOkm70H8jGW4GEO9ymh9ngbXX338Rw3oddF+0BuHHil7v46jrdE6lIO1akNa/nJJ21VtjzHWgiS00gbR1R6y8zDVhoA/vPsqwh5EYBpARv2viUQjYJafGQwr/va1QkGSnYqX+eDOmscmDw3tqxMrqjFqUfu/nx3hKB/JkCCieQGfbi0jsSrZSFW/zLY57WwRdkMy1IgtUWb83VRk/wTz6oRzAL4E+cHMweiS24YFN0a4Q6wRFijv7RqW2XoVpmR+huvDPwE/1fMy7rGttIf4pUI6SlGb5lgVul36NR1Rzh3dPTjcNnw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instruction cache flush IPIs are sent only to CPUs in cpu_online_mask, so they will not target a CPU until it calls set_cpu_online() earlier in smp_callin(). As a result, if instruction memory is modified between the CPU coming out of reset and that point, then its instruction cache may contain stale data. Therefore, the instruction cache must be flushed after the set_cpu_online() synchronization point. Fixes: 08f051eda33b ("RISC-V: Flush I$ when making a dirty page executable") Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v4: - New patch for v4 arch/riscv/kernel/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c index d162bf339beb..48af5bd3ec30 100644 --- a/arch/riscv/kernel/smpboot.c +++ b/arch/riscv/kernel/smpboot.c @@ -26,7 +26,7 @@ #include #include -#include +#include #include #include #include @@ -257,9 +257,10 @@ asmlinkage __visible void smp_callin(void) riscv_user_isa_enable(); /* - * Remote TLB flushes are ignored while the CPU is offline, so emit - * a local TLB flush right now just in case. + * Remote cache and TLB flushes are ignored while the CPU is offline, + * so flush them both right now just in case. */ + local_flush_icache_all(); local_flush_tlb_all(); complete(&cpu_running); /* From patchwork Tue Jan 2 22:00:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C029C47074 for ; Tue, 2 Jan 2024 22:01:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 969E06B02A6; Tue, 2 Jan 2024 17:01:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CBA16B02A7; Tue, 2 Jan 2024 17:01:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71C516B02A8; Tue, 2 Jan 2024 17:01:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 627256B02A6 for ; Tue, 2 Jan 2024 17:01:42 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3ECE312070C for ; Tue, 2 Jan 2024 22:01:42 +0000 (UTC) X-FDA: 81635743644.28.0D5B31B Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf21.hostedemail.com (Postfix) with ESMTP id 526131C0006 for ; Tue, 2 Jan 2024 22:01:40 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Cl8aLY14; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf21.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232900; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pynuwdwrgTDQTs9EZPEkGjuJgTxII1R0VKB5FEKmh8A=; b=HZFku7Ehmb31Yhs2NYurvSfwzGN0JfmQPJhvgXHrGq3Wt8m9nXMFqURwbbBvy1R5O60Vo4 2bgpJada9bCHoAyEiBo5RM62Il2/PJdJzcyzM0lSlRmH7L1oBABa2hkl66nYCxRPMIQLm4 apwk6rj2TbE14sL1zpX98Y0pulny7Rg= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Cl8aLY14; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf21.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232900; a=rsa-sha256; cv=none; b=lS0hOLmoV6IhjaQmOKJMSItGCAoFhgEklft2ky3Y3lhoVKza9mbty6kOQuEE3jE0TBbqf4 feVCnpKY8LKkJpwjt5oWVbQHeFAlTTzNS6vEIUaMahaE1dTGXv58m0aQK7Ww+XBhjexs41 5pSJhkDQlDhXfq6Q9/M8NGRm8E6roCc= Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-28c075ad8e7so7041791a91.2 for ; Tue, 02 Jan 2024 14:01:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232899; x=1704837699; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pynuwdwrgTDQTs9EZPEkGjuJgTxII1R0VKB5FEKmh8A=; b=Cl8aLY14pqO+9kzBdsesRsy9o52xrxkQa1aNl9DjsJV6AnK82oRF38FtlWGrTKsGE9 Zb/or2LL9ME2Wx9qCkxC5ndatN9ClCAxpcAROaMX+Zd5anbhkGAIbVRi4TXWvHbczj5t Ms7MPgQx1R8wmvqW/gUDL4tyQ5A0Q7xmSmKMnZX2JwySqOVYEuwkBt7ROcv3YanoAA/S QqY8Lp1zcaZDd/sX0lUQ2jDZYz0er5rjHD3jcukEaAXKmTi6uqYEwtaIfNjWT9WZ1ak8 N/2Lnh2Alvr+MKCwIFjalm7LmDrzIMnQihfF3UUbd22xsd+NQQvHKvxvlORD41K0IwLS u+9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232899; x=1704837699; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pynuwdwrgTDQTs9EZPEkGjuJgTxII1R0VKB5FEKmh8A=; b=ouwr2SBwlgUx/47GBKRTGDWPOCZ/qmQ0KShnjoDVSGh9kpohCRWIB6GNvKDeYzeDDJ EbpIxrWh7O5yj017r8n1XGO7CUKhYRSig0ARkPljV94EA2Ee6IeLsnedUR5tV3aopx1J iagsiFO2ValpJAKlW2+elKSmYar5uennfuI+m3EEKkxrp4wENkXLGkCQvv9GI+w5T0ZV gMEMVc/OPXCpn7ZHZQgb48dueWUngSbP4xtUinZ4DM6+44Zd5RNcSl0cn+XHT4e5lT6d ZlYH1WZKp0+HtwEcAg53B8XZ62FTEuAeEasfWHFrCrTbc2NN3Clp3nSX6whL5N1uiXN0 wEpQ== X-Gm-Message-State: AOJu0Yx2f6Dgb8nxVGj+LE6HQV/pfHh191KoMYbJZCBoVQ9atr+7AKGp B3xzE3yGPYiH9SuYTLwPeFHN+4lDuQ/rKQ== X-Google-Smtp-Source: AGHT+IEYPya8PYA19IRLK0MnT0T3isXxWKz5hXZR0znVEsgxQFvYhbUCwOfyfHQYZwqQ+xa53Aj2UQ== X-Received: by 2002:a17:90b:3ec2:b0:28c:1b98:e6e7 with SMTP id rm2-20020a17090b3ec200b0028c1b98e6e7mr8644807pjb.47.1704232899208; Tue, 02 Jan 2024 14:01:39 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:38 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 02/12] riscv: Use IPIs for remote cache/TLB flushes by default Date: Tue, 2 Jan 2024 14:00:39 -0800 Message-ID: <20240102220134.3229156-3-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: crtfduawzirxidhmz8btnkdny3bsnkmo X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 526131C0006 X-HE-Tag: 1704232900-892435 X-HE-Meta: U2FsdGVkX1/jNmwrrbwekcUPXRExiVZGieCowR1skC/MZlyXi6EciKDys0lSSkDSGseSLdGnUb1m7lRZms7/bt10RQeeno3qttYe/1YZVz7zKU1fkrU+VU05S85evIjaCsZZ4W7zaChEyNe6KtEdNtd6LQn4ea0FGOFXUV0x2XjX8MjWGBSrHKKFIR9Ttw9f+hT1X3GmkxlFW+ri4AUPVBk/hvC5WBFFBh5jyY1TfcvhxiuuyF9Zv/pQnzqZB4GvZC6tnaRrHx7TGKEgfvT1S1D87SIES2pDG7iEnW3EY1dI06boU6frF6EMj5VpwShM0jJ7SVNJA02TxoYK7GXGyp6zzmLgM7pX1mtc8iRYrYrMi7rs4/f/Gbc53VUnp0XjuqwGB34OmyyBtbdy6YJ4qBSHSXqJF+OPSuX6pTWBkCtypXE908lwGfS1rY0cQxgRf7iXSh/sRhbojunVwPyiunK+bsPxWjN4ZSkxtP7xVQxoaGhLcHzTjRSV8HyPDWKNqkdATkWbqOEZEirTWFJYYwmyL4Lq9Yq1X5Jxzcjy3yCbEER4SiHnwqFAIdTFqIQZsNWb3KXtjPonC1ThwK/kss7xytpHVFriwEaDfe8Pl6hgYovFi/AtycL9qM46zq4AXSPYT+kgOoVa8PfKcBzMPVjLS45l07E+WQ9Ig3YrSglVHSM7m1AzKZH7MnzUh1h9oEePVASYSnx7yp8VDTKtZj5jqtITJjfmykAS4TxGcYTCW6xM083j2Zlw3TNJYzKxx06a/PPzxZwDJjtZsX5/f2kz2cH+iaRdoMLmNj6dnCuOimBx9N0a8s4bqFK5z2PsGBVc1I5lK0ILN9md0xiH+hLm5xopmvb6pp+xTsdxs5yRkc9DKkOM3zEAaZmYDYlS4yn+/c+MbS5M/7xxjmMug5Bx2bYg7Od3o1Nn82xqEwTAei6flba0w1ays8bhloHhHejUT2sVob1z0W6Rn5K 6lJUfylb yVLmQ2rc5i3aphElUspnJKJcMM5ty7P8l+Q64f9/jqw9aZBO8FgKUGGXVOJsOKGpzT6X/c4DM/h86GNiPZ/6DOtYbBLHwMVyGawPJr+B02E9igyRgbGm0Fj05i52yPegOrTzTfdZmCochav6bdyPmbW9tAfzHTYvcMm+5iP+jNSlDBaalcMQcK42ZYhJwVanm0kkcqlem7xE8omGeWnRO39IV2aFn+HHtXJ4yalY5mCoSstYj9qlB+a5gA6pcl63hm1rjIJfLPwtvtoU00ttTyd5i+TgYN7+2HEeMpn0OWuzcRODK+HrukZTZDktMDucW88jKqIkc/vXSsucu5w3zIaf7pVGjTQko+FTmIhOsx5sRB4xqqOC+74xnkLycRLl14tX9bDrh6NBWM3o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: An IPI backend is always required in an SMP configuration, but an SBI implementation is not. For example, SBI will be unavailable when the kernel runs in M mode. Generally, IPIs are assumed to be faster than SBI calls due to the SBI context switch overhead. However, when SBI is used as the IPI backend, then the context switch cost must be paid anyway, and performing the cache/TLB flush directly in the SBI implementation is more efficient than inserting an interrupt to the kernel. This is the only scenario where riscv_ipi_set_virq_range()'s use_for_rfence parameter is false. Thus, it makes sense for remote fences to use IPIs by default, and make the SBI remote fence extension the special case. sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only calls riscv_ipi_set_virq_range() when no other IPI device is available. So we can move the static key and drop the use_for_rfence parameter. Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is enabled. Optherwise, IPIs must be used. Add a fallback definition of riscv_use_sbi_for_rfence() which handles this case and removes the need to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v4: - New patch for v4 arch/riscv/include/asm/sbi.h | 4 ++++ arch/riscv/include/asm/smp.h | 15 ++------------- arch/riscv/kernel/sbi-ipi.c | 11 ++++++++++- arch/riscv/kernel/smp.c | 11 +---------- arch/riscv/mm/cacheflush.c | 5 ++--- arch/riscv/mm/tlbflush.c | 31 ++++++++++++++----------------- drivers/clocksource/timer-clint.c | 2 +- 7 files changed, 34 insertions(+), 45 deletions(-) diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 0892f4421bc4..aeee0127df76 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -339,8 +339,12 @@ unsigned long riscv_cached_marchid(unsigned int cpu_id); unsigned long riscv_cached_mimpid(unsigned int cpu_id); #if IS_ENABLED(CONFIG_SMP) && IS_ENABLED(CONFIG_RISCV_SBI) +DECLARE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +#define riscv_use_sbi_for_rfence() \ + static_branch_unlikely(&riscv_sbi_for_rfence) void sbi_ipi_init(void); #else +static inline bool riscv_use_sbi_for_rfence(void) { return false; } static inline void sbi_ipi_init(void) { } #endif diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index 0d555847cde6..7ac80e9f2288 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -49,12 +49,7 @@ void riscv_ipi_disable(void); bool riscv_ipi_have_virq_range(void); /* Set the IPI interrupt numbers for arch (called by irqchip drivers) */ -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence); - -/* Check if we can use IPIs for remote FENCEs */ -DECLARE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -#define riscv_use_ipi_for_rfence() \ - static_branch_unlikely(&riscv_ipi_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr); /* Check other CPUs stop or not */ bool smp_crash_stop_failed(void); @@ -104,16 +99,10 @@ static inline bool riscv_ipi_have_virq_range(void) return false; } -static inline void riscv_ipi_set_virq_range(int virq, int nr, - bool use_for_rfence) +static inline void riscv_ipi_set_virq_range(int virq, int nr) { } -static inline bool riscv_use_ipi_for_rfence(void) -{ - return false; -} - #endif /* CONFIG_SMP */ #if defined(CONFIG_HOTPLUG_CPU) && (CONFIG_SMP) diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c index a4559695ce62..1026e22955cc 100644 --- a/arch/riscv/kernel/sbi-ipi.c +++ b/arch/riscv/kernel/sbi-ipi.c @@ -13,6 +13,9 @@ #include #include +DEFINE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); +EXPORT_SYMBOL_GPL(riscv_sbi_for_rfence); + static int sbi_ipi_virq; static void sbi_ipi_handle(struct irq_desc *desc) @@ -72,6 +75,12 @@ void __init sbi_ipi_init(void) "irqchip/sbi-ipi:starting", sbi_ipi_starting_cpu, NULL); - riscv_ipi_set_virq_range(virq, BITS_PER_BYTE, false); + riscv_ipi_set_virq_range(virq, BITS_PER_BYTE); pr_info("providing IPIs using SBI IPI extension\n"); + + /* + * Use the SBI remote fence extension to avoid + * the extra context switch needed to handle IPIs. + */ + static_branch_enable(&riscv_sbi_for_rfence); } diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 40420afbb1a0..1d06df04eb71 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -171,10 +171,7 @@ bool riscv_ipi_have_virq_range(void) return (ipi_virq_base) ? true : false; } -DEFINE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); -EXPORT_SYMBOL_GPL(riscv_ipi_for_rfence); - -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) +void riscv_ipi_set_virq_range(int virq, int nr) { int i, err; @@ -197,12 +194,6 @@ void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) /* Enabled IPIs for boot CPU immediately */ riscv_ipi_enable(); - - /* Update RFENCE static key */ - if (use_for_rfence) - static_branch_enable(&riscv_ipi_for_rfence); - else - static_branch_disable(&riscv_ipi_for_rfence); } static const char * const ipi_names[] = { diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 55a34f2020a8..47c485bc7df0 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,7 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (IS_ENABLED(CONFIG_RISCV_SBI) && !riscv_use_ipi_for_rfence()) + if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); @@ -69,8 +69,7 @@ void flush_icache_mm(struct mm_struct *mm, bool local) * with flush_icache_deferred(). */ smp_mb(); - } else if (IS_ENABLED(CONFIG_RISCV_SBI) && - !riscv_use_ipi_for_rfence()) { + } else if (riscv_use_sbi_for_rfence()) { sbi_remote_fence_i(&others); } else { on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index e6659d7368b3..09b03bf71e6a 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -73,10 +73,10 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_ipi_for_rfence()) - on_each_cpu(__ipi_flush_tlb_all, NULL, 1); - else + if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); + else + on_each_cpu(__ipi_flush_tlb_all, NULL, 1); } struct flush_tlb_range_data { @@ -96,7 +96,6 @@ static void __ipi_flush_tlb_range_asid(void *info) static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, unsigned long size, unsigned long stride) { - struct flush_tlb_range_data ftd; const struct cpumask *cmask; unsigned long asid = FLUSH_TLB_NO_ASID; bool broadcast; @@ -119,20 +118,18 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, broadcast = true; } - if (broadcast) { - if (riscv_use_ipi_for_rfence()) { - ftd.asid = asid; - ftd.start = start; - ftd.size = size; - ftd.stride = stride; - on_each_cpu_mask(cmask, - __ipi_flush_tlb_range_asid, - &ftd, 1); - } else - sbi_remote_sfence_vma_asid(cmask, - start, size, asid); - } else { + if (!broadcast) { local_flush_tlb_range_asid(start, size, stride, asid); + } else if (riscv_use_sbi_for_rfence()) { + sbi_remote_sfence_vma_asid(cmask, start, size, asid); + } else { + struct flush_tlb_range_data ftd; + + ftd.asid = asid; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } if (mm) diff --git a/drivers/clocksource/timer-clint.c b/drivers/clocksource/timer-clint.c index 9a55e733ae99..7ccc16dd6a76 100644 --- a/drivers/clocksource/timer-clint.c +++ b/drivers/clocksource/timer-clint.c @@ -251,7 +251,7 @@ static int __init clint_timer_init_dt(struct device_node *np) } irq_set_chained_handler(clint_ipi_irq, clint_ipi_interrupt); - riscv_ipi_set_virq_range(rc, BITS_PER_BYTE, true); + riscv_ipi_set_virq_range(rc, BITS_PER_BYTE); clint_clear_ipi(); #endif From patchwork Tue Jan 2 22:00:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6BBFC46CD2 for ; Tue, 2 Jan 2024 22:01:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B178C6B0071; Tue, 2 Jan 2024 17:01:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AA0906B0072; Tue, 2 Jan 2024 17:01:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91BBC6B0078; Tue, 2 Jan 2024 17:01:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7C9596B0071 for ; Tue, 2 Jan 2024 17:01:43 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5130BA1788 for ; Tue, 2 Jan 2024 22:01:43 +0000 (UTC) X-FDA: 81635743686.14.A75252A Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf13.hostedemail.com (Postfix) with ESMTP id 71EA120027 for ; Tue, 2 Jan 2024 22:01:41 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=gCqnm4il; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232901; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XA8TOHyyeKTc95PFXJRpYatfRR5oPHMuobG+iziT3HE=; b=QqWLVs1qlxuxt2tS8ivKuzsn8x0kdXQ2soFC1DkTf0LxQY54ufrG55QAe5JG7o+JoD9T6u gsCq8y2DfjMlVyu3PJw/A1kbn6H/u8KLAt2HTjT+cAxtNsOyPpN12vonL0WdgozEbJNVgI avme46BbtHzgIxTR3ID6Go5KAa1Vr/E= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=gCqnm4il; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf13.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232901; a=rsa-sha256; cv=none; b=hRnjUd5ifV5NGu95H9+xV2qGLjh2pgbQ0q0nDqZPXvCn3gxb2ETCuSAr/6c4ErO8hAVySN 6eZ28aCZg9CwOjDTt0jEMcz5kgL9Eoe+ytn685DE6HuH4rSEAq6yJUHWxcqI7npDfzQwtg mwQw4RPY4ShPUyIj1fAFhb0Bmo8cJGU= Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-28bfdf3cd12so3960751a91.3 for ; Tue, 02 Jan 2024 14:01:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232900; x=1704837700; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XA8TOHyyeKTc95PFXJRpYatfRR5oPHMuobG+iziT3HE=; b=gCqnm4ilMPvZFHQ9kC7J4ek2IovZhrVpa0gHIuSmf7Rzi1oexpz2j5Eb8JtzqSFSkd SxAXpzKQ1Zh/vMn4VdrCffxM+UYVgwjgud41MsNc+qn/lCnNe+3T1AwVonrvAXvsDfsz KMl3M3mAP5RAoU/mGL+DwaTrZVFjimuktbFaM3FmSzF66cNJjReryKICl/JEJsDdsJF3 b9+r/5jppXwC8uaa4ExBw6oHUkU4+Gr0HPEMQY/41KlBi8t9bmJs0oVLZ6DIUA/VzYnF Bso5OJ+NbhSdZgtp6gV1uyJC+bqvNplSxr9cPrJu/DnRIdtnYkjZ3ra7I84ySqECStgq A8Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232900; x=1704837700; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XA8TOHyyeKTc95PFXJRpYatfRR5oPHMuobG+iziT3HE=; b=jMP17b5BsQhH4J3GomR8uhZFrmKu2YAh3fQ5VN3TJ04zYsJoq9rB6XyVT57qfJlHxG NVP/ECbcaybp1Fwn22I6+xw1nsLPxjugmIzOoTaVWOx+zaCPh9yiWl7ztP2E6VM2zT+Q qNJt8uT87ChdwvswjnbwnxoDRxG6cZsJFKZqDeDJ9L2lZwRT7jEki/nsD/KM+ECBlv3k RUCHndJGq8f1rxG3g6F7LEFOmw5OoMLHmo50oylplwIwFH0/LZUTgz2UI9Qs2y3UVPzN 1zjDsfIY/fjg6Q6SrToujoqiWsg363tU5U4uRp8SlkwmFButwLQLgcFlYO3sQkoeSnZR 0hiA== X-Gm-Message-State: AOJu0Yxh6DEWtBCTnc0qDciWC/UW9hvDxgJ4D/lRIx7ASmG+H9b4sCUp 3yopfI1nTuvBHHD4nSo0lwpZ241A6BfG2A== X-Google-Smtp-Source: AGHT+IGXoPWHR5PafwwvCfsK03qnPYeIgtaQPhGwO0vZvHcHemW0jssRCcGF3figFCmbRtQxmnrgGQ== X-Received: by 2002:a17:90a:974c:b0:28c:accc:b1db with SMTP id i12-20020a17090a974c00b0028cacccb1dbmr2272863pjw.44.1704232900428; Tue, 02 Jan 2024 14:01:40 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:40 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 03/12] riscv: mm: Broadcast kernel TLB flushes only when needed Date: Tue, 2 Jan 2024 14:00:40 -0800 Message-ID: <20240102220134.3229156-4-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 71EA120027 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 7t4y5iq6zykxetgac8pui3ijgddfbw9w X-HE-Tag: 1704232901-624241 X-HE-Meta: U2FsdGVkX19T0547pafWRVsGauK+v8RrEWxEWijcIyz20zR95/sB9iy4xoWvHjKUT4hip+R0EmFbkNtXcXBIdwVELn/PPlwo76ZPqjMpLEtfeOIIUSzAdPWzTEx5K8FmuYKEhAxynIItICEFJ9MK5qSQIfHqDR76ASj8MMNlwn4QUEfWZwkfUwURsm1ZVdsa3q3cgSt7tvgF4P8cAaDYlJilbxbwFcq/A5bEGP0dElCo0WeH73H0JLuVAfdnRM++LpJTlAM+zzZe2hGPU5rfd3snN75yxEJPfnw/PVVl/dlelanKAlLrjhqnOSQ4HLpgfDkB/rZjiPuv9P/bPhJXfTt++b4gt6n9uOTdygkrsi1r5O5UBUjfoAG+UgedxMFrJBkmwxiVKS+REgdMgp1BjVyhQBzajN+pft814DwY/wQcxf21dUqt2/6eboKgliSGEwWCg8s9k6pl/zOkzQbNkrm4QOGwAh7QDN7gDJmA308Vq90/3O4GLOJKWqIzndQ+vnQQLYzTUjrzWt2Kb4T5jhBY2lCeIn9UKXcrMkPqDNG8JGX/trircM0MA3jZCTd4BoyUZIwHpqLIh666eB946g//kZrCR2K3n92SKmEr6+rwB/hRKUEjcS5ejff2ERQgTfWvRHdk+nOxKpvlhm3w/lcK8AWTv2IiQjmi2s2eYMnaqUiqq2ytDNkRxtd7H+RMnWuf6U6vJ6TjELoIHQaCXal073G13ISdbSF1pj0TPvU9s8kYI4XZ6cXc38dW7p5wClo+H2thd+Sz8+taN3M6haQVWuMdDDFLVA30qgVcCBNedojGqucGAP9BDzyzVu7u+cC4irxGWu8tUBDW7kdLW7BZOxuZTprhgEtB7PAz8feImxTp25CPpH6NbCr1GcnvXbF3fWTlWh8rkbdwu0bdr0SkjKij5ZMqzQUrJJ/LN2HpeIhMvG6M+vfLjifP6OtzYbG0WMFzYVIlRRb8FHh d/Dj1WZR UcAxduLyKOuKRF28kDTE403JJzr7ZhHaQzoZic90VOyvCqFEZzKOm0rCqJHbLjsO5Z4506lrfljMS9xDIDmZiDi2ZIywzlPXA6qZOK23D4f7lvwqRQRSJ9GBjhQmSlStAWgCKBHzTwWhMjFYOqRP1J/IEmR/PV9p/O2kpE7InWEH4IrhOQqI0X7tYN9ZJKkqsNUAaDH1DTWVNgZH8NSjx8nr/il6lJHFubjEUZFlYpurUXyN2IJMEhv65y0bBeOyXOiA7tP3uhcB+CDCAqqYU8FkveIqhLMCxXe+lPG2/9RZY5HD4b2KruIRRaqa8ek3hp+5hazzU/3lPRckZyLDVxgbCZYkrZOXPrunW1l7Yx+6XxZxdMM4rPGu7OEWhfQGGy6DkGC09O42b353AqaCM3169CnzpjBhi3swqVViPbBi+oTk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: __flush_tlb_range() avoids broadcasting TLB flushes when an mm context is only active on the local CPU. Apply this same optimization to TLB flushes of kernel memory when only one CPU is online. This check can be constant-folded when SMP is disabled. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v4: - New patch for v4 arch/riscv/mm/tlbflush.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 09b03bf71e6a..2f18fe6fc4f3 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -98,27 +98,23 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, { const struct cpumask *cmask; unsigned long asid = FLUSH_TLB_NO_ASID; - bool broadcast; + unsigned int cpu; if (mm) { - unsigned int cpuid; - cmask = mm_cpumask(mm); if (cpumask_empty(cmask)) return; - cpuid = get_cpu(); - /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - if (static_branch_unlikely(&use_asid_allocator)) asid = atomic_long_read(&mm->context.id) & asid_mask; } else { cmask = cpu_online_mask; - broadcast = true; } - if (!broadcast) { + cpu = get_cpu(); + + /* Check if the TLB flush needs to be sent to other CPUs. */ + if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); } else if (riscv_use_sbi_for_rfence()) { sbi_remote_sfence_vma_asid(cmask, start, size, asid); @@ -132,8 +128,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); } - if (mm) - put_cpu(); + put_cpu(); } void flush_tlb_mm(struct mm_struct *mm) From patchwork Tue Jan 2 22:00:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38909C47079 for ; Tue, 2 Jan 2024 22:01:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 747006B00A0; Tue, 2 Jan 2024 17:01:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F6BF6B00A2; Tue, 2 Jan 2024 17:01:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D4D76B00A0; Tue, 2 Jan 2024 17:01:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 38BF06B0095 for ; Tue, 2 Jan 2024 17:01:44 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1C36A1A013C for ; Tue, 2 Jan 2024 22:01:44 +0000 (UTC) X-FDA: 81635743728.08.96B858A Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf23.hostedemail.com (Postfix) with ESMTP id 6800F140005 for ; Tue, 2 Jan 2024 22:01:42 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Rt2X0DvC; spf=pass (imf23.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232902; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YDpWqQ4mc+jX/Roh6ocibeJ1LmLlK7pqWIKc4/L4AE8=; b=FLVD+73WQsV4rOL+M/6AkYhPXZYAqkIQlFdpHq/d9zgDBdFv+n+N6spIOhI7Mh3XbYS+yd Lxa2uQPTIQW8CsfKcVTz2BYXQuUhxM06FynVHXLicliLN84hj3NB8XD2kOTSUsHcqsJODD qhoeY8MyxdkvY8uEaEtnW7UUjutk9lA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232902; a=rsa-sha256; cv=none; b=vD8rDRN8Wd4mDdTveZi2/E8+zpso1ij+eFH7gq62h15haqFdiTln248qpmFqYkBZJQrEwN 3Qob47nhgWbIez0knXHxHx0xwjGGD0O8IbssSCovaqAhlW+jb+SaLp2WD2PovimhVD8JgS 3POlGes8FlTh6q0hWkiBXlPwawf5WUc= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=Rt2X0DvC; spf=pass (imf23.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-28bc7155755so3952233a91.2 for ; Tue, 02 Jan 2024 14:01:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232901; x=1704837701; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YDpWqQ4mc+jX/Roh6ocibeJ1LmLlK7pqWIKc4/L4AE8=; b=Rt2X0DvCDLIBmR1hx23s8+ATVJ+A50LuYMoJKtDYDt+SBUoYO86oDrcXwtoBGH+IYM aX7levYjnhQlHqach4hDBR6PJdQiFqXOf/XxPVJOuXjNmd1pMbGNqpWUHbYygEvmwrxP VxPeY7Km5khZkiq8cfjyo2iBkgImZPapP2EwKPccB5zF/qSm4vwaf9a4RfdghdoGiphF EmqRDph7HZuC2+HR33QPiiHszhZKAvqmiXmgh6P+EyvcMSO3lJFoig/h3m78kh/HVuLH Fgqg6YyfEm03ikXQiJKjT2FKA1nqkyEOn8XVi6P/CAQ/1y2gvxAnCXbw8OCCxfqR8d6D FlSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232901; x=1704837701; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YDpWqQ4mc+jX/Roh6ocibeJ1LmLlK7pqWIKc4/L4AE8=; b=kUjM4DEbg47Clq4c4esM39pMBncPiNfOs7c9ALjw4ypeo4CPtt2Tv+RGhiJvzMMMSn 4/NAOALifc9CUZt0wM5b51K+Y929IDu0exXLEN300yl70oMrS/m2QzwOMkVL9ru9Us31 vffV3lBff3IqUFKakHrrLB6ly3GSJaA8w1qp5hTOUnYKJKqWTe+amVy63u9vQGSDHY20 6vgsPrkv2O0S3nJVJZML21avg/IqRV77IKo4qGFkEHIWC3/V7nAX+pKAtZCW4yRexjCo +p+jbySwD+6neQR7Sykd3TK04Mzc47WNIEhC/jg32JSem+Vw1piwRdI1ozq8Ctgg/7zh QwFA== X-Gm-Message-State: AOJu0YyZ08sawX8c9zBYnpIxIzUz4o8L1Y6Bj0kYrLj14se3nP6tt6VO rK0M2m47ptlpDScgxWtG9nsQ5oXcxbNZcQ== X-Google-Smtp-Source: AGHT+IGhgK6idMxI6CkoCNiy9Y4cs5Xn9ml4r4H30wFrLDF3Z3r2uMsg0CAIjypKVmYj0k77Mk3d6A== X-Received: by 2002:a17:90a:6ba6:b0:28c:640b:ce0b with SMTP id w35-20020a17090a6ba600b0028c640bce0bmr4536054pjj.33.1704232901387; Tue, 02 Jan 2024 14:01:41 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:41 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 04/12] riscv: Only send remote fences when some other CPU is online Date: Tue, 2 Jan 2024 14:00:41 -0800 Message-ID: <20240102220134.3229156-5-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6800F140005 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 9rpoih7q3potm4ugzi1y64ya4zgd5ciz X-HE-Tag: 1704232902-716529 X-HE-Meta: U2FsdGVkX19pSs9Fg8rsnq+vA+c6Nchsamgjmi83JlrJ02R0Q4SsHDZqPzN1VbCkP0DSfjv7fr2V+nwEhxdikJeX4odv8CttjDBDo3uulgMoWuORN8kFgHg6dAKsjaORFiJUjsxKVOFX0JTmFZIF395euCrKDIvQjliPa6/yKKvTZGUVaoT8Plaf149xqJDLaQi6V2MZNGWYlqRmR3+EPk9kviyirCaXmPtypEBz3Q4kBSONR8VLpg4zcAIk/SzXHD2Iu/wyn13e2y90daitv+Ujwl2S8Cz72Gr2U1Lb9o/djXP6DXu79hJBaQfsczzmz8f3NAYj15GMitx1Sk+8XPEWjXhmHbnjGf48AMtkCi2bOFvwTR4gvzO6Mf0PXCEG09xksbIOOMWZZhsctHIwyvYZdIYNogUc1KkXwp101eWcQYbAaa1GEhtAM40eTPbOsgiwUgKqwuSjdxM33RLE484w0OfyjrVcZs3EJDKkD5Ts7LFi43vDVDL+OGN75QTRzFA5OGzfEwwT3PAQn3mJshknhqNCOhUeuVEE+qqfx3nEu8c5gnPO50RAmaOBsUGUYrBhgGPVBI5p4NOJY63/roV8UjCh9BSFHUN0j+v306b3Pv3bruWxygkKEcTs7Ifgwl3zqyOpFZTQ/IAKqCBafzQTshXIdh73LzFSUqA61UdbCzfUzVhLYnLSpyp+hv4ZNdw1kfY+y+mt0RWJN85sEx3uP1Y8XHFTyYr7AT7s3U8Upba/uvnPV+cJMnJ3KSxyjGiSjzU5iVE+JoPVLs/FEaze8Ry3UoHEPY8HeNJFs84142Pl7yRrENvxSLgawbbkQZd0QKahFusChEVosR+i37CAvO1BmoBtcuqUkSqFE16ml3Ta+rgUU7XcmTRh1jTDK+01vaM1xGgNVNVdE51kNLsiXm3ix7SWzxGYlX6Yi81U9HgoGZYaz0P1L8fib/zFiMhn1a1bsKwgMOAhFs9 AyrW4hy6 NgzP64KdYUQLIkzgQcPS0wkXI5RilEe+dQervQGzK8V1UuFsgfjmAOlO5sAm0UsJqpd9zZwQEM6sShUdFbl7KvGP4OwskBv3r49e9/dEhOc0kknk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.185171, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If no other CPU is online, a local cache or TLB flush is sufficient. These checks can be constant-folded when SMP is disabled. Signed-off-by: Samuel Holland --- Changes in v4: - New patch for v4 arch/riscv/mm/cacheflush.c | 4 +++- arch/riscv/mm/tlbflush.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index 47c485bc7df0..f7933ae88a55 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -21,7 +21,9 @@ void flush_icache_all(void) { local_flush_icache_all(); - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + return; + else if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 2f18fe6fc4f3..37b3c93e3c30 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -73,7 +73,9 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (riscv_use_sbi_for_rfence()) + if (num_online_cpus() < 2) + local_flush_tlb_all(); + else if (riscv_use_sbi_for_rfence()) sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); else on_each_cpu(__ipi_flush_tlb_all, NULL, 1); From patchwork Tue Jan 2 22:00:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4CCFC47074 for ; Tue, 2 Jan 2024 22:01:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B408D6B02B9; Tue, 2 Jan 2024 17:01:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AED2C6B02BB; Tue, 2 Jan 2024 17:01:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93FF46B02BA; Tue, 2 Jan 2024 17:01:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7A04B6B02B5 for ; Tue, 2 Jan 2024 17:01:45 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4B1D11607CE for ; Tue, 2 Jan 2024 22:01:45 +0000 (UTC) X-FDA: 81635743770.23.5A4B299 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf30.hostedemail.com (Postfix) with ESMTP id 6F4EB8000F for ; Tue, 2 Jan 2024 22:01:43 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=RCOFjuHn; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232903; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LtzHkWRF5gALofCSRDnlYgSzHUyaPaF++esfIcBOkBw=; b=NlfigpamL3YP5W3rx6flYSUVdmFYNhwBGnKyoyjY8lX5IwVbPS9HvQuIYbjGcb8ByYbLHb k/Rdj9q/maoYZu3Yd96x/1XBs+qLv+Wo9qs0Rv/mU1I3136RCKlDkXLmEj7NWTwhLsEYNR h9ZUuXW/vTBsBrZ9jJvpEWn9kYWsxow= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=RCOFjuHn; spf=pass (imf30.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232903; a=rsa-sha256; cv=none; b=5/SzWf0NDBPctuTo2EXivVGua6wtvJxEvL4utvJo1okEGk/zEZ88gW8HuSuqgXyQJQi+bO V5GhrQnirjLxm8lJtfTHr0Hq44pcf8+zU0+E2a8awTf5rDAulh9quQy0l/jK90QgMYD5wj 9dclo1GkGVY+Pq/6oMnpDc/FT2uO074= Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-28be024282bso4981174a91.3 for ; Tue, 02 Jan 2024 14:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232902; x=1704837702; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LtzHkWRF5gALofCSRDnlYgSzHUyaPaF++esfIcBOkBw=; b=RCOFjuHnCIMUTiauTxg/k8LlgYvNiEHqLW3ihkVQz+nrPSRgjpwkEjKCwNwOCxXuB+ ZML4H5oi7V2Oky06wZLpTyZZKPSjXCltN5n7LicpQaEbxOUGbQyEXI+lhfufr2yJ1nc7 FFFXrUiGXJQaUGRbMqQ/xpKf+EKu5ThHS/OwMfx1xGhIjhbh/v24h30nPQqjXhnXG1wC 0TFHoYCD9Xwg108+4uVg9Q7QEl4cglGSZsxVUuZx6exdjtdpmmLQs+mTtP+SfD9bbzFX 9zh1t3gdCxTqcA8sXlem64HaknXS9+Dk5GGjVXZYzqRbEl27y6YGtCNwlkCiQFmuw10F Jplg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232902; x=1704837702; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LtzHkWRF5gALofCSRDnlYgSzHUyaPaF++esfIcBOkBw=; b=vXdj3XSTeNss62kf2PomC1x4IjRp9qltbPCFIIHhUN3fNYI9wag+6cDDVzSa/8ez6L lu0Wnjg1xeanPlgsyJr1Ji3tIM+IoH11ZR7fM6bvkx3OgA6dJ2NlKboQ0/n1sNsIX3Fe tdvmoMz2tCeTq4lR0KfmWwwXAfWBjTHypZqxjP+c2jzF2btnBE3RBj0o8AG/97/d6anm ONoQORdd2L9fAIw25/pwGcSzdp8GaIXlNdI7mi1iRPoMOmYYMDfdK+Iala1FY7SVrDMa 1/o2fvR1qgA0xRzoiw78UzBrl7WMrQGgEp5GQVY84nk+yYhyp+FhautPo8d9NfzyhgFG rmuQ== X-Gm-Message-State: AOJu0YzlrnU+yoMOMdyDzojBDDF9FkEb+JpeWe3zz44D5zEMn3/mkyky vTdSZ3C4qB3OmetK6JV8M4rYIrF8Sg5VT9a0jMxNTF6nAwk= X-Google-Smtp-Source: AGHT+IExkJSZEsbRdHGUNd8kiI0pICSa1vMynIFUPWzzc2fqMiijvWM2jfH6XllBTjKhdLtvimrjxg== X-Received: by 2002:a17:90a:740a:b0:286:7f0d:6254 with SMTP id a10-20020a17090a740a00b002867f0d6254mr6006468pjg.63.1704232902374; Tue, 02 Jan 2024 14:01:42 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:42 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 05/12] riscv: mm: Combine the SMP and UP TLB flush code Date: Tue, 2 Jan 2024 14:00:42 -0800 Message-ID: <20240102220134.3229156-6-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6F4EB8000F X-Rspam-User: X-Stat-Signature: z6iimzwyqk4jppz3qo8bz9xs65q175ez X-Rspamd-Server: rspam01 X-HE-Tag: 1704232903-532830 X-HE-Meta: U2FsdGVkX19x+prQkUcMp9qujcnSdNmYbQh4kbNRsjrTLOQ7q7yizek8Y/ftpx6fLpxjmGRyiosF4FeRH3d3efKtc5SR1cW+q/xBViUsL6YjkPw+bGj3Z5Zrrv1Bbk0HqrJvI6LA3W+JXdhc9MQQmUkJrY/OV7lrB/5iGrchvn8vSPJRURrbOmoZuHgQ4eupK9uhVIkHQyT1Z4gPJ+d2LlYpOkaYfQU7dnFTNhyj6UWXXdHrCxCv04XUb8zw6GZwsYfe18rp34GGcA7LE2F4vhB7QVfiwSLDrrodAC0Gl97BJvrgNK2HS2e2PLmHKMSQs7oMSvYXP31NYmguccIUp+xPcpOCKOoG+ixNCJgJ95kgwL+0wIataNb4x/Va57k0ZrgQDWbzhGrR5vvPBwXHtn5PGhQmhZbQHEMa4inaQ+uqCp5jb7WCcihBOn03qZQy5oX0hoRr7+RXgUrwjr71tPenC8fHrQ2VUeACBZ51wkyr0oKJcRxgFxwFenmJvTRrrGyTltwamJXEUj3OKINYl2oDNh4qyj2UgeF4Zmx6eT7SjsEQNNNDjj3QBgO4eYlmFLQLMRqQNO5rDH96swX08DPQ0tG4IK7XpYk6PZKWUSxXZ/6gwKNPHhD0odPNLQwcrsRxN3ob1DLQjZooyj/06ZZwwDwHKfMfe0na2Q0IR1KhoPinN7RLXA4qquAWFbOU0u8I0+MW7YWPm7AVI+eOxer+seX+Bgx0XMOZCvh7HK0DJ3xrNmBcFcNZaWayS5ofzTqiNnd4czJl4MzPbZdNtcQaDklIG52o3AdXCdeW1A4AIkgS5AMtSWVDtskYwaHZG4MYZUhgZ4j3x7YiNCSrcAdzhkVrc+/XZ8zDSduARINMrLOBODChZGGhtw7OaI3Fj9fGH9pIB/C0s7u9fdbd0Fu/pf66Jo7D38eqyjvHLD5cLzalDRF4yvGCpUcAnTeyzwAKmO4Agf7hj4OGK/1 +4+87HhY otAf9RxuiNfskrIbXMleyZ/c43LuWIiht3288NuxRNd0na7UE4yNvHs/q/BnbByiJiUqfeC338/5A2hNn5GBPVShxCp6gjTFhkmhaPBeO8pLuapkhHY7NqnZmB0ZDW49iZeGzLiGKeZUrrYkuKNC4GgSJhKuGR5MzH19ahj584N76uVS9tpyaMwHk69ck9OOE1DTPJjvNQ/SkKn5QAu60rR1njW9YR9DXfmkCLl1bggnZcqKFOSrJmdv+Jhtu8ObqgV1EeNLOlZZjK/Ierqzt73iAGb0OjzumdPOxw7S85pYcxqYCTecPLjOj83ylkADrFFoyAVl0sQoXURkaG2HlVCTk9p9EkOABii0BoSU1m0fNCkG5q9TUzfSgrDHSt82p50hxDAbevvxBsaI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In SMP configurations, all TLB flushing narrower than flush_tlb_all() goes through __flush_tlb_range(). Do the same in UP configurations. This allows UP configurations to take advantage of recent improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v4: - Merge the two copies of __flush_tlb_range() and rely on the compiler to optimize out the broadcast path (both clang and gcc do this) - Merge the two copies of flush_tlb_all() and rely on constant folding Changes in v2: - Move the SMP/UP merge earlier in the series to avoid build issues - Make a copy of __flush_tlb_range() instead of adding ifdefs inside - local_flush_tlb_all() is the only function used on !MMU (smpboot.c) arch/riscv/include/asm/tlbflush.h | 29 +++-------------------------- arch/riscv/mm/Makefile | 5 +---- 2 files changed, 4 insertions(+), 30 deletions(-) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 8f3418c5f172..7712ffe2f6c4 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -27,12 +27,7 @@ static inline void local_flush_tlb_page(unsigned long addr) { ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) void flush_tlb_all(void); void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, @@ -46,26 +41,8 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end); void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - local_flush_tlb_all(); -} - -/* Flush a range of kernel pages */ -static inline void flush_tlb_kernel_range(unsigned long start, - unsigned long end) -{ - local_flush_tlb_all(); -} - -#define flush_tlb_mm(mm) flush_tlb_all() -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#endif /* CONFIG_MMU */ #endif /* _ASM_RISCV_TLBFLUSH_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 3a4dfc8babcf..96e65c571ce8 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,15 +13,12 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pgtable.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o From patchwork Tue Jan 2 22:00:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5984CC47079 for ; Tue, 2 Jan 2024 22:01:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC3966B02BA; Tue, 2 Jan 2024 17:01:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C739F6B02BB; Tue, 2 Jan 2024 17:01:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACAE46B02BC; Tue, 2 Jan 2024 17:01:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9500C6B02BA for ; Tue, 2 Jan 2024 17:01:46 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5BBA9807B2 for ; Tue, 2 Jan 2024 22:01:46 +0000 (UTC) X-FDA: 81635743812.07.FB275A9 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf28.hostedemail.com (Postfix) with ESMTP id 882ABC000F for ; Tue, 2 Jan 2024 22:01:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mqWS/RZk"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232904; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=YECNGlXChk1NQK+pNB02IlUH9WYgRiJO0H/7AQVnMSE2o8K3yMKLkhj+wt/rGDwfaKU6Su ZFgTm/VwrIubEZtEirkgrjCQIr+txdh65hiH8pB76PoRY/c5SLSQXs0HgEqZSmn+DaVNrx ew06C1LelnDy4zhJvouusLKUVIBkOqc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="mqWS/RZk"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf28.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232904; a=rsa-sha256; cv=none; b=4uYm+3RXif4ZwuqsLa9SD4w2rc3czAfDMhfBu/r6Hgeaxfu/u9uYExhaMqcK+YyxJ9ASVR QBwDSnywnzq18b2+BVRxEAyjtEv2RWVu9LYwOlDiDiJaAqWksF5FtK071NCL77TGeIUcbt 46j//cvepMLIFA0t80GMJJcG3AUw4vQ= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1d3e416f303so24955855ad.0 for ; Tue, 02 Jan 2024 14:01:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232903; x=1704837703; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=mqWS/RZkWBFHASn1szrVdQ2ZAdDGV3IDvcXDcch7wXArBB4uNZo//Koh4Yg0fzQjKX GMmzDnmvEhgumIA9vCNMM6USSRk1llvC/971caXgZldGDKLHFmP4nDfh42+uWvmMBVUz JLRKd9S43/sBqkdoadFVRGSAtJAB3t6u4wz6BjhB8xEctO/lC/WG5EXIoJOSypin2c6y OaASkXzBSokHn4yL6mFkOWnMlqO4IgR2rcIxoEDBTFUaKhybd2ZdXlQNa7o+ePU2Fe8K XO7L+2TiYyWaj8cr0UXf9tapfxWWWwCDsXfMwqjkG1Yx///TsRYdQq6LVierYdcVsSoY t5Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232903; x=1704837703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6c+wYTacIgINHvokNTIzxqzbFxSPL5G6VDrdebR45nU=; b=Cytwevv6SYSv2GSWT1/ym9sZ/ZfZyew8si8k+Py3aXWCe8MMK+M4fll7S+wF7rN0Pi GRXyhIHAAEHrPKjybc5i3mfjocQYTSAj5U/Fl+Dxx8ar8IozgEglEIwboj9JRg/csOWQ 2qjjIo6AunXhM3+iz8ObOfZa2+/RDY08KhGLj4OnACN47cyqXVlReMkIZam8jjRKcCaF HNaxBgVJx2bxACAu3V9roPTXBDGQya05z0VrxMJS6i9f7RhgAI4vRjSdMQeWYdKSxW6Q JsrX28rBlxccylZXUdg/bEWNTasgglf7oRPLs0YEl2MwrcL6rSZKL1ZWkzFYpTC9ddHX MZaQ== X-Gm-Message-State: AOJu0YwoNx3vQViV22WlM3Bta1eSWv4HN4d/bnCrveVxTQeTsWo5Z4Gu zhZJ9TWqPhrHENkNHRb0C6dnkXC6Ew9ogg== X-Google-Smtp-Source: AGHT+IE2GdHLRUcW4ROKdXldqRiWcx2M7fdl7/RqRT5d/m4h5bCpV21xGk3IklfQQqw3IhSZ5sLqag== X-Received: by 2002:a17:902:d584:b0:1d4:5f9:d3d7 with SMTP id k4-20020a170902d58400b001d405f9d3d7mr8348439plh.127.1704232903384; Tue, 02 Jan 2024 14:01:43 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:43 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 06/12] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Tue, 2 Jan 2024 14:00:43 -0800 Message-ID: <20240102220134.3229156-7-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: bqkxgfh48owzcq7jth3h7u7edsfm6qwf X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 882ABC000F X-HE-Tag: 1704232904-707802 X-HE-Meta: U2FsdGVkX1/yp1ZcchS7EZlI2pfy4n+EIUjJxIS03GOGoPJ5Jx7rLQzTaaJM8i5ZtMbi098D+MLT0vJogJdS1r6CvM0j064beG0l8hCosNPh0S9pHgnI/t/cW4QOcAug95ujtU+/Jerzjb70sPHOfPonbzm/cqWJoFbcoMfDB3lELRwJL7vM9OncE66k7mEAVRCXKixY3riBTmoH2NTs2dySnZSAGn4yEIiaDZ8Ir+yMqMT0ZwScuQN0dnlKMkC4EGYql7pj5AbSsOplo/TrXe0/+oE6oLZB9To47pPOttddPAdxWZ1paXiXNGBQfSZYvZJAKKGANInvtaG/xn+QQrCLWTcLZ1IlFceJg4JJ/3/C3dkZQCsdstMyxmVl/KjW+LvatI3qbbI3ju/CSBvzPh0beLbV6ec4UlSC5CTJCp/LSMWTHizgWxsK8OlkGS24Y0+VPMsa/0Rf6ED994WDN4ROVzD5pgxK4riohNnLWNh5cn/+f1lQD7UIA3qfA0jBY9FvKKSTIQuAopzBcbGJok/zRzUWJ0F7WD97VPhEcouc3Q/vgDsPqJwUeQTLSoWqSiDOs97CTzIaEP1f/PVQFPqV3XgD6DeRE4AJ2A8uoX2taJQQYa28kD+DnGCEg0fA8vZV7N3W2RzhR8M5uQVzUa51yHR3k2MTcMpFaSNwkHpfiajD5PasvFEt+chIwQwG6Bixb4kHZt/kYuEcS3au5e1f70jDtLXyB4I7nE5BNQI4ySmWO4VAHmwPpGZtB9wiCbzxMX74hwom82n2gJTnQCcifyR4ZxnzliqH3MhcDygN1bdi/rkGrnNNOwqHNB8fkD6pjuMK4vW1cBiqxgXVM6LZ8ioZYR8NxvpHweDFnv6+nbM1Os03cuKE9e8IX6qo5BchW8NBigwXHRBTebztgxAKUn6+KYU7f/csClxuiNPbyqchxOD45KrErvCbFKSTqSMnrOyCiDZa4OXU2dF iGDm5xZ+ 6UQ8HmrZmCTwGZ7weQQb3CGm+r+rpPK+g2F7UHJwOkeIhoB0GQxPxwhJdDf04DKgoxPV0DSyFhXbuGSYCSfC7U7yJ+E5i7fY2iAVoQ8lLWqVeGN4DTmIpoBlmy9+Uqr/jTtWYLBWOfXpc2MdlzvDiPpdV5RWOrfqNwTWUw8jvw8XqrELa8ETHZuaDyrIaq8wQtp1rBAXwbzusUKpJyriclzwPgZ9Aw67BUfNyOp9Hx4CENGb/vglXv3CQIsipcXKy4iVBl1/X4w5+5ViQGdBg2rD2smw/i5iAU/i+WrvJagNvNTRCTdMys2x8FiTins9OBhXlZyw9NQCGtSQn9vHRoj97p1V/2Qp2QoHEOxayCEJz1viddrZr7aRWdQTwp7jdFB3+gXRMJLlSLpg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Rebase on Alexandre's "riscv: tlb flush improvements" series v5 arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 19 ++++++++++++++++++- arch/riscv/mm/tlbflush.c | 23 ----------------------- 3 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 83ed25e43553..6781460ae564 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -44,11 +44,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 7712ffe2f6c4..002c4c2620f3 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -22,10 +22,27 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ASID(asid); + else + local_flush_tlb_all(); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + if (asid != FLUSH_TLB_NO_ASID) + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); + else + local_flush_tlb_page(addr); } void flush_tlb_all(void); diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 37b3c93e3c30..292d7cf3c4f6 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -7,29 +7,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); - else - local_flush_tlb_all(); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - if (asid != FLUSH_TLB_NO_ASID) - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); - else - local_flush_tlb_page(addr); -} - /* * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. From patchwork Tue Jan 2 22:00:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9FFEC46CD2 for ; Tue, 2 Jan 2024 22:01:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C309E6B02BB; Tue, 2 Jan 2024 17:01:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B8BCB6B02BC; Tue, 2 Jan 2024 17:01:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B4CA6B02BD; Tue, 2 Jan 2024 17:01:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7D69F6B02BB for ; Tue, 2 Jan 2024 17:01:47 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4C77F1A00DF for ; Tue, 2 Jan 2024 22:01:47 +0000 (UTC) X-FDA: 81635743854.22.26F2642 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf24.hostedemail.com (Postfix) with ESMTP id 5AFD2180006 for ; Tue, 2 Jan 2024 22:01:45 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="UnJSlp/4"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf24.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232905; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1tIOva8KFaIwh3i+98q+ysPtJyT8vpZl2kFy86xHGzc=; b=jrS1IZ34yYC2eahijuJVR/3QucMnoO2SvYMjaGFvK8J3c2S7jiVpoUVJu6jsXfKUDWqZ9K 6oLHMUu+WR8IAi31Km3+t5DKHaMEKaDaOQaa4Ark7fXLQHiKLNuVt6FIMiv4cmIhvz003H W9MFzGLta8EH4KdmRLaOoUafHMwykCA= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b="UnJSlp/4"; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf24.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232905; a=rsa-sha256; cv=none; b=MNl6Q/fvd8FtCHOpuE12rdUBcM3lDrbTAtaCpzd3qtS45YcaKYt+BSwGXKw3ZWQODfQoJ5 UoN3l92P5Ke51qVng84XG9vtq4uUGGnasV5ZQuD2yLIaSCYCS8lCn8+SxxKmLD4F3LO1Z3 mcqtTTAct5PynS3cRG2AunBB0lp1nM0= Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-5c66b093b86so6909333a12.0 for ; Tue, 02 Jan 2024 14:01:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232904; x=1704837704; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1tIOva8KFaIwh3i+98q+ysPtJyT8vpZl2kFy86xHGzc=; b=UnJSlp/42Xj/xJNzGoufoPWn+6OgJBoQgRfIAdlUZcl10VAAkjD1FFi2/KOQ8Zv8m9 wA9JmCvNV4Nu/O7G/0v+ZR6QkjMK0FNXq3xK7ELqZxAtR9rTzZqIry7QKAmvoxIZohfj DHY7vvDtDFDZwfdaP/GEtFn9yz025lu9w0Uni54xAxEjWgno2KRzFHcufuKNjIjcxZI0 Bh5S6tWF34LFyxsnjIpFy0fwB8OUeVLwIgXL9W57x6Aqif/NtYCR6OkCWVnwhoLYpWER 8/0X9ouG/p8zcQoMO5+/IpUBqLLo8zeyP/rCClakkHFWepcTqfbwZKAJs4P2oETe3Cur Zdag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232904; x=1704837704; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1tIOva8KFaIwh3i+98q+ysPtJyT8vpZl2kFy86xHGzc=; b=M8BkLdiDrr1saAbn5kaYR4NO+fLrIUTB2z+XLkNMj0sXdCwdGiazxKH9pVtxAuus5C r166giapdVTRAao5nSPgnrLpskIaG5X4S2ifH+JdDkxe8wGKfZ1ldxI5xYysojZRLHB5 4+EDKiawJhXdySrk85U4Gt7yA7KeopzNKmSpBMFKsVv7+cH2ytBpf8Xf/24mmWpWiLw8 5jde3saZkPRertDPi3cPNsMapZ/IcgVxoSvMz5thksbOKs0qJLuDeZgnxI0ybT0GmuxX D/PyqctY5RV3kw4OWdzjpjWso4Nd4kVz5VC1B5b3bie0BvKpAKjR46aI0eEDuNxRtzaa qIYA== X-Gm-Message-State: AOJu0YyDzrLXLInP7ph09yjusqX6uP2lurpGUBwnMk05RP1iTKQ8d2h7 bWcLHVAqbpjaBj6ln8JVNvfy6BSy7GHk1w== X-Google-Smtp-Source: AGHT+IERqoXqHCzKpOJ5RqmrnOEOyFJpME6yqKz2yoKuSVbxPLF5GZg8UdvpqpiYsRMOZbO3DMheoA== X-Received: by 2002:a17:90a:d90b:b0:28c:300d:d858 with SMTP id c11-20020a17090ad90b00b0028c300dd858mr127429pjv.20.1704232904383; Tue, 02 Jan 2024 14:01:44 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:44 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 07/12] riscv: Avoid TLB flush loops when affected by SiFive CIP-1200 Date: Tue, 2 Jan 2024 14:00:44 -0800 Message-ID: <20240102220134.3229156-8-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5AFD2180006 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: tssbgf6fbix63hkk84e66auzakkca79d X-HE-Tag: 1704232905-217807 X-HE-Meta: U2FsdGVkX19SWqmfSUfTfbMuB2625DTQ3/02PjwFJuwNXbi8YyyxwunOxoPMuzV4yPnQBozgS7bQmO4c/cWwkAWSaYgMBnIUyGXoaF9L8NCTARg/hubQnYv9axbqyjDgiRArkOPjjD33QgVajFR75XTiuz6w4SP3E5Q1gYOQbHaZXlMpVwLd5eneBaaY+Ls2Rzl5bkSC1gZhT+XAHTzgUrP+BJHASg+p2/2orEemg5GTC9fSJY/pNlIA3nmsOKsqH7cx/igoU2mS4gmhFQTYXVvUvh4rYolVQ4eO47pIPoUZHT2GzTYatk6zGowRpVNd5HBxfcSxVXV5SFXHJ7Oz7VCauqMNCR8ohBTkZEb91TegwfddJ2xumMWzgBpgIfjwd7fj4S1q7cKeGGfV2Uyq27Dj/49fMTPeX8wdtmhxtoyFzfrXbRynlTg4cOCP5v9uXfcnr+mHlD7V3vLvxFspYrPf7dSzPCVOMc1gQkBYPbjd4ktcPOMv7XhaME0SjR6CXOLFrNWlejdpxtfeZcJXq55Rbrkdv/DC+GXp386Q4+p5CiqqoeIsAAe+eCD2UVe5OptMPSZHTdBjr1hdylIRgL8fA4K3kit8InF92OB5Ib9h2INabRZmM2kBp7A8xaRWbmio/BB044iHuQTJVLndhM6GUcvtP478apkaqK+mUv1fm9BC6jLsbVvBoutCYwBEtgw5jO++3uHQnGRgrJ7AA9esA/EL8jzp0zWeL0dwYcbuOkpi2efzD1l8MCcTiRUaTGxgjnpiQzolp8xRT3LzTqzoL7VbBI3MzAIA8EX/iMIEPLJeoHPEV1ONQENHIH53oTxX83IR3pewwCUfOSTZubie5wfSOEjcM25v7HJ/AK3fzOS+dpuIG+cHQE9JxidsQgrrn6jh46xvHV6TWSlL7qpnN7/YbJXTtPDPKbyHkoiOiUulkHxteghGjwnzMORI5m3gzgKorOblUX8P5vc 9z7awnvz CUImVpt8p+j6pTyWPB8Z6zbhrWu35qq6iTIHKsNd2PS/8iyFyFQXjKsIVVh89BEKqrLXvGOUfpkzQCtQ5GWuul+8HFQXOT2bMHMUBbQCxiAffiypkRXYCcQc+m+S5HhB07CWyqbptQk/Bk26bP97HJz7BwCUsVsdJwu52KIRmR/WFLUUm1VeWzR15BsIQme/Dexe+KE5Lvds1nHJP9G3vw+RK6pAz5a3jox4sn5x4j3JVzzXSWI9/q6X+guCxakbGq3rBErPtTyXPKgjYNIq8uZz3BuC1MWx1G4Hg6YBrNBRGeie6Qyxfh6902zqfeJMrb2dcFLLdJ8iLUDEgFxvz9sMoRJUPtzBxM6kMJVk7B9BG+VQh5xb9FYYURryG9GqLfzWsXgO3OIIXZW0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000267, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since implementations affected by SiFive errata CIP-1200 always use the global variant of the sfence.vma instruction, they only need to execute the instruction once. The range-based loop only hurts performance. Signed-off-by: Samuel Holland --- Changes in v4: - Only set tlb_flush_all_threshold when CONFIG_MMU=y. Changes in v3: - New patch for v3 arch/riscv/errata/sifive/errata.c | 5 +++++ arch/riscv/include/asm/tlbflush.h | 2 ++ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c index 3d9a32d791f7..716cfedad3a2 100644 --- a/arch/riscv/errata/sifive/errata.c +++ b/arch/riscv/errata/sifive/errata.c @@ -42,6 +42,11 @@ static bool errata_cip_1200_check_func(unsigned long arch_id, unsigned long imp return false; if ((impid & 0xffffff) > 0x200630 || impid == 0x1200626) return false; + +#ifdef CONFIG_MMU + tlb_flush_all_threshold = 0; +#endif + return true; } diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 002c4c2620f3..d9913590f82e 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -58,6 +58,8 @@ void flush_tlb_kernel_range(unsigned long start, unsigned long end); void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif + +extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 292d7cf3c4f6..76b24d4ed4ab 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -11,7 +11,7 @@ * Flush entire TLB if number of entries to be flushed is greater * than the threshold below. */ -static unsigned long tlb_flush_all_threshold __read_mostly = 64; +unsigned long tlb_flush_all_threshold __read_mostly = 64; static void local_flush_tlb_range_threshold_asid(unsigned long start, unsigned long size, From patchwork Tue Jan 2 22:00:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 455E8C46CD2 for ; Tue, 2 Jan 2024 22:01:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC8AF6B02BC; Tue, 2 Jan 2024 17:01:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2DC76B02BD; Tue, 2 Jan 2024 17:01:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87D836B02BF; Tue, 2 Jan 2024 17:01:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 70AB96B02BC for ; Tue, 2 Jan 2024 17:01:48 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 49A131408C3 for ; Tue, 2 Jan 2024 22:01:48 +0000 (UTC) X-FDA: 81635743896.09.66BF6FB Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf07.hostedemail.com (Postfix) with ESMTP id 5FBB04000C for ; Tue, 2 Jan 2024 22:01:46 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=IcRwTY8z; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232906; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d7sRColwM16ZDpkHsKhbx/H1UjX1V2ZP/gk6hKZVk/k=; b=Hslo/OSibwXSDUl6CBPKZfBGQku19xxBYFw/n1H0LpnT13Zi3KnKNMO6zl9sK/Uy63YaKt hEI93xJu/mxp1BWOUWRRDKG8aGshPxZpGfTq5zqG4e0/pGhxXoa0l2/HEBzq33KLBlklRz uAJlYHWw5LmbPTZHduxF00FY6e1T3jU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232906; a=rsa-sha256; cv=none; b=3HJhw+OOoROPNmG3J1S5dGT/i77vA3eepXAHunhNJT93Ph1ukPXZkpXdQxHXAjQDlpkugt IONzt6/RIkNuZlTHy3MKlqDXJv0AHAHK8jZ+p4DTq9rff7yJAPWKL+stPDqJbnmsb15xA4 bBuGir5QvILugzuLZVQW3FHPStCQwg4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=IcRwTY8z; spf=pass (imf07.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-53fbf2c42bfso7074853a12.3 for ; Tue, 02 Jan 2024 14:01:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232905; x=1704837705; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d7sRColwM16ZDpkHsKhbx/H1UjX1V2ZP/gk6hKZVk/k=; b=IcRwTY8zCTXnyt1SomyDm63guphJcp3MyYbXjYmgaYIfecohnDjL0nWAJCHThTTZJ9 PqjSMHVrXnhU/x8gyTngI34RJX8bl1eeRf6n5ACVsDepPgIXLB77ByBUaYUKqwmKCYPG /KUXZzqpvQ9imlhbm/xCBZMeLbQYZkteVd1Q7M5bBiakA9ARaF5pir9VA4beg4t1Qr4o iat6x/aKpiEFv/OmGsOY3aWmRMdlDWGG2r+wiIJU/HvWod9hBdW87Xqnq1Azs3K5bhYb GGDBTG9mxL9k5l9NarUa4K9nhr2j6EaZCKYzUJ/1PQLi5E/WpKeBEfz9K14gVuZ6M9B5 /GGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232905; x=1704837705; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d7sRColwM16ZDpkHsKhbx/H1UjX1V2ZP/gk6hKZVk/k=; b=YIMHqhamhsnINc9rJiUcrPwrmPXJpR+FeiiPN7apyH9hPYF70BWvzZk0WgwvFITXoa oiD1Vtd9Xmo8Ga2+xMnr+a2oxVHaZ0nBmHk0qwu/+/fql9Y5pE+xhoswagtTk7PWOh/v EfKeoru0Fxl7YVeMKSrooAISJlGIZbuX6g77HS3wnSkS2YVZl9Hnf7iYtj4ote8RkTl1 l8BAQrV9R7Sib6q+PDp2civuW145B6Ai9WT9ydXeWlYv5l+IWYV8UJtc3z56BjVLtxMg TcJHFict+z8FfRwon2GmID0Ygee8Ikc18d0YHe+HYSuV477G80RFvhwuJUig9yvujlly cpMQ== X-Gm-Message-State: AOJu0YyXtYc+aIdgxIHVE+MhGJt+R/yWkl49Ih2Sa/v9o4ScxJA2ur49 FL+FeYzPaVNHn9gpUShK0JOsbMHsU4dtaQ== X-Google-Smtp-Source: AGHT+IEo3bZM2aLuSMOGwMaVus9GR4XwWujHLJvE5N9ecURPZ+yptZqxQ7nXUllBGYAXlfndapu+WA== X-Received: by 2002:a05:6a20:a107:b0:197:2fc0:7e25 with SMTP id q7-20020a056a20a10700b001972fc07e25mr2866566pzk.3.1704232905452; Tue, 02 Jan 2024 14:01:45 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:45 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 08/12] riscv: mm: Introduce cntx2asid/cntx2version helper macros Date: Tue, 2 Jan 2024 14:00:45 -0800 Message-ID: <20240102220134.3229156-9-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5FBB04000C X-Rspam-User: X-Stat-Signature: mzzd8u3wdx79wnitm4hbx4huq56q7644 X-Rspamd-Server: rspam03 X-HE-Tag: 1704232906-538272 X-HE-Meta: U2FsdGVkX18Q5960fK7+gv2iixh8ptxP6lX1GXrwH0zR7YUdTvR/VZ+1XYDSAPpHMU3ppoobVePzrIdvPcPJ7QMpGw2zqER7qmNeinriBwMfLumBWO9j24HURQ2aQHxRH4jMUDy9l6tha07gGiTEBHHgkfOS5qQsVH+JfBPPn8HFZBEbDl24gjQDroJzPAO9sHoewXku1QurZEMOIvWt/uLAt7+TxcdhfKuFK49zcke75bfCsUqXYLWWsxWlmThLob7CurABLG4ZEeTLXIhMrv/GkYCPywXSMAzxAhjG97WGad1TQAL9su9jCK8eqdct7akwsfWYB87AAk7kqRu137AOeG/H6R8iUoOrVnU/AkpKhAUPqbPEA/QNunpQh2BIVIAVq1Eui3u62KmWRgQbTAqB1B64cJITknlnCTYNQGb9UM/mCj7C0DrtFSytodswTvf0lPdC8vwSkHNZm10vCjjNw0iIr5DSgwFmGnua4cIWkr+pm7QMlqa3uMmKM6l8BinaaA4i7DmCZTXcOhyy67WgNiuJ9dj5N9VxKa/7Zy8yf+4YFSJQL2q9S3s+D5VlmR4wCASmG79j47BKATEEvJPBjDVG/rr5n9vGNcHwGUXcdkc5+W1cUuJulMGy1tYY7+cQHLFWhFdzG16Mt8zkTdfYR4x+agbqxZF9jJu0NYiEqaLVd6ziHySprbna4FFEt5jzOdfU/GREeeapVT3DHHDblm1ZXLOnSDdGF6AjZ+XtaV73z3UDtN8urDGATPL85N6OUz+Bnqk+PYqJoaE9fU+VgSauqaIG+rfDdKau1JqPSpIlphCHq9C04tAyE2nv/aTRmsDAGGlk48ZhNkZAnT9fvObZwzatjr+m7cdzGfDanoq08fSYRsWk9D8nOr1E0derbqrM1n6xXGmi7CdcxwlN+VxkrxL5mTLUHnv+LpLaZOVHbRIA7diH6yrdlFMNkENLxs4IFnk4VDr5Kv4 i9T+ekAc Q9DBgmHj9LdpbqEwr65SZC9EvYfGCIFppbMHgKzSHL9SZ/s6Kzlm2m7JyTXnOsy3ee1qHNzcvf2LN6XnqWn4QKspOM6t1a8DFVn1iLTQyqPAAf2vuaZ1/Iy7SBdgQiVpmah0cmMj08Jj7DT5pJAuvXNILwaL7R48irLCpoljb73gXIAF7lxnQleMUMaf9djmcBIMkcUAxN8BCdOkBkLsXfyshN9B+rWU+AZjDVH6v5bPgVZ2G3vzLypr0IQIh1E+U5/13x2Ipz26MPciM4Jq7Z2TfetG70mMZpmxRH0dLp+CYzBKZUyskVg7Hn9EqIGMRbckHat74fo9TGqp4c3HuzakZxhnxCxgsdoxK8vOHdGHjtTc8iX8DUyY+S+wjAfleWeNTvvohECWoI28= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000351, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When using the ASID allocator, the MM context ID contains two values: the ASID in the lower bits, and the allocator version number in the remaining bits. Use macros to make this separation more obvious. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- (no changes since v1) arch/riscv/include/asm/mmu.h | 3 +++ arch/riscv/mm/context.c | 12 ++++++------ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 355504b37f8e..a550fbf770be 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,6 +26,9 @@ typedef struct { #endif } mm_context_t; +#define cntx2asid(cntx) ((cntx) & asid_mask) +#define cntx2version(cntx) ((cntx) & ~asid_mask) + void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 217fd4de6134..43d005f63253 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -81,7 +81,7 @@ static void __flush_context(void) if (cntx == 0) cntx = per_cpu(reserved_context, i); - __set_bit(cntx & asid_mask, context_asid_map); + __set_bit(cntx2asid(cntx), context_asid_map); per_cpu(reserved_context, i) = cntx; } @@ -102,7 +102,7 @@ static unsigned long __new_context(struct mm_struct *mm) lockdep_assert_held(&context_lock); if (cntx != 0) { - unsigned long newcntx = ver | (cntx & asid_mask); + unsigned long newcntx = ver | cntx2asid(cntx); /* * If our current CONTEXT was active during a rollover, we @@ -115,7 +115,7 @@ static unsigned long __new_context(struct mm_struct *mm) * We had a valid CONTEXT in a previous life, so try to * re-use it if possible. */ - if (!__test_and_set_bit(cntx & asid_mask, context_asid_map)) + if (!__test_and_set_bit(cntx2asid(cntx), context_asid_map)) return newcntx; } @@ -168,7 +168,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) */ old_active_cntx = atomic_long_read(&per_cpu(active_context, cpu)); if (old_active_cntx && - ((cntx & ~asid_mask) == atomic_long_read(¤t_version)) && + (cntx2version(cntx) == atomic_long_read(¤t_version)) && atomic_long_cmpxchg_relaxed(&per_cpu(active_context, cpu), old_active_cntx, cntx)) goto switch_mm_fast; @@ -177,7 +177,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) /* Check that our ASID belongs to the current_version. */ cntx = atomic_long_read(&mm->context.id); - if ((cntx & ~asid_mask) != atomic_long_read(¤t_version)) { + if (cntx2version(cntx) != atomic_long_read(¤t_version)) { cntx = __new_context(mm); atomic_long_set(&mm->context.id, cntx); } @@ -191,7 +191,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) switch_mm_fast: csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | - ((cntx & asid_mask) << SATP_ASID_SHIFT) | + (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); if (need_flush_tlb) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 76b24d4ed4ab..5ec621545c69 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -85,7 +85,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, return; if (static_branch_unlikely(&use_asid_allocator)) - asid = atomic_long_read(&mm->context.id) & asid_mask; + asid = cntx2asid(atomic_long_read(&mm->context.id)); } else { cmask = cpu_online_mask; } From patchwork Tue Jan 2 22:00:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E301C47079 for ; Tue, 2 Jan 2024 22:02:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7ABF6B02BD; Tue, 2 Jan 2024 17:01:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BD9A06B02BF; Tue, 2 Jan 2024 17:01:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A04A86B02C0; Tue, 2 Jan 2024 17:01:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8CE4D6B02BD for ; Tue, 2 Jan 2024 17:01:49 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5B6311607CE for ; Tue, 2 Jan 2024 22:01:49 +0000 (UTC) X-FDA: 81635743938.23.59C7327 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by imf22.hostedemail.com (Postfix) with ESMTP id 7AAD6C000E for ; Tue, 2 Jan 2024 22:01:47 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=gDo1Z9hx; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf22.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232907; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g2cmoIK/tf9yPohv/YvO5QQvGunkgXtONjIGDQZEX58=; b=hUrxeNtK6Foc51Xdd2TCBe6x5WfYmmQp8/Xh1EnpHRcjgpeANXCKaOgChwYEsAG7UPLqcj l/EmKSdJJ26U3n9ng/a4gSdWn4uPQueNSNx8XU15QEzF9pUvd2HnF5D3VIkye0gsoEsi6d DfWI6zcVwlY1DuwShsfTKlIcRjbPqi0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=gDo1Z9hx; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf22.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232907; a=rsa-sha256; cv=none; b=xd87iGhy44glTwpsw5cHgxnUlSmafraM/VFUJKDJrPQDLf58wzQThZnjVWWDxFZGnzN2L2 lT4MN3IT/PfAfzb7+rD/A+hDrgaTdPo2wEkpa+8KaR1j9SrrhFqfEYWnhbWbAZb/QnTB4+ Ma5o+8/5U/l9B5TJ5WdgJi3blsV3K3o= Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-5cda3e35b26so2566053a12.1 for ; Tue, 02 Jan 2024 14:01:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232906; x=1704837706; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g2cmoIK/tf9yPohv/YvO5QQvGunkgXtONjIGDQZEX58=; b=gDo1Z9hxmPZQ52z6J8OsH5X3YBtzIhblefHQHZM8S8wUxrSmZMIPv34oh0u6KGS8DP lFNyrAgyP2Uo1mGdjI7A06i87d5qocasTuZfdPeJxs7zhYgU0e0004be5dRcaz5w5Xl8 5xxYQi9EnStSZkOYi9E7rpT9LYEA9Tt+zEZIn7UGRxb8P6Dco56bZyuQ93uI8WmtuOm+ LcsDuxXL0t4zQLuBvwBdAOqqoDGEuMnuhkXBqzztsVSim7rTU+W1jtp1HfMZFEMVu5ft ebopNR8b7vW5CpOulj98p/HLHusp0FCYrTBFAA7w7V3gXBMqmomIknvltDxJq29X59R5 CD5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232906; x=1704837706; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g2cmoIK/tf9yPohv/YvO5QQvGunkgXtONjIGDQZEX58=; b=u7dVWo5Sbmb4/XpFcc0Rcg0X2FieC/NVqNeKWIlHp/67jhPOB9odWvul1qdJsReTO5 tuiX/VT/aEZ1bEQ8rG9dhgB/wDPFe1TRXUakElYJzC4O13Ucxad3RiqcP74w6gkKiSAS 3UWqjJGR09ee7DY4ZRxKS6C0HawVFOUekCtQ2nDiGWOKZmfn2nXvp1euLiuIp7O4W0eC ulIoBVyLnupFxVs/LJDhfY9TmZKGJpBUZYe4fg2IXZepNqnKGb0DCwVf31rZjKCGoy56 b+Z9GK4AbDnSX83dOw3I5pH3K1YpBnsaaq51PTwgf6B6VNzkJqbViDtRl7uhDiahM2s9 n1gA== X-Gm-Message-State: AOJu0YxYyfNIG98annoEkJieIxcAllcldZokwUzR3VIkdlYYQ+d6TQ2B sA/XL6OrcgLgDqHdSBFShjq3xXR6fu8Sfw== X-Google-Smtp-Source: AGHT+IG8w/0THQ/uqOPivBsiw22hWysSlz1giatwLj9QOMmwVtysp2KaHolkndd/R0ziY6DtdT9KEQ== X-Received: by 2002:a17:90a:134f:b0:28c:b1bd:3c42 with SMTP id y15-20020a17090a134f00b0028cb1bd3c42mr2130687pjf.55.1704232906433; Tue, 02 Jan 2024 14:01:46 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:46 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 09/12] riscv: mm: Use a fixed layout for the MM context ID Date: Tue, 2 Jan 2024 14:00:46 -0800 Message-ID: <20240102220134.3229156-10-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 7AAD6C000E X-Stat-Signature: ud9iptpm3a1548t6pd9z37twwe1tm8wp X-Rspam-User: X-HE-Tag: 1704232907-206414 X-HE-Meta: U2FsdGVkX18tjH2uXQk6FXEjmzHd1qLs9pf6nXuWJoXixeXC3JG5+G9cAdx5HFmB2163f7fU+Q+8uYeIHxqFVkK1+G/Z7OrskXnao1IMz/nSpk0X2mESCphBQhzFBZjwwhAIsaGLHmV1PmZmn2QcPdys/Q9ZT7U3HAzGYeFGsU7jPPld5xfOJKEFq1N4yUk/2Cm5c2EVOpFrcs9tHXRo+9kzU/PqVkixqwuChjC26sfY4kAXB/WtYcifjtbbyv70hGsaB+0LGfszVa0nFwnUySyHQAufkTh+9y7Y8gX7PaMpY96n+z2OQw3tQb54jWzYmjL1cOBYLM/e/CjOoQ0qQy+mdbRwUyHTkvdIsQCjnnU+DmphYSqFXVlkiR2jIT67Ji7i+LXt3jP+2e/r59IIPbRapyfqLPsYjoiPJ0CS0sCzcBHFdM6aBzmzcu+b5qTwmzvxqkFDYi3iLDyHpGOkmzNQkzfdWbVf3mk9PiYWeOaBB9DW919yrG//O9c894d1hUlOnsfIW4TRixHb2BORb1X9mg2GzNw3Bz6DDQG1WMQrolIIxo8RQwf5EeYC1JtrbpCh2O/e0hB5hYzTeHmmaD0ma3CsEtyLqJV7AJaj/f/TqqU2AO8ngtWIU5/04BYuCg+hes+kZ13APZ+aSUYOg36Kvg0vBVk4SHkhEiRKpbUonqkYuhAHSzrWoVoPdEdAvepLnnw4cpAurc/PunEXf0KrJUuXjxdcnntpXZn1V3V4ro98HoGNX+HYD5d3A0TcHOaQU0cNyKkYz/WUdXYU9Bw386C6eDw9hwybJKCjCYHpF/aMB3toXTJYlZtJ5avHRuTNcnOW9pMO6UQGPBrB7LmQ5mbAVCO7xlvh1GqJPJOXk5CDdrl0rJE6xnzux7Bp/QBeB+0kmfh3RSHKvTtUUD9pnTT/kE771vZIZ3qHG4xvkmAJ8JSPPigDDVAFR1qosOmXdoaFeorKDZpoc90 OWNLzlGy HmrZd8tXNicfESg08RnVQaJgEee92+BjcMsnNzyIx9oCbvMgYpeo+gg3NMwTWEAdzGcEQ3ajNsN+p/WcnXt4zwz3W7hkACaCM2sCmETcET+YqCxNoQ40NKpzhnnsKxr5F9OSQgyAm+U2a22eQZRNG185mJeUNM0YD0CL1IPStmIcQDAcydo0Lw7AntNiKK8XEdWgzJa2PUkhki/XzbY024FbEkYGDc6WS+PBflGrgpLUwtyJNnUrfbqCQ9BTjZEuPAAuUpFkR6wk79Pm9HkhrlbWEub1TzKsot9MXjj3pmSfAz+6aLOH19JZlYj6vSKiUeq+4NPKiDeMNzh1BJOnOLdIVCEqJ4YOEez25flo9uT0T10ZDhWANKGo3MXExEc93vwgqwTqRckFacSs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, the size of the ASID field in the MM context ID dynamically depends on the number of hardware-supported ASID bits. This requires reading a global variable to extract either field from the context ID. Instead, allocate the maximum possible number of bits to the ASID field, so the layout of the context ID is known at compile-time. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- (no changes since v1) arch/riscv/include/asm/mmu.h | 4 ++-- arch/riscv/include/asm/tlbflush.h | 2 -- arch/riscv/mm/context.c | 6 ++---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index a550fbf770be..dc0273f7905f 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,8 +26,8 @@ typedef struct { #endif } mm_context_t; -#define cntx2asid(cntx) ((cntx) & asid_mask) -#define cntx2version(cntx) ((cntx) & ~asid_mask) +#define cntx2asid(cntx) ((cntx) & SATP_ASID_MASK) +#define cntx2version(cntx) ((cntx) & ~SATP_ASID_MASK) void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index d9913590f82e..5bfd37cfd8c3 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -15,8 +15,6 @@ #define FLUSH_TLB_NO_ASID ((unsigned long)-1) #ifdef CONFIG_MMU -extern unsigned long asid_mask; - static inline void local_flush_tlb_all(void) { __asm__ __volatile__ ("sfence.vma" : : : "memory"); diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 43d005f63253..b5170ac1b742 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -22,7 +22,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long asid_bits; static unsigned long num_asids; -unsigned long asid_mask; static atomic_long_t current_version; @@ -128,7 +127,7 @@ static unsigned long __new_context(struct mm_struct *mm) goto set_asid; /* We're out of ASIDs, so increment current_version */ - ver = atomic_long_add_return_relaxed(num_asids, ¤t_version); + ver = atomic_long_add_return_relaxed(BIT(SATP_ASID_BITS), ¤t_version); /* Flush everything */ __flush_context(); @@ -247,7 +246,6 @@ static int __init asids_init(void) /* Pre-compute ASID details */ if (asid_bits) { num_asids = 1 << asid_bits; - asid_mask = num_asids - 1; } /* @@ -255,7 +253,7 @@ static int __init asids_init(void) * at-least twice more than CPUs */ if (num_asids > (2 * num_possible_cpus())) { - atomic_long_set(¤t_version, num_asids); + atomic_long_set(¤t_version, BIT(SATP_ASID_BITS)); context_asid_map = bitmap_zalloc(num_asids, GFP_KERNEL); if (!context_asid_map) From patchwork Tue Jan 2 22:00:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABE01C47422 for ; Tue, 2 Jan 2024 22:02:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3FEA6B02C0; Tue, 2 Jan 2024 17:01:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AA1966B02C2; Tue, 2 Jan 2024 17:01:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CB646B02C1; Tue, 2 Jan 2024 17:01:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 78AD36B02BF for ; Tue, 2 Jan 2024 17:01:50 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5597EC0190 for ; Tue, 2 Jan 2024 22:01:50 +0000 (UTC) X-FDA: 81635743980.12.B6B0CB1 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf19.hostedemail.com (Postfix) with ESMTP id 6B1DB1A0020 for ; Tue, 2 Jan 2024 22:01:48 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=nXDLSsaC; spf=pass (imf19.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232908; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O8uQuAx5HbeGrqX4Ip/qQY7nv6lrxw4qacCJFbpKbko=; b=Fi3Pykhsrubq1twLYADK3zRDFoLcaW44uM2BpkJcRdC+I0AeD/a8KRUfBxZkl4NUrRsMXC TLdD0eRYQhhIBsDefJ7PjKNsQX4Zo585NgVkaIeWXECWfBTqSc2j2l9qcvZpLLtwkguTKF OxqBtWOJQlnntJMi8l+JukuZkz28+u8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=nXDLSsaC; spf=pass (imf19.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232908; a=rsa-sha256; cv=none; b=Z1L6MBF1siF8ZLRvJMyM+Z4KIRtEyDAb7IHQPsg4q001B5l6waX66Z3Wt+7LOkc24zEYV0 CV+8j9gi6w9TtMesgN/1uqI7JZ6CDihs35Tte26x1Jq8k1feN12e464ML6zUe4VO9M7o+7 GQvSJriWrCPyeiJBJxbLlB9SEEz05Gw= Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-28ca63fd071so1666546a91.3 for ; Tue, 02 Jan 2024 14:01:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232907; x=1704837707; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O8uQuAx5HbeGrqX4Ip/qQY7nv6lrxw4qacCJFbpKbko=; b=nXDLSsaCoyybkEIVBcdHGv6vTXEyNKn8KBHqEs+5fjjk00796Gv6u3WxcVadqsIOpf xaBbCW3X1/exti4FKh17ssur4zc3wqp7aIT5dGObdqOulr0UaNv1Sr65QyX7tUbJcgzO oj8OvMmBVqpGgeKYygKGz9iQJXT0esPqUPxH7j3hqD3WyDKeUcIKgWAKpTSO73o8OWz3 L4KK3qw1F67kg2rDX6y1qAJy0ojIpq9RSu478hWcZ1HUyU5QMpeEYGqxpOvqkamLU4nv vsUCi5wuTnWLA57zPDjSUItH7eY69LH7Kq+ZunUtonjLFmUy+uVOXFDAXLywUXvPgjSc eaiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232907; x=1704837707; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O8uQuAx5HbeGrqX4Ip/qQY7nv6lrxw4qacCJFbpKbko=; b=qr7DAkyR5Vzzo4k16yXql7cEdZdz5yg81v4KX9zaibWwknCtrIMGOV4u758eTNh2Wu qnHpAaetRHpsKbybLCBY7RpzrsJMQm/s5IRWTLsmvYGqUPGyDWMcry1SMnDNtrhutFy2 oWaRKlj8vMwoUA1nTFtD5ObbqoUwa+tgPSqnv9r8Ww8/tgBImgYeOuvUgY4li8q74lMe d+8RpeUNVs6PnE5xBfVb/8ocheHayUOPQq8QBqUg3E8D9Q2Z070IjGYn2ec7GpDrUyr3 7lp7GWPJ0DL49+Yct3fbF+yDWlWD1LxasCbQuT+FSA3WXcZvZKx7s5F3mP1Bw410fRPa JvWg== X-Gm-Message-State: AOJu0YwBrFuS1/+KMFbviRa1FJmMQ98P3IQeaPqYHOVlJvsiF9DvFnVo TgJmE0+o1tOZQ82f8dH9gaGCuYVMHxJ9BA== X-Google-Smtp-Source: AGHT+IEU78MnSVmG+tTKoY5SsP+KmgJHfZS8cB0cAVh744mS+tR8J0bWreXJl70FWHY5n2NbigpZXg== X-Received: by 2002:a17:90b:33cc:b0:28a:f2cc:e008 with SMTP id lk12-20020a17090b33cc00b0028af2cce008mr6619129pjb.54.1704232907387; Tue, 02 Jan 2024 14:01:47 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:47 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 10/12] riscv: mm: Make asid_bits a local variable Date: Tue, 2 Jan 2024 14:00:47 -0800 Message-ID: <20240102220134.3229156-11-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6B1DB1A0020 X-Rspam-User: X-Stat-Signature: 8bsddqpijj41nrs19x6wg977x9q5e4wc X-Rspamd-Server: rspam01 X-HE-Tag: 1704232908-315841 X-HE-Meta: U2FsdGVkX18EMwknaV0fr+6Ol9ZRKvNIOj1qNIhFO0li17Z1PYTdjt64T+Os/HMTpMpXVdlojeDUFTGWwaGDsGODH75doGdH9EHP927Q/8x6RZSf/EfWRlvtRG4PuKPoGtIsVa9w86LxVzXw8OhGndPMGV2UMRKAXSzXPZ97pEQnB/LR3qwg3lOhS4b7yZ2dmrpcLuN7vbBbQS9RKW4byoDjTwHUrus1qS9dnbQlpTt98bR6cTeDf1u6m6aW15XTinGXBkp9rNGQcwKIoMtm5t25JH9GqiMdBKpcpNjWEO1PD4nqiejkHzx3zdX9rbxzODfXisQYEE2+7IqMiCwftjTLHGCLjSRzMgWRg/DMg9kv4fzP4qQGOof7P/jD260TtaPuB98fcskPt2GGAggvxn1xZgXNW2dhQ3jz0nT+NKuwLKbThQAJZLm2Sxx8AxTO1Uxooz4nyTQzMBm3eGAEWVqLz+txUuAfj8lgrU1faXoneNcFpjxvPpgxtQIs9OirmzDg7AFx8CuNT2RKP9hC/Zs2K+6ghYwraxC8Kn0/EsWfSUY8kTDPlIE3xVu4MAWSgQSVQ6J7I6Ncdhbso2UcYUSb9O+3CgXjaiXfP2CXdimemrW8pCh2GtMwXizXvnOUaE2YT7p+4isF8/FMPQhJnMrAM3/aFjREDSDYCQ9BTk9QANCFgeRlREr4mOOUCdy1wXRIFVJvLUy6FEEp0c6tM6U5wRow+Cgq9bB1Pffxdwc9XFY0CAUBQ6Ln/25iq9+N5atY0ttcqjAxhG2EzXoo6yh3qjJdkYo1UiiHTf+S9G4TW4oaCs2uAdxc6N/a21BQ+uo2ZxMLuy6qiasstqsy82yGXSOgSJ0wUphbbogWRS50qXZ/6R513jTIaGk95gBrlVhpVZB8yTzOKs3yjvPLVqqlJsZvWIabMZ+BfzeT6TuJCcft46wpon/VUsDTHIzioinOp/8nx+jJSd6LUHz 3gpX5Go3 OKB8VSim6XtIgQkScQ65KSKIs0zkzDgmvius86t2cYqyAhlUpNjZyWDA5VmvvjVJvyNIJ5vZMVPMpk5r3R2rPf4MqfvuklnF1EepwZ3C187k7TRsiMZw55oKPI2cjz/FGBkwUPoHXVGFHzAygVEvCFZ5Qt5Xo0WT6CiQaW5uLfMQuNfHjSrULmegSLIgvkAq1prVDJToMEp8DSqVpCIfsJHaYI8UkW8TvRohlRX+6drOAuCql/SAwGiVNdJnLdlXb21hVZRwzuMYaNaWHzib/8zLX1PlYHwvnFJwrNhGNZ/UYHPGNT0Sh6N30qYHCfchqBQOiama4B8aiBkx2UQcQcHeOporSjNG1aODtRBeBI6ur8y2jkv1dcPDgJTyOfuf9+V4R/SgHqCnou8HUYVsQ7GEHUT9I/rsFGSLu5Vbp9K+W/J74mC0qAK8gHIgHlfaE/tiq X-Bogosity: Ham, tests=bogofilter, spamicity=0.002168, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This variable is only used inside asids_init(). Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- (no changes since v1) arch/riscv/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index b5170ac1b742..43a8bc2d5af4 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -20,7 +20,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); -static unsigned long asid_bits; static unsigned long num_asids; static atomic_long_t current_version; @@ -226,7 +225,7 @@ static inline void set_mm(struct mm_struct *prev, static int __init asids_init(void) { - unsigned long old; + unsigned long asid_bits, old; /* Figure-out number of ASID bits in HW */ old = csr_read(CSR_SATP); From patchwork Tue Jan 2 22:00:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64F6BC4707B for ; Tue, 2 Jan 2024 22:02:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 880AF6B02C9; Tue, 2 Jan 2024 17:01:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7945C6B02C5; Tue, 2 Jan 2024 17:01:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 635D76B02C7; Tue, 2 Jan 2024 17:01:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4B5616B02C4 for ; Tue, 2 Jan 2024 17:01:51 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2E266A1722 for ; Tue, 2 Jan 2024 22:01:51 +0000 (UTC) X-FDA: 81635744022.30.1E4FE82 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf01.hostedemail.com (Postfix) with ESMTP id 5599940022 for ; Tue, 2 Jan 2024 22:01:49 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=ZqgLgv4V; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf01.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232909; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MROdnZxFJAn7brM6ye/jIKPJcvVCbRqlgsjUSxtP0PY=; b=jXUJhn0ZmZFhI8Q37uUURytdt4qiwwSEplebsrwyOMDm3bXxxABKC/adXEG6EtiEifPftL rStoAVml7EWyGa4UPBUQw1vtvIbtAXd+mM8TrYgmYSN/IqxPTPIc2IpgUM1OzK2egla8EK VWBmxiDUKQyfIiQ4c1STn3pPYVgE6+o= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=ZqgLgv4V; dmarc=pass (policy=reject) header.from=sifive.com; spf=pass (imf01.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232909; a=rsa-sha256; cv=none; b=WPRbcoWlYQPECI9Gck31h/g3E0UOV0tOqTHtRnxYeiTc/CatKFuRTU43bc7ozX8vI0Werb EmrZ8wLyYj5wd4kAvV3CxGEwyUuF5CAy6aM3hmyxaBHPfdjN/Xfmv11bfdu4VfoFWf7FLS o9+7kRuj6B7H9fVmWLWSGVznqWaCVHo= Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-28c0565df34so3429746a91.0 for ; Tue, 02 Jan 2024 14:01:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232908; x=1704837708; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MROdnZxFJAn7brM6ye/jIKPJcvVCbRqlgsjUSxtP0PY=; b=ZqgLgv4V2JFcgV8TWlh09ZTu04M2Gg9eRYenerjcs+ukkA0UF5OkxokoKFb2v9vLg5 44fzqQ9Smbomj9La0fTiRp35AhGILCNuiqLNQLoTUInO5u5BY0WgGIZ2/6AYr4cw3LrB esl3b1cs7oqExwZU3UpzS2TK5HCugO8qPZW0srxH8Ru65n2rpcj2Qq1GkYgXC+b0l4Nt qfaAITxkhegSrxPCCkQFhdIIcXkj1xeILLy42Ue85saihlUOXfQWPPZ2HgPA+9z8HO2p 50BwXOiuCi8/ap87ogBSBePJu1HZR5ONS2X2ThFFovO/ID2jPDIBCrO7ziqDE+s2LUJv zWng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232908; x=1704837708; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MROdnZxFJAn7brM6ye/jIKPJcvVCbRqlgsjUSxtP0PY=; b=CpfTc+gDZmMw3wo6DgFsXOpuplATN4BB5cDbCFDLkDXA3AnuwIPOTgx/LERujguwJL J1fsXtJQ5IApt6X4BIczA3ng0WnTy+JsTMcYQTYsCz0zl2IRUirBlX1BOQfWuUj0WCwY NMvDG3esDMZdPsae/HOXyVPA1rBlSlHng1gOxhq5faaaynDA/yWU/rUH9qRsHnOoRbKV A4FZe6grni6HxeC6rJQ6TFVTGPwl49FsnEGBtW5z2ZQJ1ZR+qZNfM4agJ/AeL1kY0Z97 nWz42RbnttBHgCyx4cPMxqi0S1C99Qt1JJ4aog1x6ErgEduaJeXg+zvJ14QbfKPwB6Fn pY3Q== X-Gm-Message-State: AOJu0YxdoI9SCmF299B2W8Pxa/DLoxOpnCtRo9/iLKP8aPGB4PKDbQgT ae25lgtSfEcRAx3fb6egJxbP6eZJlDdAKiUBnCbiR8xZoYI= X-Google-Smtp-Source: AGHT+IFQ4MdRufOELZ2z13xg7yZ/1JoYbB5tE+2va7NZu/tN1uOevH1SKPh+nsWqBiMa4TjGPndiZg== X-Received: by 2002:a17:90a:5c82:b0:28c:2e1f:8bf2 with SMTP id r2-20020a17090a5c8200b0028c2e1f8bf2mr4930306pji.84.1704232908323; Tue, 02 Jan 2024 14:01:48 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:48 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 11/12] riscv: mm: Preserve global TLB entries when switching contexts Date: Tue, 2 Jan 2024 14:00:48 -0800 Message-ID: <20240102220134.3229156-12-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5599940022 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: g6upw8eudas6ft95ziaqxonmdy93ojmj X-HE-Tag: 1704232909-226413 X-HE-Meta: U2FsdGVkX18Lm90bzzV0MXPysp1vFQaWuv8J7a8EDsjGY/yXxQZ8eGvRUsmG3Hpz4KLqsHdFbLGqwrRQ3aueWTzxEnOxuAtj4ghl7ZxvXnb4udQupFMgJr5qepbiPqQaXniKJJMy1pFGcjOBc2Z4D7CpdbzWEsqC5CAqfMT7WLdI0AvaW+umWF7O4pHb774QAjiUspNhPx0BjKPL/lsnucA8F+DDoF0f+VUvE9agn8Zwz6IfMbZrTPfLYFVi3zH+yue5Xzd6iKZlF0/6aNKDh2iYoX2tv5AYaPkJLCZfGUTOuahlvOywat6dGXV5+y8iWr94uyzwbUHeRbcK7npBEo73HxFyJqd+2WX6Z79zvfN/t9ekeCf4Sp2iCbeQ5KW4edGKHbMl4d3cAxMZ0U6z0kBoDcohUgbyZyWDKLOh68uWxkRFp6PXqGrmiU75htaHLho31xMUuv6Y60Mu6FhKVyJl0XsgacWaeD7Ph7J5L2OboAOvllMvgTx/VaEUIbN0j2Mvd4JyxqaLBSGeDiB1OMOg/jroThkzyMntyCP0sa2xDEQTfJMMmHvDW15JcpLdfRHzSy4mbIjDrNhUOM6f6fkw0dXXMKM536UvS7/uYXOUSNdV78XmfXZHkXEVuf1EgJekSWjJIwOJ7DTyRsglW1BkRkzhtwJObvhe1u8OliNWr3VlEAgXJCw61vcLu3wS2lE6qq7bxSY0OxV/YXT0FUMp/INHM3qSiqBRc2UfhLgmgKfaEkZmovxuFvsi8+jZODFnV2cHmR3MIxo8grSvNI9xVAlWT2tUXrkQ2nW7uF1F/IRhV+rFF/o65qkpUXCnb85SMNcV+kMBB1Z6O+Kf265WkmhcO9nOPKZO7+VrfWAbS2u1Xg8GQgUpP02bQjm4ITyXlsDc2rYgw2YVv40ptoQ/9Bk2TVOg52q6GYwWJjsnU1J55d0w9keaEomUxw3f7aRlwTmNHAL+NaU8T8R mQOvDXSE 6IELSmtlhNfx0fXu1XGpzW0S72nLezO7MLlCfRP13DH/yuYY8BrLACqD4neNRS0HVglDxYKRU6N14BR6vBtUq+TFocMJg1L9IzI0+lS3KAmP7A+ySfJq20V8h/weaSJ3e6OUvgVPrbNRRndpLElKqGF1es9g/+k2bORoj380ES11U8HRt/a6gTOW7RrKd18TtPyrtrj5X53qMWlRwJGI1Jf80NRJK4Ey50ttLaOKfwRRwgpvXXWFt0usmxdG/xQoB45PDhI1mxVuZiu4ltN+dmlXZMeIi/Wx2BX44JEXHZrlB/Ot8ZUnnTfVXYhWdQwQO5v1YhO3rP4crteJJtwE6OOPMl7mBVQOKiHD268Z+AOsjdJGUwH8iPRGFvbEPVudxn6peaYgI5hpz66w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.004345, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the CPU does not support multiple ASIDs, all MM contexts use ASID 0. In this case, it is still beneficial to flush the TLB by ASID, as the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. This optimization is recommended by the RISC-V privileged specification: If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. It is not possible to apply this optimization when using the ASID allocator, because that code must flush the TLB for all ASIDs at once when incrementing the version number. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- (no changes since v1) arch/riscv/mm/context.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 43a8bc2d5af4..3ca9b653df7d 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -200,7 +200,7 @@ static void set_mm_noasid(struct mm_struct *mm) { /* Switch the page table and blindly nuke entire local TLB */ csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | satp_mode); - local_flush_tlb_all(); + local_flush_tlb_all_asid(0); } static inline void set_mm(struct mm_struct *prev, From patchwork Tue Jan 2 22:00:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13509539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C4E4C46CD2 for ; Tue, 2 Jan 2024 22:02:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 80A736B02D1; Tue, 2 Jan 2024 17:01:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B9206B02D8; Tue, 2 Jan 2024 17:01:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60C256B02D6; Tue, 2 Jan 2024 17:01:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 4D1A66B02D1 for ; Tue, 2 Jan 2024 17:01:52 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 25F2A1A00DF for ; Tue, 2 Jan 2024 22:01:52 +0000 (UTC) X-FDA: 81635744064.12.BA23E92 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf15.hostedemail.com (Postfix) with ESMTP id 6A68CA0013 for ; Tue, 2 Jan 2024 22:01:50 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=DSACYx13; spf=pass (imf15.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704232910; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ntXY1BdgwMpYCVbC/qFvmg3dz3t/ZzKIeqKl93B+ekg=; b=Tl2UNHI7X9KV/Kd7/IwlWJjpd9FpCU5V8o9seO92qWlVGe8Hx3QQxmvjnjhKjfeS6eFDGt xVzX/oCzQqpNtFYSZpWTTM0Hgre6WRI0SBPt0IiTcPvBSHeqodWQ2Poi8iKl4FtFOTYckP w7MoQq4yDNc/8X9d8KfcRS7BGpeH4kM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704232910; a=rsa-sha256; cv=none; b=vx/Ijnr9Y/QYeystThQiDJGdJFYlVNvUegTbsIJRjAJpK/2l8pESj+Qku0djM20NXEOR83 b9gSblCPScsFLOquK8nuz2lZfJhHseAAQwaYZ+ylGmCvWgOd0eMzc63/MX1nhytxIufwfm OXBHM9hoSz4m3t0vdPyS5j7zx3ijJYA= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=sifive.com header.s=google header.b=DSACYx13; spf=pass (imf15.hostedemail.com: domain of samuel.holland@sifive.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=samuel.holland@sifive.com; dmarc=pass (policy=reject) header.from=sifive.com Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-28c2b8d6f2aso3491273a91.2 for ; Tue, 02 Jan 2024 14:01:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704232909; x=1704837709; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ntXY1BdgwMpYCVbC/qFvmg3dz3t/ZzKIeqKl93B+ekg=; b=DSACYx13PPFLfxZGwB3A+3+kgMv6sksoUzr8Ot2sYCQ2xi8+II51TpxvaJmwTdECrm F3fvZQOtTd+6ONybY3Fgrehh6sZFkKr+LsqHBQrlMUXM1/5rtc1Wrqk446A9OgYo+yJU +Pba8qPnAaw1XpbC8rdyLhTWZtyVqtu+pBxDL+oM8Lktkd9VNg7K1cnN6trM6B2u2KVO Fu8NfBF5/QJ2cj6gZMVK4pgywkZNKX4nIzg1QYIEAEGGJfV7DT0gQuz+gwTjS8ky2Q4z zCtDO1fbnPi+PHz0zgUaXdt3OFm5y9uY0EcSJpNFe+xqEPI8vs+AJ5snD8/05JdeEvoB A+hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704232909; x=1704837709; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ntXY1BdgwMpYCVbC/qFvmg3dz3t/ZzKIeqKl93B+ekg=; b=meATExgqpgoUKhjQK+sb/pn77e1TDA1SPoSOx05vVr6SGljn5zTBhzxe/6xzry78nm zv4CCYLztH9JMPuRq14D9qnV89wJJ91r6WrsFLZa887B5u5YvkWo5BvYXWWCLs7mzi60 FGx3182G3mkWViiMvKChs1smCOyIpqWVVGnqvzUn6sFejMBz3foRZ7Q/gpRZyYq3GoSL OJm9y6CV+lykLqJAJyHa0e1rVSB0HmfXNjEcQvtiaFX3O+Msd0jZYHRPAMkQJ5U/c5w7 7YP/zcO/2iT+UavcbEiLm4mOfQOHxYJ9BUpME3bCnVvhQoyAHPkuRyGjOVQglWlFqae7 qenA== X-Gm-Message-State: AOJu0YyKNgRLJaGBsp9QkYULlLBIhnSoYH7PFzK6RDpCtqLrnAAhJxJq NIMDsfqjDznkXyVrkgo8K695CXMjbGqaMPTNekQw2htNQ3M= X-Google-Smtp-Source: AGHT+IE6m2P6rWzQ+Gmcf0Ghh7A4crRVy8Bq5ePskuN3Kp4kY6ZC52OLPfUohVtqOdFfc+/yJvJ6pQ== X-Received: by 2002:a17:90a:bc46:b0:28b:31f5:9668 with SMTP id t6-20020a17090abc4600b0028b31f59668mr5241043pjv.30.1704232909372; Tue, 02 Jan 2024 14:01:49 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id r59-20020a17090a43c100b0028ce507cd7dsm101724pjg.55.2024.01.02.14.01.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 14:01:49 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexandre Ghiti , Samuel Holland Subject: [PATCH v4 12/12] riscv: mm: Always use an ASID to flush mm contexts Date: Tue, 2 Jan 2024 14:00:49 -0800 Message-ID: <20240102220134.3229156-13-samuel.holland@sifive.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240102220134.3229156-1-samuel.holland@sifive.com> References: <20240102220134.3229156-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6A68CA0013 X-Rspam-User: X-Stat-Signature: ipyr5y96de4oaehrqj45e8wbjmnj41hu X-Rspamd-Server: rspam03 X-HE-Tag: 1704232910-477586 X-HE-Meta: U2FsdGVkX1+BfynnXJfBdd61b45WHXdYP5iiXeDQifrhu74qCkFs941xvnjaNsyBR7G23JB8d9U7oSVQ5iNFrBPmULWGUJc6EZAu5AnQqamWuj+Q4ap1yoMzLn1ybhWTrv192U3GgeloxmlaFUpyAwfYZF5eZXxiRB/0eUZdKoxC9cONlEe0qOWomDr6VPxF4RSTiH/JjvU9a6BKalz1ZAAmQHJcl6IHaGw4Rb2VReeDaGePCSQ576J/JBY3JLgKQoJx9kYIV6+fGX0fxp8fURIX+Ugq2Cg+0O4L9MnWKiWiLKW2Ru+sJMj4WRlfiv9pL9G0RQuQmHsxmu104+D3lJzRqB4Xo5T5hTlZqqsJbUN9TpytAUDKYZBSNP65MlrTZeLFZnH954Z6pjjD8DGK9NnicdhUNfXVLd9Gv9IeUoa3Y0F4Z2yXHL/MxXnrKGEBBN8eU19FB2dFjtRcDYnmOfjqnm0zsGYiEuYONyT616RROzMzH5Ufe02ziZ7Pzw8gY0mYpoLMbqv76tO3+3qOdq+ASQBX3hICqGY5ydd+a9ikHr9NKaCDkhAIQPxfwAiG4Y/9Lj8+kd5be/LgyCvXmQUuXgoM8Q95EPRmGjJFU0udSWEtdb/0gvaaLnr711dIoZqI/GwAPARm3Fdto54613B1lZS/ohqG8bsHGKuBlpVNZiJU4rlJJSQIMD/q/T3G5w5xL9+sL4CGQnUncCtwQAuB+kIROvqYFnT0F1I7DPAEMRbeXlwcnnconNZfI5Pvjr7vXuj8gkDajlGvLN/hE1dgs8QZFFnYyeVXsM3OD3eSyK91zABTdHWxw56974Jd3L9T6vMwEVPzvSl6B7lZtS3fuxMzrAKB5vvYhZa077yoPUgFzxAEPHZ/wAfBWy76sASQJQHNn63GDpR0Jpg/B5WBBvWVuWq5c5tJcoSb7EkUenyyX4NcpQZlQxRkPQFWC8df0A1cmpw/3ft2kFm QzNx6KnF E/3OhIsXIOYc3E4FLq865l0k12DJtoCT+0QX5okosJGunUsvspfCQl0RcfCy9dFDkxo6G0XNMQUgymZinEqVI0p4kwQyw3EBW7KMgPoaccubzLfg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000245, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Even if multiple ASIDs are not supported, using the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. So it is always more efficient to use the single-ASID code path. Signed-off-by: Samuel Holland Reviewed-by: Alexandre Ghiti --- Changes in v4: - There is now only one copy of __flush_tlb_range() Changes in v2: - Update both copies of __flush_tlb_range() arch/riscv/include/asm/mmu_context.h | 2 -- arch/riscv/mm/context.c | 3 +-- arch/riscv/mm/tlbflush.c | 3 +-- 3 files changed, 2 insertions(+), 6 deletions(-) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/mmu_context.h index 7030837adc1a..b0659413a080 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -33,8 +33,6 @@ static inline int init_new_context(struct task_struct *tsk, return 0; } -DECLARE_STATIC_KEY_FALSE(use_asid_allocator); - #include #endif /* _ASM_RISCV_MMU_CONTEXT_H */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 3ca9b653df7d..20057085ab8a 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -18,8 +18,7 @@ #ifdef CONFIG_MMU -DEFINE_STATIC_KEY_FALSE(use_asid_allocator); - +static DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long num_asids; static atomic_long_t current_version; diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 5ec621545c69..39d80f56d292 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -84,8 +84,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, if (cpumask_empty(cmask)) return; - if (static_branch_unlikely(&use_asid_allocator)) - asid = cntx2asid(atomic_long_read(&mm->context.id)); + asid = cntx2asid(atomic_long_read(&mm->context.id)); } else { cmask = cpu_online_mask; }