From patchwork Mon Nov 8 07:11:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 12607925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B934C433F5 for ; Mon, 8 Nov 2021 07:15:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 31EB3610A2 for ; Mon, 8 Nov 2021 07:15:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 31EB3610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=lUbcRRuOynHJrojdrSQ923PPUq7iKk/sX8GMOczpZH0=; b=xCWhcAVjlgjBYWI6CKqcr0piPk XiWnBvce7xhNQ28K5fyk7euWVtY2CRum8gF0osjC2darc396dC4hNDynqGI2hrEkU2nvxZZeSLLT3 Nf8nQHT7QpjUZ5FSRdReP/aVxEsG41nYXjfpzOwqm746cxBy57/oLB+nUIXCjnP2WlRdnty0r42uu 6A7y4ZwSiSrc9vRzGnVrzBrly7H2eMCIcfbKldRUzVLAfrsvdGtDtw/1Kp/IWd77Om5siA9T9Ua5o P1dLdSO1iRXFyUJloiT8TuuWaPR6N0tDr/fGkrqHcNEP8YB6R6wzFnrJJf/N8FZkNuULt1IdJMV5l hAD2WT4A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mjyqX-00FXXl-Du; Mon, 08 Nov 2021 07:13:37 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mjyqS-00FXWf-B8 for linux-arm-kernel@lists.infradead.org; Mon, 08 Nov 2021 07:13:35 +0000 Received: by mail-pg1-x54a.google.com with SMTP id z7-20020a63c047000000b0026b13e40309so9552513pgi.19 for ; Sun, 07 Nov 2021 23:13:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Ff0DQyyijjS/HbJd4Jx1dDiydS91TZk8zbynNORckFk=; b=fxuKjCaKJE0HCkTsIgy6Q50cMClMu5Iz5hF+VIdXaT069AUhAlvmRLj8uR8lJSi2qJ uLcPSvL1vTx8We6sDeUiMQzuhSoJ4Nunae/Kqf4zYR352X9r2YegxPnAt6E1fW/i7K/P 5vKk8nYxF9yqD7yIL/BzKnO80lIafwVUaHSwYWN1XGVgDofegEf9HrdrWV0e8Xr/TvvM UAZhTtRJdbN5RAtvIjuSgdb87us7Vge2OYuM4uG/UAqJICeNqfMuG0Jy5ATmq90Ff4xb 4OKCPjDBxQNF0dTaCOoHUAKslapEyStVGHJ3coxJlSAwi86tYm0eDyT4o/JIpDOmrQuS YMsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Ff0DQyyijjS/HbJd4Jx1dDiydS91TZk8zbynNORckFk=; b=KugSqcSdWR0QSHH/VP5NPcAcIo2p7GWUYxyobNUwyqRQfx9vjHJT4J3UsNr9L4kMWP XPqOWDkYr+HtRga/E9Ig341e6inj91ConcFE5UzO3rwmpcYinvYiGEMxqamwMYo++Z8n SPNWChdK8FqgI+BJyx4dPa7C+JD76Y7RrmrGIhsnBT6NC0hWoidg4jZZUJE/tzamwYC8 Q2reDXD0MWXAvpH5Yyv+QvoopiwIK7VULHnz1rkcItyXGNYDZgz9i0Z5fOCQRQzhTfgn Rhehgsp87CWKuUuSpChTFw80/g8Kw1JSuZxqWXtWXUpXrEAQrIa7e5RdPWmypYsFxxP0 XpEg== X-Gm-Message-State: AOAM530AAr1Sty90yAnfvvMKPTW1HklJd2YF65nx3kkP/JYx7fLLdDhf FSpGwjx7nwsUxq3Mcm+X33I/FsL/JiI= X-Google-Smtp-Source: ABdhPJw1+vD08mBGsf/uyUXTIILriyxJEePDxoIz9F9k4SGszh+IWPgIV0nho6tzixOkntLKN1Qv5XjBjyc= X-Received: from reiji-vws-sp.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3d59]) (user=reijiw job=sendgmr) by 2002:a17:902:d718:b0:13d:e2ec:1741 with SMTP id w24-20020a170902d71800b0013de2ec1741mr68261112ply.38.1636355610831; Sun, 07 Nov 2021 23:13:30 -0800 (PST) Date: Sun, 7 Nov 2021 23:11:48 -0800 In-Reply-To: <20211108071149.823570-1-reijiw@google.com> Message-Id: <20211108071149.823570-2-reijiw@google.com> Mime-Version: 1.0 References: <20211108071149.823570-1-reijiw@google.com> X-Mailer: git-send-email 2.34.0.rc0.344.g81b53c2807-goog Subject: [PATCH v2 1/2] arm64: clear_page() shouldn't use DC ZVA when DCZID_EL0.DZP == 1 From: Reiji Watanabe To: Catalin Marinas , Will Deacon Cc: Marc Zyngier , Robin Murphy , Mark Rutland , Peter Collingbourne , Evgenii Stepanov , linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211107_231332_410860_016B81F6 X-CRM114-Status: UNSURE ( 8.60 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, clear_page() uses DC ZVA instruction unconditionally. But it should make sure that DCZID_EL0.DZP, which indicates whether or not use of DC ZVA instruction is prohibited, is zero when using the instruction. Use STP as memset does instead when DCZID_EL0.DZP == 1. Fixes: f27bb139c387 ("arm64: Miscellaneous library functions") Signed-off-by: Reiji Watanabe Reviewed-by: Robin Murphy --- arch/arm64/lib/clear_page.S | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S index b84b179edba3..ea3a6c927fe8 100644 --- a/arch/arm64/lib/clear_page.S +++ b/arch/arm64/lib/clear_page.S @@ -16,6 +16,7 @@ */ SYM_FUNC_START_PI(clear_page) mrs x1, dczid_el0 + tbnz x1, #4, 2f /* Branch if DC ZVA is prohibited */ and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 @@ -25,5 +26,14 @@ SYM_FUNC_START_PI(clear_page) tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stp xzr, xzr, [x0] + stp xzr, xzr, [x0, #16] + stp xzr, xzr, [x0, #32] + stp xzr, xzr, [x0, #48] + add x0, x0, #64 + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END_PI(clear_page) EXPORT_SYMBOL(clear_page) From patchwork Mon Nov 8 07:11:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 12607929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F8B5C433F5 for ; Mon, 8 Nov 2021 07:15:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3CEFE610A2 for ; Mon, 8 Nov 2021 07:15:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3CEFE610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=d9NCBIJZkHhLXRw0yzSRYOA77ME+HUqASIWiTBvIans=; b=cRLLMRjFK0XqVEVe0mHXIAep/i 1maIY6sswp1YBu4Oh5do2YyGckOVoEZw3RemnDal3lBLvQgpXGeLPDjbkXXZH9OddBY1Mled/xkNQ hMyauFsPo79DrYG0mHulARpXvQfMOr4ULYJb3LorRIBNlFh537H19CKXJdaZw0rT2h2ye16vy71Hh uFg+TelKAlJqncrVN08QvPfjjH8x8Ef3P2PBH8I2EiKuDjMlWynkL81jOyhHzk7G9+OPtyLKZ193U MgfFzMm8v+whuU21Wl60EBu2WJ0g3WB97zO4PyID2j0qa+cnLTPid+60nLNGWFsUKGFtNgbCUslPb Kxl9Yipg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mjyqh-00FXYs-Fw; Mon, 08 Nov 2021 07:13:47 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mjyqZ-00FXXv-Fs for linux-arm-kernel@lists.infradead.org; Mon, 08 Nov 2021 07:13:40 +0000 Received: by mail-pj1-x1049.google.com with SMTP id x18-20020a17090a789200b001a7317f995cso3435016pjk.4 for ; Sun, 07 Nov 2021 23:13:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=71FBlKH/4s/ijUog8eQGS+WTSKhCNJPna+tg4xjN4Us=; b=qRv61C4qEUW5b8WUJGzs2/iMJEZ5yBIuvINcdtNsdW4u/5MfaF0oqtNGZ68eajj2XS RJ8RZCLZYKtgGzL63zRmKgV846hMPFu0K98ARWPhPE7WoSDlHsyV+ol06vy36quT+N6s 5kKbpUdxywo2mLhvHpg6szB/i+FXH28JoldCQqJsg1OKZM+dyrOPlllwCU0SqHdbyoyj nOvEEfnGp2AKrRXuTKWmcvFobeTpPINToPUflR1HMNugXV7Ess3v778OphG4Pb9AU0C/ jXhmqQw0taYnO8815/Les5brC0cAptn7kuFpmHyKW/+tXSYMK8ANbfyRv6n76bZ3ENKT cdgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=71FBlKH/4s/ijUog8eQGS+WTSKhCNJPna+tg4xjN4Us=; b=VVu/qqWPwyChiaMZxovKrv0bTrLSf7ItirrMc/HhRgYX/fW9I1Wue5OmwDAYLNGdYz 2T+dPzQeaw1CGKWAALfcn7Z15aoJ1VKdajXlJpys5SxbXiU/jXrTbcPuiAv6LZPboXCS B0OP+q/9qkYdiw1C26mDIYak4jMD0/0+IflyzSRY4U1sovIOPbuYisjBUsNX/4uCqKkf SzCmz58sTnukCUVDLcWA2C9Qa0cjYLuIs+cyln8WtQV+CPUniBwWzOAetVjSqbkPzefl 2GuzO0ic6pMqVOviWaxeaJOGSQWatfQsXqnHWe21AJhPJLHHjQElYkTSs9za8Z/IfEqp rRqw== X-Gm-Message-State: AOAM531om0YVaOphmaALDbwQQF5KePpIhr+gnqaUiJzUVsZvaAmplRhL bgo7K7IUmNmCHGhi+7BwC+N3zDZp+4s= X-Google-Smtp-Source: ABdhPJyTrPpaYiOQcvFqfY3Oo2H7V71RVmy8JRjrxAeQLOK8E/TtzaYV2Es/euNVXcxYUCzK3IaX/20q5zU= X-Received: from reiji-vws-sp.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3d59]) (user=reijiw job=sendgmr) by 2002:a17:902:7d96:b0:142:87dc:7dd3 with SMTP id a22-20020a1709027d9600b0014287dc7dd3mr660913plm.11.1636355617907; Sun, 07 Nov 2021 23:13:37 -0800 (PST) Date: Sun, 7 Nov 2021 23:11:49 -0800 In-Reply-To: <20211108071149.823570-1-reijiw@google.com> Message-Id: <20211108071149.823570-3-reijiw@google.com> Mime-Version: 1.0 References: <20211108071149.823570-1-reijiw@google.com> X-Mailer: git-send-email 2.34.0.rc0.344.g81b53c2807-goog Subject: [PATCH v2 2/2] arm64: mte: DC {GVA,GZVA} shouldn't be used when DCZID_EL0.DZP == 1 From: Reiji Watanabe To: Catalin Marinas , Will Deacon Cc: Marc Zyngier , Robin Murphy , Mark Rutland , Peter Collingbourne , Evgenii Stepanov , linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211107_231339_560789_3DEB3B36 X-CRM114-Status: GOOD ( 12.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, mte_set_mem_tag_range() and mte_zero_clear_page_tags() use DC {GVA,GZVA} unconditionally. But, they should make sure that DCZID_EL0.DZP, which indicates whether or not use of those instructions is prohibited, is zero when using those instructions. Use ST{G,ZG} instead when DCZID_EL0.DZP == 1. Fixes: 013bb59dbb7c ("arm64: mte: handle tags zeroing at page allocation time") Fixes: 3d0cca0b02ac ("kasan: speed up mte_set_mem_tag_range") Signed-off-by: Reiji Watanabe --- arch/arm64/include/asm/mte-kasan.h | 8 +++++--- arch/arm64/lib/mte.S | 8 +++++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 22420e1f8c03..26e013e540ae 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -84,10 +84,12 @@ static inline void __dc_gzva(u64 p) static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, bool init) { - u64 curr, mask, dczid_bs, end1, end2, end3; + u64 curr, mask, dczid, dczid_bs, dczid_dzp, end1, end2, end3; /* Read DC G(Z)VA block size from the system register. */ - dczid_bs = 4ul << (read_cpuid(DCZID_EL0) & 0xf); + dczid = read_cpuid(DCZID_EL0); + dczid_bs = 4ul << (dczid & 0xf); + dczid_dzp = (dczid >> 4) & 1; curr = (u64)__tag_set(addr, tag); mask = dczid_bs - 1; @@ -106,7 +108,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, */ #define SET_MEMTAG_RANGE(stg_post, dc_gva) \ do { \ - if (size >= 2 * dczid_bs) { \ + if (!dczid_dzp && size >= 2 * dczid_bs) {\ do { \ curr = stg_post(curr); \ } while (curr < end1); \ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index e83643b3995f..e62c048af337 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) * x0 - address to the beginning of the page */ SYM_FUNC_START(mte_zero_clear_page_tags) + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag mrs x1, dczid_el0 + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag 1: dc gzva, x0 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stzg x0, [x0], #16 + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END(mte_zero_clear_page_tags) /*