From patchwork Mon Dec 6 00:47:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 12694855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5593C433F5 for ; Mon, 6 Dec 2021 00:49:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+ACOxmpINlfvej7uoiZKkgIfoLXEwiRRf+qnQU3y9sA=; b=FqrOd8zdhCtpwaX8EXrFwNz8EX nhuCb9ApxkNeGaoQB9r6whCzXJDaXO59w3VV0uvuA5WCpv1MEkwPBkZ/VJskyLTQgE6CCQiiwfcMw WZCR6O4q2HxPDCDgyItDGpJQ+MJa/purU3uHuvBn+nOk8CJsEkTopPpE1ntXiaar6zoWjk+rvJzCN J3Dl7PQTjVt3aXGf1dPkYlS5selATg3MnjROSFrqw0BpLWYtF80yFopenIKJW1AKq60t1wdSc81CB J97t4P/QQ1Q+5TO7eGRzRz6C/zh4PgZAq+lzoOyG3Q0WqcjTKDaMCLVbpi5jAGNAkKFy27a3bUNBp 3i4hg5mQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mu2An-002GPT-VU; Mon, 06 Dec 2021 00:48:06 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mu2Ae-002GJm-2j for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 00:47:57 +0000 Received: by mail-pf1-x449.google.com with SMTP id p1-20020aa79e81000000b004a82ea1b82bso5630604pfq.1 for ; Sun, 05 Dec 2021 16:47:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KsOznll2QzSvZVqEJOjZuGFneXu3otpLJElYDfbF2qA=; b=Q3DNXFG66kWeNZGr5r6y3DR4gjO0BgHFhM/wMuHleJTtufKYo0KG8MLwZhlA6Y5q45 82ADroFr8NPUxUB9o95M/nCNc4KMPkArDPMepE0R5PDwRaFmJBmqGxtiV6EoLKNDwwR1 1iFVjFmzXiKBYvSmULNXcEBiD3Ae3cgQ/P71izYHF87t/uwSG88y36V9KCcdV/YCNoFR xInYTKCkY1OGX6/Iu4hUFcotqkY1qnoIskCTMIrliwsQ26TMkpxXZXs9+72aEwjf5gZF 5NEQummgPVytTWP2aLbW78aX1/yr/C0JNH04V1yamsxK6dkJZGMcMHxgV8Jaf+4OqDAg uvqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KsOznll2QzSvZVqEJOjZuGFneXu3otpLJElYDfbF2qA=; b=YHUbBaRpa6lWZCsOt0GVrQbsmEgV7CjbzWRZu751rTOHvtN1349rPR0sXpkvaxrVdN vQ7c9MTvokyCk2wRO4VYf8f7cLhUz47+GmlMxeB9eiFof3vGau52QdreGFsHBEtPT06N 0U1nK7HR6hneXF9YIGmkf2NFLXbfEJSS2gQN7rnTWVPOIBm9wmVbDlph9Vbc11puFzYB T4CMqLiGhpmYMaq4OB/alnGBr2JqUlDL6+tc7cTgtRw4+CPHx19zPhYwBfx355Sf9HPu MeuCRn/CmrgapjBHdWns6Deqh7BnOrgdH9KbGHC+jIJHxkbyyAUAFhbbvqGvFNe9+A5B N3bQ== X-Gm-Message-State: AOAM5329kEdUgUtHgF+vo8URDzAgqKWW4KPvoZ2AwHqFLBz66A5AbTKK 3IWb3hI8PxyQSSPY5N4lIvMHu+62Rfo= X-Google-Smtp-Source: ABdhPJxRJYIQILux9h9zY4HEbOzawFAJiz97BcS18IjIarPPZ5F7kk3Wtri7DZoBpdx7IUQEolDjdl86zBQ= X-Received: from reiji-vws-sp.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3d59]) (user=reijiw job=sendgmr) by 2002:a05:6a00:1392:b0:4ad:5688:3e7d with SMTP id t18-20020a056a00139200b004ad56883e7dmr7308835pfg.59.1638751674177; Sun, 05 Dec 2021 16:47:54 -0800 (PST) Date: Sun, 5 Dec 2021 16:47:35 -0800 In-Reply-To: <20211206004736.1520989-1-reijiw@google.com> Message-Id: <20211206004736.1520989-2-reijiw@google.com> Mime-Version: 1.0 References: <20211206004736.1520989-1-reijiw@google.com> X-Mailer: git-send-email 2.34.1.400.ga245620fadb-goog Subject: [PATCH v4 1/2] arm64: clear_page() shouldn't use DC ZVA when DCZID_EL0.DZP == 1 From: Reiji Watanabe To: Catalin Marinas , Will Deacon Cc: Marc Zyngier , Robin Murphy , Mark Rutland , Peter Collingbourne , Evgenii Stepanov , guanghuifeng@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211205_164756_152162_E7B548F5 X-CRM114-Status: UNSURE ( 7.93 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, clear_page() uses DC ZVA instruction unconditionally. But it should make sure that DCZID_EL0.DZP, which indicates whether or not use of DC ZVA instruction is prohibited, is zero when using the instruction. Use STNP instead when DCZID_EL0.DZP == 1. Fixes: f27bb139c387 ("arm64: Miscellaneous library functions") Signed-off-by: Reiji Watanabe Reviewed-by: Robin Murphy --- arch/arm64/lib/clear_page.S | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S index b84b179edba3..1fd5d790ab80 100644 --- a/arch/arm64/lib/clear_page.S +++ b/arch/arm64/lib/clear_page.S @@ -16,6 +16,7 @@ */ SYM_FUNC_START_PI(clear_page) mrs x1, dczid_el0 + tbnz x1, #4, 2f /* Branch if DC ZVA is prohibited */ and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 @@ -25,5 +26,14 @@ SYM_FUNC_START_PI(clear_page) tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stnp xzr, xzr, [x0] + stnp xzr, xzr, [x0, #16] + stnp xzr, xzr, [x0, #32] + stnp xzr, xzr, [x0, #48] + add x0, x0, #64 + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END_PI(clear_page) EXPORT_SYMBOL(clear_page) From patchwork Mon Dec 6 00:47:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 12694856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 13A03C433F5 for ; Mon, 6 Dec 2021 00:49:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=A+Uc4g3GC5kzHOZTE3xIxo412qPWALpMWg7q6rE5IEQ=; b=DrP9zRda8qrNJspB5vpWwYL6J7 efJgkkW6+xVnuC/xdfWgYcrd5fzjGfoXyij29OVBdh6GQ0qDaR9DSv7am5DkwUInfoYHRyOT771so 9sxSWzYLapmhqVCRmxWcI0GOg5oFbBBFaDwoZ+Nis9uYfWLzT/J26Nc0GQYMn5e7NSVag0f7jIefe A57lMu4yH+5bHoimr8icR1aGITXHulkQ+fy/2SGBkZ/9sFnwu5LZkQgm5M3wt89d85cJfLpCPcEu/ IHjPFRt+dcWsq4Nm/mxe0HjJKtMdD2lFLWIs7rkaU19+6st93153sMxyTJlcgKqLuy46lUkrGS7y/ uddTEPOQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mu2Ay-002GSU-VR; Mon, 06 Dec 2021 00:48:17 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mu2Ag-002GL9-O0 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 00:48:00 +0000 Received: by mail-pj1-x1049.google.com with SMTP id ce19-20020a17090aff1300b001a6f72e2dbdso3985231pjb.7 for ; Sun, 05 Dec 2021 16:47:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Juc0qnZj8TlomGHzjSRBMmD/3w+jNx2TwdjwTE6BS98=; b=hWASvvEZtys46W/X8EPsrSiN97sWwQWsAIj49tLDVsttqi5umZE0LsPuMSopJRUewn xOokNd7G4qpHNxLJaV/QOOR7XWu4l+XVW/9L1jfD/5otmIpvXUUq0aBv/fgf7hQDZ7gQ 9ttMrbu+Rg0RJZjNTv34F2MUu6Uo5F9ORaqpRzEdIfLp40QqznuSjpmA1O/Y9xtpd/g9 Yp0nHPsueGZUhgnNkzRALzezkN98hJq3HpF2yKZo4lz8ikZqxWPwq5npVe3nw6i1QmUX JZAH2PZs9Txrl+pV4MfbPL+TT6a6udy+W8NXKJR5F7cB428JUGD+PhRAYhSlG3qIkg3/ rlUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Juc0qnZj8TlomGHzjSRBMmD/3w+jNx2TwdjwTE6BS98=; b=Btu/jNteNmDqtAcO0MABt2Je0ax8W5QqP9O7O7UezIcnIoMVJLD6UOEscEgIPaECN2 zpHBafjsHDFPAdQ+WhD63QhYhzgTtj/Zw6zE7C7kDZ8fL11ZuFu4OrKwCoLe2cmryGml Jg7SK/oC03gxwDEs/ePPb10qG5O2Y4n60Q3MJGqTr9R3szWSbJ8OlQcXj2hNRIwcrq8O WUKI1HWNzSs37c4qEZ/JAJDlb5f5MTubSap69ZjThLhi7zaomSSLiuCwtUndaQK5UnHq YoTBnu7OM8Phka58bQBWFMZ58vnlcJEh2I5uJr3jPdsMVorc31hgC1WYA/UUfzM6wi2T x4bQ== X-Gm-Message-State: AOAM532Rijh4jZtmBqjnERzCpP6J9QhQvMLGFQzT7feJWM+cEMp0hAZW z5vpFf772Zniu+i80loLNjz/9WvTk8I= X-Google-Smtp-Source: ABdhPJytq7YmOnRIwb0exEbD+4p7g7hb2gfAKiUucUCocWiTk/gwO1Fi9Jj8cfb+NQuimHJC4pbdzVvtfMk= X-Received: from reiji-vws-sp.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3d59]) (user=reijiw job=sendgmr) by 2002:a05:6a00:b83:b0:49f:b555:1183 with SMTP id g3-20020a056a000b8300b0049fb5551183mr33536263pfj.32.1638751677472; Sun, 05 Dec 2021 16:47:57 -0800 (PST) Date: Sun, 5 Dec 2021 16:47:36 -0800 In-Reply-To: <20211206004736.1520989-1-reijiw@google.com> Message-Id: <20211206004736.1520989-3-reijiw@google.com> Mime-Version: 1.0 References: <20211206004736.1520989-1-reijiw@google.com> X-Mailer: git-send-email 2.34.1.400.ga245620fadb-goog Subject: [PATCH v4 2/2] arm64: mte: DC {GVA,GZVA} shouldn't be used when DCZID_EL0.DZP == 1 From: Reiji Watanabe To: Catalin Marinas , Will Deacon Cc: Marc Zyngier , Robin Murphy , Mark Rutland , Peter Collingbourne , Evgenii Stepanov , guanghuifeng@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211205_164758_812412_14C462A6 X-CRM114-Status: GOOD ( 12.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, mte_set_mem_tag_range() and mte_zero_clear_page_tags() use DC {GVA,GZVA} unconditionally. But, they should make sure that DCZID_EL0.DZP, which indicates whether or not use of those instructions is prohibited, is zero when using those instructions. Use ST{G,ZG,Z2G} instead when DCZID_EL0.DZP == 1. Fixes: 013bb59dbb7c ("arm64: mte: handle tags zeroing at page allocation time") Fixes: 3d0cca0b02ac ("kasan: speed up mte_set_mem_tag_range") Signed-off-by: Reiji Watanabe --- arch/arm64/include/asm/mte-kasan.h | 8 +++++--- arch/arm64/lib/mte.S | 8 +++++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 478b9bcf69ad..e4704a403237 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -84,10 +84,12 @@ static inline void __dc_gzva(u64 p) static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, bool init) { - u64 curr, mask, dczid_bs, end1, end2, end3; + u64 curr, mask, dczid, dczid_bs, dczid_dzp, end1, end2, end3; /* Read DC G(Z)VA block size from the system register. */ - dczid_bs = 4ul << (read_cpuid(DCZID_EL0) & 0xf); + dczid = read_cpuid(DCZID_EL0); + dczid_bs = 4ul << (dczid & 0xf); + dczid_dzp = (dczid >> 4) & 1; curr = (u64)__tag_set(addr, tag); mask = dczid_bs - 1; @@ -106,7 +108,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, */ #define SET_MEMTAG_RANGE(stg_post, dc_gva) \ do { \ - if (size >= 2 * dczid_bs) { \ + if (!dczid_dzp && size >= 2 * dczid_bs) {\ do { \ curr = stg_post(curr); \ } while (curr < end1); \ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index e83643b3995f..f531dcb95174 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) * x0 - address to the beginning of the page */ SYM_FUNC_START(mte_zero_clear_page_tags) + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag mrs x1, dczid_el0 + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag 1: dc gzva, x0 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stz2g x0, [x0], #(MTE_GRANULE_SIZE * 2) + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END(mte_zero_clear_page_tags) /*