From patchwork Wed Jun 30 23:15:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12352935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 401FFC11F66 for ; Wed, 30 Jun 2021 23:17:24 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 059C360230 for ; Wed, 30 Jun 2021 23:17:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 059C360230 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+iclOuHDs/uthiSVgqB80e6GW3wzN/xEy7Il0W5TlNA=; b=czrw1MkC8sSFmSALWZ+fwLEdjy JgN4CUDuZ6pcZHtEg3vQIfD1NYcHemdnUI1Az+OuIIYomWnn5FRH7/yyKqSjpKX9nQO50WCI11MmP k/zk8j5Rf2x7wPgYrAPyvFGE5smSosNdOeyICeeh+IC9/j9UO4Q49j4TEKDU4iLDoj/v79U6/Lcpw yzRlGit8fQWYikJnEEdoWu3y9sJ4OIsjmgRJU8Au5giPJJKzw5vwxq3SDYHcreu0VoCL1qYdOiNjd VsB//10uueZwUA7EwmUHKVrK4K/A46THWE1msRFzOwFPo2mYf8AJPfSSjSjYJFQcZWbmPxNCgXu6o qidkvuqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lyjQm-00FV1O-TZ; Wed, 30 Jun 2021 23:15:45 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lyjQS-00FUx1-Qf for linux-arm-kernel@lists.infradead.org; Wed, 30 Jun 2021 23:15:26 +0000 Received: by mail-qk1-x749.google.com with SMTP id 132-20020a37058a0000b02903b352139d7fso2810766qkf.23 for ; Wed, 30 Jun 2021 16:15:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=whjH7gsBVOM+2IAqPMYNVQ+hasqGrXcRvSxgNjfNq3o=; b=aK6TSvZJAZjDx0yv974jywM9aHLYqd1GBqq8Hz4/z9g7JFCcn6+fw4sLvl4KWBsDhs Ztiivghyodqgsar59HhJeiJ4TlNEnENPwKXqfLFeuDWnG7StltCk+zCz1gVTJ7W08aTk 00ekgJBLIrYsg5Mm+Sky8K/kCx3kRjPihrM9ZdYCVeBKkVlfopHu8NWuDTKuQzmFQJKJ Q6FwqqL9J/RhKHOpU3I1qH5CSxQ+bK26xkthoT89fsjANXMDOBWfHXODPKbuTG5LQwvy 4NNUbNAnHNj05Qq95QRFTyvd40yI2ecYO8efawIcsO3iXDyTJE+LJmPT4HrckhqTair4 mlsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=whjH7gsBVOM+2IAqPMYNVQ+hasqGrXcRvSxgNjfNq3o=; b=NU88Vo0JwBW7ALDd9CIInT9zymGDwt0Tl0te2550+NBtRBZsl8Vm5dwJvRy7hGqTfg AEwPQ4Bm1uE6Moy4v1+4Z+MaTdy8IA5pOlYdcgP1IXWrW5ZApFgdRxClK49D1izgA5UO i9cwuwXspuFzH9hbDQHo2KzEGt4Nr611yY8VDfuZWJ2PPizJuRoshk5R1qAGHSC3/ltP VEgYU6UUMRIVC+EPibi1vXGhovxGkt3bO1rDd0GNlWZkhIsNA88D+hvbIluv0O21mStA pM9PdJdC1brLJw1l7vWvS0GCOUhltSIQCUszn1CNWMcvQQLXl9EA48RIA9RW05x+stON uHnQ== X-Gm-Message-State: AOAM531+AoLeYPUY0vDsQz3NxN0B475oGCQLKKnGmzlq2wRDeCMrSkj+ hkjmd9nGKNMUegA/vFpTuOkVyU8= X-Google-Smtp-Source: ABdhPJx+kNG0Z17rg7Aq3ejM5mPDWUaAtBMTK3S+vWDqn3sAdyLIRz5ZrOHIz69kIEzVOK1y+KWQlxQ= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:8b00:104d:c249:d343]) (user=pcc job=sendgmr) by 2002:a05:6214:148e:: with SMTP id bn14mr3828377qvb.52.1625094922847; Wed, 30 Jun 2021 16:15:22 -0700 (PDT) Date: Wed, 30 Jun 2021 16:15:07 -0700 In-Reply-To: <20210630231509.3773172-1-pcc@google.com> Message-Id: <20210630231509.3773172-3-pcc@google.com> Mime-Version: 1.0 References: <20210630231509.3773172-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog Subject: [PATCH v8 2/4] arm64: mte: change ASYNC and SYNC TCF settings into bitfields From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210630_161524_916589_05D8B73D X-CRM114-Status: GOOD ( 17.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow the user program to specify both ASYNC and SYNC TCF modes by repurposing the existing constants as bitfields. This will allow the kernel to select one of the modes on behalf of the user program. With this patch the kernel will always select async mode, but a subsequent patch will make this configurable. Link: https://linux-review.googlesource.com/id/Icc5923c85a8ea284588cc399ae74fd19ec291230 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/mte.h | 4 ++ arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/kernel/mte.c | 78 ++++++++++++------------------ include/uapi/linux/prctl.h | 11 +++-- 4 files changed, 45 insertions(+), 51 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index bc88a1ced0d7..719687412798 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -40,6 +40,7 @@ void mte_free_tag_storage(char *storage); void mte_sync_tags(pte_t *ptep, pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void mte_thread_init_user(void); +void mte_update_sctlr_user(struct task_struct *task); void mte_thread_switch(struct task_struct *next); void mte_suspend_enter(void); void mte_suspend_exit(void); @@ -62,6 +63,9 @@ static inline void mte_copy_page_tags(void *kto, const void *kfrom) static inline void mte_thread_init_user(void) { } +static inline void mte_update_sctlr_user(struct task_struct *task) +{ +} static inline void mte_thread_switch(struct task_struct *next) { } diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 6322fb1714d5..80ceb9cbdd60 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -19,6 +19,9 @@ #define MTE_CTRL_GCR_USER_EXCL_SHIFT 0 #define MTE_CTRL_GCR_USER_EXCL_MASK 0xffff +#define MTE_CTRL_TCF_SYNC (1UL << 16) +#define MTE_CTRL_TCF_ASYNC (1UL << 17) + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index d3884d09513d..7b8c241be24e 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -197,16 +197,6 @@ static void update_gcr_el1_excl(u64 excl) sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl); } -static void set_gcr_el1_excl(u64 excl) -{ - current->thread.mte_ctrl = excl; - - /* - * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value - * by mte_set_user_gcr() in kernel_exit, - */ -} - void mte_thread_init_user(void) { if (!system_supports_mte()) @@ -216,15 +206,31 @@ void mte_thread_init_user(void) dsb(ish); write_sysreg_s(0, SYS_TFSRE0_EL1); clear_thread_flag(TIF_MTE_ASYNC_FAULT); - /* disable tag checking */ - set_task_sctlr_el1((current->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK) | - SCTLR_EL1_TCF0_NONE); - /* reset tag generation mask */ - set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK); + /* disable tag checking and reset tag generation mask */ + current->thread.mte_ctrl = MTE_CTRL_GCR_USER_EXCL_MASK; + mte_update_sctlr_user(current); + set_task_sctlr_el1(current->thread.sctlr_user); +} + +void mte_update_sctlr_user(struct task_struct *task) +{ + unsigned long sctlr = task->thread.sctlr_user; + unsigned long pref = MTE_CTRL_TCF_ASYNC; + unsigned long mte_ctrl = task->thread.mte_ctrl; + unsigned long resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; + + sctlr &= ~SCTLR_EL1_TCF0_MASK; + if (resolved_mte_tcf & MTE_CTRL_TCF_ASYNC) + sctlr |= SCTLR_EL1_TCF0_ASYNC; + else if (resolved_mte_tcf & MTE_CTRL_TCF_SYNC) + sctlr |= SCTLR_EL1_TCF0_SYNC; + task->thread.sctlr_user = sctlr; } void mte_thread_switch(struct task_struct *next) { + mte_update_sctlr_user(next); + /* * Check if an async tag exception occurred at EL1. * @@ -262,33 +268,21 @@ void mte_suspend_exit(void) long set_mte_ctrl(struct task_struct *task, unsigned long arg) { - u64 sctlr = task->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK; u64 mte_ctrl = (~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & SYS_GCR_EL1_EXCL_MASK) << MTE_CTRL_GCR_USER_EXCL_SHIFT; if (!system_supports_mte()) return 0; - switch (arg & PR_MTE_TCF_MASK) { - case PR_MTE_TCF_NONE: - sctlr |= SCTLR_EL1_TCF0_NONE; - break; - case PR_MTE_TCF_SYNC: - sctlr |= SCTLR_EL1_TCF0_SYNC; - break; - case PR_MTE_TCF_ASYNC: - sctlr |= SCTLR_EL1_TCF0_ASYNC; - break; - default: - return -EINVAL; - } + if (arg & PR_MTE_TCF_ASYNC) + mte_ctrl |= MTE_CTRL_TCF_ASYNC; + if (arg & PR_MTE_TCF_SYNC) + mte_ctrl |= MTE_CTRL_TCF_SYNC; - if (task != current) { - task->thread.sctlr_user = sctlr; - task->thread.mte_ctrl = mte_ctrl; - } else { - set_task_sctlr_el1(sctlr); - set_gcr_el1_excl(mte_ctrl); + task->thread.mte_ctrl = mte_ctrl; + if (task == current) { + mte_update_sctlr_user(task); + set_task_sctlr_el1(task->thread.sctlr_user); } return 0; @@ -305,18 +299,10 @@ long get_mte_ctrl(struct task_struct *task) return 0; ret = incl << PR_MTE_TAG_SHIFT; - - switch (task->thread.sctlr_user & SCTLR_EL1_TCF0_MASK) { - case SCTLR_EL1_TCF0_NONE: - ret |= PR_MTE_TCF_NONE; - break; - case SCTLR_EL1_TCF0_SYNC: - ret |= PR_MTE_TCF_SYNC; - break; - case SCTLR_EL1_TCF0_ASYNC: + if (mte_ctrl & MTE_CTRL_TCF_ASYNC) ret |= PR_MTE_TCF_ASYNC; - break; - } + if (mte_ctrl & MTE_CTRL_TCF_SYNC) + ret |= PR_MTE_TCF_SYNC; return ret; } diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 18a9f59dc067..d3a5afb4c1ae 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -234,14 +234,15 @@ struct prctl_mm_map { #define PR_GET_TAGGED_ADDR_CTRL 56 # define PR_TAGGED_ADDR_ENABLE (1UL << 0) /* MTE tag check fault modes */ -# define PR_MTE_TCF_SHIFT 1 -# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_NONE 0 +# define PR_MTE_TCF_SYNC (1UL << 1) +# define PR_MTE_TCF_ASYNC (1UL << 2) +# define PR_MTE_TCF_MASK (PR_MTE_TCF_SYNC | PR_MTE_TCF_ASYNC) /* MTE tag inclusion mask */ # define PR_MTE_TAG_SHIFT 3 # define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT) +/* Unused; kept only for source compatibility */ +# define PR_MTE_TCF_SHIFT 1 /* Control reclaim behavior when allocating memory */ #define PR_SET_IO_FLUSHER 57