From patchwork Sat Apr 27 05:27:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 13645569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75E74C4345F for ; Sat, 27 Apr 2024 05:28:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Mime-Version: Message-Id:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=/OwFXWgbVugz2kbo+bKzmiK8qZOOwRw6J7xlosbsonE=; b=t7p JHiOkkNWBEvPlVh4Qtiv3ZKyGIfZEvs8mVB0+XoD/lJ0TOXG2PCc4s2CAP1QvvBQPLe2VkwCgZAbY BqZa4JCSEGg06m+fnKjLmPocTBbw4LDtvXkFay+QjBcl9lcx+TpyuIfkdDmikznrxPMojXgXkVRPY P18HbYnZbDqUPTWbWjrPGap3FM23zO5zdyDtXvsrTInKWzY7b+5GLW5YGnGOFM28FofyQncPDCjQI 1QrVknn2YbvUxIuwnV4jvpnr/UbwWL7LxBGtgmhCg7ilQ2tZ1rqlCMu6FL84OUeCrUfYTDE0KNYhM dvd+erMv131Icvp2PHVRR7DPvgLrG8Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s0abc-0000000EtDF-06SX; Sat, 27 Apr 2024 05:28:12 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s0abY-0000000EtCc-34aA for linux-arm-kernel@lists.infradead.org; Sat, 27 Apr 2024 05:28:10 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6188040f3d2so58200167b3.1 for ; Fri, 26 Apr 2024 22:28:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714195686; x=1714800486; darn=lists.infradead.org; h=cc:to:from:subject:mime-version:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=7n0z/AVhzbf8S/wNcgQdXkEZVndd1PrTKFQZQ2nxF/s=; b=o0xgEmPoILEf321KLtZenfShENklptajiHvwwEgseU2SKt+DQSTsPf5bTJYUirK8S+ aN+gNE3PFVVMsMD0067ELVZ3DBJ0JSrt1NSa2lpJSfqi4587SPBf1UXIgr2RdLxfh2DZ u9X/lRvzoqr+653vmQ99bwHDwSaK/AZx5qW8OaiQem3OLqstwU7JPvMwrupRt/fhLCC7 zEzmPwM0EnkhnH8d+79CSBBjzPOTjdkNzSlVqdjc0KwlzznRgcnKTOPMj9AvqScekTuP ipsFgWMNLSuaeNH02wF2TTV3oss7K2oDohffZb2eRoaGF/37bfD+N3DkLJirFij2l9Bh 0rMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714195686; x=1714800486; h=cc:to:from:subject:mime-version:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7n0z/AVhzbf8S/wNcgQdXkEZVndd1PrTKFQZQ2nxF/s=; b=Q/FDFPoxTcaXXq4IlyjkM33CodNgeuQ360bWo6VJiwYxRCs6acYSmcTtzNozuiEIK9 JCeQ7e4eeDoma0hWkYHNWVz2h0C4LV9DwRaZknLnVJbv0/XNqDvBdfLG0rchAGvEqNR6 g1F8zAAEgkBJ7KPQviSCAAe5KwUkpsVeev1/VIZ3WzjVSvUnDiF8FDvlV8UJn8m9aWOD nByAEXZ0Gixel3j8PhXeUdHXeNbZtWRxgF6UZOHd0+8QXYDGMLHPMs4TNFjNPYfK40MZ v10qdd8V10uCn9Vrsst75rYEtuacVWVblwr0YqDSzPh0YxOBKu3++/KT1ZaewijZYW8Y Kb+g== X-Forwarded-Encrypted: i=1; AJvYcCVKr5EAEo8hxgBUgFze7gYXFVpsIrHFeK6cqXj8OxHEk8bZWmFr3yO2mAgPcdL/+LxQjvcXWfIbAk439I28/JgDTDB7uUR914KqPeV595CJuvaUxyo= X-Gm-Message-State: AOJu0YxZoHRDn+i9PFQCtJYdjdqCXHqSeEEsX3ukRV2pnG5EA4EIJpae 1IlzzRDtiPntHC6x8r5AFYd63EK7/O+48UMxDyUWy/aBR8d+9bVSod/wvz48Wvo9rg== X-Google-Smtp-Source: AGHT+IH2pDyxirnO/VcK7xLxhFlRXYI3Bv2+v31Gkboesvr7qWzonn9SgtBior/dp8+k7BZOI1cx42A= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:f683:f04c:88a2:b651]) (user=pcc job=sendgmr) by 2002:a0d:ea4a:0:b0:61a:d319:ff34 with SMTP id t71-20020a0dea4a000000b0061ad319ff34mr1130265ywe.9.1714195686205; Fri, 26 Apr 2024 22:28:06 -0700 (PDT) Date: Fri, 26 Apr 2024 22:27:51 -0700 Message-Id: <20240427052754.2014024-1-pcc@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Subject: [PATCH] arm64: Implement prctl(PR_{G,S}ET_TSC) From: Peter Collingbourne To: Catalin Marinas , Will Deacon , Marc Zyngier Cc: Peter Collingbourne , linux-arm-kernel@lists.infradead.org, Yichao Yu , "Robert O'Callahan" , Keno Fischer X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240426_222808_847433_4A05D4E8 X-CRM114-Status: GOOD ( 22.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On arm64, this prctl controls access to CNTVCT_EL0, CNTVCTSS_EL0 and CNTFRQ_EL0 via CNTKCTL_EL1.EL0VCTEN. Since this bit is also used to implement various erratum workarounds, check whether the CPU needs a workaround whenever we potentially need to change it. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/I233a1867d1ccebe2933a347552e7eae862344421 --- arch/arm64/include/asm/processor.h | 5 ++ arch/arm64/include/asm/thread_info.h | 2 + arch/arm64/kernel/process.c | 70 +++++++++++++++++++++++----- arch/arm64/kernel/traps.c | 20 +++++--- 4 files changed, 79 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index f77371232d8c6..347bd3464fcbe 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -402,5 +402,10 @@ long get_tagged_addr_ctrl(struct task_struct *task); #define GET_TAGGED_ADDR_CTRL() get_tagged_addr_ctrl(current) #endif +int get_tsc_mode(unsigned long adr); +int set_tsc_mode(unsigned int val); +#define GET_TSC_CTL(adr) get_tsc_mode((adr)) +#define SET_TSC_CTL(val) set_tsc_mode((val)) + #endif /* __ASSEMBLY__ */ #endif /* __ASM_PROCESSOR_H */ diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index e72a3bf9e5634..1114c1c3300a1 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -81,6 +81,7 @@ void arch_setup_new_exec(void); #define TIF_SME 27 /* SME in use */ #define TIF_SME_VL_INHERIT 28 /* Inherit SME vl_onexec across exec */ #define TIF_KERNEL_FPSTATE 29 /* Task is in a kernel mode FPSIMD section */ +#define TIF_TSC_SIGSEGV 30 /* SIGSEGV on counter-timer access */ #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) @@ -97,6 +98,7 @@ void arch_setup_new_exec(void); #define _TIF_SVE (1 << TIF_SVE) #define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) +#define _TIF_TSC_SIGSEGV (1 << TIF_TSC_SIGSEGV) #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 4ae31b7af6c31..1a2ae7830c179 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -43,6 +43,7 @@ #include #include +#include #include #include #include @@ -472,27 +473,49 @@ static void entry_task_switch(struct task_struct *next) } /* - * ARM erratum 1418040 handling, affecting the 32bit view of CNTVCT. - * Ensure access is disabled when switching to a 32bit task, ensure - * access is enabled when switching to a 64bit task. + * Handle sysreg updates for ARM erratum 1418040 which affects the 32bit view of + * CNTVCT, various other errata which require trapping all CNTVCT{,_EL0} + * accesses and prctl(PR_SET_TSC). Ensure access is disabled iff a workaround is + * required or PR_TSC_SIGSEGV is set. */ -static void erratum_1418040_thread_switch(struct task_struct *next) +static void update_cntkctl_el1(struct task_struct *next) { - if (!IS_ENABLED(CONFIG_ARM64_ERRATUM_1418040) || - !this_cpu_has_cap(ARM64_WORKAROUND_1418040)) - return; + struct thread_info *ti = task_thread_info(next); - if (is_compat_thread(task_thread_info(next))) + if (test_ti_thread_flag(ti, TIF_TSC_SIGSEGV) || + has_erratum_handler(read_cntvct_el0) || + (IS_ENABLED(CONFIG_ARM64_ERRATUM_1418040) && + this_cpu_has_cap(ARM64_WORKAROUND_1418040) && + is_compat_thread(ti))) sysreg_clear_set(cntkctl_el1, ARCH_TIMER_USR_VCT_ACCESS_EN, 0); else sysreg_clear_set(cntkctl_el1, 0, ARCH_TIMER_USR_VCT_ACCESS_EN); } -static void erratum_1418040_new_exec(void) +static void cntkctl_thread_switch(struct task_struct *prev, + struct task_struct *next) +{ + if ((read_ti_thread_flags(task_thread_info(prev)) & + (_TIF_32BIT | _TIF_TSC_SIGSEGV)) != + (read_ti_thread_flags(task_thread_info(next)) & + (_TIF_32BIT | _TIF_TSC_SIGSEGV))) + update_cntkctl_el1(next); +} + +static int do_set_tsc_mode(unsigned int val) { + if (val == PR_TSC_SIGSEGV) + set_thread_flag(TIF_TSC_SIGSEGV); + else if (val == PR_TSC_ENABLE) + clear_thread_flag(TIF_TSC_SIGSEGV); + else + return -EINVAL; + preempt_disable(); - erratum_1418040_thread_switch(current); + update_cntkctl_el1(current); preempt_enable(); + + return 0; } /* @@ -528,7 +551,7 @@ struct task_struct *__switch_to(struct task_struct *prev, contextidr_thread_switch(next); entry_task_switch(next); ssbs_thread_switch(next); - erratum_1418040_thread_switch(next); + cntkctl_thread_switch(prev, next); ptrauth_thread_switch_user(next); /* @@ -645,7 +668,7 @@ void arch_setup_new_exec(void) current->mm->context.flags = mmflags; ptrauth_thread_init_user(); mte_thread_init_user(); - erratum_1418040_new_exec(); + do_set_tsc_mode(PR_TSC_ENABLE); if (task_spec_ssb_noexec(current)) { arch_prctl_spec_ctrl_set(current, PR_SPEC_STORE_BYPASS, @@ -754,3 +777,26 @@ int arch_elf_adjust_prot(int prot, const struct arch_elf_state *state, return prot; } #endif + +int get_tsc_mode(unsigned long adr) +{ + unsigned int val; + + if (is_compat_task()) + return -EINVAL; + + if (test_thread_flag(TIF_TSC_SIGSEGV)) + val = PR_TSC_SIGSEGV; + else + val = PR_TSC_ENABLE; + + return put_user(val, (unsigned int __user *)adr); +} + +int set_tsc_mode(unsigned int val) +{ + if (is_compat_task()) + return -EINVAL; + + return do_set_tsc_mode(val); +} diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8c..63fe64aafb7ad 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -601,18 +601,26 @@ static void ctr_read_handler(unsigned long esr, struct pt_regs *regs) static void cntvct_read_handler(unsigned long esr, struct pt_regs *regs) { - int rt = ESR_ELx_SYS64_ISS_RT(esr); + if (test_thread_flag(TIF_TSC_SIGSEGV)) { + force_sig(SIGSEGV); + } else { + int rt = ESR_ELx_SYS64_ISS_RT(esr); - pt_regs_write_reg(regs, rt, arch_timer_read_counter()); - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + pt_regs_write_reg(regs, rt, arch_timer_read_counter()); + arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + } } static void cntfrq_read_handler(unsigned long esr, struct pt_regs *regs) { - int rt = ESR_ELx_SYS64_ISS_RT(esr); + if (test_thread_flag(TIF_TSC_SIGSEGV)) { + force_sig(SIGSEGV); + } else { + int rt = ESR_ELx_SYS64_ISS_RT(esr); - pt_regs_write_reg(regs, rt, arch_timer_get_rate()); - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + pt_regs_write_reg(regs, rt, arch_timer_get_rate()); + arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + } } static void mrs_handler(unsigned long esr, struct pt_regs *regs)