| Message ID | 20230616103150.1238132-1-mark.rutland@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: alternatives: make clean_dcache_range_nopatch() noinstr-safe | expand |
On Fri, 16 Jun 2023 11:31:50 +0100, Mark Rutland wrote: > When patching kernel alternatives, we need to be careful not to execute > kernel code which is itself subject to patching. In general, if code is > executed after the instructions in memory have been patched but prior to > the cache maintenance and barriers completing, it could lead to > UNPREDICTABLE results. > > As our regular cache maintenance routines are patched with alternatives, > we have a clean_dcache_range_nopatch() function which is *intended* to > avoid patchable code and therefore supposed to be safe in the middle of > patching alternatives. Unfortunately, it's not marked as 'noinstr', and > so can be instrumented with patchable code. > > [...] Applied to arm64 (for-next/misc), thanks! [1/1] arm64: alternatives: make clean_dcache_range_nopatch() noinstr-safe https://git.kernel.org/arm64/c/39138093f139
diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index 53d13b2e5f59e..8ff6610af4966 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -121,11 +121,11 @@ static noinstr void patch_alternative(struct alt_instr *alt, * accidentally call into the cache.S code, which is patched by us at * runtime. */ -static void clean_dcache_range_nopatch(u64 start, u64 end) +static noinstr void clean_dcache_range_nopatch(u64 start, u64 end) { u64 cur, d_size, ctr_el0; - ctr_el0 = read_sanitised_ftr_reg(SYS_CTR_EL0); + ctr_el0 = arm64_ftr_reg_ctrel0.sys_val; d_size = 4 << cpuid_feature_extract_unsigned_field(ctr_el0, CTR_EL0_DminLine_SHIFT); cur = start & ~(d_size - 1);
When patching kernel alternatives, we need to be careful not to execute kernel code which is itself subject to patching. In general, if code is executed after the instructions in memory have been patched but prior to the cache maintenance and barriers completing, it could lead to UNPREDICTABLE results. As our regular cache maintenance routines are patched with alternatives, we have a clean_dcache_range_nopatch() function which is *intended* to avoid patchable code and therefore supposed to be safe in the middle of patching alternatives. Unfortunately, it's not marked as 'noinstr', and so can be instrumented with patchable code. Additionally, it calls read_sanitised_ftr_reg() (which may be instrumented with patchable code) to find the sanitized value of CTR_EL0.DminLine, and is therefore not safe to call during patching. Luckily, since commit: 675b0563d6b26aa9 ("arm64: cpufeature: expose arm64_ftr_reg struct for CTR_EL0") ... we can read the sanitised CTR_EL0 value directly, and avoid the call to read_sanitised_ftr_reg(). This patch marks clean_dcache_range_nopatch() as noinstr, and has it read the sanitized CTR_EL0 value directly, avoiding the issues above. As a bonus, this is also an optimization. As read_sanitised_ftr_reg() performs a binary search to find the CTR_EL0 value, reading the value directly avoids this binary search per applied alternative, avoiding some unnecessary work. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> --- arch/arm64/kernel/alternative.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) Hi Catalin, This is part of a larger noinstr-safety cleanup that I'm trying to handle for the alternatives and insn patching code. The rest of that isn't ready yet, and I don't want to force that through this late (since e.g. it requires some major rework to the aarch64_insn_*() code), but as this patch is simple and also an optimization, I'd like to get it out of the way now if you're happy to pick it up. Mark.