| Message ID | 20250112155453.1104139-12-riel@surriel.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | AMD broadcast TLB invalidation | expand |
> diff > <https://lore.kernel.org/lkml/20250112155453.1104139-1-riel@surriel.com/T/#iZ2e.:..:20250112155453.1104139-12-riel::40surriel.com:1arch:x86:kernel:cpu:amd.c> > --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index > bcf73775b4f8..b7e84d43a22d 100644 --- a/arch/x86/kernel/cpu/amd.c +++ > b/arch/x86/kernel/cpu/amd.c @@ -1071,6 +1071,9 @@ static void > init_amd(struct cpuinfo_x86 *c) > /* AMD CPUs don't need fencing after x2APIC/TSC_DEADLINE MSR writes. */ > clear_cpu_cap(c, X86_FEATURE_APIC_MSRS_FENCE); > + + if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) + > msr_set_bit(MSR_EFER, _EFER_TCE); } > > #ifdef CONFIG_X86_32 I don't think this is wise. TCE is orthogonal to INVLPGB. Either Linux is safe with TCE turned on, and it should be turned on everywhere (it goes back to Fam10h CPUs IIRC), or Linux isn't safe with TCE turned on, and this needs to depend on some other condition. Or, is this a typo and did you mean to check the TCE CPUID bit, rather than the INVLPGB CPUID bit? ~Andrew
On Mon, 2025-01-13 at 11:32 +0000, Andrew Cooper wrote: > > +++ > > b/arch/x86/kernel/cpu/amd.c @@ -1071,6 +1071,9 @@ static void > > init_amd(struct cpuinfo_x86 *c) > > /* AMD CPUs don't need fencing after x2APIC/TSC_DEADLINE > > MSR writes. */ > > clear_cpu_cap(c, X86_FEATURE_APIC_MSRS_FENCE); > > + + if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) + > > msr_set_bit(MSR_EFER, _EFER_TCE); } > > > > #ifdef CONFIG_X86_32 > > I don't think this is wise. TCE is orthogonal to INVLPGB. > > Either Linux is safe with TCE turned on, and it should be turned on > everywhere (it goes back to Fam10h CPUs IIRC), or Linux isn't safe > with > TCE turned on, and this needs to depend on some other condition. > > Or, is this a typo and did you mean to check the TCE CPUID bit, > rather > than the INVLPGB CPUID bit? You're right, this should just check against X86_FEATURE_TCE, which I did not realize was a separate feature bit. I've changed this for the next version of the series. Thank you!
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 3ae84c3b8e6d..dc1c1057f26e 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -25,6 +25,7 @@ #define _EFER_SVME 12 /* Enable virtualization */ #define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */ #define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */ +#define _EFER_TCE 15 /* Enable Translation Cache Extensions */ #define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */ #define EFER_SCE (1<<_EFER_SCE) @@ -34,6 +35,7 @@ #define EFER_SVME (1<<_EFER_SVME) #define EFER_LMSLE (1<<_EFER_LMSLE) #define EFER_FFXSR (1<<_EFER_FFXSR) +#define EFER_TCE (1<<_EFER_TCE) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) /* diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index bcf73775b4f8..b7e84d43a22d 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -1071,6 +1071,9 @@ static void init_amd(struct cpuinfo_x86 *c) /* AMD CPUs don't need fencing after x2APIC/TSC_DEADLINE MSR writes. */ clear_cpu_cap(c, X86_FEATURE_APIC_MSRS_FENCE); + + if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) + msr_set_bit(MSR_EFER, _EFER_TCE); } #ifdef CONFIG_X86_32 diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index 3ae84c3b8e6d..dc1c1057f26e 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -25,6 +25,7 @@ #define _EFER_SVME 12 /* Enable virtualization */ #define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */ #define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */ +#define _EFER_TCE 15 /* Enable Translation Cache Extensions */ #define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */ #define EFER_SCE (1<<_EFER_SCE) @@ -34,6 +35,7 @@ #define EFER_SVME (1<<_EFER_SVME) #define EFER_LMSLE (1<<_EFER_LMSLE) #define EFER_FFXSR (1<<_EFER_FFXSR) +#define EFER_TCE (1<<_EFER_TCE) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) /*
With AMD TCE (translation cache extensions) only the intermediate mappings that cover the address range zapped by INVLPG / INVLPGB get invalidated, rather than all intermediate mappings getting zapped at every TLB invalidation. This can help reduce the TLB miss rate, by keeping more intermediate mappings in the cache. From the AMD manual: Translation Cache Extension (TCE) Bit. Bit 15, read/write. Setting this bit to 1 changes how the INVLPG, INVLPGB, and INVPCID instructions operate on TLB entries. When this bit is 0, these instructions remove the target PTE from the TLB as well as all upper-level table entries that are cached in the TLB, whether or not they are associated with the target PTE. When this bit is set, these instructions will remove the target PTE and only those upper-level entries that lead to the target PTE in the page table hierarchy, leaving unrelated upper-level entries intact. Signed-off-by: Rik van Riel <riel@surriel.com> --- arch/x86/include/asm/msr-index.h | 2 ++ arch/x86/kernel/cpu/amd.c | 3 +++ tools/arch/x86/include/asm/msr-index.h | 2 ++ 3 files changed, 7 insertions(+)