[01/11] objtool: Generic annotation infrastructure

Message ID	20231204093731.356358182@infradead.org (mailing list archive)
State	New, archived
Headers	show Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="VhetepAF" Message-Id: <20231204093731.356358182@infradead.org> User-Agent: quilt/0.65 Date: Mon, 04 Dec 2023 10:37:03 +0100 From: Peter Zijlstra <peterz@infradead.org> To: Sean Christopherson <seanjc@google.com>, Paolo Bonzini <pbonzini@redhat.com>, Josh Poimboeuf <jpoimboe@redhat.com>, Thomas Gleixner <tglx@linutronix.de> Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, x86@kernel.org, kvm@vger.kernel.org Subject: [PATCH 01/11] objtool: Generic annotation infrastructure References: <20231204093702.989848513@infradead.org> Precedence: bulk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8
Series	x86/kvm/emulate: Avoid RET for FASTOPs \| expand [00/11] x86/kvm/emulate: Avoid RET for FASTOPs [01/11] objtool: Generic annotation infrastructure [02/11] objtool: Convert ANNOTATE_NOENDBR to ANNOTATE [03/11] objtool: Convert ANNOTATE_RETPOLINE_SAFE to ANNOTATE [04/11] objtool: Convert instrumentation_{begin,end}() to ANNOTATE [05/11] objtool: Convert VALIDATE_UNRET_BEGIN to ANNOTATE [06/11] objtool: Convert ANNOTATE_IGNORE_ALTERNATIVE to ANNOTATE [07/11] objtool: Convert ANNOTATE_INTRA_FUNCTION_CALLS to ANNOTATE [08/11] objtool: Collapse annotate sequences [09/11] x86/kvm/emulate: Implement test_cc() in C [10/11] x86/nospec: JMP_NOSPEC [11/11] x86/kvm/emulate: Avoid RET for fastops

Message ID

20231204093731.356358182@infradead.org (mailing list archive)

State

New, archived

Headers

Message-Id: <20231204093731.356358182@infradead.org>
User-Agent: quilt/0.65
Date: Mon, 04 Dec 2023 10:37:03 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Sean Christopherson <seanjc@google.com>,
 Paolo Bonzini <pbonzini@redhat.com>,
 Josh Poimboeuf <jpoimboe@redhat.com>,
 Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org,
 peterz@infradead.org,
 x86@kernel.org,
 kvm@vger.kernel.org
Subject: [PATCH 01/11] objtool: Generic annotation infrastructure
References: <20231204093702.989848513@infradead.org>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8

Series

x86/kvm/emulate: Avoid RET for FASTOPs | expand

Avoid endless .discard.foo sections for each annotation, create a single .discard.annotate section that takes an annotation type along with the instruction. Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> ---

Comments

Peter Zijlstra Nov. 8, 2024, 2:16 p.m. UTC | #1

On Mon, Dec 04, 2023 at 10:37:03AM +0100, Peter Zijlstra wrote:
> Avoid endless .discard.foo sections for each annotation, create a
> single .discard.annotate section that takes an annotation type along
> with the instruction.
> 
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> --- a/include/linux/objtool.h
> +++ b/include/linux/objtool.h
> @@ -57,6 +57,13 @@
>  	".long 998b\n\t"						\
>  	".popsection\n\t"
>  
> +#define ASM_ANNOTATE(x)						\
> +	"911:\n\t"						\
> +	".pushsection .discard.annotate,\"M\",@progbits,8\n\t"	\
> +	".long 911b - .\n\t"					\
> +	".long " __stringify(x) "\n\t"				\
> +	".popsection\n\t"
> +
>  #else /* __ASSEMBLY__ */
>  
>  /*
> @@ -146,6 +153,14 @@
>  	.popsection
>  .endm
>  
> +.macro ANNOTATE type:req
> +.Lhere_\@:
> +	.pushsection .discard.annotate,"M",@progbits,8
> +	.long	.Lhere_\@ - .
> +	.long	\type
> +	.popsection
> +.endm
> +
>  #endif /* __ASSEMBLY__ */
>  
>  #else /* !CONFIG_OBJTOOL */
> @@ -167,6 +182,8 @@
>  .endm
>  .macro REACHABLE
>  .endm
> +.macro ANNOTATE
> +.endm
>  #endif
>  
>  #endif /* CONFIG_OBJTOOL */
> --- a/tools/objtool/check.c
> +++ b/tools/objtool/check.c
> @@ -2308,6 +2308,41 @@ static int read_unwind_hints(struct objt
>  	return 0;
>  }
>  
> +static int read_annotate(struct objtool_file *file, void (*func)(int type, struct instruction *insn))
> +{
> +	struct section *rsec, *sec;
> +	struct instruction *insn;
> +	struct reloc *reloc;
> +	int type;
> +
> +	rsec = find_section_by_name(file->elf, ".rela.discard.annotate");
> +	if (!rsec)
> +		return 0;
> +
> +	sec = find_section_by_name(file->elf, ".discard.annotate");
> +	if (!sec)
> +		return 0;
> +
> +	for_each_reloc(rsec, reloc) {
> +		insn = find_insn(file, reloc->sym->sec,
> +				 reloc->sym->offset + reloc_addend(reloc));
> +		if (!insn) {
> +			WARN("bad .discard.annotate entry: %d", reloc_idx(reloc));
> +			return -1;
> +		}
> +
> +		type = *(u32 *)(sec->data->d_buf + (reloc_idx(reloc) * sec->sh.sh_entsize) + 4);
> +
> +		func(type, insn);
> +	}
> +
> +	return 0;
> +}

So... ld.lld hates this :-(

From an LLVM=-19 build we can see that:

$ readelf -WS tmp-build/arch/x86/kvm/vmx/vmenter.o | grep annotate
  [13] .discard.annotate PROGBITS        0000000000000000 00028c 000018 08   M  0   0  1

$ readelf -WS tmp-build/arch/x86/kvm/kvm-intel.o | grep annotate
  [ 3] .discard.annotate PROGBITS        0000000000000000 069fe0 0089d0 00   M  0   0  1

Which tells us that the translation unit itself has a sh_entsize of 8,
while the linked object has sh_entsize of 0.

This then completely messes up the indexing objtool does, which relies
on it being a sane number.

GCC/binutils very much does not do this, it retains the 8.

Dear clang folks, help?

Nathan Chancellor Nov. 8, 2024, 2:57 p.m. UTC | #2

On Fri, Nov 08, 2024 at 03:16:00PM +0100, Peter Zijlstra wrote:
> On Mon, Dec 04, 2023 at 10:37:03AM +0100, Peter Zijlstra wrote:
> > Avoid endless .discard.foo sections for each annotation, create a
> > single .discard.annotate section that takes an annotation type along
> > with the instruction.
> > 
> > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> > ---
> > --- a/include/linux/objtool.h
> > +++ b/include/linux/objtool.h
> > @@ -57,6 +57,13 @@
> >  	".long 998b\n\t"						\
> >  	".popsection\n\t"
> >  
> > +#define ASM_ANNOTATE(x)						\
> > +	"911:\n\t"						\
> > +	".pushsection .discard.annotate,\"M\",@progbits,8\n\t"	\
> > +	".long 911b - .\n\t"					\
> > +	".long " __stringify(x) "\n\t"				\
> > +	".popsection\n\t"
> > +
> >  #else /* __ASSEMBLY__ */
> >  
> >  /*
> > @@ -146,6 +153,14 @@
> >  	.popsection
> >  .endm
> >  
> > +.macro ANNOTATE type:req
> > +.Lhere_\@:
> > +	.pushsection .discard.annotate,"M",@progbits,8
> > +	.long	.Lhere_\@ - .
> > +	.long	\type
> > +	.popsection
> > +.endm
> > +
> >  #endif /* __ASSEMBLY__ */
> >  
> >  #else /* !CONFIG_OBJTOOL */
> > @@ -167,6 +182,8 @@
> >  .endm
> >  .macro REACHABLE
> >  .endm
> > +.macro ANNOTATE
> > +.endm
> >  #endif
> >  
> >  #endif /* CONFIG_OBJTOOL */
> > --- a/tools/objtool/check.c
> > +++ b/tools/objtool/check.c
> > @@ -2308,6 +2308,41 @@ static int read_unwind_hints(struct objt
> >  	return 0;
> >  }
> >  
> > +static int read_annotate(struct objtool_file *file, void (*func)(int type, struct instruction *insn))
> > +{
> > +	struct section *rsec, *sec;
> > +	struct instruction *insn;
> > +	struct reloc *reloc;
> > +	int type;
> > +
> > +	rsec = find_section_by_name(file->elf, ".rela.discard.annotate");
> > +	if (!rsec)
> > +		return 0;
> > +
> > +	sec = find_section_by_name(file->elf, ".discard.annotate");
> > +	if (!sec)
> > +		return 0;
> > +
> > +	for_each_reloc(rsec, reloc) {
> > +		insn = find_insn(file, reloc->sym->sec,
> > +				 reloc->sym->offset + reloc_addend(reloc));
> > +		if (!insn) {
> > +			WARN("bad .discard.annotate entry: %d", reloc_idx(reloc));
> > +			return -1;
> > +		}
> > +
> > +		type = *(u32 *)(sec->data->d_buf + (reloc_idx(reloc) * sec->sh.sh_entsize) + 4);
> > +
> > +		func(type, insn);
> > +	}
> > +
> > +	return 0;
> > +}
> 
> So... ld.lld hates this :-(
> 
> From an LLVM=-19 build we can see that:
> 
> $ readelf -WS tmp-build/arch/x86/kvm/vmx/vmenter.o | grep annotate
>   [13] .discard.annotate PROGBITS        0000000000000000 00028c 000018 08   M  0   0  1
> 
> $ readelf -WS tmp-build/arch/x86/kvm/kvm-intel.o | grep annotate
>   [ 3] .discard.annotate PROGBITS        0000000000000000 069fe0 0089d0 00   M  0   0  1
> 
> Which tells us that the translation unit itself has a sh_entsize of 8,
> while the linked object has sh_entsize of 0.
> 
> This then completely messes up the indexing objtool does, which relies
> on it being a sane number.
> 
> GCC/binutils very much does not do this, it retains the 8.
> 
> Dear clang folks, help?

Perhaps Fangrui has immediate thoughts, since this appears to be an
ld.lld thing? Otherwise, I will see if I can dig into this in the next
couple of weeks (I have an LF webinar on Wednesday that I am still
prepping for). Is this reproducible with just defconfig or something
else?

Cheers,
Nathan

Peter Zijlstra Nov. 8, 2024, 3:13 p.m. UTC | #3

On Fri, Nov 08, 2024 at 07:57:16AM -0700, Nathan Chancellor wrote:
> On Fri, Nov 08, 2024 at 03:16:00PM +0100, Peter Zijlstra wrote:
> > On Mon, Dec 04, 2023 at 10:37:03AM +0100, Peter Zijlstra wrote:
> > > Avoid endless .discard.foo sections for each annotation, create a
> > > single .discard.annotate section that takes an annotation type along
> > > with the instruction.
> > > 
> > > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> > > ---
> > > --- a/include/linux/objtool.h
> > > +++ b/include/linux/objtool.h
> > > @@ -57,6 +57,13 @@
> > >  	".long 998b\n\t"						\
> > >  	".popsection\n\t"
> > >  
> > > +#define ASM_ANNOTATE(x)						\
> > > +	"911:\n\t"						\
> > > +	".pushsection .discard.annotate,\"M\",@progbits,8\n\t"	\
> > > +	".long 911b - .\n\t"					\
> > > +	".long " __stringify(x) "\n\t"				\
> > > +	".popsection\n\t"
> > > +
> > >  #else /* __ASSEMBLY__ */
> > >  
> > >  /*
> > > @@ -146,6 +153,14 @@
> > >  	.popsection
> > >  .endm
> > >  
> > > +.macro ANNOTATE type:req
> > > +.Lhere_\@:
> > > +	.pushsection .discard.annotate,"M",@progbits,8
> > > +	.long	.Lhere_\@ - .
> > > +	.long	\type
> > > +	.popsection
> > > +.endm
> > > +
> > >  #endif /* __ASSEMBLY__ */
> > >  
> > >  #else /* !CONFIG_OBJTOOL */
> > > @@ -167,6 +182,8 @@
> > >  .endm
> > >  .macro REACHABLE
> > >  .endm
> > > +.macro ANNOTATE
> > > +.endm
> > >  #endif
> > >  
> > >  #endif /* CONFIG_OBJTOOL */
> > > --- a/tools/objtool/check.c
> > > +++ b/tools/objtool/check.c
> > > @@ -2308,6 +2308,41 @@ static int read_unwind_hints(struct objt
> > >  	return 0;
> > >  }
> > >  
> > > +static int read_annotate(struct objtool_file *file, void (*func)(int type, struct instruction *insn))
> > > +{
> > > +	struct section *rsec, *sec;
> > > +	struct instruction *insn;
> > > +	struct reloc *reloc;
> > > +	int type;
> > > +
> > > +	rsec = find_section_by_name(file->elf, ".rela.discard.annotate");
> > > +	if (!rsec)
> > > +		return 0;
> > > +
> > > +	sec = find_section_by_name(file->elf, ".discard.annotate");
> > > +	if (!sec)
> > > +		return 0;
> > > +
> > > +	for_each_reloc(rsec, reloc) {
> > > +		insn = find_insn(file, reloc->sym->sec,
> > > +				 reloc->sym->offset + reloc_addend(reloc));
> > > +		if (!insn) {
> > > +			WARN("bad .discard.annotate entry: %d", reloc_idx(reloc));
> > > +			return -1;
> > > +		}
> > > +
> > > +		type = *(u32 *)(sec->data->d_buf + (reloc_idx(reloc) * sec->sh.sh_entsize) + 4);
> > > +
> > > +		func(type, insn);
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > 
> > So... ld.lld hates this :-(
> > 
> > From an LLVM=-19 build we can see that:
> > 
> > $ readelf -WS tmp-build/arch/x86/kvm/vmx/vmenter.o | grep annotate
> >   [13] .discard.annotate PROGBITS        0000000000000000 00028c 000018 08   M  0   0  1
> > 
> > $ readelf -WS tmp-build/arch/x86/kvm/kvm-intel.o | grep annotate
> >   [ 3] .discard.annotate PROGBITS        0000000000000000 069fe0 0089d0 00   M  0   0  1
> > 
> > Which tells us that the translation unit itself has a sh_entsize of 8,
> > while the linked object has sh_entsize of 0.
> > 
> > This then completely messes up the indexing objtool does, which relies
> > on it being a sane number.
> > 
> > GCC/binutils very much does not do this, it retains the 8.
> > 
> > Dear clang folks, help?
> 
> Perhaps Fangrui has immediate thoughts, since this appears to be an
> ld.lld thing? Otherwise, I will see if I can dig into this in the next
> couple of weeks (I have an LF webinar on Wednesday that I am still
> prepping for). Is this reproducible with just defconfig or something
> else?

I took the .config from the report you pointed me at yesterday.

  https://lore.kernel.org/202411071743.HZsCuurm-lkp@intel.com/

And specifically the kvm build targets from:

$ make O=tmp-build/ LLVM=-19 arch/x86/kvm/

show this problem.

I just ran a defconfig, and that seems to behave properly. Notably,
vmlinux.o (definitely a link target) has entsize=8 for the relevant
section.

I'm not sure how the kvm targets might be 'special'.

Peter Zijlstra Nov. 8, 2024, 7:03 p.m. UTC | #4

On Fri, Nov 08, 2024 at 03:16:00PM +0100, Peter Zijlstra wrote:

> From an LLVM=-19 build we can see that:
> 
> $ readelf -WS tmp-build/arch/x86/kvm/vmx/vmenter.o | grep annotate
>   [13] .discard.annotate PROGBITS        0000000000000000 00028c 000018 08   M  0   0  1
> 
> $ readelf -WS tmp-build/arch/x86/kvm/kvm-intel.o | grep annotate
>   [ 3] .discard.annotate PROGBITS        0000000000000000 069fe0 0089d0 00   M  0   0  1
> 
> Which tells us that the translation unit itself has a sh_entsize of 8,
> while the linked object has sh_entsize of 0.
> 
> This then completely messes up the indexing objtool does, which relies
> on it being a sane number.
> 
> GCC/binutils very much does not do this, it retains the 8.

Anyway, for now I've added:

+       if (sec->sh.sh_entsize != 8) {
+               static bool warn = false;
+               if (!warn) {
+                       WARN("%s: dodgy linker, sh_entsize != 8", sec->name);
+                       warn = true;
+               }
+               sec->sh.sh_entsize = 8;
+       }

To objtool, this allows it function correctly and prints this reminder
to for us to figure out the linker story.

--- a/include/linux/objtool.h
+++ b/include/linux/objtool.h
@@ -57,6 +57,13 @@ 
 	".long 998b\n\t"						\
 	".popsection\n\t"
 
+#define ASM_ANNOTATE(x)						\
+	"911:\n\t"						\
+	".pushsection .discard.annotate,\"M\",@progbits,8\n\t"	\
+	".long 911b - .\n\t"					\
+	".long " __stringify(x) "\n\t"				\
+	".popsection\n\t"
+
 #else /* __ASSEMBLY__ */
 
 /*
@@ -146,6 +153,14 @@ 
 	.popsection
 .endm
 
+.macro ANNOTATE type:req
+.Lhere_\@:
+	.pushsection .discard.annotate,"M",@progbits,8
+	.long	.Lhere_\@ - .
+	.long	\type
+	.popsection
+.endm
+
 #endif /* __ASSEMBLY__ */
 
 #else /* !CONFIG_OBJTOOL */
@@ -167,6 +182,8 @@ 
 .endm
 .macro REACHABLE
 .endm
+.macro ANNOTATE
+.endm
 #endif
 
 #endif /* CONFIG_OBJTOOL */
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -2308,6 +2308,41 @@  static int read_unwind_hints(struct objt
 	return 0;
 }
 
+static int read_annotate(struct objtool_file *file, void (*func)(int type, struct instruction *insn))
+{
+	struct section *rsec, *sec;
+	struct instruction *insn;
+	struct reloc *reloc;
+	int type;
+
+	rsec = find_section_by_name(file->elf, ".rela.discard.annotate");
+	if (!rsec)
+		return 0;
+
+	sec = find_section_by_name(file->elf, ".discard.annotate");
+	if (!sec)
+		return 0;
+
+	for_each_reloc(rsec, reloc) {
+		insn = find_insn(file, reloc->sym->sec,
+				 reloc->sym->offset + reloc_addend(reloc));
+		if (!insn) {
+			WARN("bad .discard.annotate entry: %d", reloc_idx(reloc));
+			return -1;
+		}
+
+		type = *(u32 *)(sec->data->d_buf + (reloc_idx(reloc) * sec->sh.sh_entsize) + 4);
+
+		func(type, insn);
+	}
+
+	return 0;
+}
+
+static void __annotate_nop(int type, struct instruction *insn)
+{
+}
+
 static int read_noendbr_hints(struct objtool_file *file)
 {
 	struct instruction *insn;
@@ -2602,6 +2637,8 @@  static int decode_sections(struct objtoo
 	if (ret)
 		return ret;
 
+	ret = read_annotate(file, __annotate_nop);
+
 	/*
 	 * Must be before read_unwind_hints() since that needs insn->noendbr.
 	 */

[01/11] objtool: Generic annotation infrastructure

Commit Message

Comments

Patch