mbox series

[v5,0/3] Kbuild: DWARF v5 support

Message ID 20210115210616.404156-1-ndesaulniers@google.com (mailing list archive)
Headers show
Series Kbuild: DWARF v5 support | expand

Message

Nick Desaulniers Jan. 15, 2021, 9:06 p.m. UTC
DWARF v5 is the latest standard of the DWARF debug info format.

DWARF5 wins significantly in terms of size when mixed with compression
(CONFIG_DEBUG_INFO_COMPRESSED).

Link: http://www.dwarfstd.org/doc/DWARF5.pdf

Patch 1 is a cleanup from Masahiro and isn't DWARF v5 specific.
Patch 2 is a cleanup that lays the ground work and isn't DWARF
v5 specific.
Patch 3 implements Kconfig and Kbuild support for DWARFv5.

Changes from v4:
* drop set -e from script as per Nathan.
* add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
* Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
  Masahiro. Sorry it took me a few tries to understand the point (I
  might still not), but it looks much cleaner this way. Sorry Nathan, I
  did not carry forward your previous reviews as a result, but I would
  appreciate if you could look again.
* Add Nathan's reviewed by tag to patch 1.
* Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
  binutils addition later, and BTF issue.
* I still happen to see a pahole related error spew for the combination
  of:
  * LLVM=1
  * LLVM_IAS=1
  * CONFIG_DEBUG_INFO_DWARF4
  * CONFIG_DEBUG_INFO_BTF
  Though they're non-fatal to the build. I'm not sure yet why removing
  any one of the above prevents the warning spew. Maybe we'll need a v6.

Changes from v3:

Changes as per Arvind:
* only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
* add -gdwarf-5 to Kconfig shell script.
* only run Kconfig shell script for Clang.

Apologies to Sedat and Nathan; I appreciate previous testing/review, but
I did no carry forward your Tested-by and Reviewed-by tags, as the
patches have changed too much IMO.

Changes from v2:
* Drop two of the earlier patches that have been accepted already.
* Add measurements with GCC 10.2 to commit message.
* Update help text as per Arvind with help from Caroline.
* Improve case/wording between DWARF Versions as per Masahiro.

Changes from the RFC:
* split patch in 3 patch series, include Fangrui's patch, too.
* prefer `DWARF vX` format, as per Fangrui.
* use spaces between assignment in Makefile as per Masahiro.
* simplify setting dwarf-version-y as per Masahiro.
* indent `prompt` in Kconfig change as per Masahiro.
* remove explicit default in Kconfig as per Masahiro.
* add comments to test_dwarf5_support.sh.
* change echo in test_dwarf5_support.sh as per Masahiro.
* remove -u from test_dwarf5_support.sh as per Masahiro.
* add a -gdwarf-5 cc-option check to Kconfig as per Jakub.

*** BLURB HERE ***

Masahiro Yamada (1):
  Remove $(cc-option,-gdwarf-4) dependency from CONFIG_DEBUG_INFO_DWARF4

Nick Desaulniers (2):
  Kbuild: make DWARF version a choice
  Kbuild: implement support for DWARF v5

 Makefile                          | 13 +++++++---
 include/asm-generic/vmlinux.lds.h |  6 ++++-
 lib/Kconfig.debug                 | 42 +++++++++++++++++++++++++------
 scripts/test_dwarf5_support.sh    |  8 ++++++
 4 files changed, 57 insertions(+), 12 deletions(-)
 create mode 100755 scripts/test_dwarf5_support.sh

Comments

Sedat Dilek Jan. 15, 2021, 9:53 p.m. UTC | #1
On Fri, Jan 15, 2021 at 10:06 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> DWARF5 wins significantly in terms of size when mixed with compression
> (CONFIG_DEBUG_INFO_COMPRESSED).
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
>
> Patch 1 is a cleanup from Masahiro and isn't DWARF v5 specific.
> Patch 2 is a cleanup that lays the ground work and isn't DWARF
> v5 specific.
> Patch 3 implements Kconfig and Kbuild support for DWARFv5.
>
> Changes from v4:
> * drop set -e from script as per Nathan.
> * add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
> * Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
>   Masahiro. Sorry it took me a few tries to understand the point (I
>   might still not), but it looks much cleaner this way. Sorry Nathan, I
>   did not carry forward your previous reviews as a result, but I would
>   appreciate if you could look again.
> * Add Nathan's reviewed by tag to patch 1.
> * Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
>   binutils addition later, and BTF issue.
> * I still happen to see a pahole related error spew for the combination
>   of:
>   * LLVM=1
>   * LLVM_IAS=1
>   * CONFIG_DEBUG_INFO_DWARF4
>   * CONFIG_DEBUG_INFO_BTF
>   Though they're non-fatal to the build. I'm not sure yet why removing
>   any one of the above prevents the warning spew. Maybe we'll need a v6.
>

En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
CONFIG_DEBUG_INFO_DWARF4.
So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.

I had some other small nits commented in the single patches.

As requested in your previous patch-series, feel free to add my:

Tested-by: Sedat Dilek <sedat.dilek@gmail.com>

- Sedat -

> Changes from v3:
>
> Changes as per Arvind:
> * only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
> * add -gdwarf-5 to Kconfig shell script.
> * only run Kconfig shell script for Clang.
>
> Apologies to Sedat and Nathan; I appreciate previous testing/review, but
> I did no carry forward your Tested-by and Reviewed-by tags, as the
> patches have changed too much IMO.
>
> Changes from v2:
> * Drop two of the earlier patches that have been accepted already.
> * Add measurements with GCC 10.2 to commit message.
> * Update help text as per Arvind with help from Caroline.
> * Improve case/wording between DWARF Versions as per Masahiro.
>
> Changes from the RFC:
> * split patch in 3 patch series, include Fangrui's patch, too.
> * prefer `DWARF vX` format, as per Fangrui.
> * use spaces between assignment in Makefile as per Masahiro.
> * simplify setting dwarf-version-y as per Masahiro.
> * indent `prompt` in Kconfig change as per Masahiro.
> * remove explicit default in Kconfig as per Masahiro.
> * add comments to test_dwarf5_support.sh.
> * change echo in test_dwarf5_support.sh as per Masahiro.
> * remove -u from test_dwarf5_support.sh as per Masahiro.
> * add a -gdwarf-5 cc-option check to Kconfig as per Jakub.
>
> *** BLURB HERE ***
>
> Masahiro Yamada (1):
>   Remove $(cc-option,-gdwarf-4) dependency from CONFIG_DEBUG_INFO_DWARF4
>
> Nick Desaulniers (2):
>   Kbuild: make DWARF version a choice
>   Kbuild: implement support for DWARF v5
>
>  Makefile                          | 13 +++++++---
>  include/asm-generic/vmlinux.lds.h |  6 ++++-
>  lib/Kconfig.debug                 | 42 +++++++++++++++++++++++++------
>  scripts/test_dwarf5_support.sh    |  8 ++++++
>  4 files changed, 57 insertions(+), 12 deletions(-)
>  create mode 100755 scripts/test_dwarf5_support.sh
>
> --
> 2.30.0.284.gd98b1dd5eaa7-goog
>
Yonghong Song Jan. 15, 2021, 11:24 p.m. UTC | #2
On 1/15/21 1:53 PM, Sedat Dilek wrote:
> On Fri, Jan 15, 2021 at 10:06 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
>>
>> DWARF v5 is the latest standard of the DWARF debug info format.
>>
>> DWARF5 wins significantly in terms of size when mixed with compression
>> (CONFIG_DEBUG_INFO_COMPRESSED).
>>
>> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
>>
>> Patch 1 is a cleanup from Masahiro and isn't DWARF v5 specific.
>> Patch 2 is a cleanup that lays the ground work and isn't DWARF
>> v5 specific.
>> Patch 3 implements Kconfig and Kbuild support for DWARFv5.
>>
>> Changes from v4:
>> * drop set -e from script as per Nathan.
>> * add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
>> * Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
>>    Masahiro. Sorry it took me a few tries to understand the point (I
>>    might still not), but it looks much cleaner this way. Sorry Nathan, I
>>    did not carry forward your previous reviews as a result, but I would
>>    appreciate if you could look again.
>> * Add Nathan's reviewed by tag to patch 1.
>> * Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
>>    binutils addition later, and BTF issue.
>> * I still happen to see a pahole related error spew for the combination
>>    of:
>>    * LLVM=1
>>    * LLVM_IAS=1
>>    * CONFIG_DEBUG_INFO_DWARF4
>>    * CONFIG_DEBUG_INFO_BTF
>>    Though they're non-fatal to the build. I'm not sure yet why removing
>>    any one of the above prevents the warning spew. Maybe we'll need a v6.
>>
> 
> En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> CONFIG_DEBUG_INFO_DWARF4.
> So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.

I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
It is not there before and adding this may suddenly break some users.

If certain combination of gcc/llvm does not work for 
CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
should fix.

> 
> I had some other small nits commented in the single patches.
> 
> As requested in your previous patch-series, feel free to add my:
> 
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> 
> - Sedat -
> 
>> Changes from v3:
>>
>> Changes as per Arvind:
>> * only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
>> * add -gdwarf-5 to Kconfig shell script.
>> * only run Kconfig shell script for Clang.
>>
>> Apologies to Sedat and Nathan; I appreciate previous testing/review, but
>> I did no carry forward your Tested-by and Reviewed-by tags, as the
>> patches have changed too much IMO.
>>
>> Changes from v2:
>> * Drop two of the earlier patches that have been accepted already.
>> * Add measurements with GCC 10.2 to commit message.
>> * Update help text as per Arvind with help from Caroline.
>> * Improve case/wording between DWARF Versions as per Masahiro.
>>
>> Changes from the RFC:
>> * split patch in 3 patch series, include Fangrui's patch, too.
>> * prefer `DWARF vX` format, as per Fangrui.
>> * use spaces between assignment in Makefile as per Masahiro.
>> * simplify setting dwarf-version-y as per Masahiro.
>> * indent `prompt` in Kconfig change as per Masahiro.
>> * remove explicit default in Kconfig as per Masahiro.
>> * add comments to test_dwarf5_support.sh.
>> * change echo in test_dwarf5_support.sh as per Masahiro.
>> * remove -u from test_dwarf5_support.sh as per Masahiro.
>> * add a -gdwarf-5 cc-option check to Kconfig as per Jakub.
>>
>> *** BLURB HERE ***
>>
>> Masahiro Yamada (1):
>>    Remove $(cc-option,-gdwarf-4) dependency from CONFIG_DEBUG_INFO_DWARF4
>>
>> Nick Desaulniers (2):
>>    Kbuild: make DWARF version a choice
>>    Kbuild: implement support for DWARF v5
>>
>>   Makefile                          | 13 +++++++---
>>   include/asm-generic/vmlinux.lds.h |  6 ++++-
>>   lib/Kconfig.debug                 | 42 +++++++++++++++++++++++++------
>>   scripts/test_dwarf5_support.sh    |  8 ++++++
>>   4 files changed, 57 insertions(+), 12 deletions(-)
>>   create mode 100755 scripts/test_dwarf5_support.sh
>>
>> --
>> 2.30.0.284.gd98b1dd5eaa7-goog
>>
Nick Desaulniers Jan. 15, 2021, 11:34 p.m. UTC | #3
On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > CONFIG_DEBUG_INFO_DWARF4.
> > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.

Can you privately send me your configs that repro? Maybe I can isolate
it to a set of configs?

>
> I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> It is not there before and adding this may suddenly break some users.
>
> If certain combination of gcc/llvm does not work for
> CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> should fix.

Is there a place I should report bugs?

>
> >
> > I had some other small nits commented in the single patches.
> >
> > As requested in your previous patch-series, feel free to add my:
> >
> > Tested-by: Sedat Dilek <sedat.dilek@gmail.com>

Yeah, I'll keep it if v6 is just commit message changes.
Andrii Nakryiko Jan. 15, 2021, 11:42 p.m. UTC | #4
On Fri, Jan 15, 2021 at 3:34 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> >
> >
> >
> > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > CONFIG_DEBUG_INFO_DWARF4.
> > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
>
> Can you privately send me your configs that repro? Maybe I can isolate
> it to a set of configs?

Why privately? To reproduce and fix the issue we (BPF and pahole
community) would need the config as well.


>
> >
> > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > It is not there before and adding this may suddenly break some users.
> >
> > If certain combination of gcc/llvm does not work for
> > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > should fix.
>
> Is there a place I should report bugs?

bpf@vger.kernel.org (BPF in general) and dwarves@vger.kernel.org
(pahole, which seems to be emitting these warnings and having problems
with DWARF5).


>
> >
> > >
> > > I had some other small nits commented in the single patches.
> > >
> > > As requested in your previous patch-series, feel free to add my:
> > >
> > > Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
>
> Yeah, I'll keep it if v6 is just commit message changes.
>
> --
> Thanks,
> ~Nick Desaulniers
Yonghong Song Jan. 15, 2021, 11:43 p.m. UTC | #5
On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 1/15/21 1:53 PM, Sedat Dilek wrote:
>>> En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
>>> CONFIG_DEBUG_INFO_DWARF4.
>>> So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> 
> Can you privately send me your configs that repro? Maybe I can isolate
> it to a set of configs?
> 
>>
>> I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
>> It is not there before and adding this may suddenly break some users.
>>
>> If certain combination of gcc/llvm does not work for
>> CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
>> should fix.
> 
> Is there a place I should report bugs?

You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>, 
dwarves@vger.kernel.org and bpf@vger.kernel.org.

> 
>>
>>>
>>> I had some other small nits commented in the single patches.
>>>
>>> As requested in your previous patch-series, feel free to add my:
>>>
>>> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> 
> Yeah, I'll keep it if v6 is just commit message changes.
>
Arnaldo Carvalho de Melo Jan. 17, 2021, 8:15 p.m. UTC | #6
Em Fri, Jan 15, 2021 at 03:43:06PM -0800, Yonghong Song escreveu:
> 
> 
> On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> > On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> > > 
> > > 
> > > 
> > > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > > CONFIG_DEBUG_INFO_DWARF4.
> > > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> > 
> > Can you privately send me your configs that repro? Maybe I can isolate
> > it to a set of configs?
> > 
> > > 
> > > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > > It is not there before and adding this may suddenly break some users.
> > > 
> > > If certain combination of gcc/llvm does not work for
> > > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > > should fix.
> > 
> > Is there a place I should report bugs?
> 
> You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>,
> dwarves@vger.kernel.org and bpf@vger.kernel.org.

I'm coming back from vacation, will try to read the messages and see if
I can fix this.

Thanks,

- Arnaldo
 
> > 
> > > 
> > > > 
> > > > I had some other small nits commented in the single patches.
> > > > 
> > > > As requested in your previous patch-series, feel free to add my:
> > > > 
> > > > Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> > 
> > Yeah, I'll keep it if v6 is just commit message changes.
> >
Nick Desaulniers Feb. 4, 2021, 1:31 a.m. UTC | #7
On Sun, Jan 17, 2021 at 12:14 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Fri, Jan 15, 2021 at 03:43:06PM -0800, Yonghong Song escreveu:
> >
> >
> > On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> > > On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> > > >
> > > >
> > > >
> > > > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > > > CONFIG_DEBUG_INFO_DWARF4.
> > > > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> > >
> > > Can you privately send me your configs that repro? Maybe I can isolate
> > > it to a set of configs?
> > >
> > > >
> > > > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > > > It is not there before and adding this may suddenly break some users.
> > > >
> > > > If certain combination of gcc/llvm does not work for
> > > > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > > > should fix.
> > >
> > > Is there a place I should report bugs?
> >
> > You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>,
> > dwarves@vger.kernel.org and bpf@vger.kernel.org.
>
> I'm coming back from vacation, will try to read the messages and see if
> I can fix this.

IDK about DWARF v4; that seems to work for me.  I was previously observing
https://bugzilla.redhat.com/show_bug.cgi?id=1922698
with DWARF v5.  I just re-pulled the latest pahole, rebuilt, and no
longer see that warning.

I now observe a different set.  I plan on attending "BPF office hours
tomorrow morning," but if anyone wants a sneak peak of the errors and
how to reproduce:
https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781


(FWIW: some other folks are hitting issues now with kernel's lack of
DWARF v5 support: https://bugzilla.redhat.com/show_bug.cgi?id=1922707)
Andrii Nakryiko Feb. 4, 2021, 2:57 a.m. UTC | #8
On Wed, Feb 3, 2021 at 5:31 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Sun, Jan 17, 2021 at 12:14 PM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > Em Fri, Jan 15, 2021 at 03:43:06PM -0800, Yonghong Song escreveu:
> > >
> > >
> > > On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> > > > On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > > > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > > > > CONFIG_DEBUG_INFO_DWARF4.
> > > > > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> > > >
> > > > Can you privately send me your configs that repro? Maybe I can isolate
> > > > it to a set of configs?
> > > >
> > > > >
> > > > > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > > > > It is not there before and adding this may suddenly break some users.
> > > > >
> > > > > If certain combination of gcc/llvm does not work for
> > > > > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > > > > should fix.
> > > >
> > > > Is there a place I should report bugs?
> > >
> > > You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>,
> > > dwarves@vger.kernel.org and bpf@vger.kernel.org.
> >
> > I'm coming back from vacation, will try to read the messages and see if
> > I can fix this.
>
> IDK about DWARF v4; that seems to work for me.  I was previously observing
> https://bugzilla.redhat.com/show_bug.cgi?id=1922698
> with DWARF v5.  I just re-pulled the latest pahole, rebuilt, and no
> longer see that warning.
>
> I now observe a different set.  I plan on attending "BPF office hours
> tomorrow morning," but if anyone wants a sneak peak of the errors and
> how to reproduce:
> https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
>

Is there another (easy) way to get your patch set without the b4 tool?
Is your patch set present in some patchworks instance, so that I can
download it in mbox format, for example?

>
> (FWIW: some other folks are hitting issues now with kernel's lack of
> DWARF v5 support: https://bugzilla.redhat.com/show_bug.cgi?id=1922707)
> --
> Thanks,
> ~Nick Desaulniers
Nick Desaulniers Feb. 4, 2021, 3:13 a.m. UTC | #9
On Wed, Feb 3, 2021 at 6:58 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Wed, Feb 3, 2021 at 5:31 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
> >
> > On Sun, Jan 17, 2021 at 12:14 PM Arnaldo Carvalho de Melo
> > <acme@kernel.org> wrote:
> > >
> > > Em Fri, Jan 15, 2021 at 03:43:06PM -0800, Yonghong Song escreveu:
> > > >
> > > >
> > > > On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> > > > > On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > > > > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > > > > > CONFIG_DEBUG_INFO_DWARF4.
> > > > > > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> > > > >
> > > > > Can you privately send me your configs that repro? Maybe I can isolate
> > > > > it to a set of configs?
> > > > >
> > > > > >
> > > > > > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > > > > > It is not there before and adding this may suddenly break some users.
> > > > > >
> > > > > > If certain combination of gcc/llvm does not work for
> > > > > > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > > > > > should fix.
> > > > >
> > > > > Is there a place I should report bugs?
> > > >
> > > > You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>,
> > > > dwarves@vger.kernel.org and bpf@vger.kernel.org.
> > >
> > > I'm coming back from vacation, will try to read the messages and see if
> > > I can fix this.
> >
> > IDK about DWARF v4; that seems to work for me.  I was previously observing
> > https://bugzilla.redhat.com/show_bug.cgi?id=1922698
> > with DWARF v5.  I just re-pulled the latest pahole, rebuilt, and no
> > longer see that warning.
> >
> > I now observe a different set.  I plan on attending "BPF office hours
> > tomorrow morning," but if anyone wants a sneak peak of the errors and
> > how to reproduce:
> > https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
> >
>
> Is there another (easy) way to get your patch set without the b4 tool?
> Is your patch set present in some patchworks instance, so that I can
> download it in mbox format, for example?

$ wget https://lore.kernel.org/lkml/20210130004401.2528717-2-ndesaulniers@google.com/raw
-O - | git am
$ wget https://lore.kernel.org/lkml/20210130004401.2528717-3-ndesaulniers@google.com/raw
-O - | git am

If you haven't tried b4 yet, it's quite nice.  Hard to go back.  Lore
also has mbox.gz links.  Not sure about patchwork.

>
> >
> > (FWIW: some other folks are hitting issues now with kernel's lack of
> > DWARF v5 support: https://bugzilla.redhat.com/show_bug.cgi?id=1922707)
Sedat Dilek Feb. 4, 2021, 8:42 a.m. UTC | #10
On Thu, Feb 4, 2021 at 3:58 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
...
>
> Is there another (easy) way to get your patch set without the b4 tool?
> Is your patch set present in some patchworks instance, so that I can
> download it in mbox format, for example?
>

Just to promote the b4 tool - we have some cool wiki in [1].

Personally, I got in touch with b4 when dealing with the
ClangBuiltLinux project.

Note: Sometimes it takes a bit for the patch(set) to be available from
the LORE link.

- Sedat -

[1] https://github.com/ClangBuiltLinux/linux/wiki/Command-line-tips-and-tricks#fetching-a-single-patch-or-series-from-lkml
Sedat Dilek Feb. 4, 2021, 8:59 a.m. UTC | #11
On Thu, Feb 4, 2021 at 9:42 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Thu, Feb 4, 2021 at 3:58 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> ...
> >
> > Is there another (easy) way to get your patch set without the b4 tool?
> > Is your patch set present in some patchworks instance, so that I can
> > download it in mbox format, for example?
> >
>
> Just to promote the b4 tool - we have some cool wiki in [1].
>
> Personally, I got in touch with b4 when dealing with the
> ClangBuiltLinux project.
>
> Note: Sometimes it takes a bit for the patch(set) to be available from
> the LORE link.
>
> - Sedat -
>
> [1] https://github.com/ClangBuiltLinux/linux/wiki/Command-line-tips-and-tricks#fetching-a-single-patch-or-series-from-lkml

Honestly, when behind a proxy-server I did not find a trick to use b4
here (DNS resolution problems).
I tried with proxychains-ng - NOPE.

In the meantime I updated the CBL-wiki page and added v8 as an example :-).

- Sedat -
Andrii Nakryiko Feb. 4, 2021, 11:54 p.m. UTC | #12
On Wed, Feb 3, 2021 at 7:13 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Wed, Feb 3, 2021 at 6:58 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Wed, Feb 3, 2021 at 5:31 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
> > >
> > > On Sun, Jan 17, 2021 at 12:14 PM Arnaldo Carvalho de Melo
> > > <acme@kernel.org> wrote:
> > > >
> > > > Em Fri, Jan 15, 2021 at 03:43:06PM -0800, Yonghong Song escreveu:
> > > > >
> > > > >
> > > > > On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> > > > > > On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > > > > > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > > > > > > CONFIG_DEBUG_INFO_DWARF4.
> > > > > > > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> > > > > >
> > > > > > Can you privately send me your configs that repro? Maybe I can isolate
> > > > > > it to a set of configs?
> > > > > >
> > > > > > >
> > > > > > > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > > > > > > It is not there before and adding this may suddenly break some users.
> > > > > > >
> > > > > > > If certain combination of gcc/llvm does not work for
> > > > > > > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > > > > > > should fix.
> > > > > >
> > > > > > Is there a place I should report bugs?
> > > > >
> > > > > You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>,
> > > > > dwarves@vger.kernel.org and bpf@vger.kernel.org.
> > > >
> > > > I'm coming back from vacation, will try to read the messages and see if
> > > > I can fix this.
> > >
> > > IDK about DWARF v4; that seems to work for me.  I was previously observing
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1922698
> > > with DWARF v5.  I just re-pulled the latest pahole, rebuilt, and no
> > > longer see that warning.
> > >
> > > I now observe a different set.  I plan on attending "BPF office hours
> > > tomorrow morning," but if anyone wants a sneak peak of the errors and
> > > how to reproduce:
> > > https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
> > >
> >
> > Is there another (easy) way to get your patch set without the b4 tool?
> > Is your patch set present in some patchworks instance, so that I can
> > download it in mbox format, for example?
>
> $ wget https://lore.kernel.org/lkml/20210130004401.2528717-2-ndesaulniers@google.com/raw
> -O - | git am
> $ wget https://lore.kernel.org/lkml/20210130004401.2528717-3-ndesaulniers@google.com/raw
> -O - | git am
>
> If you haven't tried b4 yet, it's quite nice.  Hard to go back.  Lore
> also has mbox.gz links.  Not sure about patchwork.
>

Ok, I managed to apply that on linux-next, but I can't get past this:

ld.lld: error: undefined symbol: pa_trampoline_start
>>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)

ld.lld: error: undefined symbol: pa_trampoline_header
>>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)

ld.lld: error: undefined symbol: pa_trampoline_pgd
>>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)
>>> referenced by trampoline_64.S:142 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:142)
>>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)

ld.lld: error: undefined symbol: pa_wakeup_start
>>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)

ld.lld: error: undefined symbol: pa_wakeup_header
>>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)

ld.lld: error: undefined symbol: pa_machine_real_restart_asm
>>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)

ld.lld: error: undefined symbol: pa_startup_32
>>> referenced by trampoline_64.S:77 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:77)
>>>               arch/x86/realmode/rm/trampoline_64.o:(trampoline_start)

ld.lld: error: undefined symbol: pa_tr_flags
>>> referenced by trampoline_64.S:124 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:124)
>>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)

ld.lld: error: undefined symbol: pa_tr_cr4
>>> referenced by trampoline_64.S:138 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:138)
>>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)

ld.lld: error: undefined symbol: pa_tr_efer
>>> referenced by trampoline_64.S:146 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:146)
>>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
>>> referenced by trampoline_64.S:147 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:147)
>>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)

ld.lld: error: undefined symbol: pa_startup_64
>>> referenced by trampoline_64.S:161 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:161)
>>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)

ld.lld: error: undefined symbol: pa_tr_gdt
>>> referenced by arch/x86/realmode/rm/trampoline_64.o:(tr_gdt)
>>> referenced by reboot.S:28 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:28)
>>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)

ld.lld: error: undefined symbol: pa_machine_real_restart_paging_off
>>> referenced by reboot.S:34 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:34)
>>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)

ld.lld: error: undefined symbol: pa_machine_real_restart_idt
>>> referenced by reboot.S:47 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:47)
>>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)

ld.lld: error: undefined symbol: pa_machine_real_restart_gdt
>>> referenced by reboot.S:54 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:54)
>>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)
>>> referenced by arch/x86/realmode/rm/reboot.o:(machine_real_restart_gdt)

ld.lld: error: undefined symbol: pa_wakeup_gdt
>>> referenced by arch/x86/realmode/rm/wakeup_asm.o:(wakeup_gdt)
  CC      arch/x86/mm/numa_64.o
  CC      arch/x86/mm/amdtopology.o
  HOSTCC  arch/x86/entry/vdso/vdso2c
make[4]: *** [arch/x86/realmode/rm/realmode.elf] Error 1
make[3]: *** [arch/x86/realmode/rm/realmode.bin] Error 2
make[2]: *** [arch/x86/realmode] Error 2
make[2]: *** Waiting for unfinished jobs....


Hopefully Arnaldo will have better luck.



> >
> > >
> > > (FWIW: some other folks are hitting issues now with kernel's lack of
> > > DWARF v5 support: https://bugzilla.redhat.com/show_bug.cgi?id=1922707)
>
>
> --
> Thanks,
> ~Nick Desaulniers
Nick Desaulniers Feb. 5, 2021, 12:04 a.m. UTC | #13
Moving a bunch of folks + lists to BCC.

On Thu, Feb 4, 2021 at 3:54 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Wed, Feb 3, 2021 at 7:13 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
> >
> > On Wed, Feb 3, 2021 at 6:58 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Wed, Feb 3, 2021 at 5:31 PM Nick Desaulniers <ndesaulniers@google.com> wrote:
> > > >
> > > > On Sun, Jan 17, 2021 at 12:14 PM Arnaldo Carvalho de Melo
> > > > <acme@kernel.org> wrote:
> > > > >
> > > > > Em Fri, Jan 15, 2021 at 03:43:06PM -0800, Yonghong Song escreveu:
> > > > > >
> > > > > >
> > > > > > On 1/15/21 3:34 PM, Nick Desaulniers wrote:
> > > > > > > On Fri, Jan 15, 2021 at 3:24 PM Yonghong Song <yhs@fb.com> wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 1/15/21 1:53 PM, Sedat Dilek wrote:
> > > > > > > > > En plus, I encountered breakage with GCC v10.2.1 and LLVM=1 and
> > > > > > > > > CONFIG_DEBUG_INFO_DWARF4.
> > > > > > > > > So might be good to add a "depends on !DEBUG_INFO_BTF" in this combination.
> > > > > > >
> > > > > > > Can you privately send me your configs that repro? Maybe I can isolate
> > > > > > > it to a set of configs?
> > > > > > >
> > > > > > > >
> > > > > > > > I suggested not to add !DEBUG_INFO_BTF to CONFIG_DEBUG_INFO_DWARF4.
> > > > > > > > It is not there before and adding this may suddenly break some users.
> > > > > > > >
> > > > > > > > If certain combination of gcc/llvm does not work for
> > > > > > > > CONFIG_DEBUG_INFO_DWARF4 with pahole, this is a bug bpf community
> > > > > > > > should fix.
> > > > > > >
> > > > > > > Is there a place I should report bugs?
> > > > > >
> > > > > > You can send bug report to Arnaldo Carvalho de Melo <acme@kernel.org>,
> > > > > > dwarves@vger.kernel.org and bpf@vger.kernel.org.
> > > > >
> > > > > I'm coming back from vacation, will try to read the messages and see if
> > > > > I can fix this.
> > > >
> > > > IDK about DWARF v4; that seems to work for me.  I was previously observing
> > > > https://bugzilla.redhat.com/show_bug.cgi?id=1922698
> > > > with DWARF v5.  I just re-pulled the latest pahole, rebuilt, and no
> > > > longer see that warning.
> > > >
> > > > I now observe a different set.  I plan on attending "BPF office hours
> > > > tomorrow morning," but if anyone wants a sneak peak of the errors and
> > > > how to reproduce:
> > > > https://gist.github.com/nickdesaulniers/ae8c9efbe4da69b1cf0dce138c1d2781
> > > >
> > >
> > > Is there another (easy) way to get your patch set without the b4 tool?
> > > Is your patch set present in some patchworks instance, so that I can
> > > download it in mbox format, for example?
> >
> > $ wget https://lore.kernel.org/lkml/20210130004401.2528717-2-ndesaulniers@google.com/raw
> > -O - | git am
> > $ wget https://lore.kernel.org/lkml/20210130004401.2528717-3-ndesaulniers@google.com/raw
> > -O - | git am
> >
> > If you haven't tried b4 yet, it's quite nice.  Hard to go back.  Lore
> > also has mbox.gz links.  Not sure about patchwork.
> >
>
> Ok, I managed to apply that on linux-next, but I can't get past this:
>
> ld.lld: error: undefined symbol: pa_trampoline_start
> >>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)

Thanks for testing and the report. Do you have a .config you can send
me to reproduce?

>
> ld.lld: error: undefined symbol: pa_trampoline_header
> >>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)
>
> ld.lld: error: undefined symbol: pa_trampoline_pgd
> >>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)
> >>> referenced by trampoline_64.S:142 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:142)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
>
> ld.lld: error: undefined symbol: pa_wakeup_start
> >>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)
>
> ld.lld: error: undefined symbol: pa_wakeup_header
> >>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)
>
> ld.lld: error: undefined symbol: pa_machine_real_restart_asm
> >>> referenced by arch/x86/realmode/rm/header.o:(real_mode_header)
>
> ld.lld: error: undefined symbol: pa_startup_32
> >>> referenced by trampoline_64.S:77 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:77)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(trampoline_start)
>
> ld.lld: error: undefined symbol: pa_tr_flags
> >>> referenced by trampoline_64.S:124 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:124)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
>
> ld.lld: error: undefined symbol: pa_tr_cr4
> >>> referenced by trampoline_64.S:138 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:138)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
>
> ld.lld: error: undefined symbol: pa_tr_efer
> >>> referenced by trampoline_64.S:146 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:146)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
> >>> referenced by trampoline_64.S:147 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:147)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
>
> ld.lld: error: undefined symbol: pa_startup_64
> >>> referenced by trampoline_64.S:161 (/data/users/andriin/linux/arch/x86/realmode/rm/trampoline_64.S:161)
> >>>               arch/x86/realmode/rm/trampoline_64.o:(startup_32)
>
> ld.lld: error: undefined symbol: pa_tr_gdt
> >>> referenced by arch/x86/realmode/rm/trampoline_64.o:(tr_gdt)
> >>> referenced by reboot.S:28 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:28)
> >>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)
>
> ld.lld: error: undefined symbol: pa_machine_real_restart_paging_off
> >>> referenced by reboot.S:34 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:34)
> >>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)
>
> ld.lld: error: undefined symbol: pa_machine_real_restart_idt
> >>> referenced by reboot.S:47 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:47)
> >>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)
>
> ld.lld: error: undefined symbol: pa_machine_real_restart_gdt
> >>> referenced by reboot.S:54 (/data/users/andriin/linux/arch/x86/realmode/rm/reboot.S:54)
> >>>               arch/x86/realmode/rm/reboot.o:(machine_real_restart_asm)
> >>> referenced by arch/x86/realmode/rm/reboot.o:(machine_real_restart_gdt)
>
> ld.lld: error: undefined symbol: pa_wakeup_gdt
> >>> referenced by arch/x86/realmode/rm/wakeup_asm.o:(wakeup_gdt)
>   CC      arch/x86/mm/numa_64.o
>   CC      arch/x86/mm/amdtopology.o
>   HOSTCC  arch/x86/entry/vdso/vdso2c
> make[4]: *** [arch/x86/realmode/rm/realmode.elf] Error 1
> make[3]: *** [arch/x86/realmode/rm/realmode.bin] Error 2
> make[2]: *** [arch/x86/realmode] Error 2
> make[2]: *** Waiting for unfinished jobs....
>
>
> Hopefully Arnaldo will have better luck.
>
>
>
> > >
> > > >
> > > > (FWIW: some other folks are hitting issues now with kernel's lack of
> > > > DWARF v5 support: https://bugzilla.redhat.com/show_bug.cgi?id=1922707)
> >
> >
> > --
> > Thanks,
> > ~Nick Desaulniers