diff mbox series

[01/12] scripts: subarch.include: fix SUBARCH on MacOS hosts

Message ID 20240807-macos-build-support-v1-1-4cd1ded85694@samsung.com (mailing list archive)
State Handled Elsewhere
Delegated to: Paul Moore
Headers show
Series Enable build system on macOS hosts | expand

Commit Message

Daniel Gomez via B4 Relay Aug. 6, 2024, 11:09 p.m. UTC 
From: Nick Desaulniers <nick.desaulniers@gmail.com>

When building the Linux kernel on an aarch64 MacOS based host, if we don't
specify a value for ARCH when invoking make, we default to arm and thus
multi_v7_defconfig rather than the expected arm64 and arm64's defconfig.

This is because subarch.include invokes `uname -m` which on MacOS hosts
evaluates to `arm64` but on Linux hosts evaluates to `aarch64`,

This allows us to build ARCH=arm64 natively on MacOS (as in ARCH need
not be specified on an aarch64-based system).

Utilize a negative lookahead regular expression to avoid matching arm64.

Add a separate expression to support for armv.* as per error reported by
Nicolas Schier [1].

[1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t

Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com>
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
 scripts/subarch.include | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Nicolas Schier Aug. 7, 2024, 3:07 p.m. UTC | #1 
On Wed, Aug 07, 2024 at 01:09:15AM +0200, Daniel Gomez via B4 Relay wrote:
> From: Nick Desaulniers <nick.desaulniers@gmail.com>
> 
> When building the Linux kernel on an aarch64 MacOS based host, if we don't
> specify a value for ARCH when invoking make, we default to arm and thus
> multi_v7_defconfig rather than the expected arm64 and arm64's defconfig.
> 
> This is because subarch.include invokes `uname -m` which on MacOS hosts
> evaluates to `arm64` but on Linux hosts evaluates to `aarch64`,
> 
> This allows us to build ARCH=arm64 natively on MacOS (as in ARCH need
> not be specified on an aarch64-based system).
> 
> Utilize a negative lookahead regular expression to avoid matching arm64.
> 
> Add a separate expression to support for armv.* as per error reported by
> Nicolas Schier [1].
> 
> [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t
> 
> Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com>
> Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
> ---
>  scripts/subarch.include | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/subarch.include b/scripts/subarch.include
> index 4bd327d0ae42..5d84ad8c0dee 100644
> --- a/scripts/subarch.include
> +++ b/scripts/subarch.include
> @@ -6,7 +6,8 @@
>  
>  SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
>  				  -e s/sun4u/sparc64/ \
> -				  -e s/arm.*/arm/ -e s/sa110/arm/ \
> +				  -e s/armv.*/arm/ \
> +				  -e s/arm\(?:\(?!64\).*\)/arm/ -e s/sa110/arm/ \
>  				  -e s/s390x/s390/ \
>  				  -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
>  				  -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \
> 

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Masahiro Yamada Aug. 23, 2024, 4:12 p.m. UTC | #2 
On Wed, Aug 7, 2024 at 8:10 AM Daniel Gomez via B4 Relay
<devnull+da.gomez.samsung.com@kernel.org> wrote:
>
> From: Nick Desaulniers <nick.desaulniers@gmail.com>
>
> When building the Linux kernel on an aarch64 MacOS based host, if we don't
> specify a value for ARCH when invoking make, we default to arm and thus
> multi_v7_defconfig rather than the expected arm64 and arm64's defconfig.
>
> This is because subarch.include invokes `uname -m` which on MacOS hosts
> evaluates to `arm64` but on Linux hosts evaluates to `aarch64`,
>
> This allows us to build ARCH=arm64 natively on MacOS (as in ARCH need
> not be specified on an aarch64-based system).
>
> Utilize a negative lookahead regular expression to avoid matching arm64.


Does sed support "negative lookahead regular expression"?

>
> Add a separate expression to support for armv.* as per error reported by
> Nicolas Schier [1].
>
> [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t
>
> Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com>
> Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
> ---
>  scripts/subarch.include | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/subarch.include b/scripts/subarch.include
> index 4bd327d0ae42..5d84ad8c0dee 100644
> --- a/scripts/subarch.include
> +++ b/scripts/subarch.include
> @@ -6,7 +6,8 @@
>
>  SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
>                                   -e s/sun4u/sparc64/ \
> -                                 -e s/arm.*/arm/ -e s/sa110/arm/ \
> +                                 -e s/armv.*/arm/ \
> +                                 -e s/arm\(?:\(?!64\).*\)/arm/ -e s/sa110/arm/ \


s/arm\(?:\(?!64\).*\)/arm/

In sed, this expression does not seem to match anything.

(or please give me some matching examples if I miss something)





Nocolas already provided correct code:

> [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t






>                                   -e s/s390x/s390/ \
>                                   -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
>                                   -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \
>
> --
> Git-146)
>
>


--
Best Regards


Masahiro Yamada
Daniel Gomez Aug. 23, 2024, 10:14 p.m. UTC | #3 
On Fri, Aug 23, 2024 at 6:13 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Wed, Aug 7, 2024 at 8:10 AM Daniel Gomez via B4 Relay
> <devnull+da.gomez.samsung.com@kernel.org> wrote:
> >
> > From: Nick Desaulniers <nick.desaulniers@gmail.com>
> >
> > When building the Linux kernel on an aarch64 MacOS based host, if we don't
> > specify a value for ARCH when invoking make, we default to arm and thus
> > multi_v7_defconfig rather than the expected arm64 and arm64's defconfig.
> >
> > This is because subarch.include invokes `uname -m` which on MacOS hosts
> > evaluates to `arm64` but on Linux hosts evaluates to `aarch64`,
> >
> > This allows us to build ARCH=arm64 natively on MacOS (as in ARCH need
> > not be specified on an aarch64-based system).
> >
> > Utilize a negative lookahead regular expression to avoid matching arm64.
>
>
> Does sed support "negative lookahead regular expression"?

I think they removed support for PCRE. I've found this:

commit 261c7f145d015d9acb79dc650d27e4a23b839c23
Author: Assaf Gordon <assafgordon@gmail.com>
Date:   Tue Aug 21 14:25:57 2018 -0600

    maint: remove REG_PERL code

    Perl-regexp syntax (PCRE) in GNU Sed is shelved indefinitely.
    See https://bugs.gnu.org/22801 , https://bugs.gnu.org/22647 .
    Remove all (unused) REG_PERL related code.

    * sed/sed.c, sed/sed.h, sed/regexp.c, sed/compile.c: Remove REG_PERL code.

git tag --contains 261c7f145d015d9acb79dc650d27e4a23b839c23
v4.6
v4.7
v4.8
v4.9

And my sed version is (Debian):

sed --version
sed (GNU sed) 4.9
Packaged by Debian
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
Paolo Bonzini, Jim Meyering, and Assaf Gordon.

This sed program was built with SELinux support.
SELinux is disabled on this system.

GNU sed home page: <https://www.gnu.org/software/sed/>.
General help using GNU software: <https://www.gnu.org/gethelp/>.
E-mail bug reports to: <bug-sed@gnu.org>.

sed version (Homebrew):
sed --version
sed (GNU sed) 4.9
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
Paolo Bonzini, Jim Meyering, and Assaf Gordon.

This sed program was built without SELinux support.

GNU sed home page: <https://www.gnu.org/software/sed/>.
General help using GNU software: <https://www.gnu.org/gethelp/>.
E-mail bug reports to: <bug-sed@gnu.org>.

>
> >
> > Add a separate expression to support for armv.* as per error reported by
> > Nicolas Schier [1].
> >
> > [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t
> >
> > Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com>
> > Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
> > ---
> >  scripts/subarch.include | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/subarch.include b/scripts/subarch.include
> > index 4bd327d0ae42..5d84ad8c0dee 100644
> > --- a/scripts/subarch.include
> > +++ b/scripts/subarch.include
> > @@ -6,7 +6,8 @@
> >
> >  SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
> >                                   -e s/sun4u/sparc64/ \
> > -                                 -e s/arm.*/arm/ -e s/sa110/arm/ \
> > +                                 -e s/armv.*/arm/ \
> > +                                 -e s/arm\(?:\(?!64\).*\)/arm/ -e s/sa110/arm/ \
>
>
> s/arm\(?:\(?!64\).*\)/arm/
>
> In sed, this expression does not seem to match anything.

You are correct. I've removed the expression and saw no difference.
See below with my test case:
>
> (or please give me some matching examples if I miss something)

cat Makefile
MACHINE ?= "aarch64"
SUBARCH0 := $(shell echo $(MACHINE) | sed \
                                  -e s/arm.*/arm/ \
                                  -e s/aarch64.*/arm64/)

SUBARCH1 := $(shell echo $(MACHINE) | sed \
                                  -e s/armv.*/arm/ \
                                  -e s/aarch64.*/arm64/)

SUBARCH2 := $(shell echo $(MACHINE) | sed \
                                  -e /^arm64$/!s/arm.*/arm/ \
                                  -e s/aarch64.*/arm64/)

test:
        @echo "MACHINE=$(MACHINE)"
        @echo "SUBARCH0=$(SUBARCH0)"
        @echo "SUBARCH1=$(SUBARCH1)"
        @echo "SUBARCH2=$(SUBARCH2)"
        @echo "---"

SUBARCH0 represents the current upstream expressions for arm/arm64.
SUBARCH1 is my proposal in case we need to cover only armv* for 32-bit
arm (I think that is incomplete?) and SUBARCH2 is Nicolas' proposal
(which I can't make it work in the test Makefile).

Running the above Makefile, I get:

make test MACHINE=armv4 && make test MACHINE=arm7 && make test
MACHINE=armhf && make test MACHINE=aarch64 && make test MACHINE=arm64
MACHINE=armv4
SUBARCH0=arm
SUBARCH1=arm
SUBARCH2=armv4
---
MACHINE=arm7
SUBARCH0=arm
SUBARCH1=arm7
SUBARCH2=arm7
---
MACHINE=armhf
SUBARCH0=arm
SUBARCH1=armhf
SUBARCH2=armhf
---
MACHINE=aarch64
SUBARCH0=arm64
SUBARCH1=arm64
SUBARCH2=arm64
---
MACHINE=arm64
SUBARCH0=arm
SUBARCH1=arm64
SUBARCH2=arm64
---
>
>
>
>
>
> Nocolas already provided correct code:
>
> > [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t

I think it is even more simple if we just make this change:

-                                 -e s/arm.*/arm/ -e s/sa110/arm/ \
+                                 -e s/armv.*/arm/ \

Does armv.* cover all arm32 machines? I see armhf, arm7, arm8 and
armv*, is it correct?

And thanks for checking!

>
>
>
>
>
>
> >                                   -e s/s390x/s390/ \
> >                                   -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
> >                                   -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \
> >
> > --
> > Git-146)
> >
> >
>
>
> --
> Best Regards
>
>
> Masahiro Yamada
Daniel Gomez Aug. 24, 2024, 7:34 p.m. UTC | #4 
On Sat, Aug 24, 2024 at 12:14 AM Daniel Gomez (Samsung)
<d+samsung@kruces.com> wrote:
>
> On Fri, Aug 23, 2024 at 6:13 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Wed, Aug 7, 2024 at 8:10 AM Daniel Gomez via B4 Relay
> > <devnull+da.gomez.samsung.com@kernel.org> wrote:
> > >
> > > From: Nick Desaulniers <nick.desaulniers@gmail.com>
> > >
> > > When building the Linux kernel on an aarch64 MacOS based host, if we don't
> > > specify a value for ARCH when invoking make, we default to arm and thus
> > > multi_v7_defconfig rather than the expected arm64 and arm64's defconfig.
> > >
> > > This is because subarch.include invokes `uname -m` which on MacOS hosts
> > > evaluates to `arm64` but on Linux hosts evaluates to `aarch64`,
> > >
> > > This allows us to build ARCH=arm64 natively on MacOS (as in ARCH need
> > > not be specified on an aarch64-based system).
> > >
> > > Utilize a negative lookahead regular expression to avoid matching arm64.
> >
> >
> > Does sed support "negative lookahead regular expression"?
>
> I think they removed support for PCRE. I've found this:
>
> commit 261c7f145d015d9acb79dc650d27e4a23b839c23
> Author: Assaf Gordon <assafgordon@gmail.com>
> Date:   Tue Aug 21 14:25:57 2018 -0600
>
>     maint: remove REG_PERL code
>
>     Perl-regexp syntax (PCRE) in GNU Sed is shelved indefinitely.
>     See https://bugs.gnu.org/22801 , https://bugs.gnu.org/22647 .
>     Remove all (unused) REG_PERL related code.
>
>     * sed/sed.c, sed/sed.h, sed/regexp.c, sed/compile.c: Remove REG_PERL code.
>
> git tag --contains 261c7f145d015d9acb79dc650d27e4a23b839c23
> v4.6
> v4.7
> v4.8
> v4.9
>
> And my sed version is (Debian):
>
> sed --version
> sed (GNU sed) 4.9
> Packaged by Debian
> Copyright (C) 2022 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Written by Jay Fenlason, Tom Lord, Ken Pizzini,
> Paolo Bonzini, Jim Meyering, and Assaf Gordon.
>
> This sed program was built with SELinux support.
> SELinux is disabled on this system.
>
> GNU sed home page: <https://www.gnu.org/software/sed/>.
> General help using GNU software: <https://www.gnu.org/gethelp/>.
> E-mail bug reports to: <bug-sed@gnu.org>.
>
> sed version (Homebrew):
> sed --version
> sed (GNU sed) 4.9
> Copyright (C) 2022 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Written by Jay Fenlason, Tom Lord, Ken Pizzini,
> Paolo Bonzini, Jim Meyering, and Assaf Gordon.
>
> This sed program was built without SELinux support.
>
> GNU sed home page: <https://www.gnu.org/software/sed/>.
> General help using GNU software: <https://www.gnu.org/gethelp/>.
> E-mail bug reports to: <bug-sed@gnu.org>.
>
> >
> > >
> > > Add a separate expression to support for armv.* as per error reported by
> > > Nicolas Schier [1].
> > >
> > > [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t
> > >
> > > Signed-off-by: Nick Desaulniers <nick.desaulniers@gmail.com>
> > > Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
> > > ---
> > >  scripts/subarch.include | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/scripts/subarch.include b/scripts/subarch.include
> > > index 4bd327d0ae42..5d84ad8c0dee 100644
> > > --- a/scripts/subarch.include
> > > +++ b/scripts/subarch.include
> > > @@ -6,7 +6,8 @@
> > >
> > >  SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
> > >                                   -e s/sun4u/sparc64/ \
> > > -                                 -e s/arm.*/arm/ -e s/sa110/arm/ \
> > > +                                 -e s/armv.*/arm/ \
> > > +                                 -e s/arm\(?:\(?!64\).*\)/arm/ -e s/sa110/arm/ \
> >
> >
> > s/arm\(?:\(?!64\).*\)/arm/
> >
> > In sed, this expression does not seem to match anything.
>
> You are correct. I've removed the expression and saw no difference.
> See below with my test case:
> >
> > (or please give me some matching examples if I miss something)
>
> cat Makefile
> MACHINE ?= "aarch64"
> SUBARCH0 := $(shell echo $(MACHINE) | sed \
>                                   -e s/arm.*/arm/ \
>                                   -e s/aarch64.*/arm64/)
>
> SUBARCH1 := $(shell echo $(MACHINE) | sed \
>                                   -e s/armv.*/arm/ \
>                                   -e s/aarch64.*/arm64/)
>
> SUBARCH2 := $(shell echo $(MACHINE) | sed \
>                                   -e /^arm64$/!s/arm.*/arm/ \
>                                   -e s/aarch64.*/arm64/)
>
> test:
>         @echo "MACHINE=$(MACHINE)"
>         @echo "SUBARCH0=$(SUBARCH0)"
>         @echo "SUBARCH1=$(SUBARCH1)"
>         @echo "SUBARCH2=$(SUBARCH2)"
>         @echo "---"
>
> SUBARCH0 represents the current upstream expressions for arm/arm64.
> SUBARCH1 is my proposal in case we need to cover only armv* for 32-bit
> arm (I think that is incomplete?) and SUBARCH2 is Nicolas' proposal
> (which I can't make it work in the test Makefile).

To make Nicolas's expression work in Makefile I just need to pass 2 $ like this:

diff -u Makefile.old Makefile
--- Makefile.old  2024-08-24 21:25:28.525267566 +0200
+++ Makefile    2024-08-24 21:28:32.640477991 +0200
@@ -8,7 +8,7 @@
                                  -e s/aarch64.*/arm64/)

 SUBARCH2 := $(shell echo $(MACHINE) | sed \
-                                 -e /^arm64$/!s/arm.*/arm/ \
+                                 -e /^arm64$$/!s/arm.*/arm/ \
                                  -e s/aarch64.*/arm64/)

 test:

And all test cases passed. So, I will include this change for v2.

>
> Running the above Makefile, I get:
>
> make test MACHINE=armv4 && make test MACHINE=arm7 && make test
> MACHINE=armhf && make test MACHINE=aarch64 && make test MACHINE=arm64
> MACHINE=armv4
> SUBARCH0=arm
> SUBARCH1=arm
> SUBARCH2=armv4
> ---
> MACHINE=arm7
> SUBARCH0=arm
> SUBARCH1=arm7
> SUBARCH2=arm7
> ---
> MACHINE=armhf
> SUBARCH0=arm
> SUBARCH1=armhf
> SUBARCH2=armhf
> ---
> MACHINE=aarch64
> SUBARCH0=arm64
> SUBARCH1=arm64
> SUBARCH2=arm64
> ---
> MACHINE=arm64
> SUBARCH0=arm
> SUBARCH1=arm64
> SUBARCH2=arm64
> ---
> >
> >
> >
> >
> >
> > Nocolas already provided correct code:
> >
> > > [1] https://lore.kernel.org/all/Y3MRvtwdjIwMHvRo@bergen.fjasle.eu/#t
>
> I think it is even more simple if we just make this change:
>
> -                                 -e s/arm.*/arm/ -e s/sa110/arm/ \
> +                                 -e s/armv.*/arm/ \
>
> Does armv.* cover all arm32 machines? I see armhf, arm7, arm8 and
> armv*, is it correct?
>
> And thanks for checking!
>
> >
> >
> >
> >
> >
> >
> > >                                   -e s/s390x/s390/ \
> > >                                   -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
> > >                                   -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \
> > >
> > > --
> > > Git-146)
> > >
> > >
> >
> >
> > --
> > Best Regards
> >
> >
> > Masahiro Yamada
diff mbox series

Patch

diff --git a/scripts/subarch.include b/scripts/subarch.include
index 4bd327d0ae42..5d84ad8c0dee 100644
--- a/scripts/subarch.include
+++ b/scripts/subarch.include
@@ -6,7 +6,8 @@ 
 
 SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \
 				  -e s/sun4u/sparc64/ \
-				  -e s/arm.*/arm/ -e s/sa110/arm/ \
+				  -e s/armv.*/arm/ \
+				  -e s/arm\(?:\(?!64\).*\)/arm/ -e s/sa110/arm/ \
 				  -e s/s390x/s390/ \
 				  -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
 				  -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \