[v2,8/8] Documentation: add howto build in macos

Commit Message

Daniel Gomez via B4 Relay Sept. 6, 2024, 11:01 a.m. UTC

From: Daniel Gomez <[e-mail]>

Add documentation under kbuild/llvm to inform about the experimental
support for building the Linux kernel in macOS hosts environments.

Signed-off-by: Daniel Gomez <[e-mail]>
---
 Documentation/kbuild/llvm.rst | 78 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)

Comments

Masahiro Yamada Sept. 7, 2024, 8:33 a.m. UTC | #1

On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay
<[e-mail]> wrote:
>
> From: Daniel Gomez <[e-mail]>
>
> Add documentation under kbuild/llvm to inform about the experimental
> support for building the Linux kernel in macOS hosts environments.
>
> Signed-off-by: Daniel Gomez <[e-mail]>


Instead, you can add this instruction to:

https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md





> ---
>  Documentation/kbuild/llvm.rst | 78 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 78 insertions(+)
>
> diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
> index 6dc66b4f31a7..de3bde925793 100644
> --- a/Documentation/kbuild/llvm.rst
> +++ b/Documentation/kbuild/llvm.rst
> @@ -186,6 +186,84 @@ yet. Bug reports are always welcome at the issue tracker below!
>       - Supported
>       - ``LLVM=1``
>
> +Experimental Build in macOS
> +---------------------------
> +
> +Building on macOS with LLVM is experimental. This section provides steps to
> +install dependencies via Homebrew, set up the environment, and start the build
> +process.
> +
> +1. **Create a Case-Sensitive Volume**
> +
> +   For fetching and building the project, you need a case-sensitive volume. Use the following
> +   command to create one:
> +
> +   .. code-block:: shell
> +
> +      diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux
> +
> +   Replace `/dev/disk<N>` with the appropriate disk identifier.
> +
> +2. **Install Build Dependencies**
> +
> +Use Homebrew to install the required build dependencies.
> +
> +- **Core Utilities**: `coreutils`, `findutils`, `gnu-sed`, `gnu-tar`, `grep`,
> +  `llvm`, `make`, and `pkg-config`.
> +
> +   .. code-block:: shell
> +
> +      brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config
> +
> +- **Bee Headers**: Install byteswap, elf and endian headers using the
> +  `Bee Headers Project <https://github.com/bee-headers/headers>`_.
> +
> +   .. code-block:: shell
> +
> +      brew tap bee-headers/bee-headers
> +      brew install bee-headers/bee-headers/bee-headers
> +
> +   After installation, verify the `CFLAGS` with `pkg-config`:
> +
> +   .. code-block:: shell
> +
> +      pkg-config --cflags bee-headers
> +      -I/opt/homebrew/Cellar/bee-headers/0.1/include
> +
> +3. **Configure the PATH**
> +
> +   Include all the required GNU tools and LLVM in your `PATH`. This ensures that
> +   the necessary tools are available during the build process.
> +
> +   .. code-block:: shell
> +
> +      PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH"
> +      PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH"
> +      PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH"
> +      PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH"
> +      PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH"
> +      PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH"
> +      PATH="/opt/homebrew/opt/llvm/bin:$PATH"
> +
> +Building the Project
> +--------------------
> +
> +Once the environment is set up, you can start the build process using LLVM. Run
> +the following commands to initiate the build:
> +
> +.. code-block:: shell
> +
> +   make LLVM=1 allyesconfig
> +   make LLVM=1 -j$(nproc)
> +
> +Supported in macOS
> +~~~~~~~~~~~~~~~~~~
> +
> +At the moment, only arm64 is supported and tested with `allyesconfig` Makefile
> +configuration target. Other Kconfig options not included in `allyesconfig`
> +target and architectures may be supported as well as support in macOS is based
> +on LLVM effort and maintenance.
> +
>  Getting Help
>  ------------
>
>
> --
> 2.46.0
>
>


--
Best Regards
Masahiro Yamada

Daniel Gomez Sept. 7, 2024, 9:32 a.m. UTC | #2

On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <[e-mail]> wrote:
>
> On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay
> <[e-mail]> wrote:
> >
> > From: Daniel Gomez <[e-mail]>
> >
> > Add documentation under kbuild/llvm to inform about the experimental
> > support for building the Linux kernel in macOS hosts environments.
> >
> > Signed-off-by: Daniel Gomez <[e-mail]>
>
>
> Instead, you can add this instruction to:
>
> https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md

Sure, that can be done as well. But the effort here is to have this
integrated. So, I think documentation should be in-tree.

>
>
>
>
>
> > ---
> >  Documentation/kbuild/llvm.rst | 78 +++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 78 insertions(+)
> >
> > diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
> > index 6dc66b4f31a7..de3bde925793 100644
> > --- a/Documentation/kbuild/llvm.rst
> > +++ b/Documentation/kbuild/llvm.rst
> > @@ -186,6 +186,84 @@ yet. Bug reports are always welcome at the issue tracker below!
> >       - Supported
> >       - ``LLVM=1``
> >
> > +Experimental Build in macOS
> > +---------------------------
> > +
> > +Building on macOS with LLVM is experimental. This section provides steps to
> > +install dependencies via Homebrew, set up the environment, and start the build
> > +process.
> > +
> > +1. **Create a Case-Sensitive Volume**
> > +
> > +   For fetching and building the project, you need a case-sensitive volume. Use the following
> > +   command to create one:
> > +
> > +   .. code-block:: shell
> > +
> > +      diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux
> > +
> > +   Replace `/dev/disk<N>` with the appropriate disk identifier.
> > +
> > +2. **Install Build Dependencies**
> > +
> > +Use Homebrew to install the required build dependencies.
> > +
> > +- **Core Utilities**: `coreutils`, `findutils`, `gnu-sed`, `gnu-tar`, `grep`,
> > +  `llvm`, `make`, and `pkg-config`.
> > +
> > +   .. code-block:: shell
> > +
> > +      brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config
> > +
> > +- **Bee Headers**: Install byteswap, elf and endian headers using the
> > +  `Bee Headers Project <https://github.com/bee-headers/headers>`_.
> > +
> > +   .. code-block:: shell
> > +
> > +      brew tap bee-headers/bee-headers
> > +      brew install bee-headers/bee-headers/bee-headers
> > +
> > +   After installation, verify the `CFLAGS` with `pkg-config`:
> > +
> > +   .. code-block:: shell
> > +
> > +      pkg-config --cflags bee-headers
> > +      -I/opt/homebrew/Cellar/bee-headers/0.1/include
> > +
> > +3. **Configure the PATH**
> > +
> > +   Include all the required GNU tools and LLVM in your `PATH`. This ensures that
> > +   the necessary tools are available during the build process.
> > +
> > +   .. code-block:: shell
> > +
> > +      PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH"
> > +      PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH"
> > +      PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH"
> > +      PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH"
> > +      PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH"
> > +      PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH"
> > +      PATH="/opt/homebrew/opt/llvm/bin:$PATH"
> > +
> > +Building the Project
> > +--------------------
> > +
> > +Once the environment is set up, you can start the build process using LLVM. Run
> > +the following commands to initiate the build:
> > +
> > +.. code-block:: shell
> > +
> > +   make LLVM=1 allyesconfig
> > +   make LLVM=1 -j$(nproc)
> > +
> > +Supported in macOS
> > +~~~~~~~~~~~~~~~~~~
> > +
> > +At the moment, only arm64 is supported and tested with `allyesconfig` Makefile
> > +configuration target. Other Kconfig options not included in `allyesconfig`
> > +target and architectures may be supported as well as support in macOS is based
> > +on LLVM effort and maintenance.
> > +
> >  Getting Help
> >  ------------
> >
> >
> > --
> > 2.46.0
> >
> >
>
>
> --
> Best Regards
> Masahiro Yamada

Masahiro Yamada Sept. 8, 2024, 1:29 a.m. UTC | #3

On Sat, Sep 7, 2024 at 6:32 PM Daniel Gomez (Samsung)
<[e-mail]> wrote:
>
> On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <[e-mail]> wrote:
> >
> > On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay
> > <[e-mail]> wrote:
> > >
> > > From: Daniel Gomez <[e-mail]>
> > >
> > > Add documentation under kbuild/llvm to inform about the experimental
> > > support for building the Linux kernel in macOS hosts environments.
> > >
> > > Signed-off-by: Daniel Gomez <[e-mail]>
> >
> >
> > Instead, you can add this instruction to:
> >
> > https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md
>
> Sure, that can be done as well. But the effort here is to have this
> integrated. So, I think documentation should be in-tree.



I do not think so.


Most people are not compile-testing on macOS.

This is an unofficial tip, which you can advertise
somewhere else.

Marc Zyngier Sept. 8, 2024, 9:03 a.m. UTC | #4

On Sat, 07 Sep 2024 10:32:20 +0100,
"Daniel Gomez (Samsung)" <[e-mail]> wrote:
> 
> On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <[e-mail]> wrote:
> >
> > On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay
> > <[e-mail]> wrote:
> > >
> > > From: Daniel Gomez <[e-mail]>
> > >
> > > Add documentation under kbuild/llvm to inform about the experimental
> > > support for building the Linux kernel in macOS hosts environments.
> > >
> > > Signed-off-by: Daniel Gomez <[e-mail]>
> >
> >
> > Instead, you can add this instruction to:
> >
> > https://github.com/bee-headers/homebrew-bee-headers/blob/main/README.md
> 
> Sure, that can be done as well. But the effort here is to have this
> integrated. So, I think documentation should be in-tree.

I think this ship sailed the moment you ended-up with an external
dependency.

Having looked at this series (and in particular patch #4 which falls
under my remit), I can't help but think that the whole thing should
simply live as a wrapper around the pristine build system instead of
hacking things inside of it. You already pull external dependencies
(the include files). Just add a script that sets things up
(environment variables that already exist) and calls 'make' in the
kernel tree.

I also dislike that this is forcing "native" developers to cater for
an operating system they are unlikely to have access to. If I break
this hack tomorrow by adding a new dependency that MacOS doesn't
provide, how do I fix it? Should I drop my changes on the floor?

As an alternative, and since you already have to create a special
file-system to contain your kernel tree, you may as well run Linux in
a VM, which I am told works pretty well (QEMU supports HVF, and there
are plenty of corporate-friendly alternatives). This would solve your
problem once and for all.

Please don't take the above the wrong way. I'm sympathetic to what you
are trying to do. But this is IMO going in the wrong direction.

Thanks,

	M.

Daniel Gomez Sept. 24, 2024, 8:51 a.m. UTC | #5

On 9/8/2024 11:03 AM, Marc Zyngier wrote:
> On Sat, 07 Sep 2024 10:32:20 +0100,
> "Daniel Gomez (Samsung)" <[e-mail]> wrote:
>>
>> On Sat, Sep 7, 2024 at 10:33 AM Masahiro Yamada <[e-mail]> wrote:
>>>
>>> On Fri, Sep 6, 2024 at 8:01 PM Daniel Gomez via B4 Relay
>>> <[e-mail]> wrote:
>>>>
>>>> From: Daniel Gomez <[e-mail]>
>>>>
>>>> Add documentation under kbuild/llvm to inform about the experimental
>>>> support for building the Linux kernel in macOS hosts environments.
>>>>
>>>> Signed-off-by: Daniel Gomez <[e-mail]>
>>>
>>>
>>> Instead, you can add this instruction to:
>>>
>>> https://protect2.fireeye.com/v1/url?k=f33af8a2-92b1ed94-f33b73ed-74fe485cbff1-7d382b34bfd617fc&q=1&e=c7a3e869-d48e-4168-88a9-03cd717797f0&u=https%3A%2F%2Fgithub.com%2Fbee-headers%2Fhomebrew-bee-headers%2Fblob%2Fmain%2FREADME.md
>>
>> Sure, that can be done as well. But the effort here is to have this
>> integrated. So, I think documentation should be in-tree.
> 
> I think this ship sailed the moment you ended-up with an external
> dependency.

The external dependency is not different in Linux hosts. We depend on 
byteswap.h, elf.h and endian.h headers. However, these are provided by 
glibc and musl, and macOS (at least in arm64/Homebrew) does not provide 
any of these.

To fix the dependency with these missing headers, it was suggested in v1 
[1][2] to create a "development kit" for this hosts. And that is what 
Bee Headers [3][4] aims to provide.

[1]https://lore.kernel.org/all/20240807-mottled-stoic-degu-d1e4cb@lindesnes/
[2]https://lore.kernel.org/all/[e-mail]/

[3] https://github.com/bee-headers/headers
[4] https://github.com/bee-headers/homebrew-bee-headers

I don't see any other alternative to provide the missing headers other 
than the suggested. Please let me know if you think other paths can be 
explored and tested.

> 
> Having looked at this series (and in particular patch #4 which falls
> under my remit), I can't help but think that the whole thing should
> simply live as a wrapper around the pristine build system instead of
> hacking things inside of it. You already pull external dependencies
> (the include files). Just add a script that sets things up
> (environment variables that already exist) and calls 'make' in the
> kernel tree.

Agree. This aligns very well with other feedback.

I've added a script to the Bee Headers [5] to help users init their 
shell/Makefile environment.

[5] 
https://github.com/bee-headers/homebrew-bee-headers/blob/main/bee-headers.rb#L28

> 
> I also dislike that this is forcing "native" developers to cater for
> an operating system they are unlikely to have access to. If I break
> this hack tomorrow by adding a new dependency that MacOS doesn't
> provide, how do I fix it? Should I drop my changes on the floor?
> 
> As an alternative, and since you already have to create a special
> file-system to contain your kernel tree, you may as well run Linux in
> a VM, which I am told works pretty well (QEMU supports HVF, and there
> are plenty of corporate-friendly alternatives). This would solve your
> problem once and for all.

This is a use case I'm trying to avoid with this series. QEMU HVF works 
very well indeed but extracting the built objects/vmlinux/*.ko etc is 
tedious and slow. Building natively allows users to boot a VM with QEMU 
and -kernel argument directly. This is way faster than the suggested 
alternative. In addition, lldb can be used to debug the kernel from the 
host.

> 
> Please don't take the above the wrong way. I'm sympathetic to what you
> are trying to do. But this is IMO going in the wrong direction.

First patch has been merged already and after some fixes in the build 
system from Masahiro, this series only requires a small change in 
xe_gen_wa_oob to make this work if users of this provide the missing 
headers and set up the build environment properly.

I will send a v3 later today with the remaining patch.

Thanks Marc for sharing your views!

> 
> Thanks,
> 
> 	M.
>

Patch

diff --git a/Documentation/kbuild/llvm.rst b/Documentation/kbuild/llvm.rst
index 6dc66b4f31a7..de3bde925793 100644
--- a/Documentation/kbuild/llvm.rst
+++ b/Documentation/kbuild/llvm.rst
@@ -186,6 +186,84 @@  yet. Bug reports are always welcome at the issue tracker below!
      - Supported
      - ``LLVM=1``
 
+Experimental Build in macOS
+---------------------------
+
+Building on macOS with LLVM is experimental. This section provides steps to
+install dependencies via Homebrew, set up the environment, and start the build
+process.
+
+1. **Create a Case-Sensitive Volume**
+
+   For fetching and building the project, you need a case-sensitive volume. Use the following
+   command to create one:
+
+   .. code-block:: shell
+
+      diskutil apfs addVolume /dev/disk<N> "Case-sensitive APFS" linux
+
+   Replace `/dev/disk<N>` with the appropriate disk identifier.
+
+2. **Install Build Dependencies**
+
+Use Homebrew to install the required build dependencies.
+
+- **Core Utilities**: `coreutils`, `findutils`, `gnu-sed`, `gnu-tar`, `grep`,
+  `llvm`, `make`, and `pkg-config`.
+
+   .. code-block:: shell
+
+      brew install coreutils findutils gnu-sed gnu-tar grep llvm make pkg-config
+
+- **Bee Headers**: Install byteswap, elf and endian headers using the
+  `Bee Headers Project <https://github.com/bee-headers/headers>`_.
+
+   .. code-block:: shell
+
+      brew tap bee-headers/bee-headers
+      brew install bee-headers/bee-headers/bee-headers
+
+   After installation, verify the `CFLAGS` with `pkg-config`:
+
+   .. code-block:: shell
+
+      pkg-config --cflags bee-headers
+      -I/opt/homebrew/Cellar/bee-headers/0.1/include
+
+3. **Configure the PATH**
+
+   Include all the required GNU tools and LLVM in your `PATH`. This ensures that
+   the necessary tools are available during the build process.
+
+   .. code-block:: shell
+
+      PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH"
+      PATH="/opt/homebrew/opt/findutils/libexec/gnubin:$PATH"
+      PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH"
+      PATH="/opt/homebrew/opt/gnu-tar/libexec/gnubin:$PATH"
+      PATH="/opt/homebrew/opt/grep/libexec/gnubin:$PATH"
+      PATH="/opt/homebrew/opt/make/libexec/gnubin:$PATH"
+      PATH="/opt/homebrew/opt/llvm/bin:$PATH"
+
+Building the Project
+--------------------
+
+Once the environment is set up, you can start the build process using LLVM. Run
+the following commands to initiate the build:
+
+.. code-block:: shell
+
+   make LLVM=1 allyesconfig
+   make LLVM=1 -j$(nproc)
+
+Supported in macOS
+~~~~~~~~~~~~~~~~~~
+
+At the moment, only arm64 is supported and tested with `allyesconfig` Makefile
+configuration target. Other Kconfig options not included in `allyesconfig`
+target and architectures may be supported as well as support in macOS is based
+on LLVM effort and maintenance.
+
 Getting Help
 ------------