[v7] kbuild: add script and target to generate pacman package

Commit Message

Thomas Weißschuh July 20, 2024, 9:18 a.m. UTC

pacman is the package manager used by Arch Linux and its derivates.
Creating native packages from the kernel tree has multiple advantages:

* The package triggers the correct hooks for initramfs generation and
  bootloader configuration
* Uninstallation is complete and also invokes the relevant hooks
* New UAPI headers can be installed without any manual bookkeeping

The PKGBUILD file is a modified version of the one used for the
downstream Arch Linux "linux" package.
Extra steps that should not be necessary for a development kernel have
been removed and an UAPI header package has been added.

Signed-off-by: Thomas Weißschuh <[e-mail]>
---
Changes in v7:
- Make pkgbase configurable
- Add more information to pkgdesc
- Drop base-devel from makedepends
- Drop now unneeded "chmod"
- Make sure KERNELRELEASE and KBUILD_BUILD_VERSION are correct
- Add MAINTAINERS entry
- Use absolute path to BUILDDIR again, as makepkg doesn't work otherwise
- Drop Reviewed-by and Tested-by as a fair amount of changes has been
  done since
- Link to v6: https://lore.kernel.org/r/[e-mail]

Changes in v6:
- Drop reference to srctree/Makefile
- Drop $(realpath $(srctree))
- Make use of the fact that $(objtree) is always "."
- Align coding style to kernel and drop vim config line
- Drop indirection through `$MAKE run-command`
- Unify shell variable syntax to "${var}"
- Add explanations to custom variables
- Add makedepends
- Link to v5: https://lore.kernel.org/r/[e-mail]

Changes in v5:
- Rebase onto kbuild/for-next
- Use new path to build-version script (from kbuild/for-next)
- Ensure submake jobserver delegation works
- Simplify $modulesdir/pkgbase file creation
- Add Reviewed-by from Nicolas
- Link to v4: https://lore.kernel.org/r/[e-mail]

Changes in v4:
- Update MRPROPER_FILES
- Unify shell variable syntax
- Link to v3: https://lore.kernel.org/r/[e-mail]

Changes in v3:
- Enforce matching architectures for installation
- Add Reviewed-by and Tested-by from Nathan
- Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v2:
- Replace ${MAKE} with $MAKE for consistency with other variables
- Use $MAKE for "-s image_name"
- Avoid permission warnings from build directory
- Clarify reason for /build symlink removal
- Install System.map and config
- Install dtbs where available
- Allow cross-build through arch=any
- Sort Contributor/Maintainer chronologically
- Disable some unneeded makepkg options
- Use DEPMOD=true for consistency with rpm-package
- Link to v1: https://lore.kernel.org/r/[e-mail]
---
 .gitignore               |   6 +++
 MAINTAINERS              |   7 +++
 Makefile                 |   2 +-
 scripts/Makefile.package |  14 ++++++
 scripts/package/PKGBUILD | 108 +++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 136 insertions(+), 1 deletion(-)


---
base-commit: 6e6ef2da3a28f3e02fd204b4f8821030b61f8cd4
change-id: 20240625-kbuild-pacman-pkg-b4f87e19d036

Best regards,

Comments

Nathan Chancellor July 21, 2024, 3:32 a.m. UTC | #1

Hi Thomas,

On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> pacman is the package manager used by Arch Linux and its derivates.
> Creating native packages from the kernel tree has multiple advantages:
> 
> * The package triggers the correct hooks for initramfs generation and
>   bootloader configuration
> * Uninstallation is complete and also invokes the relevant hooks
> * New UAPI headers can be installed without any manual bookkeeping
> 
> The PKGBUILD file is a modified version of the one used for the
> downstream Arch Linux "linux" package.
> Extra steps that should not be necessary for a development kernel have
> been removed and an UAPI header package has been added.
> 
> Signed-off-by: Thomas Weißschuh <[e-mail]>
> ---

I think this looks really solid now, thanks again for the PACMAN_PKGBASE
addition.

I tested building Arch Linux's configuration for x86_64 and booting it
in a VM. I built Fedora's configuration for aarch64 just to test the
cross building aspect and making sure the result of various bits that we
added that would not affect x86 (such as the dtb installation) looked
reasonable.

Reviewed-by: Nathan Chancellor <[e-mail]>
Tested-by: Nathan Chancellor <[e-mail]>

> Changes in v7:
> - Make pkgbase configurable
> - Add more information to pkgdesc
> - Drop base-devel from makedepends
> - Drop now unneeded "chmod"
> - Make sure KERNELRELEASE and KBUILD_BUILD_VERSION are correct
> - Add MAINTAINERS entry
> - Use absolute path to BUILDDIR again, as makepkg doesn't work otherwise
> - Drop Reviewed-by and Tested-by as a fair amount of changes has been
>   done since
> - Link to v6: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v6:
> - Drop reference to srctree/Makefile
> - Drop $(realpath $(srctree))
> - Make use of the fact that $(objtree) is always "."
> - Align coding style to kernel and drop vim config line
> - Drop indirection through `$MAKE run-command`
> - Unify shell variable syntax to "${var}"
> - Add explanations to custom variables
> - Add makedepends
> - Link to v5: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v5:
> - Rebase onto kbuild/for-next
> - Use new path to build-version script (from kbuild/for-next)
> - Ensure submake jobserver delegation works
> - Simplify $modulesdir/pkgbase file creation
> - Add Reviewed-by from Nicolas
> - Link to v4: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v4:
> - Update MRPROPER_FILES
> - Unify shell variable syntax
> - Link to v3: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v3:
> - Enforce matching architectures for installation
> - Add Reviewed-by and Tested-by from Nathan
> - Link to v2: https://lore.kernel.org/r/[e-mail]
> 
> Changes in v2:
> - Replace ${MAKE} with $MAKE for consistency with other variables
> - Use $MAKE for "-s image_name"
> - Avoid permission warnings from build directory
> - Clarify reason for /build symlink removal
> - Install System.map and config
> - Install dtbs where available
> - Allow cross-build through arch=any
> - Sort Contributor/Maintainer chronologically
> - Disable some unneeded makepkg options
> - Use DEPMOD=true for consistency with rpm-package
> - Link to v1: https://lore.kernel.org/r/[e-mail]
> ---
>  .gitignore               |   6 +++
>  MAINTAINERS              |   7 +++
>  Makefile                 |   2 +-
>  scripts/Makefile.package |  14 ++++++
>  scripts/package/PKGBUILD | 108 +++++++++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 136 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitignore b/.gitignore
> index c59dc60ba62e..7902adf4f7f1 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -92,6 +92,12 @@ modules.order
>  #
>  /tar-install/
>  
> +#
> +# pacman files (make pacman-pkg)
> +#
> +/PKGBUILD
> +/pacman/
> +
>  #
>  # We don't want to ignore the following even if they are dot-files
>  #
> diff --git a/MAINTAINERS b/MAINTAINERS
> index da5352dbd4f3..16f8e13aa4c6 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -11997,6 +11997,13 @@ F:	include/uapi/linux/nfsd/
>  F:	include/uapi/linux/sunrpc/
>  F:	net/sunrpc/
>  
> +KERNEL PACMAN PACKAGING (in addition to generic KERNEL BUILD)
> +M:	Thomas Weißschuh <[e-mail]>
> +R:	Christian Heusel <[e-mail]>
> +R:	Nathan Chancellor <[e-mail]>
> +S:	Maintained
> +F:	scripts/package/PKGBUILD
> +
>  KERNEL REGRESSIONS
>  M:	Thorsten Leemhuis <[e-mail]>
>  L:	[e-mail]
> diff --git a/Makefile b/Makefile
> index c97d6404b891..943899656977 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1487,7 +1487,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
>  # Directories & files removed with 'make mrproper'
>  MRPROPER_FILES += include/config include/generated          \
>  		  arch/$(SRCARCH)/include/generated .objdiff \
> -		  debian snap tar-install \
> +		  debian snap tar-install PKGBUILD pacman \
>  		  .config .config.old .version \
>  		  Module.symvers \
>  		  certs/signing_key.pem \
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index bf016af8bf8a..94357f47d2fa 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -141,6 +141,19 @@ snap-pkg:
>  	cd $(objtree)/snap && \
>  	snapcraft --target-arch=$(UTS_MACHINE)
>  
> +# pacman-pkg
> +# ---------------------------------------------------------------------------
> +
> +PHONY += pacman-pkg
> +pacman-pkg:
> +	@ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> +	+objtree="$(realpath $(objtree))" \
> +		BUILDDIR="$(realpath pacman)" \
> +		CARCH="$(UTS_MACHINE)" \
> +		KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> +		KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> +		makepkg
> +
>  # dir-pkg tar*-pkg - tarball targets
>  # ---------------------------------------------------------------------------
>  
> @@ -221,6 +234,7 @@ help:
>  	@echo '  bindeb-pkg          - Build only the binary kernel deb package'
>  	@echo '  snap-pkg            - Build only the binary kernel snap package'
>  	@echo '                        (will connect to external hosts)'
> +	@echo '  pacman-pkg          - Build only the binary kernel pacman package'
>  	@echo '  dir-pkg             - Build the kernel as a plain directory structure'
>  	@echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
>  	@echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
> diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
> new file mode 100644
> index 000000000000..663ce300dd06
> --- /dev/null
> +++ b/scripts/package/PKGBUILD
> @@ -0,0 +1,108 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +# Maintainer: Thomas Weißschuh <[e-mail]>
> +# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
> +
> +pkgbase=${PACMAN_PKGBASE:-linux-upstream}
> +pkgname=("${pkgbase}" "${pkgbase}-api-headers")
> +if grep -q CONFIG_MODULES=y include/config/auto.conf; then
> +	pkgname+=("${pkgbase}-headers")
> +fi
> +pkgver="${KERNELRELEASE//-/_}"
> +# The PKGBUILD is evaluated multiple times.
> +# Running scripts/build-version from here would introduce inconsistencies.
> +pkgrel="${KBUILD_REVISION}"
> +pkgdesc='Upstream Linux'
> +url='https://www.kernel.org/'
> +# Enable flexible cross-compilation
> +arch=(${CARCH})
> +license=(GPL-2.0-only)
> +makedepends=(
> +	bc
> +	bison
> +	cpio
> +	flex
> +	gettext
> +	kmod
> +	libelf
> +	openssl
> +	pahole
> +	perl
> +	python
> +	rsync
> +	tar
> +)
> +options=(!debug !strip !buildflags !makeflags)
> +
> +build() {
> +	# MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
> +	# Bypass this override with a custom variable.
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +
> +	${MAKE} KERNELRELEASE="${KERNELRELEASE}" KBUILD_BUILD_VERSION="${pkgrel}"
> +}
> +
> +_package() {
> +	pkgdesc="The ${pkgdesc} kernel and modules"
> +
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +	local modulesdir="${pkgdir}/usr/${MODLIB}"
> +
> +	echo "Installing boot image..."
> +	# systemd expects to find the kernel here to allow hibernation
> +	# https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
> +	install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
> +
> +	# Used by mkinitcpio to name the kernel
> +	echo "${pkgbase}" > "${modulesdir}/pkgbase"
> +
> +	echo "Installing modules..."
> +	${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
> +		DEPMOD=true modules_install
> +
> +	if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
> +		echo "Installing dtbs..."
> +		${MAKE} INSTALL_DTBS_PATH="${modulesdir}/dtb" dtbs_install
> +	fi
> +
> +	# remove build link, will be part of -headers package
> +	rm -f "${modulesdir}/build"
> +}
> +
> +_package-headers() {
> +	pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel"
> +
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +	local builddir="${pkgdir}/usr/${MODLIB}/build"
> +
> +	echo "Installing build files..."
> +	"${srctree}/scripts/package/install-extmod-build" "${builddir}"
> +
> +	echo "Installing System.map and config..."
> +	cp System.map "${builddir}/System.map"
> +	cp .config "${builddir}/.config"
> +
> +	echo "Adding symlink..."
> +	mkdir -p "${pkgdir}/usr/src"
> +	ln -sr "${builddir}" "${pkgdir}/usr/src/${pkgbase}"
> +}
> +
> +_package-api-headers() {
> +	pkgdesc="Kernel headers sanitized for use in userspace"
> +	provides=(linux-api-headers)
> +	conflicts=(linux-api-headers)
> +
> +	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
> +	cd "${objtree}"
> +
> +	${MAKE} headers_install INSTALL_HDR_PATH="${pkgdir}/usr"
> +}
> +
> +for _p in "${pkgname[@]}"; do
> +	eval "package_$_p() {
> +		$(declare -f "_package${_p#$pkgbase}")
> +		_package${_p#$pkgbase}
> +	}"
> +done
> 
> ---
> base-commit: 6e6ef2da3a28f3e02fd204b4f8821030b61f8cd4
> change-id: 20240625-kbuild-pacman-pkg-b4f87e19d036
> 
> Best regards,
> -- 
> Thomas Weißschuh <[e-mail]>
>

Masahiro Yamada July 21, 2024, 6:58 a.m. UTC | #2

On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:
>
> Hi Thomas,
>
> On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> > pacman is the package manager used by Arch Linux and its derivates.
> > Creating native packages from the kernel tree has multiple advantages:
> >
> > * The package triggers the correct hooks for initramfs generation and
> >   bootloader configuration
> > * Uninstallation is complete and also invokes the relevant hooks
> > * New UAPI headers can be installed without any manual bookkeeping
> >
> > The PKGBUILD file is a modified version of the one used for the
> > downstream Arch Linux "linux" package.
> > Extra steps that should not be necessary for a development kernel have
> > been removed and an UAPI header package has been added.
> >
> > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > ---
>
> I think this looks really solid now, thanks again for the PACMAN_PKGBASE
> addition.
>
> I tested building Arch Linux's configuration for x86_64 and booting it
> in a VM. I built Fedora's configuration for aarch64 just to test the
> cross building aspect and making sure the result of various bits that we
> added that would not affect x86 (such as the dtb installation) looked
> reasonable.
>
> Reviewed-by: Nathan Chancellor <[e-mail]>
> Tested-by: Nathan Chancellor <[e-mail]>


I believe this is a separate issue, but
Debian/Ubuntu provides a 'makepkg' package, which fails
with 'User defined signal 1' error.


After 'sudo apt install makepkg',


masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
/etc/os-release
PRETTY_NAME="Ubuntu 24.04 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
makepkg --version
makepkg (pacman) 6.0.2
Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.

This is free software; see the source for copying conditions.
There is NO WARRANTY, to the extent permitted by law.
masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
log --oneline -2
5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
linux-next specific files for 20240719
masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
make defconfig  pacman-pkg
*** Default configuration is based on 'x86_64_defconfig'
#
# No change to .config
#
objtree="/home/masahiro/ref/linux-next" \
BUILDDIR="" \
CARCH="x86_64" \
KBUILD_MAKEFLAGS="rR --no-print-directory" \
KBUILD_REVISION="4" \
makepkg

==> ERROR: An unknown error has occurred. Exiting...
User defined signal 1
make[3]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138
make[2]: *** [Makefile:1538: pacman-pkg] Error 2
make[1]: *** [/home/masahiro/ref/linux-next/Makefile:347:
__build_one_by_one] Error 2
make: *** [Makefile:224: __sub-make] Error 2



Do you know anything?

Nathan Chancellor July 21, 2024, 12:11 p.m. UTC | #3

Hi Masahiro,

On Sun, Jul 21, 2024 at 03:58:49PM +0900, Masahiro Yamada wrote:
> On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:
> > On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> > > pacman is the package manager used by Arch Linux and its derivates.
> > > Creating native packages from the kernel tree has multiple advantages:
> > >
> > > * The package triggers the correct hooks for initramfs generation and
> > >   bootloader configuration
> > > * Uninstallation is complete and also invokes the relevant hooks
> > > * New UAPI headers can be installed without any manual bookkeeping
> > >
> > > The PKGBUILD file is a modified version of the one used for the
> > > downstream Arch Linux "linux" package.
> > > Extra steps that should not be necessary for a development kernel have
> > > been removed and an UAPI header package has been added.
> > >
> > > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > > ---
> >
> > I think this looks really solid now, thanks again for the PACMAN_PKGBASE
> > addition.
> >
> > I tested building Arch Linux's configuration for x86_64 and booting it
> > in a VM. I built Fedora's configuration for aarch64 just to test the
> > cross building aspect and making sure the result of various bits that we
> > added that would not affect x86 (such as the dtb installation) looked
> > reasonable.
> >
> > Reviewed-by: Nathan Chancellor <[e-mail]>
> > Tested-by: Nathan Chancellor <[e-mail]>
> 
> I believe this is a separate issue, but
> Debian/Ubuntu provides a 'makepkg' package, which fails
> with 'User defined signal 1' error.
> 
> After 'sudo apt install makepkg',
> 
> masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
> /etc/os-release
> PRETTY_NAME="Ubuntu 24.04 LTS"
> NAME="Ubuntu"
> VERSION_ID="24.04"
> VERSION="24.04 LTS (Noble Numbat)"
> VERSION_CODENAME=noble
> ID=ubuntu
> ID_LIKE=debian
> HOME_URL="https://www.ubuntu.com/"
> SUPPORT_URL="https://help.ubuntu.com/"
> BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
> PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
> UBUNTU_CODENAME=noble
> LOGO=ubuntu-logo
> masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> makepkg --version
> makepkg (pacman) 6.0.2
> Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
> Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.
> 
> This is free software; see the source for copying conditions.
> There is NO WARRANTY, to the extent permitted by law.
> masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
> log --oneline -2
> 5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
> 41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
> linux-next specific files for 20240719
> masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> make defconfig  pacman-pkg
> *** Default configuration is based on 'x86_64_defconfig'
> #
> # No change to .config
> #
> objtree="/home/masahiro/ref/linux-next" \
> BUILDDIR="" \

It is not related to this issue but I don't think this should be empty.
'realpath pacman' does not appear to work here, I was able to fix this
with the following diff:

diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index 94357f47d2fa..b0fd44a40075 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -148,7 +148,7 @@ PHONY += pacman-pkg
 pacman-pkg:
 	@ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
 	+objtree="$(realpath $(objtree))" \
-		BUILDDIR="$(realpath pacman)" \
+		BUILDDIR="$(realpath $(objtree))/pacman" \
 		CARCH="$(UTS_MACHINE)" \
 		KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
 		KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \


> CARCH="x86_64" \
> KBUILD_MAKEFLAGS="rR --no-print-directory" \
> KBUILD_REVISION="4" \
> makepkg
> 
> ==> ERROR: An unknown error has occurred. Exiting...
> User defined signal 1
> make[3]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138
> make[2]: *** [Makefile:1538: pacman-pkg] Error 2
> make[1]: *** [/home/masahiro/ref/linux-next/Makefile:347:
> __build_one_by_one] Error 2
> make: *** [Makefile:224: __sub-make] Error 2
> 
> Do you know anything?

Adding '-x' to the interpreter line in /usr/bin/makepkg shows that
pacman is implicitly required for makepkg to function, so you'll need to
install pacman-package-manager as well. However, even with it installed,
the build will fail because you will be unable to perform the dependency
checking (since pacman won't be managing the packages).

I think the traditional solution for this situation (building a
distribution's package on a distribution other than the one the package
is being built for) is using an OPTS variable to allow the user to pass
in the dependency checking skip flag (that's what I do for RPM builds on
Arch Linux), so perhaps MAKEPKGOPTS so that you can use MAKEPKGOPTS=-d?
With this diff and that variable value, the build starts within makepkg
for me.

diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index b0fd44a40075..4a80584ec771 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -152,7 +152,7 @@ pacman-pkg:
 		CARCH="$(UTS_MACHINE)" \
 		KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
 		KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
-		makepkg
+		makepkg $(MAKEPKGOPTS)
 
 # dir-pkg tar*-pkg - tarball targets
 # ---------------------------------------------------------------------------

Cheers,
Nathan

Masahiro Yamada July 21, 2024, 1:42 p.m. UTC | #4

On Sun, Jul 21, 2024 at 9:11 PM Nathan Chancellor <[e-mail]> wrote:
>
> Hi Masahiro,
>
> On Sun, Jul 21, 2024 at 03:58:49PM +0900, Masahiro Yamada wrote:
> > On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:
> > > On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> > > > pacman is the package manager used by Arch Linux and its derivates.
> > > > Creating native packages from the kernel tree has multiple advantages:
> > > >
> > > > * The package triggers the correct hooks for initramfs generation and
> > > >   bootloader configuration
> > > > * Uninstallation is complete and also invokes the relevant hooks
> > > > * New UAPI headers can be installed without any manual bookkeeping
> > > >
> > > > The PKGBUILD file is a modified version of the one used for the
> > > > downstream Arch Linux "linux" package.
> > > > Extra steps that should not be necessary for a development kernel have
> > > > been removed and an UAPI header package has been added.
> > > >
> > > > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > > > ---
> > >
> > > I think this looks really solid now, thanks again for the PACMAN_PKGBASE
> > > addition.
> > >
> > > I tested building Arch Linux's configuration for x86_64 and booting it
> > > in a VM. I built Fedora's configuration for aarch64 just to test the
> > > cross building aspect and making sure the result of various bits that we
> > > added that would not affect x86 (such as the dtb installation) looked
> > > reasonable.
> > >
> > > Reviewed-by: Nathan Chancellor <[e-mail]>
> > > Tested-by: Nathan Chancellor <[e-mail]>
> >
> > I believe this is a separate issue, but
> > Debian/Ubuntu provides a 'makepkg' package, which fails
> > with 'User defined signal 1' error.
> >
> > After 'sudo apt install makepkg',
> >
> > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
> > /etc/os-release
> > PRETTY_NAME="Ubuntu 24.04 LTS"
> > NAME="Ubuntu"
> > VERSION_ID="24.04"
> > VERSION="24.04 LTS (Noble Numbat)"
> > VERSION_CODENAME=noble
> > ID=ubuntu
> > ID_LIKE=debian
> > HOME_URL="https://www.ubuntu.com/"
> > SUPPORT_URL="https://help.ubuntu.com/"
> > BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
> > PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
> > UBUNTU_CODENAME=noble
> > LOGO=ubuntu-logo
> > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > makepkg --version
> > makepkg (pacman) 6.0.2
> > Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
> > Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.
> >
> > This is free software; see the source for copying conditions.
> > There is NO WARRANTY, to the extent permitted by law.
> > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
> > log --oneline -2
> > 5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
> > 41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
> > linux-next specific files for 20240719
> > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > make defconfig  pacman-pkg
> > *** Default configuration is based on 'x86_64_defconfig'
> > #
> > # No change to .config
> > #
> > objtree="/home/masahiro/ref/linux-next" \
> > BUILDDIR="" \
>
> It is not related to this issue but I don't think this should be empty.
> 'realpath pacman' does not appear to work here, I was able to fix this
> with the following diff:
>
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index 94357f47d2fa..b0fd44a40075 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -148,7 +148,7 @@ PHONY += pacman-pkg
>  pacman-pkg:
>         @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
>         +objtree="$(realpath $(objtree))" \
> -               BUILDDIR="$(realpath pacman)" \
> +               BUILDDIR="$(realpath $(objtree))/pacman" \


Right.

$(realpath pacman) expands to empty
if 'pacman' does not exist yet.
Your fix is correct.








>                 CARCH="$(UTS_MACHINE)" \
>                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
>                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
>
>
> > CARCH="x86_64" \
> > KBUILD_MAKEFLAGS="rR --no-print-directory" \
> > KBUILD_REVISION="4" \
> > makepkg
> >
> > ==> ERROR: An unknown error has occurred. Exiting...
> > User defined signal 1
> > make[3]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138
> > make[2]: *** [Makefile:1538: pacman-pkg] Error 2
> > make[1]: *** [/home/masahiro/ref/linux-next/Makefile:347:
> > __build_one_by_one] Error 2
> > make: *** [Makefile:224: __sub-make] Error 2
> >
> > Do you know anything?
>
> Adding '-x' to the interpreter line in /usr/bin/makepkg shows that
> pacman is implicitly required for makepkg to function, so you'll need to
> install pacman-package-manager as well. However, even with it installed,
> the build will fail because you will be unable to perform the dependency
> checking (since pacman won't be managing the packages).
>
> I think the traditional solution for this situation (building a
> distribution's package on a distribution other than the one the package
> is being built for) is using an OPTS variable to allow the user to pass
> in the dependency checking skip flag (that's what I do for RPM builds on
> Arch Linux), so perhaps MAKEPKGOPTS so that you can use MAKEPKGOPTS=-d?
> With this diff and that variable value, the build starts within makepkg
> for me.
>
> diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> index b0fd44a40075..4a80584ec771 100644
> --- a/scripts/Makefile.package
> +++ b/scripts/Makefile.package
> @@ -152,7 +152,7 @@ pacman-pkg:
>                 CARCH="$(UTS_MACHINE)" \
>                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
>                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> -               makepkg
> +               makepkg $(MAKEPKGOPTS)


Thank you for delving into this!

With -d option, I was able to compile it.

I previously compile-tested it in an Arch Linux Docker container,
but this is more convenient.


Thomas, will you send v8 with Nathan's suggestion?

Or, I can offer to fold the diff if Thomas agrees.

Thomas Weißschuh July 21, 2024, 2:57 p.m. UTC | #5

On 2024-07-21 22:42:12+0000, Masahiro Yamada wrote:
> On Sun, Jul 21, 2024 at 9:11 PM Nathan Chancellor <[e-mail]> wrote:
> >
> > Hi Masahiro,
> >
> > On Sun, Jul 21, 2024 at 03:58:49PM +0900, Masahiro Yamada wrote:
> > > On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:
> > > > On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> > > > > pacman is the package manager used by Arch Linux and its derivates.
> > > > > Creating native packages from the kernel tree has multiple advantages:
> > > > >
> > > > > * The package triggers the correct hooks for initramfs generation and
> > > > >   bootloader configuration
> > > > > * Uninstallation is complete and also invokes the relevant hooks
> > > > > * New UAPI headers can be installed without any manual bookkeeping
> > > > >
> > > > > The PKGBUILD file is a modified version of the one used for the
> > > > > downstream Arch Linux "linux" package.
> > > > > Extra steps that should not be necessary for a development kernel have
> > > > > been removed and an UAPI header package has been added.
> > > > >
> > > > > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > > > > ---
> > > >
> > > > I think this looks really solid now, thanks again for the PACMAN_PKGBASE
> > > > addition.
> > > >
> > > > I tested building Arch Linux's configuration for x86_64 and booting it
> > > > in a VM. I built Fedora's configuration for aarch64 just to test the
> > > > cross building aspect and making sure the result of various bits that we
> > > > added that would not affect x86 (such as the dtb installation) looked
> > > > reasonable.
> > > >
> > > > Reviewed-by: Nathan Chancellor <[e-mail]>
> > > > Tested-by: Nathan Chancellor <[e-mail]>
> > >
> > > I believe this is a separate issue, but
> > > Debian/Ubuntu provides a 'makepkg' package, which fails
> > > with 'User defined signal 1' error.
> > >
> > > After 'sudo apt install makepkg',
> > >
> > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
> > > /etc/os-release
> > > PRETTY_NAME="Ubuntu 24.04 LTS"
> > > NAME="Ubuntu"
> > > VERSION_ID="24.04"
> > > VERSION="24.04 LTS (Noble Numbat)"
> > > VERSION_CODENAME=noble
> > > ID=ubuntu
> > > ID_LIKE=debian
> > > HOME_URL="https://www.ubuntu.com/"
> > > SUPPORT_URL="https://help.ubuntu.com/"
> > > BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
> > > PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
> > > UBUNTU_CODENAME=noble
> > > LOGO=ubuntu-logo
> > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > makepkg --version
> > > makepkg (pacman) 6.0.2
> > > Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
> > > Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.
> > >
> > > This is free software; see the source for copying conditions.
> > > There is NO WARRANTY, to the extent permitted by law.
> > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
> > > log --oneline -2
> > > 5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
> > > 41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
> > > linux-next specific files for 20240719
> > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > make defconfig  pacman-pkg
> > > *** Default configuration is based on 'x86_64_defconfig'
> > > #
> > > # No change to .config
> > > #
> > > objtree="/home/masahiro/ref/linux-next" \
> > > BUILDDIR="" \
> >
> > It is not related to this issue but I don't think this should be empty.
> > 'realpath pacman' does not appear to work here, I was able to fix this
> > with the following diff:
> >
> > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > index 94357f47d2fa..b0fd44a40075 100644
> > --- a/scripts/Makefile.package
> > +++ b/scripts/Makefile.package
> > @@ -148,7 +148,7 @@ PHONY += pacman-pkg
> >  pacman-pkg:
> >         @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> >         +objtree="$(realpath $(objtree))" \
> > -               BUILDDIR="$(realpath pacman)" \
> > +               BUILDDIR="$(realpath $(objtree))/pacman" \
> 
> 
> Right.
> 
> $(realpath pacman) expands to empty
> if 'pacman' does not exist yet.
> Your fix is correct.

I'm wondering how it worked for me, as I specifically tested this case.
But I'm fine with the proposal.

Shouldn't it be "$(realpath ./pacman)" though?

> 
> >                 CARCH="$(UTS_MACHINE)" \
> >                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> >                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> >
> >
> > > CARCH="x86_64" \
> > > KBUILD_MAKEFLAGS="rR --no-print-directory" \
> > > KBUILD_REVISION="4" \
> > > makepkg
> > >
> > > ==> ERROR: An unknown error has occurred. Exiting...
> > > User defined signal 1
> > > make[3]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138
> > > make[2]: *** [Makefile:1538: pacman-pkg] Error 2
> > > make[1]: *** [/home/masahiro/ref/linux-next/Makefile:347:
> > > __build_one_by_one] Error 2
> > > make: *** [Makefile:224: __sub-make] Error 2
> > >
> > > Do you know anything?
> >
> > Adding '-x' to the interpreter line in /usr/bin/makepkg shows that
> > pacman is implicitly required for makepkg to function, so you'll need to
> > install pacman-package-manager as well. However, even with it installed,
> > the build will fail because you will be unable to perform the dependency
> > checking (since pacman won't be managing the packages).

It seems very weird that there is no dependency, not even a soft one,
from makepkg to pacman.

> > I think the traditional solution for this situation (building a
> > distribution's package on a distribution other than the one the package
> > is being built for) is using an OPTS variable to allow the user to pass
> > in the dependency checking skip flag (that's what I do for RPM builds on
> > Arch Linux), so perhaps MAKEPKGOPTS so that you can use MAKEPKGOPTS=-d?
> > With this diff and that variable value, the build starts within makepkg
> > for me.
> >
> > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > index b0fd44a40075..4a80584ec771 100644
> > --- a/scripts/Makefile.package
> > +++ b/scripts/Makefile.package
> > @@ -152,7 +152,7 @@ pacman-pkg:
> >                 CARCH="$(UTS_MACHINE)" \
> >                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> >                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > -               makepkg
> > +               makepkg $(MAKEPKGOPTS)
> 
> 
> Thank you for delving into this!

Thanks to both of you!

> With -d option, I was able to compile it.
> 
> I previously compile-tested it in an Arch Linux Docker container,
> but this is more convenient.
> 
> 
> Thomas, will you send v8 with Nathan's suggestion?
> 
> Or, I can offer to fold the diff if Thomas agrees.

If you are fine with folding the diff, please do!


Thanks,
Thomas

Masahiro Yamada July 21, 2024, 4:02 p.m. UTC | #6

On Sun, Jul 21, 2024 at 11:57 PM Thomas Weißschuh <[e-mail]> wrote:
>
> On 2024-07-21 22:42:12+0000, Masahiro Yamada wrote:
> > On Sun, Jul 21, 2024 at 9:11 PM Nathan Chancellor <[e-mail]> wrote:
> > >
> > > Hi Masahiro,
> > >
> > > On Sun, Jul 21, 2024 at 03:58:49PM +0900, Masahiro Yamada wrote:
> > > > On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:
> > > > > On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> > > > > > pacman is the package manager used by Arch Linux and its derivates.
> > > > > > Creating native packages from the kernel tree has multiple advantages:
> > > > > >
> > > > > > * The package triggers the correct hooks for initramfs generation and
> > > > > >   bootloader configuration
> > > > > > * Uninstallation is complete and also invokes the relevant hooks
> > > > > > * New UAPI headers can be installed without any manual bookkeeping
> > > > > >
> > > > > > The PKGBUILD file is a modified version of the one used for the
> > > > > > downstream Arch Linux "linux" package.
> > > > > > Extra steps that should not be necessary for a development kernel have
> > > > > > been removed and an UAPI header package has been added.
> > > > > >
> > > > > > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > > > > > ---
> > > > >
> > > > > I think this looks really solid now, thanks again for the PACMAN_PKGBASE
> > > > > addition.
> > > > >
> > > > > I tested building Arch Linux's configuration for x86_64 and booting it
> > > > > in a VM. I built Fedora's configuration for aarch64 just to test the
> > > > > cross building aspect and making sure the result of various bits that we
> > > > > added that would not affect x86 (such as the dtb installation) looked
> > > > > reasonable.
> > > > >
> > > > > Reviewed-by: Nathan Chancellor <[e-mail]>
> > > > > Tested-by: Nathan Chancellor <[e-mail]>
> > > >
> > > > I believe this is a separate issue, but
> > > > Debian/Ubuntu provides a 'makepkg' package, which fails
> > > > with 'User defined signal 1' error.
> > > >
> > > > After 'sudo apt install makepkg',
> > > >
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
> > > > /etc/os-release
> > > > PRETTY_NAME="Ubuntu 24.04 LTS"
> > > > NAME="Ubuntu"
> > > > VERSION_ID="24.04"
> > > > VERSION="24.04 LTS (Noble Numbat)"
> > > > VERSION_CODENAME=noble
> > > > ID=ubuntu
> > > > ID_LIKE=debian
> > > > HOME_URL="https://www.ubuntu.com/"
> > > > SUPPORT_URL="https://help.ubuntu.com/"
> > > > BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
> > > > PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
> > > > UBUNTU_CODENAME=noble
> > > > LOGO=ubuntu-logo
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > > makepkg --version
> > > > makepkg (pacman) 6.0.2
> > > > Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
> > > > Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.
> > > >
> > > > This is free software; see the source for copying conditions.
> > > > There is NO WARRANTY, to the extent permitted by law.
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
> > > > log --oneline -2
> > > > 5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
> > > > 41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
> > > > linux-next specific files for 20240719
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > > make defconfig  pacman-pkg
> > > > *** Default configuration is based on 'x86_64_defconfig'
> > > > #
> > > > # No change to .config
> > > > #
> > > > objtree="/home/masahiro/ref/linux-next" \
> > > > BUILDDIR="" \
> > >
> > > It is not related to this issue but I don't think this should be empty.
> > > 'realpath pacman' does not appear to work here, I was able to fix this
> > > with the following diff:
> > >
> > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > index 94357f47d2fa..b0fd44a40075 100644
> > > --- a/scripts/Makefile.package
> > > +++ b/scripts/Makefile.package
> > > @@ -148,7 +148,7 @@ PHONY += pacman-pkg
> > >  pacman-pkg:
> > >         @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > >         +objtree="$(realpath $(objtree))" \
> > > -               BUILDDIR="$(realpath pacman)" \
> > > +               BUILDDIR="$(realpath $(objtree))/pacman" \
> >
> >
> > Right.
> >
> > $(realpath pacman) expands to empty
> > if 'pacman' does not exist yet.
> > Your fix is correct.
>
> I'm wondering how it worked for me, as I specifically tested this case.
> But I'm fine with the proposal.
>
> Shouldn't it be "$(realpath ./pacman)" though?


No.

Aa I said, $(realpath ./pacman) expands to an empty string
if ./pacman does not exist.


masahiro@zoe:~$ echo '$(info $(realpath ./pacman))' | make -f -

make: *** No targets.  Stop.
masahiro@zoe:~$ mkdir pacman
masahiro@zoe:~$ echo '$(info $(realpath ./pacman))' | make -f -
/home/masahiro/pacman
make: *** No targets.  Stop.








>
> >
> > >                 CARCH="$(UTS_MACHINE)" \
> > >                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > >                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > >
> > >
> > > > CARCH="x86_64" \
> > > > KBUILD_MAKEFLAGS="rR --no-print-directory" \
> > > > KBUILD_REVISION="4" \
> > > > makepkg
> > > >
> > > > ==> ERROR: An unknown error has occurred. Exiting...
> > > > User defined signal 1
> > > > make[3]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138
> > > > make[2]: *** [Makefile:1538: pacman-pkg] Error 2
> > > > make[1]: *** [/home/masahiro/ref/linux-next/Makefile:347:
> > > > __build_one_by_one] Error 2
> > > > make: *** [Makefile:224: __sub-make] Error 2
> > > >
> > > > Do you know anything?
> > >
> > > Adding '-x' to the interpreter line in /usr/bin/makepkg shows that
> > > pacman is implicitly required for makepkg to function, so you'll need to
> > > install pacman-package-manager as well. However, even with it installed,
> > > the build will fail because you will be unable to perform the dependency
> > > checking (since pacman won't be managing the packages).
>
> It seems very weird that there is no dependency, not even a soft one,
> from makepkg to pacman.
>
> > > I think the traditional solution for this situation (building a
> > > distribution's package on a distribution other than the one the package
> > > is being built for) is using an OPTS variable to allow the user to pass
> > > in the dependency checking skip flag (that's what I do for RPM builds on
> > > Arch Linux), so perhaps MAKEPKGOPTS so that you can use MAKEPKGOPTS=-d?
> > > With this diff and that variable value, the build starts within makepkg
> > > for me.
> > >
> > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > index b0fd44a40075..4a80584ec771 100644
> > > --- a/scripts/Makefile.package
> > > +++ b/scripts/Makefile.package
> > > @@ -152,7 +152,7 @@ pacman-pkg:
> > >                 CARCH="$(UTS_MACHINE)" \
> > >                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > >                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > > -               makepkg
> > > +               makepkg $(MAKEPKGOPTS)
> >
> >
> > Thank you for delving into this!
>
> Thanks to both of you!
>
> > With -d option, I was able to compile it.
> >
> > I previously compile-tested it in an Arch Linux Docker container,
> > but this is more convenient.
> >
> >
> > Thomas, will you send v8 with Nathan's suggestion?
> >
> > Or, I can offer to fold the diff if Thomas agrees.
>
> If you are fine with folding the diff, please do!
>
>
> Thanks,
> Thomas
>

Thomas Weißschuh July 21, 2024, 5:21 p.m. UTC | #7

On 2024-07-22 01:02:43+0000, Masahiro Yamada wrote:
> On Sun, Jul 21, 2024 at 11:57 PM Thomas Weißschuh <[e-mail]> wrote:
> > On 2024-07-21 22:42:12+0000, Masahiro Yamada wrote:
> > > On Sun, Jul 21, 2024 at 9:11 PM Nathan Chancellor <[e-mail]> wrote:
> > > > On Sun, Jul 21, 2024 at 03:58:49PM +0900, Masahiro Yamada wrote:
> > > > > On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:

<snip>

> > > > > I believe this is a separate issue, but
> > > > > Debian/Ubuntu provides a 'makepkg' package, which fails
> > > > > with 'User defined signal 1' error.
> > > > >
> > > > > After 'sudo apt install makepkg',
> > > > >
> > > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
> > > > > /etc/os-release
> > > > > PRETTY_NAME="Ubuntu 24.04 LTS"
> > > > > NAME="Ubuntu"
> > > > > VERSION_ID="24.04"
> > > > > VERSION="24.04 LTS (Noble Numbat)"
> > > > > VERSION_CODENAME=noble
> > > > > ID=ubuntu
> > > > > ID_LIKE=debian
> > > > > HOME_URL="https://www.ubuntu.com/"
> > > > > SUPPORT_URL="https://help.ubuntu.com/"
> > > > > BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
> > > > > PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
> > > > > UBUNTU_CODENAME=noble
> > > > > LOGO=ubuntu-logo
> > > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > > > makepkg --version
> > > > > makepkg (pacman) 6.0.2
> > > > > Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
> > > > > Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.
> > > > >
> > > > > This is free software; see the source for copying conditions.
> > > > > There is NO WARRANTY, to the extent permitted by law.
> > > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
> > > > > log --oneline -2
> > > > > 5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
> > > > > 41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
> > > > > linux-next specific files for 20240719
> > > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > > > make defconfig  pacman-pkg
> > > > > *** Default configuration is based on 'x86_64_defconfig'
> > > > > #
> > > > > # No change to .config
> > > > > #
> > > > > objtree="/home/masahiro/ref/linux-next" \
> > > > > BUILDDIR="" \
> > > >
> > > > It is not related to this issue but I don't think this should be empty.
> > > > 'realpath pacman' does not appear to work here, I was able to fix this
> > > > with the following diff:
> > > >
> > > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > > index 94357f47d2fa..b0fd44a40075 100644
> > > > --- a/scripts/Makefile.package
> > > > +++ b/scripts/Makefile.package
> > > > @@ -148,7 +148,7 @@ PHONY += pacman-pkg
> > > >  pacman-pkg:
> > > >         @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > > >         +objtree="$(realpath $(objtree))" \
> > > > -               BUILDDIR="$(realpath pacman)" \
> > > > +               BUILDDIR="$(realpath $(objtree))/pacman" \
> > >
> > >
> > > Right.
> > >
> > > $(realpath pacman) expands to empty
> > > if 'pacman' does not exist yet.
> > > Your fix is correct.
> >
> > I'm wondering how it worked for me, as I specifically tested this case.
> > But I'm fine with the proposal.
> >
> > Shouldn't it be "$(realpath ./pacman)" though?
> 
> 
> No.
> 
> Aa I said, $(realpath ./pacman) expands to an empty string
> if ./pacman does not exist.

My bad, I misread the braces and thought the code was doing
BUILDDIR="$(realpath $(objtree)/pacman)"


Thomas

Masahiro Yamada July 21, 2024, 6:42 p.m. UTC | #8

On Sun, Jul 21, 2024 at 11:57 PM Thomas Weißschuh <[e-mail]> wrote:
>
> On 2024-07-21 22:42:12+0000, Masahiro Yamada wrote:
> > On Sun, Jul 21, 2024 at 9:11 PM Nathan Chancellor <[e-mail]> wrote:
> > >
> > > Hi Masahiro,
> > >
> > > On Sun, Jul 21, 2024 at 03:58:49PM +0900, Masahiro Yamada wrote:
> > > > On Sun, Jul 21, 2024 at 12:32 PM Nathan Chancellor <[e-mail]> wrote:
> > > > > On Sat, Jul 20, 2024 at 11:18:12AM +0200, Thomas Weißschuh wrote:
> > > > > > pacman is the package manager used by Arch Linux and its derivates.
> > > > > > Creating native packages from the kernel tree has multiple advantages:
> > > > > >
> > > > > > * The package triggers the correct hooks for initramfs generation and
> > > > > >   bootloader configuration
> > > > > > * Uninstallation is complete and also invokes the relevant hooks
> > > > > > * New UAPI headers can be installed without any manual bookkeeping
> > > > > >
> > > > > > The PKGBUILD file is a modified version of the one used for the
> > > > > > downstream Arch Linux "linux" package.
> > > > > > Extra steps that should not be necessary for a development kernel have
> > > > > > been removed and an UAPI header package has been added.
> > > > > >
> > > > > > Signed-off-by: Thomas Weißschuh <[e-mail]>
> > > > > > ---
> > > > >
> > > > > I think this looks really solid now, thanks again for the PACMAN_PKGBASE
> > > > > addition.
> > > > >
> > > > > I tested building Arch Linux's configuration for x86_64 and booting it
> > > > > in a VM. I built Fedora's configuration for aarch64 just to test the
> > > > > cross building aspect and making sure the result of various bits that we
> > > > > added that would not affect x86 (such as the dtb installation) looked
> > > > > reasonable.
> > > > >
> > > > > Reviewed-by: Nathan Chancellor <[e-mail]>
> > > > > Tested-by: Nathan Chancellor <[e-mail]>
> > > >
> > > > I believe this is a separate issue, but
> > > > Debian/Ubuntu provides a 'makepkg' package, which fails
> > > > with 'User defined signal 1' error.
> > > >
> > > > After 'sudo apt install makepkg',
> > > >
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ cat
> > > > /etc/os-release
> > > > PRETTY_NAME="Ubuntu 24.04 LTS"
> > > > NAME="Ubuntu"
> > > > VERSION_ID="24.04"
> > > > VERSION="24.04 LTS (Noble Numbat)"
> > > > VERSION_CODENAME=noble
> > > > ID=ubuntu
> > > > ID_LIKE=debian
> > > > HOME_URL="https://www.ubuntu.com/"
> > > > SUPPORT_URL="https://help.ubuntu.com/"
> > > > BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
> > > > PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
> > > > UBUNTU_CODENAME=noble
> > > > LOGO=ubuntu-logo
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > > makepkg --version
> > > > makepkg (pacman) 6.0.2
> > > > Copyright (c) 2006-2021 Pacman Development Team <[e-mail]>.
> > > > Copyright (C) 2002-2006 Judd Vinet <[e-mail]>.
> > > >
> > > > This is free software; see the source for copying conditions.
> > > > There is NO WARRANTY, to the extent permitted by law.
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$ git
> > > > log --oneline -2
> > > > 5dcaebb67ad9 (HEAD) kbuild: add script and target to generate pacman package
> > > > 41c196e567fb (tag: next-20240719, origin/master, origin/HEAD) Add
> > > > linux-next specific files for 20240719
> > > > masahiro@zoe:~/ref/linux-next((HEAD detached from origin/master))$
> > > > make defconfig  pacman-pkg
> > > > *** Default configuration is based on 'x86_64_defconfig'
> > > > #
> > > > # No change to .config
> > > > #
> > > > objtree="/home/masahiro/ref/linux-next" \
> > > > BUILDDIR="" \
> > >
> > > It is not related to this issue but I don't think this should be empty.
> > > 'realpath pacman' does not appear to work here, I was able to fix this
> > > with the following diff:
> > >
> > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > index 94357f47d2fa..b0fd44a40075 100644
> > > --- a/scripts/Makefile.package
> > > +++ b/scripts/Makefile.package
> > > @@ -148,7 +148,7 @@ PHONY += pacman-pkg
> > >  pacman-pkg:
> > >         @ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
> > >         +objtree="$(realpath $(objtree))" \
> > > -               BUILDDIR="$(realpath pacman)" \
> > > +               BUILDDIR="$(realpath $(objtree))/pacman" \
> >
> >
> > Right.
> >
> > $(realpath pacman) expands to empty
> > if 'pacman' does not exist yet.
> > Your fix is correct.
>
> I'm wondering how it worked for me, as I specifically tested this case.
> But I'm fine with the proposal.
>
> Shouldn't it be "$(realpath ./pacman)" though?
>
> >
> > >                 CARCH="$(UTS_MACHINE)" \
> > >                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > >                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > >
> > >
> > > > CARCH="x86_64" \
> > > > KBUILD_MAKEFLAGS="rR --no-print-directory" \
> > > > KBUILD_REVISION="4" \
> > > > makepkg
> > > >
> > > > ==> ERROR: An unknown error has occurred. Exiting...
> > > > User defined signal 1
> > > > make[3]: *** [scripts/Makefile.package:150: pacman-pkg] Error 138
> > > > make[2]: *** [Makefile:1538: pacman-pkg] Error 2
> > > > make[1]: *** [/home/masahiro/ref/linux-next/Makefile:347:
> > > > __build_one_by_one] Error 2
> > > > make: *** [Makefile:224: __sub-make] Error 2
> > > >
> > > > Do you know anything?
> > >
> > > Adding '-x' to the interpreter line in /usr/bin/makepkg shows that
> > > pacman is implicitly required for makepkg to function, so you'll need to
> > > install pacman-package-manager as well. However, even with it installed,
> > > the build will fail because you will be unable to perform the dependency
> > > checking (since pacman won't be managing the packages).
>
> It seems very weird that there is no dependency, not even a soft one,
> from makepkg to pacman.
>
> > > I think the traditional solution for this situation (building a
> > > distribution's package on a distribution other than the one the package
> > > is being built for) is using an OPTS variable to allow the user to pass
> > > in the dependency checking skip flag (that's what I do for RPM builds on
> > > Arch Linux), so perhaps MAKEPKGOPTS so that you can use MAKEPKGOPTS=-d?
> > > With this diff and that variable value, the build starts within makepkg
> > > for me.
> > >
> > > diff --git a/scripts/Makefile.package b/scripts/Makefile.package
> > > index b0fd44a40075..4a80584ec771 100644
> > > --- a/scripts/Makefile.package
> > > +++ b/scripts/Makefile.package
> > > @@ -152,7 +152,7 @@ pacman-pkg:
> > >                 CARCH="$(UTS_MACHINE)" \
> > >                 KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
> > >                 KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
> > > -               makepkg
> > > +               makepkg $(MAKEPKGOPTS)
> >
> >
> > Thank you for delving into this!
>
> Thanks to both of you!
>
> > With -d option, I was able to compile it.
> >
> > I previously compile-tested it in an Arch Linux Docker container,
> > but this is more convenient.
> >
> >
> > Thomas, will you send v8 with Nathan's suggestion?
> >
> > Or, I can offer to fold the diff if Thomas agrees.
>
> If you are fine with folding the diff, please do!
>


Applied with Nathan's diff.
Thanks!

Patch

diff --git a/.gitignore b/.gitignore
index c59dc60ba62e..7902adf4f7f1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -92,6 +92,12 @@  modules.order
 #
 /tar-install/
 
+#
+# pacman files (make pacman-pkg)
+#
+/PKGBUILD
+/pacman/
+
 #
 # We don't want to ignore the following even if they are dot-files
 #
diff --git a/MAINTAINERS b/MAINTAINERS
index da5352dbd4f3..16f8e13aa4c6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11997,6 +11997,13 @@  F:	include/uapi/linux/nfsd/
 F:	include/uapi/linux/sunrpc/
 F:	net/sunrpc/
 
+KERNEL PACMAN PACKAGING (in addition to generic KERNEL BUILD)
+M:	Thomas Weißschuh <[e-mail]>
+R:	Christian Heusel <[e-mail]>
+R:	Nathan Chancellor <[e-mail]>
+S:	Maintained
+F:	scripts/package/PKGBUILD
+
 KERNEL REGRESSIONS
 M:	Thorsten Leemhuis <[e-mail]>
 L:	[e-mail]
diff --git a/Makefile b/Makefile
index c97d6404b891..943899656977 100644
--- a/Makefile
+++ b/Makefile
@@ -1487,7 +1487,7 @@  CLEAN_FILES += vmlinux.symvers modules-only.symvers \
 # Directories & files removed with 'make mrproper'
 MRPROPER_FILES += include/config include/generated          \
 		  arch/$(SRCARCH)/include/generated .objdiff \
-		  debian snap tar-install \
+		  debian snap tar-install PKGBUILD pacman \
 		  .config .config.old .version \
 		  Module.symvers \
 		  certs/signing_key.pem \
diff --git a/scripts/Makefile.package b/scripts/Makefile.package
index bf016af8bf8a..94357f47d2fa 100644
--- a/scripts/Makefile.package
+++ b/scripts/Makefile.package
@@ -141,6 +141,19 @@  snap-pkg:
 	cd $(objtree)/snap && \
 	snapcraft --target-arch=$(UTS_MACHINE)
 
+# pacman-pkg
+# ---------------------------------------------------------------------------
+
+PHONY += pacman-pkg
+pacman-pkg:
+	@ln -srf $(srctree)/scripts/package/PKGBUILD $(objtree)/PKGBUILD
+	+objtree="$(realpath $(objtree))" \
+		BUILDDIR="$(realpath pacman)" \
+		CARCH="$(UTS_MACHINE)" \
+		KBUILD_MAKEFLAGS="$(MAKEFLAGS)" \
+		KBUILD_REVISION="$(shell $(srctree)/scripts/build-version)" \
+		makepkg
+
 # dir-pkg tar*-pkg - tarball targets
 # ---------------------------------------------------------------------------
 
@@ -221,6 +234,7 @@  help:
 	@echo '  bindeb-pkg          - Build only the binary kernel deb package'
 	@echo '  snap-pkg            - Build only the binary kernel snap package'
 	@echo '                        (will connect to external hosts)'
+	@echo '  pacman-pkg          - Build only the binary kernel pacman package'
 	@echo '  dir-pkg             - Build the kernel as a plain directory structure'
 	@echo '  tar-pkg             - Build the kernel as an uncompressed tarball'
 	@echo '  targz-pkg           - Build the kernel as a gzip compressed tarball'
diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD
new file mode 100644
index 000000000000..663ce300dd06
--- /dev/null
+++ b/scripts/package/PKGBUILD
@@ -0,0 +1,108 @@ 
+# SPDX-License-Identifier: GPL-2.0-only
+# Maintainer: Thomas Weißschuh <[e-mail]>
+# Contributor: Jan Alexander Steffens (heftig) <[e-mail]>
+
+pkgbase=${PACMAN_PKGBASE:-linux-upstream}
+pkgname=("${pkgbase}" "${pkgbase}-api-headers")
+if grep -q CONFIG_MODULES=y include/config/auto.conf; then
+	pkgname+=("${pkgbase}-headers")
+fi
+pkgver="${KERNELRELEASE//-/_}"
+# The PKGBUILD is evaluated multiple times.
+# Running scripts/build-version from here would introduce inconsistencies.
+pkgrel="${KBUILD_REVISION}"
+pkgdesc='Upstream Linux'
+url='https://www.kernel.org/'
+# Enable flexible cross-compilation
+arch=(${CARCH})
+license=(GPL-2.0-only)
+makedepends=(
+	bc
+	bison
+	cpio
+	flex
+	gettext
+	kmod
+	libelf
+	openssl
+	pahole
+	perl
+	python
+	rsync
+	tar
+)
+options=(!debug !strip !buildflags !makeflags)
+
+build() {
+	# MAKEFLAGS from makepkg.conf override the ones inherited from kbuild.
+	# Bypass this override with a custom variable.
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+
+	${MAKE} KERNELRELEASE="${KERNELRELEASE}" KBUILD_BUILD_VERSION="${pkgrel}"
+}
+
+_package() {
+	pkgdesc="The ${pkgdesc} kernel and modules"
+
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+	local modulesdir="${pkgdir}/usr/${MODLIB}"
+
+	echo "Installing boot image..."
+	# systemd expects to find the kernel here to allow hibernation
+	# https://github.com/systemd/systemd/commit/edda44605f06a41fb86b7ab8128dcf99161d2344
+	install -Dm644 "$(${MAKE} -s image_name)" "${modulesdir}/vmlinuz"
+
+	# Used by mkinitcpio to name the kernel
+	echo "${pkgbase}" > "${modulesdir}/pkgbase"
+
+	echo "Installing modules..."
+	${MAKE} INSTALL_MOD_PATH="${pkgdir}/usr" INSTALL_MOD_STRIP=1 \
+		DEPMOD=true modules_install
+
+	if [ -d "${srctree}/arch/${SRCARCH}/boot/dts" ]; then
+		echo "Installing dtbs..."
+		${MAKE} INSTALL_DTBS_PATH="${modulesdir}/dtb" dtbs_install
+	fi
+
+	# remove build link, will be part of -headers package
+	rm -f "${modulesdir}/build"
+}
+
+_package-headers() {
+	pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel"
+
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+	local builddir="${pkgdir}/usr/${MODLIB}/build"
+
+	echo "Installing build files..."
+	"${srctree}/scripts/package/install-extmod-build" "${builddir}"
+
+	echo "Installing System.map and config..."
+	cp System.map "${builddir}/System.map"
+	cp .config "${builddir}/.config"
+
+	echo "Adding symlink..."
+	mkdir -p "${pkgdir}/usr/src"
+	ln -sr "${builddir}" "${pkgdir}/usr/src/${pkgbase}"
+}
+
+_package-api-headers() {
+	pkgdesc="Kernel headers sanitized for use in userspace"
+	provides=(linux-api-headers)
+	conflicts=(linux-api-headers)
+
+	export MAKEFLAGS="${KBUILD_MAKEFLAGS}"
+	cd "${objtree}"
+
+	${MAKE} headers_install INSTALL_HDR_PATH="${pkgdir}/usr"
+}
+
+for _p in "${pkgname[@]}"; do
+	eval "package_$_p() {
+		$(declare -f "_package${_p#$pkgbase}")
+		_package${_p#$pkgbase}
+	}"
+done