| Message ID | 20240131104851.2311358-1-john.g.garry@oracle.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Introduce uts_release | expand |
On Wed, Jan 31, 2024 at 10:48:47AM +0000, John Garry wrote: > When hacking it is a waste of time and compute energy that we need to > rebuild much kernel code just for changing the head git commit, like this: > > > touch include/generated/utsrelease.h > > time make -j3 > mkdir -p /home/john/mnt_sda4/john/kernel-dev2/tools/objtool && make O=/home/john/mnt_sda4/john/kernel-dev2 subdir=tools/objtool --no-print-directory -C objtool > INSTALL libsubcmd_headers > CALL scripts/checksyscalls.sh > CC init/version.o > AR init/built-in.a > CC kernel/sys.o > CC kernel/module/main.o > AR kernel/module/built-in.a > CC drivers/base/firmware_loader/main.o > CC kernel/trace/trace.o > AR drivers/base/firmware_loader/built-in.a > AR drivers/base/built-in.a > CC net/ethtool/ioctl.o > AR kernel/trace/built-in.a > AR kernel/built-in.a > AR net/ethtool/built-in.a > AR net/built-in.a > AR drivers/built-in.a > AR built-in.a > ... > > Files like drivers/base/firmware_loader/main.c needs to be recompiled as > it includes generated/utsrelease.h for UTS_RELEASE macro, and utsrelease.h > is regenerated when the head commit changes. > > Introduce global char uts_release[] in init/version.c, which this > mentioned code can use instead of UTS_RELEASE, meaning that we don't need > to rebuild for changing the head commit - only init/version.c needs to be > rebuilt. Whether all the references to UTS_RELEASE in the codebase are > proper is a different matter. > > For an x86_64 defconfig build for this series on my old laptop, here is > before and after rebuild time: > > before: > real 0m53.591s > user 1m1.842s > sys 0m9.161s > > after: > real 0m37.481s > user 0m46.461s > sys 0m7.199s > > Sending as an RFC as I need to test more of the conversions and I would > like to also convert more UTS_RELEASE users to prove this is proper > approach. I like it, I also think that v4l2 includes this as well as all of those drivers seem to rebuild when this changes, does that not happen for you too? Anyway, if the firmware changes work, I'm all for this, thanks for taking it on! thanks, greg k-h
On 31/01/2024 16:22, Greg KH wrote: >> before: >> real 0m53.591s >> user 1m1.842s >> sys 0m9.161s >> >> after: >> real 0m37.481s >> user 0m46.461s >> sys 0m7.199s >> >> Sending as an RFC as I need to test more of the conversions and I would >> like to also convert more UTS_RELEASE users to prove this is proper >> approach. > I like it, I also think that v4l2 includes this as well as all of those > drivers seem to rebuild when this changes, does that not happen for you > too? I didn't see that. Were you were building for arm64? I can see some v4l2 configs enabled there for the vanilla defconfig (but none for x86-64). > > Anyway, if the firmware changes work, I'm all for this, thanks for > taking it on! cheers. BTW, I just noticed that utsrelase.h might not be updated for my allmodconfig build when I change the head commit - I'll investigate why ... John
On Wed, Jan 31, 2024 at 05:16:09PM +0000, John Garry wrote: > On 31/01/2024 16:22, Greg KH wrote: > > > before: > > > real 0m53.591s > > > user 1m1.842s > > > sys 0m9.161s > > > > > > after: > > > real 0m37.481s > > > user 0m46.461s > > > sys 0m7.199s > > > > > > Sending as an RFC as I need to test more of the conversions and I would > > > like to also convert more UTS_RELEASE users to prove this is proper > > > approach. > > I like it, I also think that v4l2 includes this as well as all of those > > drivers seem to rebuild when this changes, does that not happen for you > > too? > > I didn't see that. Were you were building for arm64? I can see some v4l2 > configs enabled there for the vanilla defconfig (but none for x86-64). Building for x86, maybe it's one of the other LINUX_VERSION type defines we have, sorry, can't remember, it's been a long time since I looked into it. thanks, greg k-h
On Wed, Jan 31, 2024 at 7:49 PM John Garry <john.g.garry@oracle.com> wrote: > > When hacking it is a waste of time and compute energy that we need to > rebuild much kernel code just for changing the head git commit, like this: > > > touch include/generated/utsrelease.h > > time make -j3 > mkdir -p /home/john/mnt_sda4/john/kernel-dev2/tools/objtool && make O=/home/john/mnt_sda4/john/kernel-dev2 subdir=tools/objtool --no-print-directory -C objtool > INSTALL libsubcmd_headers > CALL scripts/checksyscalls.sh > CC init/version.o > AR init/built-in.a > CC kernel/sys.o > CC kernel/module/main.o > AR kernel/module/built-in.a > CC drivers/base/firmware_loader/main.o > CC kernel/trace/trace.o > AR drivers/base/firmware_loader/built-in.a > AR drivers/base/built-in.a > CC net/ethtool/ioctl.o > AR kernel/trace/built-in.a > AR kernel/built-in.a > AR net/ethtool/built-in.a > AR net/built-in.a > AR drivers/built-in.a > AR built-in.a > ... > > Files like drivers/base/firmware_loader/main.c needs to be recompiled as > it includes generated/utsrelease.h for UTS_RELEASE macro, and utsrelease.h > is regenerated when the head commit changes. > > Introduce global char uts_release[] in init/version.c, which this > mentioned code can use instead of UTS_RELEASE, meaning that we don't need > to rebuild for changing the head commit - only init/version.c needs to be > rebuilt. Whether all the references to UTS_RELEASE in the codebase are > proper is a different matter. > > For an x86_64 defconfig build for this series on my old laptop, here is > before and after rebuild time: > > before: > real 0m53.591s > user 1m1.842s > sys 0m9.161s > > after: > real 0m37.481s > user 0m46.461s > sys 0m7.199s > > Sending as an RFC as I need to test more of the conversions and I would > like to also convert more UTS_RELEASE users to prove this is proper > approach. > > John Garry (4): > init: Add uts_release > tracing: Use uts_release > net: ethtool: Use uts_release > firmware_loader: Use uts_release > > drivers/base/firmware_loader/main.c | 39 +++++++++++++++++++++++------ > include/linux/utsname.h | 1 + > init/version.c | 3 +++ > kernel/trace/trace.c | 4 +-- > net/ethtool/ioctl.c | 4 +-- > 5 files changed, 39 insertions(+), 12 deletions(-) > > -- > 2.35.3 > As you see, several drivers store UTS_RELEASE in their driver data, and even print it in debug print. I do not see why it is useful. As you discussed in 3/4, if UTS_RELEASE is unneeded, it is better to get rid of it. If such version information is useful for drivers, the intention is whether the version of the module, or the version of vmlinux. That is a question. They differ when CONFIG_MODVERSION. When module developers intend to printk the git version from which the module was compiled from, presumably they want to use UTS_RELEASE, which was expanded at the compile time of the module. If you replace it with uts_release, it is the git version of vmlinux. Of course, the replacement is safe for always-builtin code. Lastly, we can avoid using UTS_RELEASE without relying on your patch. For example, commit 3a3a11e6e5a2bc0595c7e36ae33c861c9e8c75b1 replaced UTS_RELEASE with init_uts_ns.name.release So, is your uts_release a shorthand of init_uts_ns.name.release? I think what you can contribute are: - Explore the UTS_RELEASE users, and check if you can get rid of it. - Where UTS_RELEASE is useful, consider if it is possible to replace it with init_uts_ns.name.release
On Sat, 3 Feb 2024 00:01:26 +0900 Masahiro Yamada wrote: > I do not see why it is useful. > As you discussed in 3/4, if UTS_RELEASE is unneeded, > it is better to get rid of it. To be clear - the discussion on 3/4 was about the fact that netdev already prints UTS_RELEASE into the version member of driver info struct, as a default. So the drivers no longer have to. But there's no user-observable change there.
On 02/02/2024 15:01, Masahiro Yamada wrote: >> -- >> 2.35.3 > > As you see, several drivers store UTS_RELEASE in their driver data, > and even print it in debug print. > > > I do not see why it is useful. I would tend to agree, and mentioned that earlier. > As you discussed in 3/4, if UTS_RELEASE is unneeded, > it is better to get rid of it. Jakub replied about this. > > > If such version information is useful for drivers, the intention is > whether the version of the module, or the version of vmlinux. > That is a question. > They differ when CONFIG_MODVERSION. > I think often this information in UTS_RELEASE is shared as informative only, so the user can conveniently know the specific kernel git version. > > When module developers intend to printk the git version > from which the module was compiled from, > presumably they want to use UTS_RELEASE, which > was expanded at the compile time of the module. > > If you replace it with uts_release, it is the git version > of vmlinux. > > > Of course, the replacement is safe for always-builtin code. > > > > Lastly, we can avoid using UTS_RELEASE without relying > on your patch. > > > > For example, commit 3a3a11e6e5a2bc0595c7e36ae33c861c9e8c75b1 > replaced UTS_RELEASE with init_uts_ns.name.release > > > So, is your uts_release a shorthand of init_uts_ns.name.release? Yes - well that both are strings containing UTS_RELEASE. Using a struct sub-member is bit ungainly, but I suppose that we should not be making life easy for people using this. However we already have init_utsname in: static inline struct new_utsname *init_utsname(void) { return &init_uts_ns.name; } So could use init_utsname()->release, which is a bit nicer. > > > > I think what you can contribute are: > > - Explore the UTS_RELEASE users, and check if you can get rid of it. Unfortunately I expect resistance for this. I also expect places like FW loader it is necessary. And when this is used in sysfs, people will say that it is part of the ABI now. How about I send the patch to update to use init_uts_ns and mention also that it would be better to not use at all, if possible? I can cc you. > > - Where UTS_RELEASE is useful, consider if it is possible > to replace it with init_uts_ns.name.release ok, but, as above, could use init_utsname()->release also Thanks, John
On Mon, Feb 5, 2024 at 5:25 PM John Garry <john.g.garry@oracle.com> wrote: > > On 02/02/2024 15:01, Masahiro Yamada wrote: > >> -- > >> 2.35.3 > > > > As you see, several drivers store UTS_RELEASE in their driver data, > > and even print it in debug print. > > > > > > I do not see why it is useful. > > I would tend to agree, and mentioned that earlier. > > > As you discussed in 3/4, if UTS_RELEASE is unneeded, > > it is better to get rid of it. > > Jakub replied about this. > > > > > > > If such version information is useful for drivers, the intention is > > whether the version of the module, or the version of vmlinux. > > That is a question. > > They differ when CONFIG_MODVERSION. > > > > I think often this information in UTS_RELEASE is shared as informative > only, so the user can conveniently know the specific kernel git version. > > > > > When module developers intend to printk the git version > > from which the module was compiled from, > > presumably they want to use UTS_RELEASE, which > > was expanded at the compile time of the module. > > > > If you replace it with uts_release, it is the git version > > of vmlinux. > > > > > > Of course, the replacement is safe for always-builtin code. > > > > > > > > Lastly, we can avoid using UTS_RELEASE without relying > > on your patch. > > > > > > > > For example, commit 3a3a11e6e5a2bc0595c7e36ae33c861c9e8c75b1 > > replaced UTS_RELEASE with init_uts_ns.name.release > > > > > > So, is your uts_release a shorthand of init_uts_ns.name.release? > > Yes - well that both are strings containing UTS_RELEASE. Using a struct > sub-member is bit ungainly, but I suppose that we should not be making > life easy for people using this. > > However we already have init_utsname in: > > static inline struct new_utsname *init_utsname(void) > { > return &init_uts_ns.name; > } > > So could use init_utsname()->release, which is a bit nicer. > > > > > > > > > I think what you can contribute are: > > > > - Explore the UTS_RELEASE users, and check if you can get rid of it. > > Unfortunately I expect resistance for this. I also expect places like FW > loader it is necessary. And when this is used in sysfs, people will say > that it is part of the ABI now. > > How about I send the patch to update to use init_uts_ns and mention also > that it would be better to not use at all, if possible? I can cc you. OK. As I mentioned in the previous reply, the replacement is safe for builtin code. When you touch modular code, please pay a little more care, because UTS_RELEASE and init_utsname()->release may differ when CONFIG_MODVERSIONS=y. > > > > > - Where UTS_RELEASE is useful, consider if it is possible > > to replace it with init_uts_ns.name.release > > ok, but, as above, could use init_utsname()->release also I am fine with it. init_utsname()->release is more commonly used (but less common than UTS_RELEASE) $ git grep 'init_utsname()->release' | wc 28 92 2065 $ git grep 'init_uts_ns.name.release' | wc 7 34 587 $ git grep 'UTS_RELEASE' | wc 57 304 4741
On 05/02/2024 23:10, Masahiro Yamada wrote: >>> I think what you can contribute are: >>> >>> - Explore the UTS_RELEASE users, and check if you can get rid of it. >> Unfortunately I expect resistance for this. I also expect places like FW >> loader it is necessary. And when this is used in sysfs, people will say >> that it is part of the ABI now. >> >> How about I send the patch to update to use init_uts_ns and mention also >> that it would be better to not use at all, if possible? I can cc you. > > OK. > > > As I mentioned in the previous reply, the replacement is safe > for builtin code. > > When you touch modular code, please pay a little more care, > because UTS_RELEASE and init_utsname()->release > may differ when CONFIG_MODVERSIONS=y. > Are you saying that we may have a different release version kernel and module built with CONFIG_MODVERSIONS=y, and the module was using UTS_RELEASE for something? That something may be like setting some info in a sysfs file, like in this example: static ssize_t target_core_item_version_show(struct config_item *item, char *page) { return sprintf(page, "Target Engine Core ConfigFS Infrastructure %s" " on %s/%s on "UTS_RELEASE"\n", TARGET_CORE_VERSION, utsname()->sysname, utsname()->machine); } And the intention is to use the module codebase release version and not the kernel codebase release version. Hence utsname() is used for .sysname and .machine, but not .release . Thanks, John
On 08/02/2024 10:08, John Garry wrote: > On 05/02/2024 23:10, Masahiro Yamada wrote: >>>> I think what you can contribute are: >>>> >>>> - Explore the UTS_RELEASE users, and check if you can get rid of it. >>> Unfortunately I expect resistance for this. I also expect places like FW >>> loader it is necessary. And when this is used in sysfs, people will say >>> that it is part of the ABI now. >>> >>> How about I send the patch to update to use init_uts_ns and mention also >>> that it would be better to not use at all, if possible? I can cc you. >> >> OK. >> >> >> As I mentioned in the previous reply, the replacement is safe >> for builtin code. >> >> When you touch modular code, please pay a little more care, >> because UTS_RELEASE and init_utsname()->release >> may differ when CONFIG_MODVERSIONS=y. >> > > Are you saying that we may have a different release version kernel and > module built with CONFIG_MODVERSIONS=y, and the module was using > UTS_RELEASE for something? That something may be like setting some info > in a sysfs file, like in this example: > > static ssize_t target_core_item_version_show(struct config_item *item, > char *page) > { > return sprintf(page, "Target Engine Core ConfigFS Infrastructure %s" > " on %s/%s on "UTS_RELEASE"\n", TARGET_CORE_VERSION, > utsname()->sysname, utsname()->machine); > } > > And the intention is to use the module codebase release version and not > the kernel codebase release version. Hence utsname() is used for > .sysname and .machine, but not .release . Hi Masahiro, Can you comment on whether I am right about CONFIG_MODVERSIONS, above? Thanks, John
On Wed, Feb 21, 2024 at 6:01 PM John Garry <john.g.garry@oracle.com> wrote: > > On 08/02/2024 10:08, John Garry wrote: > > On 05/02/2024 23:10, Masahiro Yamada wrote: > >>>> I think what you can contribute are: > >>>> > >>>> - Explore the UTS_RELEASE users, and check if you can get rid of it. > >>> Unfortunately I expect resistance for this. I also expect places like FW > >>> loader it is necessary. And when this is used in sysfs, people will say > >>> that it is part of the ABI now. > >>> > >>> How about I send the patch to update to use init_uts_ns and mention also > >>> that it would be better to not use at all, if possible? I can cc you. > >> > >> OK. > >> > >> > >> As I mentioned in the previous reply, the replacement is safe > >> for builtin code. > >> > >> When you touch modular code, please pay a little more care, > >> because UTS_RELEASE and init_utsname()->release > >> may differ when CONFIG_MODVERSIONS=y. > >> > > > > Are you saying that we may have a different release version kernel and > > module built with CONFIG_MODVERSIONS=y, and the module was using > > UTS_RELEASE for something? That something may be like setting some info > > in a sysfs file, like in this example: > > > > static ssize_t target_core_item_version_show(struct config_item *item, > > char *page) > > { > > return sprintf(page, "Target Engine Core ConfigFS Infrastructure %s" > > " on %s/%s on "UTS_RELEASE"\n", TARGET_CORE_VERSION, > > utsname()->sysname, utsname()->machine); > > } > > > > And the intention is to use the module codebase release version and not > > the kernel codebase release version. Hence utsname() is used for > > .sysname and .machine, but not .release . > > Hi Masahiro, > > Can you comment on whether I am right about CONFIG_MODVERSIONS, above? > > Thanks, > John Your understanding about CONFIG_MODVERSIONS is correct.