| Message ID | 1384201760-16785-1-git-send-email-jason@lakedaemon.net (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 11/11/2013 01:29 PM, Jason Cooper wrote: > Consumers of the Linux kernel's build products are beginning to hardcode > the filenames of the dtbs generated. Since the dtb filenames are > currently the dts filename s/dts/dtb/, this prevents the kernel > community from renaming dts files as needed. My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?
On Mon, Nov 11, 2013 at 02:24:11PM -0700, Stephen Warren wrote: > On 11/11/2013 01:29 PM, Jason Cooper wrote: > > Consumers of the Linux kernel's build products are beginning to hardcode > > the filenames of the dtbs generated. Since the dtb filenames are > > currently the dts filename s/dts/dtb/, this prevents the kernel > > community from renaming dts files as needed. > > My take is that the DTB filenames are part of the ABI, and therefore the > DTS filenames are also part of the ABI. Why would we want to rename them? This idea originated from a conversation with Olof about starting to pre-pend the manufacturer name to the dts files: https://lkml.org/lkml/2013/11/8/378 imho, at some point there is going to be an ABI breakage wrt dtb names when the devicetree moves to a separate repository. All users will have to rework their scripts at that point. If that assumption is correct, why not put in place a _good_ ABI that allows us to be flexible as needed and specific for end users? If you're still with me on this idea, shouldn't we put this in place now (as symlinks, thus retaining old names) so that we can switch over more easily later? thx, Jason.
On Mon, Nov 11, 2013 at 3:24 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: > On 11/11/2013 01:29 PM, Jason Cooper wrote: >> Consumers of the Linux kernel's build products are beginning to hardcode >> the filenames of the dtbs generated. Since the dtb filenames are >> currently the dts filename s/dts/dtb/, this prevents the kernel >> community from renaming dts files as needed. > > My take is that the DTB filenames are part of the ABI, and therefore the > DTS filenames are also part of the ABI. Why would we want to rename them? I agree with the ABI part, but for long term I think compatible strings are a better choice for the ABI than filenames. A link provides for a way to transition. Whether a platform can tolerate a filename change is really up to the platform maintainer like other ABI changes. Rob
On 11/12/2013 08:29 AM, Rob Herring wrote: > On Mon, Nov 11, 2013 at 3:24 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: >> On 11/11/2013 01:29 PM, Jason Cooper wrote: >>> Consumers of the Linux kernel's build products are beginning to hardcode >>> the filenames of the dtbs generated. Since the dtb filenames are >>> currently the dts filename s/dts/dtb/, this prevents the kernel >>> community from renaming dts files as needed. >> >> My take is that the DTB filenames are part of the ABI, and therefore the >> DTS filenames are also part of the ABI. Why would we want to rename them? > > I agree with the ABI part, but for long term I think compatible > strings are a better choice for the ABI than filenames. A link > provides for a way to transition. But this change isn't making the compatible value be the ABI, it's still keeping the filename as the ABI, but creating a different way of choosing the filename. In other words, if the compatible value is the ABI, then the algorithm is: * Search for all available DTBs * Parse each DTB to find one with a matching compatible value Whereas if the filename is the ABI, the algorithm is: * Calculate the filename * Load that one specific file, without looking at its content Some additional thoughts: * The filename is only relevant to the bootloader (or whatever locates/selects a DTB). The kernel/OS just gets a single DTB passed to it and uses it, without any naming metadata[1]. * Perhaps different bootloaders work different ways. For example, I've set up U-Boot on all Tegra devices so that if U-Boot expands the string "${soc}-${board}.dtb" using its environment, it'll generate the correct DTB filename. * Other boot-loaders might like "${compatible_parsed_from_the_dtb}" to be the filename; that's Jason's proposal. * Other SW (which I mention in [1]) might want to locate DTBs based on ARM machine ID rather than compatible value. * As such, we're really talking about an indexing process when installing/packaging/searching DTBs. Shouldn't this file renaming happen as part of that process, not the build process? * When we create symlinks or any kind of index, I think we want to create links for all entries in the compatible property (or other index source). Consider if we have two boards: Cardhu revision A02 Cardhu revision A04 Initially we think they're the same because we didn't read the schematics well enough, but then later we split them. That means we'll start out with one file: nvidia,tegra30-cardhu.dtb then later have instead: nvidia,tegra30-cardhu-a02.dtb nvidia,tegra30-cardhu-a04.dtb Similarly, perhaps one DTB is (initially thought to be?) suitable for multiple ARM machine IDs, etc. So, bootloaders would have to search for $vendor,$soc-$board-$rev.dtb then fall back to $vendor,$soc-$board.dtb (both those values are in the compatible properties in those DTBs). But then "nvidia,tegra30" is in the compatible property for many boards. I wonder how we resolve that? [1] I'm ignoring the idea that was proposed of passing n DTBs to the kernel and having it select one based on machine ID. So by "the kernel" I mean the code that runs after the DTB is selected.
Stephen, On Tue, Nov 12, 2013 at 11:38:47AM -0700, Stephen Warren wrote: > On 11/12/2013 08:29 AM, Rob Herring wrote: > > On Mon, Nov 11, 2013 at 3:24 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: > >> On 11/11/2013 01:29 PM, Jason Cooper wrote: > >>> Consumers of the Linux kernel's build products are beginning to hardcode > >>> the filenames of the dtbs generated. Since the dtb filenames are > >>> currently the dts filename s/dts/dtb/, this prevents the kernel > >>> community from renaming dts files as needed. > >> > >> My take is that the DTB filenames are part of the ABI, and therefore the > >> DTS filenames are also part of the ABI. Why would we want to rename them? > > > > I agree with the ABI part, but for long term I think compatible > > strings are a better choice for the ABI than filenames. A link > > provides for a way to transition. > > But this change isn't making the compatible value be the ABI, it's still > keeping the filename as the ABI, but creating a different way of > choosing the filename. Right, which provides a path towards a slightly more sane ABI. If we choose to implement this, or another variant, we get to shape a migration path towards an ABI we designed. As opposed to locked in to one we didn't even see coming. > In other words, if the compatible value is the ABI, then the algorithm is: > > * Search for all available DTBs > * Parse each DTB to find one with a matching compatible value > > Whereas if the filename is the ABI, the algorithm is: > > * Calculate the filename > * Load that one specific file, without looking at its content > > Some additional thoughts: > > * The filename is only relevant to the bootloader (or whatever > locates/selects a DTB). The kernel/OS just gets a single DTB passed to > it and uses it, without any naming metadata[1]. As I see it (and perhaps this suffers from my limited view of the embedded space, eg fairly marvell-specific), there are two scenarios: 1) The end goal: bootloaders load a single DTB from a chunk of flash and pass it off to the kernel/next stage, whatever. Either there is no filename, just a flash partition, or the vendor has chosen some dtb naming scheme suitable for their needs, eg OMAP SoCs pulling the bootloader from an SD card would need to name the dtb contained on the same SD card. board.dtb? I'm assuming here that the SD card files are board specific (x-loader, uboot, env, dtb) 2) right now: distributions are creating utilities, such a flash-kernel in Debian, that pick from a slew of dtbs, append it to the zImage, and create the uImage. The bootloaders in these installs don't have support for fdt. Our problem (as I see it) is that #2 is creating an ABI out of something we never realized would become an ABI, the filenames created by 'make dtbs'. And since the dtb name is derived directly from the dts name, we're getting locked into a bunch of filenames that are less than ideal because we didn't know what we didn't know when we started this two+ years ago. > * Perhaps different bootloaders work different ways. For example, I've > set up U-Boot on all Tegra devices so that if U-Boot expands the string > "${soc}-${board}.dtb" using its environment, it'll generate the correct > DTB filename. Why does the bootloader have to choose? Is this a development scenario or a deployed scenario? > * Other boot-loaders might like "${compatible_parsed_from_the_dtb}" to > be the filename; that's Jason's proposal. > > * Other SW (which I mention in [1]) might want to locate DTBs based on > ARM machine ID rather than compatible value. fyi: https://github.com/zonque/pxa-impedance-matcher.git > * As such, we're really talking about an indexing process when > installing/packaging/searching DTBs. Shouldn't this file renaming happen > as part of that process, not the build process? I agree, but it's not. Hence the patch. > * When we create symlinks or any kind of index, I think we want to > create links for all entries in the compatible property (or other index > source). We can't, you'll get collisions. In my globalscale,mirabox.dtb example, the next string is 'marvell,armada370'. That matches many boards which _aren't_ compatible with the globalscale,mirabox.dtb. > Consider if we have two boards: > > Cardhu revision A02 > Cardhu revision A04 > > Initially we think they're the same because we didn't read the > schematics well enough, but then later we split them. That means we'll > start out with one file: > > nvidia,tegra30-cardhu.dtb We've seen this as well with a couple of boards. > then later have instead: > > nvidia,tegra30-cardhu-a02.dtb > nvidia,tegra30-cardhu-a04.dtb > > Similarly, perhaps one DTB is (initially thought to be?) suitable for > multiple ARM machine IDs, etc. > > So, bootloaders would have to search for $vendor,$soc-$board-$rev.dtb > then fall back to $vendor,$soc-$board.dtb (both those values are in the > compatible properties in those DTBs). I disagree here. I don't think this decision/discovery process should be done by the bootloader. It should be done by the distribution installer/firmware upgrade utility. The risk of breaking boot is too high. > But then "nvidia,tegra30" is in the compatible property for many boards. > I wonder how we resolve that? At least during installer/upgrade the distro has the opportunity to query the user, "Hey, this used to have nvidia,tegra30-cardhu.dtb, but now I need to choose between nvidia,tegra30-cardhu-a02.dtb and nvidia,tegra30-cardhu-a04.dtb. Please see http://wiki.mydistro.org/devicetree/upgrade/nvidia,tegra30-cardhu for howto determine which variant matches your board." Ideally, there would be programmatic way for the upgrader to determine which variant it is on, but failing that, it could fall back to the above. thx, Jason.
> 2) right now: distributions are creating utilities, such a flash-kernel > in Debian, that pick from a slew of dtbs, append it to the zImage, and > create the uImage. The bootloaders in these installs don't have support > for fdt. > > Our problem (as I see it) is that #2 is creating an ABI out of something > we never realized would become an ABI, the filenames created by 'make > dtbs'. Well, flash-kernel could be taught how to parse .dtb files to get the compatibility strings out of them, rather than use filenames. It might still be early enough to give distributions some guidelines about what DT folks consider ABI and what is not. How they should find a fitting .dtb file and how they should not. Andrew
On Tue, Nov 12, 2013 at 08:40:09PM +0100, Andrew Lunn wrote: > > 2) right now: distributions are creating utilities, such a flash-kernel > > in Debian, that pick from a slew of dtbs, append it to the zImage, and > > create the uImage. The bootloaders in these installs don't have support > > for fdt. > > > > Our problem (as I see it) is that #2 is creating an ABI out of something > > we never realized would become an ABI, the filenames created by 'make > > dtbs'. > > Well, flash-kernel could be taught how to parse .dtb files to get the > compatibility strings out of them, rather than use filenames. > > It might still be early enough to give distributions some guidelines > about what DT folks consider ABI and what is not. How they should find > a fitting .dtb file and how they should not. True, but this isn't a 'search /boot/config-X for CONFIG_Y so we can configure for Z' *cough* *grub* *cough*. flash-kernel currently doesn't have any more correct way to select dtbs than by filename. It's normal to look for arch/<arch>/boot/zImage after a build. Module names are not 1-1 with object files or source files, they are unique, and static. Our failure was not providing a similar ABI for the dtbs. imho, we messed up, so we should fix it. thx, Jason.
On 11/12/2013 12:30 PM, Jason Cooper wrote: > Stephen, > > On Tue, Nov 12, 2013 at 11:38:47AM -0700, Stephen Warren wrote: >> On 11/12/2013 08:29 AM, Rob Herring wrote: >>> On Mon, Nov 11, 2013 at 3:24 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: >>>> On 11/11/2013 01:29 PM, Jason Cooper wrote: >>>>> Consumers of the Linux kernel's build products are beginning to hardcode >>>>> the filenames of the dtbs generated. Since the dtb filenames are >>>>> currently the dts filename s/dts/dtb/, this prevents the kernel >>>>> community from renaming dts files as needed. >>>> >>>> My take is that the DTB filenames are part of the ABI, and therefore the >>>> DTS filenames are also part of the ABI. Why would we want to rename them? >>> >>> I agree with the ABI part, but for long term I think compatible >>> strings are a better choice for the ABI than filenames. A link >>> provides for a way to transition. >> >> But this change isn't making the compatible value be the ABI, it's still >> keeping the filename as the ABI, but creating a different way of >> choosing the filename. > > Right, which provides a path towards a slightly more sane ABI. If we > choose to implement this, or another variant, we get to shape a > migration path towards an ABI we designed. As opposed to locked in to > one we didn't even see coming. I don't really agree here; I specifically named all the Tegra DTB/.dts files in a sane fashion precisely so that U-Boot could easily determine which one to load. It's hard to see how this wasn't a predictable issue. ... >> * Perhaps different bootloaders work different ways. For example, I've >> set up U-Boot on all Tegra devices so that if U-Boot expands the string >> "${soc}-${board}.dtb" using its environment, it'll generate the correct >> DTB filename. > > Why does the bootloader have to choose? Is this a development scenario or > a deployed scenario? I would expect it to work in both scenarios. Assuming the DTB is in a filesystem rather than in a singleton location in flash, the DTB must have a filename. It'd be far better to store each board's DTB as a separate filename, rather than have some process copy "the" DTB to a singleton filename at install time. Doing so allows the same filesystem to work on multiple boards unmodified. As an example, I move my SD card between ~10 different Tegra boards without touching its content, and U-Boot loads the correct DTB based on the board it's running on. Pengutronix demo'd a single SD card being moved between a couple of entirely unrelated boards at ELC-E too. Given that assumption, the bootloader must somehow determine which filename to load at boot time, rather than some one-time installation process doing the picking operation. ... >> * As such, we're really talking about an indexing process when >> installing/packaging/searching DTBs. Shouldn't this file renaming happen >> as part of that process, not the build process? > > I agree, but it's not. Hence the patch. Well, you could patch either:-) ... >> So, bootloaders would have to search for $vendor,$soc-$board-$rev.dtb >> then fall back to $vendor,$soc-$board.dtb (both those values are in the >> compatible properties in those DTBs). > > I disagree here. I don't think this decision/discovery process should > be done by the bootloader. It should be done by the distribution > installer/firmware upgrade utility. The risk of breaking boot is too > high. That means distros have to have board identification logic, and explicitly have code that selects the correct DT. Thus, distros would have to be explicitly ported to each board (since IDing the board would need some kind of board specific logic). Isn't the push towards single-zImage and DT explicitly so that distros require zero board-specific knowledge? BTW, since the outcome of this discussion possibly affects distro installers and kernel packages, perhaps the discussion should take place on, or be CC'd to, cross-distro@lists.linaro.org?
On Tue, Nov 12, 2013 at 01:15:51PM -0700, Stephen Warren wrote: > On 11/12/2013 12:30 PM, Jason Cooper wrote: > > On Tue, Nov 12, 2013 at 11:38:47AM -0700, Stephen Warren wrote: > >> On 11/12/2013 08:29 AM, Rob Herring wrote: > >>> On Mon, Nov 11, 2013 at 3:24 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: > >>>> On 11/11/2013 01:29 PM, Jason Cooper wrote: > >>>>> Consumers of the Linux kernel's build products are beginning to hardcode > >>>>> the filenames of the dtbs generated. Since the dtb filenames are > >>>>> currently the dts filename s/dts/dtb/, this prevents the kernel > >>>>> community from renaming dts files as needed. > >>>> > >>>> My take is that the DTB filenames are part of the ABI, and therefore the > >>>> DTS filenames are also part of the ABI. Why would we want to rename them? > >>> > >>> I agree with the ABI part, but for long term I think compatible > >>> strings are a better choice for the ABI than filenames. A link > >>> provides for a way to transition. > >> > >> But this change isn't making the compatible value be the ABI, it's still > >> keeping the filename as the ABI, but creating a different way of > >> choosing the filename. > > > > Right, which provides a path towards a slightly more sane ABI. If we > > choose to implement this, or another variant, we get to shape a > > migration path towards an ABI we designed. As opposed to locked in to > > one we didn't even see coming. > > I don't really agree here; I specifically named all the Tegra DTB/.dts > files in a sane fashion precisely so that U-Boot could easily determine > which one to load. It's hard to see how this wasn't a predictable issue. This is the difference between a commercial effort and a hobbyist effort. I think I have a plan for how we can both get what we want. I could change the argument to 'dtc ... -L arch/arm/boot ...'. Thus: $ cd arch/arm/boot $ ls -l *.dtb # minus a few columns ... globalscale,mirabox.dtb -> dts/armada-370-mirabox.dtb ... This way, aftermarket users (debian installers, etc) could go to the same place they go for zImages, and find a standard named dtb. They wouldn't need to parse the dtbs, just look at the current compatible string for the board they are installing on and find the matching file. For commercial efforts, nothing changes, the well-thought out *.dtb filenames are still in the same place they always were. A self-imposed vendor ABI, if you will. Then, in situations where dts filenames need to change, that can happen (typically per vendor) because the ABI, from the kernel maintainers pov, is the symlinks in arch/arm/boot/. As for the migration, there wouldn't need to be one beyond this first step of providing symlinks. That way, vendor-preferred filenames stay as they are. Would that work better for you? thx, Jason.
On 11/14/2013 09:28 AM, Jason Cooper wrote: > On Tue, Nov 12, 2013 at 01:15:51PM -0700, Stephen Warren wrote: >> On 11/12/2013 12:30 PM, Jason Cooper wrote: >>> On Tue, Nov 12, 2013 at 11:38:47AM -0700, Stephen Warren wrote: >>>> On 11/12/2013 08:29 AM, Rob Herring wrote: >>>>> On Mon, Nov 11, 2013 at 3:24 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: >>>>>> On 11/11/2013 01:29 PM, Jason Cooper wrote: >>>>>>> Consumers of the Linux kernel's build products are beginning to hardcode >>>>>>> the filenames of the dtbs generated. Since the dtb filenames are >>>>>>> currently the dts filename s/dts/dtb/, this prevents the kernel >>>>>>> community from renaming dts files as needed. >>>>>> >>>>>> My take is that the DTB filenames are part of the ABI, and therefore the >>>>>> DTS filenames are also part of the ABI. Why would we want to rename them? >>>>> >>>>> I agree with the ABI part, but for long term I think compatible >>>>> strings are a better choice for the ABI than filenames. A link >>>>> provides for a way to transition. >>>> >>>> But this change isn't making the compatible value be the ABI, it's still >>>> keeping the filename as the ABI, but creating a different way of >>>> choosing the filename. >>> >>> Right, which provides a path towards a slightly more sane ABI. If we >>> choose to implement this, or another variant, we get to shape a >>> migration path towards an ABI we designed. As opposed to locked in to >>> one we didn't even see coming. >> >> I don't really agree here; I specifically named all the Tegra DTB/.dts >> files in a sane fashion precisely so that U-Boot could easily determine >> which one to load. It's hard to see how this wasn't a predictable issue. > > This is the difference between a commercial effort and a hobbyist > effort. I think I have a plan for how we can both get what we want. I > could change the argument to 'dtc ... -L arch/arm/boot ...'. Thus: > > $ cd arch/arm/boot > $ ls -l *.dtb # minus a few columns > ... > globalscale,mirabox.dtb -> dts/armada-370-mirabox.dtb > ... I don't think the symlinks should be in arch/arm/boot, since that's not where the source files are. They should either exist in arch/arm/boot/dts (since that's the source file location, and that generally determines the object/binary/output file location in the kernel build system), or in some user-specified directory, created as a side-effect of e.g. "make dtbs_install". There have been some small discussions of a dtb install target before, but I don't remember the details. > This way, aftermarket users (debian installers, etc) could go to the > same place they go for zImages, and find a standard named dtb. They > wouldn't need to parse the dtbs, just look at the current compatible > string for the board they are installing on and find the matching file. > For commercial efforts, nothing changes, the well-thought out *.dtb > filenames are still in the same place they always were. A self-imposed > vendor ABI, if you will. > > Then, in situations where dts filenames need to change, that can happen > (typically per vendor) because the ABI, from the kernel maintainers pov, > is the symlinks in arch/arm/boot/. > > As for the migration, there wouldn't need to be one beyond this first > step of providing symlinks. That way, vendor-preferred filenames stay > as they are. > > Would that work better for you? I don't think a distro should need to know the DTB filename at all. Rather, the/a bootloader should know which platform it's running on, and provide a variable/... to the boot script/... that defines the DTB filename. That would completely remove all the knowledge of DTB filenames from distros. I don't think there's anything stopping you from renaming dts/armada-370-mirabox.dts to globalscale,mirabox.dts, as a one-time ABI-breaking event, if you want the compatible value to be the filename for those DTs. Is there a reason you'd want the .dts and .dtb filenames to differ? You'd mentioned flexibility in renaming the .dts files, but why would you need that flexibility? Perhaps if we had to, we could just rename all .dts to conform to the new naming rule and then I can deal with the bootloader fallout for Tegra:-(
On Thu, Nov 14, 2013 at 12:00:24PM -0700, Stephen Warren wrote: > I don't think a distro should need to know the DTB filename at all. > Rather, the/a bootloader should know which platform it's running on, and > provide a variable/... to the boot script/... that defines the DTB > filename. That would completely remove all the knowledge of DTB > filenames from distros. At some point the distro has to install a dtb into some cannonical place so the bootloader can find it.. Are you thinking that distros will have to ship a /boot/dtbs/*.dtb with every dtb from the kernel build? What happens when you install two different versions of the kernel? What about boot schemes that can load a kernel version dependent dtb? I've said this before - I really think the kernel community will do itself a big favor if it encourages boot schemes that can handle a kernel version dependent dtb, rather that design schemes that absolutely can't allow for that. Jason
On Thu, Nov 14, 2013 at 11:28:59AM -0500, Jason Cooper wrote: > This is the difference between a commercial effort and a hobbyist > effort. I think I have a plan for how we can both get what we want. I > could change the argument to 'dtc ... -L arch/arm/boot ...'. Thus: > > $ cd arch/arm/boot > $ ls -l *.dtb # minus a few columns > ... > globalscale,mirabox.dtb -> dts/armada-370-mirabox.dtb > ... > > > This way, aftermarket users (debian installers, etc) could go to the > same place they go for zImages, and find a standard named dtb. They > wouldn't need to parse the dtbs, just look at the current compatible > string for the board they are installing on and find the matching file. There's little point saying that they should be in arch/arm/boot. Unless the distro is installing from a kernel source tree on every platform, the location of the dtbs really doesn't matter. Let's look at the typical distro build process: - take the source - patch it - build it - install it, relatively placing files according to their ultimate destination - packaging the result up, adding metadata and scripts to aid installation So, how does affecting where things are in the build tree help after all that process? What matters for DT files are: (a) their names (b) their location after installation Where it appears in the build tree is neither here nor there, and IMHO should not be of concern to any distro - just the same as picking the zImage/uImage/whatever out of arch/arm/boot is none of their business. We have ways and means to deal with the installation of kernel images and modules. There's well defined makefile targets for this, and make variables which control where and how these get put. For the kernel image, you can specify the script to be run (which is called with the location of the *Image file) for copying. Pulling it manually out of arch/arm/boot is really a failure of the distro packaging people to get their process right. The same goes for the DT files: if we don't provide a method for the kernel build system to install these in a sane way, then that's our failing. However, distros should be asking for this, and not picking the files out of arch/arm/boot/dts. As for the commercial vs hobbist, there's very little difference here. If you're a hobbist, building natively, that's where having the right /sbin/installkernel present is a godsend, especially if you're running some distro. If you don't, most likely you understand what's required to install a kernel. Last point - on the Carrier-1, I have a fully automated setup which I can fire off a build, walk away, and it will build a kernel including modules, install it locally on my x86 PC in ~/systems/imx6 - including appending the DT file, tar up the appropriate files, automatically copy the tar file over, unpack it, update the initramfs for ubuntu 12.04, update the vfat /boot filesystem appropriately, and reboot the platform. Yes, I know the target system with that. That's not my point. The point I'm making is that that level of automation is possible without excessive amounts of messing about with kernel build internals - that all runs with standard unmodified kbuild. So, maintain a sense of perspective and separation of stages please.
On Thu, Nov 14, 2013 at 12:16:22PM -0700, Jason Gunthorpe wrote: > On Thu, Nov 14, 2013 at 12:00:24PM -0700, Stephen Warren wrote: > > > I don't think a distro should need to know the DTB filename at all. > > Rather, the/a bootloader should know which platform it's running on, and > > provide a variable/... to the boot script/... that defines the DTB > > filename. That would completely remove all the knowledge of DTB > > filenames from distros. > > At some point the distro has to install a dtb into some cannonical > place so the bootloader can find it.. > > Are you thinking that distros will have to ship a /boot/dtbs/*.dtb > with every dtb from the kernel build? Why are we even having this discussion? Where distros want to install dtbs is their policy. What we should do is make it _possible_ for them to have policy, not define the policy for them. The problem here is this. Consider how often these files change, and how often there's differences in them. Sensible distros allow you to have several kernels installed so that you can switch to other versions if, for example, you run into problems. Distros have two main choices - either to ship the DTBs separately assuming that they'll always be compatible and correct, or ship them with the kernel itself (in which case we've lost the plot about what DT is supposed to give us.) That's where the problem lies: what if a distro wants to have installed more than one set of DTBs corresponding to $old_kernel and $new_kernel. This really isn't something for us kernel developers to decide. It's a distro question and it needs distro people to think about it and put proposals forward for what they'd like to see. In my experience, that won't happen - they like hiding away and sorting out their own problems. The communication between distros and kernel folk has always tended to be extremely poor.
On 11/14/2013 12:16 PM, Jason Gunthorpe wrote: > On Thu, Nov 14, 2013 at 12:00:24PM -0700, Stephen Warren wrote: > >> I don't think a distro should need to know the DTB filename at all. >> Rather, the/a bootloader should know which platform it's running on, and >> provide a variable/... to the boot script/... that defines the DTB >> filename. That would completely remove all the knowledge of DTB >> filenames from distros. > > At some point the distro has to install a dtb into some cannonical > place so the bootloader can find it.. > > Are you thinking that distros will have to ship a /boot/dtbs/*.dtb > with every dtb from the kernel build? Yes. How else would the distro support booting the install on arbitrary boards? > What happens when you install two different versions of the kernel? /boot/zImage-$version /boot/dtbs-$version/*.dtb > What about boot schemes that can load a kernel version dependent dtb? I think that's solved by installing the DTB files in a kernel-version-specific directory.
On Thu, Nov 14, 2013 at 3:37 PM, Stephen Warren <swarren@wwwdotorg.org> wrote: > On 11/14/2013 12:16 PM, Jason Gunthorpe wrote: >> On Thu, Nov 14, 2013 at 12:00:24PM -0700, Stephen Warren wrote: >> >>> I don't think a distro should need to know the DTB filename at all. >>> Rather, the/a bootloader should know which platform it's running on, and >>> provide a variable/... to the boot script/... that defines the DTB >>> filename. That would completely remove all the knowledge of DTB >>> filenames from distros. >> >> At some point the distro has to install a dtb into some cannonical >> place so the bootloader can find it.. Right, and this place can be *anywhere* with *any name* in the scripted case. In a non-scripted case, it just needs to match the bootloader name and where it differs from the one in the kernel right now, this would be hellish. To use GlobalScale as an example, since the patch did - if Marvell or GlobalScale ship a U-Boot that boots the board, that has a fixed name and *then* rename their tree in the kernel source, they should be shot. The kernel community should be careful about it if they're not Marvell or GlobalScale and didn't check with the bootloader situation.. but when it happens, this is *entirely* a packaging problem for your package maintainers. *ENTIRELY*. In the case where it gets flashed into the board rather than left on a filesystem, then it doesn't matter one jot what the filename is as long as it goes to the right offset - which is going to end up hardcoded in your distro, right? So stop hardcoding it (flash-kernel does a particularly crappy, but eventually usable job of this under Debian/Ubuntu right now already so they're safe). >> Are you thinking that distros will have to ship a /boot/dtbs/*.dtb >> with every dtb from the kernel build? > > Yes. How else would the distro support booting the install on arbitrary > boards? There are LOTS of ways. Mark and Russell insist on me being more concise, so I'll simply say the distros have to do more work and there's not really any way around it. Specifying "requirements for a distro boot" means making everything comply with that, which is a lot of work for distros anyway. May as well make your distro work around the problems where they actually have some control - distro scripting, distro board support, and not esoteric kernel build behaviors, or having distro teams work on fixing non-compliant bootloaders. Ta, Matt Sealey <neko@bakuhatsu.net>
On Mon, 11 Nov 2013 20:29:20 +0000, Jason Cooper <jason@lakedaemon.net> wrote: > Consumers of the Linux kernel's build products are beginning to hardcode > the filenames of the dtbs generated. Since the dtb filenames are > currently the dts filename s/dts/dtb/, this prevents the kernel > community from renaming dts files as needed. > > Let's provide a consistent naming structure for consumers to script > against. Or at least, as consistent as the dts properties themselves. > > With this patch, adding the '-L' option to the dtc commandline will > cause dtc to create a symlink to the generated dtb, using the board > compatible string as the filename, eg: > > globalscale,mirabox.dtb -> armada-370-mirabox.dtb This whole thread is just crazy. The filename is not an ABI. Anything digging into the kernel build tree is a) wrong, and b) userspace tooling consuming the kernel build products which is /not/ ABI and can be changed If we want to have a tool or script for creating a well formed directory full of .dtb files because it would be helpful to U-Boot or something else, then that is fine. Make it part of the "make install" path. g.
On Fri, Nov 15, 2013 at 09:12:15PM +0900, Grant Likely wrote: > On Mon, 11 Nov 2013 20:29:20 +0000, Jason Cooper <jason@lakedaemon.net> wrote: > > Consumers of the Linux kernel's build products are beginning to hardcode > > the filenames of the dtbs generated. Since the dtb filenames are > > currently the dts filename s/dts/dtb/, this prevents the kernel > > community from renaming dts files as needed. > > > > Let's provide a consistent naming structure for consumers to script > > against. Or at least, as consistent as the dts properties themselves. > > > > With this patch, adding the '-L' option to the dtc commandline will > > cause dtc to create a symlink to the generated dtb, using the board > > compatible string as the filename, eg: > > > > globalscale,mirabox.dtb -> armada-370-mirabox.dtb > > This whole thread is just crazy. The filename is not an ABI. Anything > digging into the kernel build tree is a) wrong, and b) userspace tooling > consuming the kernel build products which is /not/ ABI and can be changed whew. I still think there is a need for an install target. The fact that our changing filenames might break things is an indicator of that. > If we want to have a tool or script for creating a well formed directory > full of .dtb files because it would be helpful to U-Boot or something > else, then that is fine. Make it part of the "make install" path. I'm currently reworking the patch(es) to implement 'make dtbs-install'. I'll hopefully have them ready this weekend or sooner. I'm taking the same tack as install.sh (/sbin/installdtbs takes precedence). thx, Jason.
On Thu, Nov 14, 2013 at 07:26:29PM +0000, Russell King - ARM Linux wrote: > We have ways and means to deal with the installation of kernel images > and modules. There's well defined makefile targets for this, and > make variables which control where and how these get put. For the > kernel image, you can specify the script to be run (which is called > with the location of the *Image file) for copying. Pulling it manually > out of arch/arm/boot is really a failure of the distro packaging people > to get their process right. > > The same goes for the DT files: if we don't provide a method for the > kernel build system to install these in a sane way, then that's our > failing. However, distros should be asking for this, and not picking > the files out of arch/arm/boot/dts. Exactly. V2 hopefully this weekend implementing 'make dtbs-install' thx, Jason.
On Thu, 14 Nov 2013 14:37:48 -0700, Stephen Warren <swarren@wwwdotorg.org> wrote: > On 11/14/2013 12:16 PM, Jason Gunthorpe wrote: > > On Thu, Nov 14, 2013 at 12:00:24PM -0700, Stephen Warren wrote: > > > >> I don't think a distro should need to know the DTB filename at all. > >> Rather, the/a bootloader should know which platform it's running on, and > >> provide a variable/... to the boot script/... that defines the DTB > >> filename. That would completely remove all the knowledge of DTB > >> filenames from distros. > > > > At some point the distro has to install a dtb into some cannonical > > place so the bootloader can find it.. > > > > Are you thinking that distros will have to ship a /boot/dtbs/*.dtb > > with every dtb from the kernel build? > > Yes. How else would the distro support booting the install on arbitrary > boards? > > > What happens when you install two different versions of the kernel? > > /boot/zImage-$version > /boot/dtbs-$version/*.dtb > > > What about boot schemes that can load a kernel version dependent dtb? > > I think that's solved by installing the DTB files in a > kernel-version-specific directory. That's certainly the way I would solve it. g.
Hi Jason, On Fri, Nov 15, 2013 at 4:23 PM, Jason Cooper <jason@lakedaemon.net> wrote: > On Thu, Nov 14, 2013 at 07:26:29PM +0000, Russell King - ARM Linux wrote: >> We have ways and means to deal with the installation of kernel images >> and modules. There's well defined makefile targets for this, and >> make variables which control where and how these get put. For the >> kernel image, you can specify the script to be run (which is called >> with the location of the *Image file) for copying. Pulling it manually >> out of arch/arm/boot is really a failure of the distro packaging people >> to get their process right. >> >> The same goes for the DT files: if we don't provide a method for the >> kernel build system to install these in a sane way, then that's our >> failing. However, distros should be asking for this, and not picking >> the files out of arch/arm/boot/dts. > > Exactly. V2 hopefully this weekend implementing 'make dtbs-install' Just a small comment, I think the make target should be called 'dtbs_install and not of 'dtbs-install' to be consistent with make {modules,firmware,headers}_install > > thx, > > Jason. > Best regards, Javier
On Fri, Nov 15, 2013 at 06:09:50PM +0100, Javier Martinez Canillas wrote: > Hi Jason, > > On Fri, Nov 15, 2013 at 4:23 PM, Jason Cooper <jason@lakedaemon.net> wrote: > > On Thu, Nov 14, 2013 at 07:26:29PM +0000, Russell King - ARM Linux wrote: > >> We have ways and means to deal with the installation of kernel images > >> and modules. There's well defined makefile targets for this, and > >> make variables which control where and how these get put. For the > >> kernel image, you can specify the script to be run (which is called > >> with the location of the *Image file) for copying. Pulling it manually > >> out of arch/arm/boot is really a failure of the distro packaging people > >> to get their process right. > >> > >> The same goes for the DT files: if we don't provide a method for the > >> kernel build system to install these in a sane way, then that's our > >> failing. However, distros should be asking for this, and not picking > >> the files out of arch/arm/boot/dts. > > > > Exactly. V2 hopefully this weekend implementing 'make dtbs-install' > > Just a small comment, I think the make target should be called > 'dtbs_install and not of 'dtbs-install' to be consistent with make > {modules,firmware,headers}_install Yep, typo on my part. thx, Jason.
On Fri, 15 Nov 2013 10:21:42 -0500, Jason Cooper <jason@lakedaemon.net> wrote: > On Fri, Nov 15, 2013 at 09:12:15PM +0900, Grant Likely wrote: > > On Mon, 11 Nov 2013 20:29:20 +0000, Jason Cooper <jason@lakedaemon.net> wrote: > > > Consumers of the Linux kernel's build products are beginning to hardcode > > > the filenames of the dtbs generated. Since the dtb filenames are > > > currently the dts filename s/dts/dtb/, this prevents the kernel > > > community from renaming dts files as needed. > > > > > > Let's provide a consistent naming structure for consumers to script > > > against. Or at least, as consistent as the dts properties themselves. > > > > > > With this patch, adding the '-L' option to the dtc commandline will > > > cause dtc to create a symlink to the generated dtb, using the board > > > compatible string as the filename, eg: > > > > > > globalscale,mirabox.dtb -> armada-370-mirabox.dtb > > > > This whole thread is just crazy. The filename is not an ABI. Anything > > digging into the kernel build tree is a) wrong, and b) userspace tooling > > consuming the kernel build products which is /not/ ABI and can be changed > > whew. I still think there is a need for an install target. The fact > that our changing filenames might break things is an indicator of that. > > > If we want to have a tool or script for creating a well formed directory > > full of .dtb files because it would be helpful to U-Boot or something > > else, then that is fine. Make it part of the "make install" path. > > I'm currently reworking the patch(es) to implement 'make dtbs-install'. > I'll hopefully have them ready this weekend or sooner. I'm taking the > same tack as install.sh (/sbin/installdtbs takes precedence). Cool, I think that is a good approach. g.
Detailed changelog can be found in each patch.
The kernel community never created a 'dtbs_install' make target for properly
handling the build output of the kernel's devicetree files. Currently, the dtb
files are simply their source file, s/dts/dtb/. Users have recently begun
retrieving these files from inside the kernel tree. Any change to the dts
filenames breaks the scripts of those users.
Before this gets too far along, we'd like to create a standardized method of
installing the dtb files for users to find and use. In addition, some vendors
have done a diligent job naming their devicetree source files appropriately and
we don't want to break their setups.
This patch series solves both problems by creating a dtbs_install make target
which calls arch/arm/boot/installdtbs.sh. If /sbin/installdtbs or
~/bin/installdtbs is present, they will be run instead of the default behavior.
The default action is to install a given dtb to
/lib/modules/${kernel_version}/devicetree/${board_compatible}.dtb
Ideally, the devicetree file should be independent of the kernel version,
however, we aren't there yet. So, while the devicetree files live in the
kernel source tree, we'll install them as above. Once they are in their own
repository, this install target can be removed, and users can adjust their
workflow to getting the dtbs from a new location.
Jason Cooper (2):
dtc: add 'compat' output option, prints board string
kbuild: dtbs_install: new make target
Makefile | 3 ++-
arch/arm/Makefile | 4 ++++
arch/arm/boot/installdtbs.sh | 44 ++++++++++++++++++++++++++++++++++++++++++++
scripts/dtc/dtc.c | 3 +++
scripts/dtc/dtc.h | 1 +
scripts/dtc/flattree.c | 9 +++++++++
6 files changed, 63 insertions(+), 1 deletion(-)
create mode 100644 arch/arm/boot/installdtbs.sh
Detailed changelog can be found in each patch.
The kernel community never created a 'dtbs_install' make target for properly
handling the build output of the kernel's devicetree files. Currently, the dtb
files are simply their source file, s/dts/dtb/. Users have recently begun
retrieving these files from inside the kernel tree. Any change to the dts
filenames breaks the scripts of those users.
Before this gets too far along, we'd like to create a standardized method of
installing the dtb files for users to find and use. In addition, some vendors
have done a diligent job naming their devicetree source files appropriately and
we don't want to break their setups.
This patch series solves both problems by creating a dtbs_install make target
which calls arch/arm/boot/installdtbs.sh. If /sbin/installdtbs or
~/bin/installdtbs is present, they will be run instead of the default behavior.
The default action is to install a given dtb to
/lib/modules/${kernel_version}/devicetree/${board_compatible}.dtb
Ideally, the devicetree file should be independent of the kernel version,
however, we aren't there yet. So, while the devicetree files live in the
kernel source tree, we'll install them as above. Once they are in their own
repository, this install target can be removed, and users can adjust their
workflow to getting the dtbs from a new location.
Jason Cooper (2):
scripts: dtc: build fdtget for extracting properties from dtbs
kbuild: dtbs_install: new make target
Makefile | 3 ++-
arch/arm/Makefile | 4 ++++
arch/arm/boot/installdtbs.sh | 44 ++++++++++++++++++++++++++++++++++++++++++++
scripts/dtc/.gitignore | 2 +-
scripts/dtc/Makefile | 10 +++++++++-
5 files changed, 60 insertions(+), 3 deletions(-)
create mode 100644 arch/arm/boot/installdtbs.sh
On Mon, 18 Nov 2013 18:38:14 +0000, Jason Cooper <jason@lakedaemon.net> wrote: > Consumers of the Linux kernel's build products are beginning to hardcode > the filenames of the dtbs generated. Since the dtb filenames are > currently the dts filename s/dts/dtb/, this prevents the kernel > community from renaming dts files as needed. > > Let's provide a consistent naming structure for consumers to script > against. Or at least, as consistent as the dts properties themselves. > > With this patch, adding the '-O compat' option to the dtc commandline > will cause dtc to parse the provided file, and print out the board > compatible string to stdout. Can you use the fdtget tool here instead? It will parse and return property values for a given .dtb. It certainly makes sense to post this here for the purpose of review, but any changes to dtc need to be against the upstream dtc repository. g. > This will facilitate an 'installdtbs.sh' script in the kernel for naming > dtb files by their compatible string, eg: > > $ dtc -I dtb -O compat arch/arm/boot/dts/armada-370-mirabox.dtb > globalscale,mirabox > > This change will also simplify distribution install scripts that need to > search through many dtbs to find the right one for a target board. > > Signed-off-by: Jason Cooper <jason@lakedaemon.net> > --- > changes since v1: > - made patch against in-tree dtc code to facilitate testing > - dtc prints compatible string to stdout now, instead of creating symlink > - should be more flexible for end-users > > scripts/dtc/dtc.c | 3 +++ > scripts/dtc/dtc.h | 1 + > scripts/dtc/flattree.c | 9 +++++++++ > 3 files changed, 13 insertions(+) > > diff --git a/scripts/dtc/dtc.c b/scripts/dtc/dtc.c > index a375683c1534..89264bb0a3dd 100644 > --- a/scripts/dtc/dtc.c > +++ b/scripts/dtc/dtc.c > @@ -68,6 +68,7 @@ static void __attribute__ ((noreturn)) usage(void) > fprintf(stderr, "\t\tOutput formats are:\n"); > fprintf(stderr, "\t\t\tdts - device tree source text\n"); > fprintf(stderr, "\t\t\tdtb - device tree blob\n"); > + fprintf(stderr, "\t\t\tcompat - print board compatible string\n"); > fprintf(stderr, "\t\t\tasm - assembler source\n"); > fprintf(stderr, "\t-V <output version>\n"); > fprintf(stderr, "\t\tBlob version to produce, defaults to %d (relevant for dtb\n\t\tand asm output only)\n", DEFAULT_FDT_VERSION); > @@ -250,6 +251,8 @@ int main(int argc, char *argv[]) > dt_to_blob(outf, bi, outversion); > } else if (streq(outform, "asm")) { > dt_to_asm(outf, bi, outversion); > + } else if (streq(outform, "compat")) { > + dt_to_compat(bi); > } else if (streq(outform, "null")) { > /* do nothing */ > } else { > diff --git a/scripts/dtc/dtc.h b/scripts/dtc/dtc.h > index 3e42a071070e..d4e47c697c2f 100644 > --- a/scripts/dtc/dtc.h > +++ b/scripts/dtc/dtc.h > @@ -255,6 +255,7 @@ void process_checks(int force, struct boot_info *bi); > > void dt_to_blob(FILE *f, struct boot_info *bi, int version); > void dt_to_asm(FILE *f, struct boot_info *bi, int version); > +void dt_to_compat(struct boot_info *bi); > > struct boot_info *dt_from_blob(const char *fname); > > diff --git a/scripts/dtc/flattree.c b/scripts/dtc/flattree.c > index 665dad7bb465..bdbd3d7e8964 100644 > --- a/scripts/dtc/flattree.c > +++ b/scripts/dtc/flattree.c > @@ -577,6 +577,15 @@ void dt_to_asm(FILE *f, struct boot_info *bi, int version) > data_free(strbuf); > } > > +void dt_to_compat(struct boot_info *bi) > +{ > + struct property *prop; > + > + prop = get_property(bi->dt, "compatible"); > + > + printf("%s\n", prop->val.val); > +} > + > struct inbuf { > char *base, *limit, *ptr; > }; > -- > 1.8.4.3 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Mon, 18 Nov 2013 21:21:06 +0000, Jason Cooper <jason@lakedaemon.net> wrote: > Unlike other build products in the Linux kernel, the devicetree blobs > are simply the name of their source file, s/dts/dtb/. There is also no > 'make *install' mechanism to put them in a standard place with a > standard naming structure. > > Unfortunately, users have begun scripting pulling the needed dtbs from > within the kernel tree, thus hardcoding the dtbs names. In turn, this > means any changes to the dts filenames breaks these scripts. > > This patch is an attempt to fix this problem. Akin to 'make install', > this creates a new make target, dtbs_install. The script that gets > called defers to a vendor or distribution supplied installdtbs binary, > if found in the system. Otherwise, the default action is to install a > given dtb into > > /lib/modules/${kernel_version}/devicetree/${board_compat}.dtb > > This solves several problems for us. The board compatible string should > be unique, this enforces/highlights that. While devicetrees are > stablilizing, the fact is, devicetrees aren't (yet) independent of > kernel version. This install target facilitates keeping track of that. > > Once the devicetree files are moved to their own repository, this > install target can be removed as users will be modifying their scripts > anyway. > > Signed-off-by: Jason Cooper <jason@lakedaemon.net> > --- > changes since v2: > - use fdtget instead of a modified dtc to get the board compat string > > changes since v1: > - added this patch > > Makefile | 3 ++- > arch/arm/Makefile | 4 ++++ > arch/arm/boot/installdtbs.sh | 44 ++++++++++++++++++++++++++++++++++++++++++++ Place the script in scripts/ please. There is no need for it to be arm specific. > 3 files changed, 50 insertions(+), 1 deletion(-) > create mode 100644 arch/arm/boot/installdtbs.sh > > diff --git a/Makefile b/Makefile > index 920ad07180c9..29d609e972d6 100644 > --- a/Makefile > +++ b/Makefile > @@ -339,6 +339,7 @@ OBJDUMP = $(CROSS_COMPILE)objdump > AWK = awk > GENKSYMS = scripts/genksyms/genksyms > INSTALLKERNEL := installkernel > +INSTALLDTBS := installdtbs > DEPMOD = /sbin/depmod > PERL = perl > CHECK = sparse > @@ -391,7 +392,7 @@ KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(S > export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION > export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC > export CPP AR NM STRIP OBJCOPY OBJDUMP > -export MAKE AWK GENKSYMS INSTALLKERNEL PERL UTS_MACHINE > +export MAKE AWK GENKSYMS INSTALLKERNEL INSTALLDTBS PERL UTS_MACHINE > export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS > > export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS > diff --git a/arch/arm/Makefile b/arch/arm/Makefile > index c99b1086d83d..c16ebb1e74b0 100644 > --- a/arch/arm/Makefile > +++ b/arch/arm/Makefile > @@ -314,6 +314,10 @@ PHONY += dtbs > dtbs: scripts > $(Q)$(MAKE) $(build)=$(boot)/dts MACHINE=$(MACHINE) dtbs > > +dtbs_install: dtbs > + $(CONFIG_SHELL) $(srctree)/$(boot)/installdtbs.sh $(KERNELRELEASE) \ > + "$(MODLIB)/devicetree" "$(srctree)/$(boot)/dts" > + > # We use MRPROPER_FILES and CLEAN_FILES now > archclean: > $(Q)$(MAKE) $(clean)=$(boot) > diff --git a/arch/arm/boot/installdtbs.sh b/arch/arm/boot/installdtbs.sh > new file mode 100644 > index 000000000000..3adef376188f > --- /dev/null > +++ b/arch/arm/boot/installdtbs.sh > @@ -0,0 +1,44 @@ > +#!/bin/sh > +# > +# This file is subject to the terms and conditions of the GNU General Public > +# License. See the file "COPYING" in the main directory of this archive > +# for more details. > +# > +# Copyright (C) 1995 by Linus Torvalds > +# > +# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin > +# > +# Further adapted from arch/x86/boot/install.sh by Jason Cooper > +# > +# "make dtbs_install" script > +# > +# Arguments: > +# $1 - kernel version > +# $2 - default install path (blank if root directory) > +# $3 - directory containing dtbs > +# > + > +# User may have a custom install script > + > +if [ -x ~/bin/${INSTALLDTBS} ]; then exec ~/bin/${INSTALLDTBS} "$@"; fi > +if [ -x /sbin/${INSTALLDTBS} ]; then exec /sbin/${INSTALLDTBS} "$@"; fi > + > +FDTGET=./scripts/dtc/fdtget > + > +# Default install > +[ -d "$2" ] && rm -rf "$2" > + > +mkdir -p "$2" > + > +for dtb in `find "$3" -name "*.dtb"`; do > + # we use fdtget to parse the dtb, get the board compatible string, > + # and then copy the dtb to $2/${board_compatible}.dtb > + compat="`$FDTGET -t s "$dtb" / compatible | cut -d ' ' -f 1`" > + > + if [ -e "$2/${compat}.dtb" ]; then > + echo "Install $dtb: $2/${compat}.dtb already exists!" 1>&2 > + exit 1 > + else > + cp "$dtb" "$2/${compat}.dtb" > + fi It sounded to me like the question of whether to rename the .dtbs is still up in the air. Personally, I don't think the naming convention is as big an issue as has been expressed. If some of the names are a little non-specific, is that really a big problem? Once a filename is chosen for a device, then it belongs to that device. New hardware can always choose a more-specific name. Regardless, I'll let the debate proceed a bit to see if some consensus can be formed before acking this patch. In general though the script looks fine to me. g.
diff --git a/dtc.c b/dtc.c index e3c96536fd9d..cb2bb1b7ce1f 100644 --- a/dtc.c +++ b/dtc.c @@ -49,7 +49,7 @@ static void fill_fullpaths(struct node *tree, const char *prefix) /* Usage related data. */ static const char usage_synopsis[] = "dtc [options] <input file>"; -static const char usage_short_opts[] = "qI:O:o:V:d:R:S:p:fb:i:H:sW:E:hv"; +static const char usage_short_opts[] = "qI:O:o:V:d:R:S:p:fb:i:H:sW:E:Lhv"; static struct option const usage_long_opts[] = { {"quiet", no_argument, NULL, 'q'}, {"in-format", a_argument, NULL, 'I'}, @@ -67,6 +67,7 @@ static struct option const usage_long_opts[] = { {"phandle", a_argument, NULL, 'H'}, {"warning", a_argument, NULL, 'W'}, {"error", a_argument, NULL, 'E'}, + {"sym-link", no_argument, NULL, 'L'}, {"help", no_argument, NULL, 'h'}, {"version", no_argument, NULL, 'v'}, {NULL, no_argument, NULL, 0x0}, @@ -97,6 +98,7 @@ static const char * const usage_opts_help[] = { "\t\tboth - Both \"linux,phandle\" and \"phandle\" properties", "\n\tEnable/disable warnings (prefix with \"no-\")", "\n\tEnable/disable errors (prefix with \"no-\")", + "\n\tCreate a symlink to the dtb named by board compatible string", "\n\tPrint this help and exit", "\n\tPrint version and exit", NULL, @@ -109,7 +111,7 @@ int main(int argc, char *argv[]) const char *outform = "dts"; const char *outname = "-"; const char *depname = NULL; - int force = 0, sort = 0; + int force = 0, sort = 0, mksymlink = 0; const char *arg; int opt; FILE *outf = NULL; @@ -184,6 +186,9 @@ int main(int argc, char *argv[]) case 'E': parse_checks_option(false, true, optarg); break; + case 'L': + mksymlink = 1; + break; case 'h': usage(NULL); @@ -247,6 +252,9 @@ int main(int argc, char *argv[]) dt_to_source(outf, bi); } else if (streq(outform, "dtb")) { dt_to_blob(outf, bi, outversion); + if (mksymlink) { + dt_to_symlink(bi, outname); + } } else if (streq(outform, "asm")) { dt_to_asm(outf, bi, outversion); } else if (streq(outform, "null")) { diff --git a/dtc.h b/dtc.h index 264a20cf66a8..0cdb558fead1 100644 --- a/dtc.h +++ b/dtc.h @@ -254,6 +254,7 @@ void process_checks(int force, struct boot_info *bi); void dt_to_blob(FILE *f, struct boot_info *bi, int version); void dt_to_asm(FILE *f, struct boot_info *bi, int version); +void dt_to_symlink(struct boot_info *bi, const char *outname); struct boot_info *dt_from_blob(const char *fname); diff --git a/flattree.c b/flattree.c index 665dad7bb465..e1720dec4389 100644 --- a/flattree.c +++ b/flattree.c @@ -577,6 +577,36 @@ void dt_to_asm(FILE *f, struct boot_info *bi, int version) data_free(strbuf); } +void dt_to_symlink(struct boot_info *bi, const char *outname) +{ + struct property *prop; + const char *board; + int symlen; + char *symname; + char *dname; + char *bname; + + prop = get_property(bi->dt, "compatible"); + board = prop->val.val; + + symlen = strlen(outname) + prop->val.len; + symname = xmalloc(symlen); + + dname = xdirname(outname); + bname = xbasename(outname); + + snprintf(symname, symlen, "%s/%s.dtb", dname, board); + + /* create the symlink */ + if (symlink(bname, symname) == -1) { + die("Couldn't create symlink %s: %s\n", symlink, strerror(errno)); + } + + free(symname); + free(dname); + free(bname); +} + struct inbuf { char *base, *limit, *ptr; }; diff --git a/srcpos.c b/srcpos.c index c20bc5315bc1..5f9e032330ea 100644 --- a/srcpos.c +++ b/srcpos.c @@ -34,7 +34,7 @@ struct search_path { static struct search_path *search_path_head, **search_path_tail; -static char *dirname(const char *path) +char *xdirname(const char *path) { const char *slash = strrchr(path, '/'); @@ -49,6 +49,22 @@ static char *dirname(const char *path) return NULL; } +char *xbasename(const char *path) +{ + const char *slash = strrchr(path, '/'); + + if (slash) { + int len = strlen(path) - (slash - path); + char *base = xmalloc(len + 1); + + memcpy(base, slash + 1, len); + base[len] = '\0'; + return base; + } + + return NULL; +} + FILE *depfile; /* = NULL */ struct srcfile_state *current_srcfile; /* = NULL */ @@ -150,7 +166,7 @@ void srcfile_push(const char *fname) srcfile = xmalloc(sizeof(*srcfile)); srcfile->f = srcfile_relative_open(fname, &srcfile->name); - srcfile->dir = dirname(srcfile->name); + srcfile->dir = xdirname(srcfile->name); srcfile->prev = current_srcfile; srcfile->lineno = 1; diff --git a/srcpos.h b/srcpos.h index 93a27123c2e9..a6b6ad308d52 100644 --- a/srcpos.h +++ b/srcpos.h @@ -57,6 +57,8 @@ FILE *srcfile_relative_open(const char *fname, char **fullnamep); void srcfile_push(const char *fname); int srcfile_pop(void); +char *xdirname(const char *path); +char *xbasename(const char *path); /** * Add a new directory to the search path for input files *
Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed. Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves. With this patch, adding the '-L' option to the dtc commandline will cause dtc to create a symlink to the generated dtb, using the board compatible string as the filename, eg: globalscale,mirabox.dtb -> armada-370-mirabox.dtb Signed-off-by: Jason Cooper <jason@lakedaemon.net> --- All, I'm sending this RFC to see if this is how we want to go about this. If it's acceptable, I'll resend to the dtc maintainers thx, Jason. dtc.c | 12 ++++++++++-- dtc.h | 1 + flattree.c | 30 ++++++++++++++++++++++++++++++ srcpos.c | 20 ++++++++++++++++++-- srcpos.h | 2 ++ 5 files changed, 61 insertions(+), 4 deletions(-)