| Message ID | 20200626002710.110200-2-rajatja@google.com (mailing list archive) |
|---|---|
| State | Changes Requested, archived |
| Headers | show |
| Series | [1/2] pci: Add pci device even if the driver failed to attach | expand |
On Thu, Jun 25, 2020 at 05:27:10PM -0700, Rajat Jain wrote: > Introduce a PCI parameter that disables the automatic attachment of > untrusted devices to their drivers. You didn't document this new api anywhere :(
On Fri, Jun 26, 2020 at 10:27 AM Rajat Jain <rajatja@google.com> wrote: > > Introduce a PCI parameter that disables the automatic attachment of > untrusted devices to their drivers. > > Signed-off-by: Rajat Jain <rajatja@google.com> > --- > Context: > > I set out to implement the approach outlined in > https://lkml.org/lkml/2020/6/9/1331 > https://lkml.org/lkml/2020/6/15/1453 > > But to my surprise, I found that the new hotplugged PCI devices > were getting automatically attached to drivers even though > /sys/bus/pci/drivers_autoprobe was set to 0. > > I realized that the device core's "drivers_autoprobe": > > * only disables the *initial* probe of the device (i.e. from > device_add()). If a subsystem calls device_attach() explicitly > for its devices like PCI subsystem does, the drivers_autoprobe > setting does not matter. The core will attach device to the driver. > This looks like correct semantic behavior to me because PCI is > explicitly calling device_attach(), which is a way to explicitly > ask the core to find and attach a driver for a device. Right, but we're doing using device_attach() largely because the driver core doesn't provide any mechanism for deferring the initial probe. I didn't think there was any deeper reason for it, but while looking I noticed that the initial probe can be async and device_attach() forces probing to be synchronous. That has the side effect of serialising all PCI device probing which might be intentional to avoid device renaming due to the change in probe order. Userspace is better at dealing with device names changing now days, but you might still get some people mad at you for changing it. > 2) Make the drivers_autoprobe property available to PCI to use > (currently it is private to device core). The PCI could use this > to determine whether or not to call device_attach(). This still > leaves the other problem (of not being able to set > drivers_autoprobe via command line open). > > 3) I found the pci_dev->match_driver, which seemed similar to what I > am trying to do, but can't be controlled from userspace. I considered > populating that field based on drivers_autoprobe (still need (2)). > But the problem is that there is the AMD IOMMU driver which is setting > this independently, so setting the match_driver based on > drivers_autoprobe may not be a good idea. Huh, that's pretty weird. Even with that hack you should be able trigger the bug they're working around by removing the IOMMU device in sysfs and doing a rescan. I wouldn't worry much about making match_device user controllable since you would need to work pretty hard for it to be an issue. Oliver
On Thu, Jun 25, 2020 at 05:27:10PM -0700, Rajat Jain wrote: > Introduce a PCI parameter that disables the automatic attachment of > untrusted devices to their drivers. > > Signed-off-by: Rajat Jain <rajatja@google.com> > --- > Context: > > I set out to implement the approach outlined in > https://lkml.org/lkml/2020/6/9/1331 > https://lkml.org/lkml/2020/6/15/1453 > > But to my surprise, I found that the new hotplugged PCI devices > were getting automatically attached to drivers even though > /sys/bus/pci/drivers_autoprobe was set to 0. > > I realized that the device core's "drivers_autoprobe": > > * only disables the *initial* probe of the device (i.e. from > device_add()). If a subsystem calls device_attach() explicitly > for its devices like PCI subsystem does, the drivers_autoprobe > setting does not matter. The core will attach device to the driver. > This looks like correct semantic behavior to me because PCI is > explicitly calling device_attach(), which is a way to explicitly > ask the core to find and attach a driver for a device. > > * "drivers_autoprobe" cannot be controlled at boot time (to restrict > any drivers before userspace comes up). > > The options I considered were: > > 1) Change device_attach() so that it takes into consideration the > drivers_autoprobe property. Not sure if this is semantically correct > thing to do though. If I do this, then the only way a driver can > be attached to the drivers would be via userspace > (/sys/bus/pci/drivers/bind) (Good for our use case though!). This is the correct thing to do here, haven't I been asking you do move this logic into the driver core so that all busses can use it? > 2) Make the drivers_autoprobe property available to PCI to use > (currently it is private to device core). The PCI could use this > to determine whether or not to call device_attach(). This still > leaves the other problem (of not being able to set > drivers_autoprobe via command line open). Ick, command lines are horrible, don't do that if at all possible. On some systems they are not able to be changed which can be good or bad... > 3) I found the pci_dev->match_driver, which seemed similar to what I > am trying to do, but can't be controlled from userspace. I considered > populating that field based on drivers_autoprobe (still need (2)). > But the problem is that there is the AMD IOMMU driver which is setting > this independently, so setting the match_driver based on > drivers_autoprobe may not be a good idea. May be we can populate it > for untrusted devicesi, based on the parameter that I'm introducing? > > 4) This patch was my option 4 that helps fix both the problems for me. I suggest putting some of the above text in the changelog, as it has a lot of good context, while your existing changelog is pretty sparse and does not explain anything... > > drivers/pci/bus.c | 11 ++++++++--- > drivers/pci/pci.c | 9 +++++++++ > drivers/pci/pci.h | 1 + > 3 files changed, 18 insertions(+), 3 deletions(-) > > diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c > index 3cef835b375fd..336aeeb4c4ebf 100644 > --- a/drivers/pci/bus.c > +++ b/drivers/pci/bus.c > @@ -321,9 +321,14 @@ void pci_bus_add_device(struct pci_dev *dev) > pci_bridge_d3_update(dev); > > dev->match_driver = true; > - retval = device_attach(&dev->dev); > - if (retval < 0 && retval != -EPROBE_DEFER) > - pci_warn(dev, "device attach failed (%d)\n", retval); > + > + if (dev->untrusted && pci_dont_attach_untrusted_devs) { > + pci_info(dev, "not attaching untrusted device\n"); > + } else { > + retval = device_attach(&dev->dev); > + if (retval < 0 && retval != -EPROBE_DEFER) > + pci_warn(dev, "device attach failed (%d)\n", retval); > + } > > pci_dev_assign_added(dev, true); > } > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index ce096272f52b1..dec1f9ef27d71 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -127,6 +127,13 @@ static bool pcie_ats_disabled; > /* If set, the PCI config space of each device is printed during boot. */ > bool pci_early_dump; > > +/* > + * If set, the devices with "untrusted" flag shall not be attached automatically > + * Userspace will need to attach them manually: > + * echo <pci device> > /sys/bus/pci/drivers/<driver>/bind > + */ > +bool pci_dont_attach_untrusted_devs; > + > bool pci_ats_disabled(void) > { > return pcie_ats_disabled; > @@ -6522,6 +6529,8 @@ static int __init pci_setup(char *str) > pci_add_flags(PCI_SCAN_ALL_PCIE_DEVS); > } else if (!strncmp(str, "disable_acs_redir=", 18)) { > disable_acs_redir_param = str + 18; > + } else if (!strcmp(str, "dont_attach_untrusted_devs")) { > + pci_dont_attach_untrusted_devs = true; > } else { > pr_err("PCI: Unknown option `%s'\n", str); > } > diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h > index 6d3f758671064..30ffad047d926 100644 > --- a/drivers/pci/pci.h > +++ b/drivers/pci/pci.h > @@ -13,6 +13,7 @@ > > extern const unsigned char pcie_link_speed[]; > extern bool pci_early_dump; > +extern bool pci_dont_attach_untrusted_devs; > > bool pcie_cap_has_lnkctl(const struct pci_dev *dev); > bool pcie_cap_has_rtctl(const struct pci_dev *dev); > -- > 2.27.0.212.ge8ba1cc988-goog > What happened to the split of "trust" and "internal/external" logic that we discussed before? This seems to ignore all of that and go straight to some form of "we know what we trust, so all is fine!". It's not obvious what this is really doing here at all, sorry... greg k-h
Hello, Thanks for taking a look. On Fri, Jun 26, 2020 at 7:18 AM Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > > On Thu, Jun 25, 2020 at 05:27:10PM -0700, Rajat Jain wrote: > > Introduce a PCI parameter that disables the automatic attachment of > > untrusted devices to their drivers. > > > > Signed-off-by: Rajat Jain <rajatja@google.com> > > --- > > Context: > > > > I set out to implement the approach outlined in > > https://lkml.org/lkml/2020/6/9/1331 > > https://lkml.org/lkml/2020/6/15/1453 > > > > But to my surprise, I found that the new hotplugged PCI devices > > were getting automatically attached to drivers even though > > /sys/bus/pci/drivers_autoprobe was set to 0. > > > > I realized that the device core's "drivers_autoprobe": > > > > * only disables the *initial* probe of the device (i.e. from > > device_add()). If a subsystem calls device_attach() explicitly > > for its devices like PCI subsystem does, the drivers_autoprobe > > setting does not matter. The core will attach device to the driver. > > This looks like correct semantic behavior to me because PCI is > > explicitly calling device_attach(), which is a way to explicitly > > ask the core to find and attach a driver for a device. > > > > * "drivers_autoprobe" cannot be controlled at boot time (to restrict > > any drivers before userspace comes up). > > > > The options I considered were: > > > > 1) Change device_attach() so that it takes into consideration the > > drivers_autoprobe property. Not sure if this is semantically correct > > thing to do though. If I do this, then the only way a driver can > > be attached to the drivers would be via userspace > > (/sys/bus/pci/drivers/bind) (Good for our use case though!). > > This is the correct thing to do here, haven't I been asking you do move > this logic into the driver core so that all busses can use it? (please see below) > > > 2) Make the drivers_autoprobe property available to PCI to use > > (currently it is private to device core). The PCI could use this > > to determine whether or not to call device_attach(). This still > > leaves the other problem (of not being able to set > > drivers_autoprobe via command line open). > > Ick, command lines are horrible, don't do that if at all possible. On > some systems they are not able to be changed which can be good or bad... (please see below) > > > 3) I found the pci_dev->match_driver, which seemed similar to what I > > am trying to do, but can't be controlled from userspace. I considered > > populating that field based on drivers_autoprobe (still need (2)). > > But the problem is that there is the AMD IOMMU driver which is setting > > this independently, so setting the match_driver based on > > drivers_autoprobe may not be a good idea. May be we can populate it > > for untrusted devicesi, based on the parameter that I'm introducing? > > > > 4) This patch was my option 4 that helps fix both the problems for me. > > I suggest putting some of the above text in the changelog, as it has a > lot of good context, while your existing changelog is pretty sparse and > does not explain anything... Will do. > > > > > > drivers/pci/bus.c | 11 ++++++++--- > > drivers/pci/pci.c | 9 +++++++++ > > drivers/pci/pci.h | 1 + > > 3 files changed, 18 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c > > index 3cef835b375fd..336aeeb4c4ebf 100644 > > --- a/drivers/pci/bus.c > > +++ b/drivers/pci/bus.c > > @@ -321,9 +321,14 @@ void pci_bus_add_device(struct pci_dev *dev) > > pci_bridge_d3_update(dev); > > > > dev->match_driver = true; > > - retval = device_attach(&dev->dev); > > - if (retval < 0 && retval != -EPROBE_DEFER) > > - pci_warn(dev, "device attach failed (%d)\n", retval); > > + > > + if (dev->untrusted && pci_dont_attach_untrusted_devs) { > > + pci_info(dev, "not attaching untrusted device\n"); > > + } else { > > + retval = device_attach(&dev->dev); > > + if (retval < 0 && retval != -EPROBE_DEFER) > > + pci_warn(dev, "device attach failed (%d)\n", retval); > > + } > > > > pci_dev_assign_added(dev, true); > > } > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > > index ce096272f52b1..dec1f9ef27d71 100644 > > --- a/drivers/pci/pci.c > > +++ b/drivers/pci/pci.c > > @@ -127,6 +127,13 @@ static bool pcie_ats_disabled; > > /* If set, the PCI config space of each device is printed during boot. */ > > bool pci_early_dump; > > > > +/* > > + * If set, the devices with "untrusted" flag shall not be attached automatically > > + * Userspace will need to attach them manually: > > + * echo <pci device> > /sys/bus/pci/drivers/<driver>/bind > > + */ > > +bool pci_dont_attach_untrusted_devs; > > + > > bool pci_ats_disabled(void) > > { > > return pcie_ats_disabled; > > @@ -6522,6 +6529,8 @@ static int __init pci_setup(char *str) > > pci_add_flags(PCI_SCAN_ALL_PCIE_DEVS); > > } else if (!strncmp(str, "disable_acs_redir=", 18)) { > > disable_acs_redir_param = str + 18; > > + } else if (!strcmp(str, "dont_attach_untrusted_devs")) { > > + pci_dont_attach_untrusted_devs = true; > > } else { > > pr_err("PCI: Unknown option `%s'\n", str); > > } > > diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h > > index 6d3f758671064..30ffad047d926 100644 > > --- a/drivers/pci/pci.h > > +++ b/drivers/pci/pci.h > > @@ -13,6 +13,7 @@ > > > > extern const unsigned char pcie_link_speed[]; > > extern bool pci_early_dump; > > +extern bool pci_dont_attach_untrusted_devs; > > > > bool pcie_cap_has_lnkctl(const struct pci_dev *dev); > > bool pcie_cap_has_rtctl(const struct pci_dev *dev); > > -- > > 2.27.0.212.ge8ba1cc988-goog > > > > What happened to the split of "trust" and "internal/external" logic that > we discussed before? a) I think what was decided was introducing a device core "location" property that can be exposed to userspace to help it to decide whether or not to attach a driver to a device. Yes, that is still the plan. (Mild sidenote: userspace may not need to distinguish between internal and external devices if it can assume that no internal PCI devices will show up after "echo 0 > /sys/bus/pci/drivers_autoprobe". But nevertheless...) b) Note that even with (a) in place, we still need a parameter that can ensure that drivers are not bound to external devices at boot, *before* userspace gets a chance to disable "drivers_autoprobe". https://lkml.org/lkml/2020/6/15/1453 Is it OK to add such a parameter in device core? Thanks, Rajat > This seems to ignore all of that and go straight > to some form of "we know what we trust, so all is fine!". > > It's not obvious what this is really doing here at all, sorry... > > greg k-h
On Fri, Jun 26, 2020 at 11:53:34AM -0700, Rajat Jain wrote: > a) I think what was decided was introducing a device core "location" > property that can be exposed to userspace to help it to decide whether > or not to attach a driver to a device. Yes, that is still the plan. Great, but this patch ignores that and starts to add policy :( > (Mild sidenote: userspace may not need to distinguish between internal > and external devices if it can assume that no internal PCI devices > will show up after "echo 0 > /sys/bus/pci/drivers_autoprobe". But > nevertheless...) It can not assume that. > b) Note that even with (a) in place, we still need a parameter that > can ensure that drivers are not bound to external devices at boot, > *before* userspace gets a chance to disable "drivers_autoprobe". Why do you think you need that? I kind of doubt you really want this, but ick, if you really do, make it a policy decision that you bake into the kernel as a build option, so that no one else has to use it :) > https://lkml.org/lkml/2020/6/15/1453 Ick, please use lore.kernel.org, we don't control lkml.org and it's not all that reliable. > Is it OK to add such a parameter in device core? You don't have internal/external/wherever in the driver core yet, so don't start adding policy before you get that... thanks, greg k-h
diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index 3cef835b375fd..336aeeb4c4ebf 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -321,9 +321,14 @@ void pci_bus_add_device(struct pci_dev *dev) pci_bridge_d3_update(dev); dev->match_driver = true; - retval = device_attach(&dev->dev); - if (retval < 0 && retval != -EPROBE_DEFER) - pci_warn(dev, "device attach failed (%d)\n", retval); + + if (dev->untrusted && pci_dont_attach_untrusted_devs) { + pci_info(dev, "not attaching untrusted device\n"); + } else { + retval = device_attach(&dev->dev); + if (retval < 0 && retval != -EPROBE_DEFER) + pci_warn(dev, "device attach failed (%d)\n", retval); + } pci_dev_assign_added(dev, true); } diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index ce096272f52b1..dec1f9ef27d71 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -127,6 +127,13 @@ static bool pcie_ats_disabled; /* If set, the PCI config space of each device is printed during boot. */ bool pci_early_dump; +/* + * If set, the devices with "untrusted" flag shall not be attached automatically + * Userspace will need to attach them manually: + * echo <pci device> > /sys/bus/pci/drivers/<driver>/bind + */ +bool pci_dont_attach_untrusted_devs; + bool pci_ats_disabled(void) { return pcie_ats_disabled; @@ -6522,6 +6529,8 @@ static int __init pci_setup(char *str) pci_add_flags(PCI_SCAN_ALL_PCIE_DEVS); } else if (!strncmp(str, "disable_acs_redir=", 18)) { disable_acs_redir_param = str + 18; + } else if (!strcmp(str, "dont_attach_untrusted_devs")) { + pci_dont_attach_untrusted_devs = true; } else { pr_err("PCI: Unknown option `%s'\n", str); } diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 6d3f758671064..30ffad047d926 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -13,6 +13,7 @@ extern const unsigned char pcie_link_speed[]; extern bool pci_early_dump; +extern bool pci_dont_attach_untrusted_devs; bool pcie_cap_has_lnkctl(const struct pci_dev *dev); bool pcie_cap_has_rtctl(const struct pci_dev *dev);
Introduce a PCI parameter that disables the automatic attachment of untrusted devices to their drivers. Signed-off-by: Rajat Jain <rajatja@google.com> --- Context: I set out to implement the approach outlined in https://lkml.org/lkml/2020/6/9/1331 https://lkml.org/lkml/2020/6/15/1453 But to my surprise, I found that the new hotplugged PCI devices were getting automatically attached to drivers even though /sys/bus/pci/drivers_autoprobe was set to 0. I realized that the device core's "drivers_autoprobe": * only disables the *initial* probe of the device (i.e. from device_add()). If a subsystem calls device_attach() explicitly for its devices like PCI subsystem does, the drivers_autoprobe setting does not matter. The core will attach device to the driver. This looks like correct semantic behavior to me because PCI is explicitly calling device_attach(), which is a way to explicitly ask the core to find and attach a driver for a device. * "drivers_autoprobe" cannot be controlled at boot time (to restrict any drivers before userspace comes up). The options I considered were: 1) Change device_attach() so that it takes into consideration the drivers_autoprobe property. Not sure if this is semantically correct thing to do though. If I do this, then the only way a driver can be attached to the drivers would be via userspace (/sys/bus/pci/drivers/bind) (Good for our use case though!). 2) Make the drivers_autoprobe property available to PCI to use (currently it is private to device core). The PCI could use this to determine whether or not to call device_attach(). This still leaves the other problem (of not being able to set drivers_autoprobe via command line open). 3) I found the pci_dev->match_driver, which seemed similar to what I am trying to do, but can't be controlled from userspace. I considered populating that field based on drivers_autoprobe (still need (2)). But the problem is that there is the AMD IOMMU driver which is setting this independently, so setting the match_driver based on drivers_autoprobe may not be a good idea. May be we can populate it for untrusted devicesi, based on the parameter that I'm introducing? 4) This patch was my option 4 that helps fix both the problems for me. drivers/pci/bus.c | 11 ++++++++--- drivers/pci/pci.c | 9 +++++++++ drivers/pci/pci.h | 1 + 3 files changed, 18 insertions(+), 3 deletions(-)