| Message ID | 20181112160628.86620-5-mika.westerberg@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | PCI / iommu / thunderbolt: IOMMU based DMA protection | expand |
> -----Original Message----- > From: Mika Westerberg <mika.westerberg@linux.intel.com> > Sent: Monday, November 12, 2018 10:06 AM > To: iommu@lists.linux-foundation.org > Cc: Joerg Roedel; David Woodhouse; Lu Baolu; Ashok Raj; Bjorn Helgaas; Rafael J. > Wysocki; Jacob jun Pan; Andreas Noever; Michael Jamet; Yehezkel Bernat; Lukas > Wunner; Christian Kellner; Limonciello, Mario; Anthony Wong; Mika Westerberg; > linux-acpi@vger.kernel.org; linux-pci@vger.kernel.org; linux- > kernel@vger.kernel.org > Subject: [PATCH 4/4] thunderbolt: Export IOMMU based DMA protection support > to userspace > > > > > Recent systems shipping with Windows 10 version 1803 or later may > support a feature called Kernel DMA protection [1]. In practice this > means that Thunderbolt connected devices are placed behind an IOMMU > during the whole time it is connected (including during boot) making > Thunderbolt security levels redundant. Some of these systems still have > Thunderbolt security level set to "user" in order to support OS > downgrade (the older version of the OS might not support IOMMU based DMA > protection so connecting a device still relies on user approval then). > > Export this information to userspace by introducing a new sysfs > attribute (iommu_dma_protection). Based on it userspace tools can make > more accurate decision whether or not authorize the connected device. > > In addition update Thunderbolt documentation regarding IOMMU based DMA > protection. > > [1] https://docs.microsoft.com/en-us/windows/security/information- > protection/kernel-dma-protection-for-thunderbolt > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > --- > .../ABI/testing/sysfs-bus-thunderbolt | 9 ++++++++ > Documentation/admin-guide/thunderbolt.rst | 23 +++++++++++++++++++ > drivers/thunderbolt/domain.c | 17 ++++++++++++++ > 3 files changed, 49 insertions(+) > > diff --git a/Documentation/ABI/testing/sysfs-bus-thunderbolt > b/Documentation/ABI/testing/sysfs-bus-thunderbolt > index 151584a1f950..b21fba14689b 100644 > --- a/Documentation/ABI/testing/sysfs-bus-thunderbolt > +++ b/Documentation/ABI/testing/sysfs-bus-thunderbolt > @@ -21,6 +21,15 @@ Description: Holds a comma separated list of device > unique_ids that > If a device is authorized automatically during boot its > boot attribute is set to 1. > > +What: /sys/bus/thunderbolt/devices/.../domainX/iommu_dma_protection > +Date: Mar 2019 > +KernelVersion: 4.21 > +Contact: thunderbolt-software@lists.01.org > +Description: This attribute tells whether the system uses IOMMU > + for DMA protection. Value of 1 means IOMMU is used 0 means > + it is not (DMA protection is solely based on Thunderbolt > + security levels). > + > What: /sys/bus/thunderbolt/devices/.../domainX/security > Date: Sep 2017 > KernelVersion: 4.13 > diff --git a/Documentation/admin-guide/thunderbolt.rst b/Documentation/admin- > guide/thunderbolt.rst > index 35fccba6a9a6..ccac2596a49f 100644 > --- a/Documentation/admin-guide/thunderbolt.rst > +++ b/Documentation/admin-guide/thunderbolt.rst > @@ -133,6 +133,29 @@ If the user still wants to connect the device they can > either approve > the device without a key or write a new key and write 1 to the > ``authorized`` file to get the new key stored on the device NVM. > > +DMA protection utilizing IOMMU > +------------------------------ > +Recent systems shipping with Windows 10 version 1803 or later may support a > +feature called `Kernel DMA Protection for Thunderbolt 3`_. This means that Keep in mind there will be systems that ship with Linux that enable this feature too ;) It might be better to make it time frame and platform firmware oriented as it's entirely possible for an OEM to have a field firmware upgrade that may enable this functionality from the platform and allow an end user to upgrade to a sufficiently protected kernel or Windows OS to take advantage of it. > +Thunderbolt security is handled by an IOMMU so connected devices cannot > +access memory regions outside of what is allocated for them by drivers. > +When Linux is running on such system it automatically enables IOMMU if not > +enabled by the user already. These systems can be identified by reading > +``1`` from ``/sys/bus/thunderbolt/devices/domainX/iommu_dma_protection`` > +attribute. > + > +The driver does not do anything special in this case but because DMA > +protection is handled by the IOMMU, security levels (if set) are > +redundant. For this reason some systems ship with security level set to > +``none``. Other systems have security level set to ``user`` in order to > +support downgrade to older Windows, so users who want to automatically > +authorize devices when IOMMU DMA protection is enabled can use the > +following ``udev`` rule:: > + > + ACTION=="add", SUBSYSTEM=="thunderbolt", > ATTRS{iommu_dma_protection}=="1", ATTR{authorized}=="0", > ATTR{authorized}="1" > + > +.. _Kernel DMA Protection for Thunderbolt 3: https://docs.microsoft.com/en- > us/windows/security/information-protection/kernel-dma-protection-for- > thunderbolt > + > Upgrading NVM on Thunderbolt device or host > ------------------------------------------- > Since most of the functionality is handled in firmware running on a > diff --git a/drivers/thunderbolt/domain.c b/drivers/thunderbolt/domain.c > index 93e562f18d40..7416bdbd8576 100644 > --- a/drivers/thunderbolt/domain.c > +++ b/drivers/thunderbolt/domain.c > @@ -7,7 +7,9 @@ > */ > > #include <linux/device.h> > +#include <linux/dmar.h> > #include <linux/idr.h> > +#include <linux/iommu.h> > #include <linux/module.h> > #include <linux/pm_runtime.h> > #include <linux/slab.h> > @@ -236,6 +238,20 @@ static ssize_t boot_acl_store(struct device *dev, struct > device_attribute *attr, > } > static DEVICE_ATTR_RW(boot_acl); > > +static ssize_t iommu_dma_protection_show(struct device *dev, > + struct device_attribute *attr, > + char *buf) > +{ > + /* > + * Kernel DMA protection is a feature where Thunderbolt security is > + * handled natively using IOMMU. It is enabled when IOMMU is > + * enabled and ACPI DMAR table has DMAR_PLATFORM_OPT_IN set. > + */ > + return sprintf(buf, "%d\n", > + iommu_present(&pci_bus_type) && dmar_platform_optin()); > +} > +static DEVICE_ATTR_RO(iommu_dma_protection); > + > static ssize_t security_show(struct device *dev, struct device_attribute *attr, > char *buf) > { > @@ -251,6 +267,7 @@ static DEVICE_ATTR_RO(security); > > static struct attribute *domain_attrs[] = { > &dev_attr_boot_acl.attr, > + &dev_attr_iommu_dma_protection.attr, > &dev_attr_security.attr, > NULL, > }; > -- > 2.19.1
On Mon, Nov 12, 2018 at 6:06 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > Recent systems shipping with Windows 10 version 1803 or later may > support a feature called Kernel DMA protection [1]. In practice this > means that Thunderbolt connected devices are placed behind an IOMMU > during the whole time it is connected (including during boot) making > Thunderbolt security levels redundant. Some of these systems still have > Thunderbolt security level set to "user" in order to support OS > downgrade (the older version of the OS might not support IOMMU based DMA > protection so connecting a device still relies on user approval then). > > Export this information to userspace by introducing a new sysfs > attribute (iommu_dma_protection). Based on it userspace tools can make > more accurate decision whether or not authorize the connected device. > > In addition update Thunderbolt documentation regarding IOMMU based DMA > protection. > > [1] https://docs.microsoft.com/en-us/windows/security/information-protection/kernel-dma-protection-for-thunderbolt > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > --- I can't comment on the IOMMU side, but the Thunderbolt side looks good to me. Just one point: Have you considered the option to add this property per (TBT?) device? If the kernel may decide to enable/disable the IOMMU or AST per device, maybe it should be on this level. Or maybe the IOMMU decision isn't going to change (it's system-wide) and the AST decision will be communicated per device by a new sysfs attribute anyway, if needed?
On Mon, Nov 12, 2018 at 04:22:25PM +0000, Mario.Limonciello@dell.com wrote: > > +DMA protection utilizing IOMMU > > +------------------------------ > > +Recent systems shipping with Windows 10 version 1803 or later may support a > > +feature called `Kernel DMA Protection for Thunderbolt 3`_. This means that > > Keep in mind there will be systems that ship with Linux that enable this feature too ;) Yes, you are absolutely right. I sometimes forgot that fact ;-) > It might be better to make it time frame and platform firmware oriented as it's > entirely possible for an OEM to have a field firmware upgrade that may enable this > functionality from the platform and allow an end user to upgrade to a sufficiently > protected kernel or Windows OS to take advantage of it. I agree. I will update this in the next version.
On Mon, Nov 12, 2018 at 06:59:02PM +0200, Yehezkel Bernat wrote: > On Mon, Nov 12, 2018 at 6:06 PM Mika Westerberg > <mika.westerberg@linux.intel.com> wrote: > > > > Recent systems shipping with Windows 10 version 1803 or later may > > support a feature called Kernel DMA protection [1]. In practice this > > means that Thunderbolt connected devices are placed behind an IOMMU > > during the whole time it is connected (including during boot) making > > Thunderbolt security levels redundant. Some of these systems still have > > Thunderbolt security level set to "user" in order to support OS > > downgrade (the older version of the OS might not support IOMMU based DMA > > protection so connecting a device still relies on user approval then). > > > > Export this information to userspace by introducing a new sysfs > > attribute (iommu_dma_protection). Based on it userspace tools can make > > more accurate decision whether or not authorize the connected device. > > > > In addition update Thunderbolt documentation regarding IOMMU based DMA > > protection. > > > > [1] https://docs.microsoft.com/en-us/windows/security/information-protection/kernel-dma-protection-for-thunderbolt > > > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > --- > > I can't comment on the IOMMU side, but the Thunderbolt side looks good to me. Thanks! > Just one point: > Have you considered the option to add this property per (TBT?) device? No. ;-) You mean that one device uses security levels and another IOMMU? I don't think it is possible without having some sort of table in the IOMMU driver telling which devices it needs identity map and which not. Also not sure what would be the benefit? > If the kernel may decide to enable/disable the IOMMU or AST per device, maybe > it should be on this level. > Or maybe the IOMMU decision isn't going to change (it's system-wide) and the AST > decision will be communicated per device by a new sysfs attribute anyway, if > needed? Not sure what you mean by "AST"? The IOMMU decision is pretty much system-wide.
On Tue, Nov 13, 2018 at 12:56 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > > Just one point: > > Have you considered the option to add this property per (TBT?) device? > > No. ;-) > > You mean that one device uses security levels and another IOMMU? I don't > think it is possible without having some sort of table in the IOMMU > driver telling which devices it needs identity map and which not. Also > not sure what would be the benefit? For performance, of course. If some devices are considered safe (maybe a list communicated by platform firmware), the kernel may decide to configure them to passthrough the IOMMU (I think I remember there is such an option, but maybe I'm wrong.) > > If the kernel may decide to enable/disable the IOMMU or AST per device, maybe > > it should be on this level. > > Or maybe the IOMMU decision isn't going to change (it's system-wide) and the AST > > decision will be communicated per device by a new sysfs attribute anyway, if > > needed? > > Not sure what you mean by "AST"? The IOMMU decision is pretty much > system-wide. Sorry, I meant ATS, Address Translation Service, mentioned in patch 3 in this series, and possibly be enabled for some devices for performance, as mentioned there. So if needed, this will be another attribute, and definitely per-device, isn't it?
On Tue, Nov 13, 2018 at 01:13:31PM +0200, Yehezkel Bernat wrote: > On Tue, Nov 13, 2018 at 12:56 PM Mika Westerberg > <mika.westerberg@linux.intel.com> wrote: > > > > > Just one point: > > > Have you considered the option to add this property per (TBT?) device? > > > > No. ;-) > > > > You mean that one device uses security levels and another IOMMU? I don't > > think it is possible without having some sort of table in the IOMMU > > driver telling which devices it needs identity map and which not. Also > > not sure what would be the benefit? > > For performance, of course. If some devices are considered safe (maybe a list > communicated by platform firmware), the kernel may decide to configure them to > passthrough the IOMMU (I think I remember there is such an option, but maybe I'm > wrong.) At least I'm not aware of such an option. Windows for example enables IOMMU for everything and I think macOS does the same. In Linux (with these patches) we put all internal devices already passthrough mode so things like internal graphics should not be affected. eGPUs are different thing, though. > > > If the kernel may decide to enable/disable the IOMMU or AST per device, maybe > > > it should be on this level. > > > Or maybe the IOMMU decision isn't going to change (it's system-wide) and the AST > > > decision will be communicated per device by a new sysfs attribute anyway, if > > > needed? > > > > Not sure what you mean by "AST"? The IOMMU decision is pretty much > > system-wide. > > Sorry, I meant ATS, Address Translation Service, mentioned in patch 3 in this > series, and possibly be enabled for some devices for performance, as mentioned > there. Thanks for clarifying :) > So if needed, this will be another attribute, and definitely > per-device, isn't it? Yes, we may want to add a way enable ATS for external devices (perhaps global option or per-device attribute) if it turns out to cause performance issues. However, I think it can be done later if needed. I have not seen a single device actually supporting ATS (with the exception of the "hacked" one in the linked thesis of patch 3/4 ;-))
On Tue, Nov 13, 2018 at 1:40 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > On Tue, Nov 13, 2018 at 01:13:31PM +0200, Yehezkel Bernat wrote: > > On Tue, Nov 13, 2018 at 12:56 PM Mika Westerberg > > <mika.westerberg@linux.intel.com> wrote: > > > > > > > Just one point: > > > > Have you considered the option to add this property per (TBT?) device? > > > > > > No. ;-) > > > > > > You mean that one device uses security levels and another IOMMU? I don't > > > think it is possible without having some sort of table in the IOMMU > > > driver telling which devices it needs identity map and which not. Also > > > not sure what would be the benefit? > > > > For performance, of course. If some devices are considered safe (maybe a list > > communicated by platform firmware), the kernel may decide to configure them to > > passthrough the IOMMU (I think I remember there is such an option, but maybe I'm > > wrong.) > > At least I'm not aware of such an option. Windows for example enables > IOMMU for everything and I think macOS does the same. In Linux (with > these patches) we put all internal devices already passthrough mode so > things like internal graphics should not be affected. eGPUs are > different thing, though. So your point here is "currently we do the IOMMU decisions system-wide; we can always add a per-device attribute if needed"? Fair enough. So for this patch, Reviewed-by: Yehezkel Bernat <YehezkelShB@gmail.com>
On Tue, Nov 13, 2018 at 5:20 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > On Tue, Nov 13, 2018 at 04:42:58PM +0200, Yehezkel Bernat wrote: > > On Tue, Nov 13, 2018 at 1:40 PM Mika Westerberg > > <mika.westerberg@linux.intel.com> wrote: > > > > > > On Tue, Nov 13, 2018 at 01:13:31PM +0200, Yehezkel Bernat wrote: > > > > On Tue, Nov 13, 2018 at 12:56 PM Mika Westerberg > > > > <mika.westerberg@linux.intel.com> wrote: > > > > > > > > > > > Just one point: > > > > > > Have you considered the option to add this property per (TBT?) device? > > > > > > > > > > No. ;-) > > > > > > > > > > You mean that one device uses security levels and another IOMMU? I don't > > > > > think it is possible without having some sort of table in the IOMMU > > > > > driver telling which devices it needs identity map and which not. Also > > > > > not sure what would be the benefit? > > > > > > > > For performance, of course. If some devices are considered safe (maybe a list > > > > communicated by platform firmware), the kernel may decide to configure them to > > > > passthrough the IOMMU (I think I remember there is such an option, but maybe I'm > > > > wrong.) > > > > > > At least I'm not aware of such an option. Windows for example enables > > > IOMMU for everything and I think macOS does the same. In Linux (with > > > these patches) we put all internal devices already passthrough mode so > > > things like internal graphics should not be affected. eGPUs are > > > different thing, though. > > > > So your point here is "currently we do the IOMMU decisions system-wide; we can > > always add a per-device attribute if needed"? > > Well, let me elaborate a bit :) I think it is possible to put certain > devices into IOMMU passthrough mode even if they are "external" but I'm > not that familiar with the guts of Intel IOMMU (Baolu or Ashok are the > experts). Assuming it is possible we could have a table or similar in > the kernel that can be used to identify devices that are allowed to be > in passthrough mode. > > I'm not sure if it can be per-device sysfs attribute because at the time > the PCIe device is enumerated it is already put to its IOMMU group. Good point. But I thought about per-TBT-device decision. If the platform is configured for IOMMU+"user" security level, while approving the device the user may want to set also in which IOMMU group to put all the PCIe devices connected to it. The same goes if kernel is supposed to auto-approve such devices based on an internal table. The point is that we can think on a configuration where the devices aren't tunneled yet and the decision about IOMMU can still be changed. As you mentioned this isn't the common configuration anyway, so it probably doesn't worth all this hassle.
On Tue, Nov 13, 2018 at 05:38:53PM +0200, Yehezkel Bernat wrote: > Good point. But I thought about per-TBT-device decision. If the platform is > configured for IOMMU+"user" security level, while approving the device the user > may want to set also in which IOMMU group to put all the PCIe devices connected > to it. The same goes if kernel is supposed to auto-approve such devices based on > an internal table. The point is that we can think on a configuration where the > devices aren't tunneled yet and the decision about IOMMU can still be changed. Right, some of these systems have security level set to "user" so there we could have a way to put the device into passthrough mode before it appears on the PCIe bus. That would require some sort of API on the IOMMU side, though. > As you mentioned this isn't the common configuration anyway, so it probably > doesn't worth all this hassle. AFAIK mixing the two is not something they are going to be supporting in Windows so I would not expect it to be common. I think the ultimate goal is to move away from security levels towards IOMMU DMA protection so in future I would expect more and more systems with IOMMU enabled + security level set to "none". So I agree with you that it probably is not worth doing at least without having more data about real performance issues around this. ;-)
diff --git a/Documentation/ABI/testing/sysfs-bus-thunderbolt b/Documentation/ABI/testing/sysfs-bus-thunderbolt index 151584a1f950..b21fba14689b 100644 --- a/Documentation/ABI/testing/sysfs-bus-thunderbolt +++ b/Documentation/ABI/testing/sysfs-bus-thunderbolt @@ -21,6 +21,15 @@ Description: Holds a comma separated list of device unique_ids that If a device is authorized automatically during boot its boot attribute is set to 1. +What: /sys/bus/thunderbolt/devices/.../domainX/iommu_dma_protection +Date: Mar 2019 +KernelVersion: 4.21 +Contact: thunderbolt-software@lists.01.org +Description: This attribute tells whether the system uses IOMMU + for DMA protection. Value of 1 means IOMMU is used 0 means + it is not (DMA protection is solely based on Thunderbolt + security levels). + What: /sys/bus/thunderbolt/devices/.../domainX/security Date: Sep 2017 KernelVersion: 4.13 diff --git a/Documentation/admin-guide/thunderbolt.rst b/Documentation/admin-guide/thunderbolt.rst index 35fccba6a9a6..ccac2596a49f 100644 --- a/Documentation/admin-guide/thunderbolt.rst +++ b/Documentation/admin-guide/thunderbolt.rst @@ -133,6 +133,29 @@ If the user still wants to connect the device they can either approve the device without a key or write a new key and write 1 to the ``authorized`` file to get the new key stored on the device NVM. +DMA protection utilizing IOMMU +------------------------------ +Recent systems shipping with Windows 10 version 1803 or later may support a +feature called `Kernel DMA Protection for Thunderbolt 3`_. This means that +Thunderbolt security is handled by an IOMMU so connected devices cannot +access memory regions outside of what is allocated for them by drivers. +When Linux is running on such system it automatically enables IOMMU if not +enabled by the user already. These systems can be identified by reading +``1`` from ``/sys/bus/thunderbolt/devices/domainX/iommu_dma_protection`` +attribute. + +The driver does not do anything special in this case but because DMA +protection is handled by the IOMMU, security levels (if set) are +redundant. For this reason some systems ship with security level set to +``none``. Other systems have security level set to ``user`` in order to +support downgrade to older Windows, so users who want to automatically +authorize devices when IOMMU DMA protection is enabled can use the +following ``udev`` rule:: + + ACTION=="add", SUBSYSTEM=="thunderbolt", ATTRS{iommu_dma_protection}=="1", ATTR{authorized}=="0", ATTR{authorized}="1" + +.. _Kernel DMA Protection for Thunderbolt 3: https://docs.microsoft.com/en-us/windows/security/information-protection/kernel-dma-protection-for-thunderbolt + Upgrading NVM on Thunderbolt device or host ------------------------------------------- Since most of the functionality is handled in firmware running on a diff --git a/drivers/thunderbolt/domain.c b/drivers/thunderbolt/domain.c index 93e562f18d40..7416bdbd8576 100644 --- a/drivers/thunderbolt/domain.c +++ b/drivers/thunderbolt/domain.c @@ -7,7 +7,9 @@ */ #include <linux/device.h> +#include <linux/dmar.h> #include <linux/idr.h> +#include <linux/iommu.h> #include <linux/module.h> #include <linux/pm_runtime.h> #include <linux/slab.h> @@ -236,6 +238,20 @@ static ssize_t boot_acl_store(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RW(boot_acl); +static ssize_t iommu_dma_protection_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + /* + * Kernel DMA protection is a feature where Thunderbolt security is + * handled natively using IOMMU. It is enabled when IOMMU is + * enabled and ACPI DMAR table has DMAR_PLATFORM_OPT_IN set. + */ + return sprintf(buf, "%d\n", + iommu_present(&pci_bus_type) && dmar_platform_optin()); +} +static DEVICE_ATTR_RO(iommu_dma_protection); + static ssize_t security_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -251,6 +267,7 @@ static DEVICE_ATTR_RO(security); static struct attribute *domain_attrs[] = { &dev_attr_boot_acl.attr, + &dev_attr_iommu_dma_protection.attr, &dev_attr_security.attr, NULL, };
Recent systems shipping with Windows 10 version 1803 or later may support a feature called Kernel DMA protection [1]. In practice this means that Thunderbolt connected devices are placed behind an IOMMU during the whole time it is connected (including during boot) making Thunderbolt security levels redundant. Some of these systems still have Thunderbolt security level set to "user" in order to support OS downgrade (the older version of the OS might not support IOMMU based DMA protection so connecting a device still relies on user approval then). Export this information to userspace by introducing a new sysfs attribute (iommu_dma_protection). Based on it userspace tools can make more accurate decision whether or not authorize the connected device. In addition update Thunderbolt documentation regarding IOMMU based DMA protection. [1] https://docs.microsoft.com/en-us/windows/security/information-protection/kernel-dma-protection-for-thunderbolt Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> --- .../ABI/testing/sysfs-bus-thunderbolt | 9 ++++++++ Documentation/admin-guide/thunderbolt.rst | 23 +++++++++++++++++++ drivers/thunderbolt/domain.c | 17 ++++++++++++++ 3 files changed, 49 insertions(+)