Message ID | 20240528120501.3382554-1-arnd@kernel.org (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | vfio/qat: add PCI_IOV dependency | expand |
> From: Arnd Bergmann <arnd@kernel.org> > Sent: Tuesday, May 28, 2024 8:05 PM > > From: Arnd Bergmann <arnd@arndb.de> > > The newly added driver depends on the crypto driver, but it uses exported > symbols that are only available when IOV is also turned on: > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_pci_open_device': > main.c:(.text+0xd7): undefined reference to `qat_vfmig_open' > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_pci_release_dev': > main.c:(.text+0x122): undefined reference to `qat_vfmig_cleanup' > x86_64-linux-ld: main.c:(.text+0x12d): undefined reference to > `qat_vfmig_destroy' > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_resume_write': > main.c:(.text+0x308): undefined reference to `qat_vfmig_load_setup' > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_save_device_data': > main.c:(.text+0x64c): undefined reference to `qat_vfmig_save_state' > x86_64-linux-ld: main.c:(.text+0x677): undefined reference to > `qat_vfmig_save_setup' > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_pci_aer_reset_done': > main.c:(.text+0x82d): undefined reference to `qat_vfmig_reset' > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_pci_close_device': > main.c:(.text+0x862): undefined reference to `qat_vfmig_close' > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > `qat_vf_pci_set_device_state': > main.c:(.text+0x9af): undefined reference to `qat_vfmig_suspend' > x86_64-linux-ld: main.c:(.text+0xa14): undefined reference to > `qat_vfmig_save_state' > x86_64-linux-ld: main.c:(.text+0xb37): undefined reference to > `qat_vfmig_resume' > x86_64-linux-ld: main.c:(.text+0xbc7): undefined reference to > `qat_vfmig_load_state' at a glance those undefined symbols don't use any symbol under IOV. They are just wrappers to certain callbacks registered by by respective qat drivers which support migration. Probably they'd better be moved out of CONFIG_PCI_IOV in "drivers/crypto/intel/qat/qat_common/Makefile" to remove this dependency in vfio variant driver. > > Add this as a second dependency. > > Fixes: bb208810b1ab ("vfio/qat: Add vfio_pci driver for Intel QAT SR-IOV VF > devices") > Signed-off-by: Arnd Bergmann <arnd@arndb.de> > --- > drivers/vfio/pci/qat/Kconfig | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/vfio/pci/qat/Kconfig b/drivers/vfio/pci/qat/Kconfig > index bf52cfa4b595..fae9d6cb8ccb 100644 > --- a/drivers/vfio/pci/qat/Kconfig > +++ b/drivers/vfio/pci/qat/Kconfig > @@ -1,8 +1,9 @@ > # SPDX-License-Identifier: GPL-2.0-only > config QAT_VFIO_PCI > tristate "VFIO support for QAT VF PCI devices" > - select VFIO_PCI_CORE > depends on CRYPTO_DEV_QAT_4XXX > + depends on PCI_IOV > + select VFIO_PCI_CORE > help > This provides migration support for Intel(R) QAT Virtual Function > using the VFIO framework. > -- > 2.39.2
On Wednesday, May 29, 2024 10:25 AM, Tian, Kevin <kevin.tian@intel.com> > To: Arnd Bergmann <arnd@kernel.org>; Zeng, Xin <xin.zeng@intel.com>; > Cabiddu, Giovanni <giovanni.cabiddu@intel.com>; Alex Williamson > <alex.williamson@redhat.com>; Cao, Yahui <yahui.cao@intel.com> > Cc: Arnd Bergmann <arnd@arndb.de>; Jason Gunthorpe <jgg@ziepe.ca>; > Yishai Hadas <yishaih@nvidia.com>; Shameer Kolothum > <shameerali.kolothum.thodi@huawei.com>; kvm@vger.kernel.org; qat- > linux <qat-linux@intel.com>; linux-kernel@vger.kernel.org > Subject: RE: [PATCH] vfio/qat: add PCI_IOV dependency > > > From: Arnd Bergmann <arnd@kernel.org> > > Sent: Tuesday, May 28, 2024 8:05 PM > > > > From: Arnd Bergmann <arnd@arndb.de> > > > > The newly added driver depends on the crypto driver, but it uses exported > > symbols that are only available when IOV is also turned on: > > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_pci_open_device': > > main.c:(.text+0xd7): undefined reference to `qat_vfmig_open' > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_pci_release_dev': > > main.c:(.text+0x122): undefined reference to `qat_vfmig_cleanup' > > x86_64-linux-ld: main.c:(.text+0x12d): undefined reference to > > `qat_vfmig_destroy' > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_resume_write': > > main.c:(.text+0x308): undefined reference to `qat_vfmig_load_setup' > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_save_device_data': > > main.c:(.text+0x64c): undefined reference to `qat_vfmig_save_state' > > x86_64-linux-ld: main.c:(.text+0x677): undefined reference to > > `qat_vfmig_save_setup' > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_pci_aer_reset_done': > > main.c:(.text+0x82d): undefined reference to `qat_vfmig_reset' > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_pci_close_device': > > main.c:(.text+0x862): undefined reference to `qat_vfmig_close' > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > `qat_vf_pci_set_device_state': > > main.c:(.text+0x9af): undefined reference to `qat_vfmig_suspend' > > x86_64-linux-ld: main.c:(.text+0xa14): undefined reference to > > `qat_vfmig_save_state' > > x86_64-linux-ld: main.c:(.text+0xb37): undefined reference to > > `qat_vfmig_resume' > > x86_64-linux-ld: main.c:(.text+0xbc7): undefined reference to > > `qat_vfmig_load_state' > > at a glance those undefined symbols don't use any symbol under > IOV. They are just wrappers to certain callbacks registered by > by respective qat drivers which support migration. > > Probably they'd better be moved out of CONFIG_PCI_IOV in > "drivers/crypto/intel/qat/qat_common/Makefile" to remove > this dependency in vfio variant driver. > Thanks, Kevin :-). This dependency is like the relationship between the QAT vfio variant driver and macro CRYPTO_DEV_QAT_4XXX. The variant driver doesn't directly reference the symbols exported by module qat_4xxx which is protected by CRYPTO_DEV_QAT_4XXX, but requires the module qat_4xxx at runtime so far. Alex suggested to put CRYPTO_DEV_QAT_4XXX as the dependency of this variant driver. For CONFIG_PCI_IOV, if it is disabled, this variant driver doesn't serve the user as well since no VFs will be created by QAT PF driver. To keep the consistency, it might be right to make it as the dependency of this variant driver as Arnd pointed out. What do you think? Thanks, Xin > > > > Add this as a second dependency. > > > > Fixes: bb208810b1ab ("vfio/qat: Add vfio_pci driver for Intel QAT SR-IOV VF > > devices") > > Signed-off-by: Arnd Bergmann <arnd@arndb.de> > > --- > > drivers/vfio/pci/qat/Kconfig | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/vfio/pci/qat/Kconfig b/drivers/vfio/pci/qat/Kconfig > > index bf52cfa4b595..fae9d6cb8ccb 100644 > > --- a/drivers/vfio/pci/qat/Kconfig > > +++ b/drivers/vfio/pci/qat/Kconfig > > @@ -1,8 +1,9 @@ > > # SPDX-License-Identifier: GPL-2.0-only > > config QAT_VFIO_PCI > > tristate "VFIO support for QAT VF PCI devices" > > - select VFIO_PCI_CORE > > depends on CRYPTO_DEV_QAT_4XXX > > + depends on PCI_IOV > > + select VFIO_PCI_CORE > > help > > This provides migration support for Intel(R) QAT Virtual Function > > using the VFIO framework. > > -- > > 2.39.2
> From: Zeng, Xin <xin.zeng@intel.com> > Sent: Wednesday, May 29, 2024 11:11 AM > > On Wednesday, May 29, 2024 10:25 AM, Tian, Kevin <kevin.tian@intel.com> > > To: Arnd Bergmann <arnd@kernel.org>; Zeng, Xin <xin.zeng@intel.com>; > > Cabiddu, Giovanni <giovanni.cabiddu@intel.com>; Alex Williamson > > <alex.williamson@redhat.com>; Cao, Yahui <yahui.cao@intel.com> > > Cc: Arnd Bergmann <arnd@arndb.de>; Jason Gunthorpe <jgg@ziepe.ca>; > > Yishai Hadas <yishaih@nvidia.com>; Shameer Kolothum > > <shameerali.kolothum.thodi@huawei.com>; kvm@vger.kernel.org; qat- > > linux <qat-linux@intel.com>; linux-kernel@vger.kernel.org > > Subject: RE: [PATCH] vfio/qat: add PCI_IOV dependency > > > > > From: Arnd Bergmann <arnd@kernel.org> > > > Sent: Tuesday, May 28, 2024 8:05 PM > > > > > > From: Arnd Bergmann <arnd@arndb.de> > > > > > > The newly added driver depends on the crypto driver, but it uses > exported > > > symbols that are only available when IOV is also turned on: > > > > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_pci_open_device': > > > main.c:(.text+0xd7): undefined reference to `qat_vfmig_open' > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_pci_release_dev': > > > main.c:(.text+0x122): undefined reference to `qat_vfmig_cleanup' > > > x86_64-linux-ld: main.c:(.text+0x12d): undefined reference to > > > `qat_vfmig_destroy' > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_resume_write': > > > main.c:(.text+0x308): undefined reference to `qat_vfmig_load_setup' > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_save_device_data': > > > main.c:(.text+0x64c): undefined reference to `qat_vfmig_save_state' > > > x86_64-linux-ld: main.c:(.text+0x677): undefined reference to > > > `qat_vfmig_save_setup' > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_pci_aer_reset_done': > > > main.c:(.text+0x82d): undefined reference to `qat_vfmig_reset' > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_pci_close_device': > > > main.c:(.text+0x862): undefined reference to `qat_vfmig_close' > > > x86_64-linux-ld: drivers/vfio/pci/qat/main.o: in function > > > `qat_vf_pci_set_device_state': > > > main.c:(.text+0x9af): undefined reference to `qat_vfmig_suspend' > > > x86_64-linux-ld: main.c:(.text+0xa14): undefined reference to > > > `qat_vfmig_save_state' > > > x86_64-linux-ld: main.c:(.text+0xb37): undefined reference to > > > `qat_vfmig_resume' > > > x86_64-linux-ld: main.c:(.text+0xbc7): undefined reference to > > > `qat_vfmig_load_state' > > > > at a glance those undefined symbols don't use any symbol under > > IOV. They are just wrappers to certain callbacks registered by > > by respective qat drivers which support migration. > > > > Probably they'd better be moved out of CONFIG_PCI_IOV in > > "drivers/crypto/intel/qat/qat_common/Makefile" to remove > > this dependency in vfio variant driver. > > > > Thanks, Kevin :-). This dependency is like the relationship between the QAT > vfio > variant driver and macro CRYPTO_DEV_QAT_4XXX. The variant driver doesn't > directly reference the symbols exported by module qat_4xxx which is > protected > by CRYPTO_DEV_QAT_4XXX, but requires the module qat_4xxx at runtime so > far. > Alex suggested to put CRYPTO_DEV_QAT_4XXX as the dependency of this > variant > driver. > For CONFIG_PCI_IOV, if it is disabled, this variant driver doesn't serve the > user as > well since no VFs will be created by QAT PF driver. To keep the consistency, it > might > be right to make it as the dependency of this variant driver as Arnd pointed > out. > What do you think? > Following this rationale then we need also make PCI_IOV a dependency for mlx5 and hisilicon given they are for VF migration too?
On Wednesday, May 29, 2024 1:36 PM, Tian, Kevin <kevin.tian@intel.com> wrote: > To: Zeng, Xin <xin.zeng@intel.com>; Arnd Bergmann <arnd@kernel.org>; > Cabiddu, Giovanni <giovanni.cabiddu@intel.com>; Alex Williamson > <alex.williamson@redhat.com>; Cao, Yahui <yahui.cao@intel.com> > Cc: Arnd Bergmann <arnd@arndb.de>; Jason Gunthorpe <jgg@ziepe.ca>; > Yishai Hadas <yishaih@nvidia.com>; Shameer Kolothum > <shameerali.kolothum.thodi@huawei.com>; kvm@vger.kernel.org; qat- > linux <qat-linux@intel.com>; linux-kernel@vger.kernel.org > Subject: RE: [PATCH] vfio/qat: add PCI_IOV dependency > > > From: Zeng, Xin <xin.zeng@intel.com> > > Sent: Wednesday, May 29, 2024 11:11 AM > > > > On Wednesday, May 29, 2024 10:25 AM, Tian, Kevin <kevin.tian@intel.com> > > > To: Arnd Bergmann <arnd@kernel.org>; Zeng, Xin <xin.zeng@intel.com>; > > > Cabiddu, Giovanni <giovanni.cabiddu@intel.com>; Alex Williamson > > > <alex.williamson@redhat.com>; Cao, Yahui <yahui.cao@intel.com> > > > Cc: Arnd Bergmann <arnd@arndb.de>; Jason Gunthorpe <jgg@ziepe.ca>; > > > Yishai Hadas <yishaih@nvidia.com>; Shameer Kolothum > > > <shameerali.kolothum.thodi@huawei.com>; kvm@vger.kernel.org; qat- > > > linux <qat-linux@intel.com>; linux-kernel@vger.kernel.org > > > Subject: RE: [PATCH] vfio/qat: add PCI_IOV dependency > > > > > > > From: Arnd Bergmann <arnd@kernel.org> > > > > Sent: Tuesday, May 28, 2024 8:05 PM > > > > > > > > From: Arnd Bergmann <arnd@arndb.de> > > > > > > > > The newly added driver depends on the crypto driver, but it uses > > exported > > > > symbols that are only available when IOV is also turned on: > > > > > > at a glance those undefined symbols don't use any symbol under > > > IOV. They are just wrappers to certain callbacks registered by > > > by respective qat drivers which support migration. > > > > > > Probably they'd better be moved out of CONFIG_PCI_IOV in > > > "drivers/crypto/intel/qat/qat_common/Makefile" to remove > > > this dependency in vfio variant driver. > > > > > > > Thanks, Kevin :-). This dependency is like the relationship between the QAT > > vfio > > variant driver and macro CRYPTO_DEV_QAT_4XXX. The variant driver > doesn't > > directly reference the symbols exported by module qat_4xxx which is > > protected > > by CRYPTO_DEV_QAT_4XXX, but requires the module qat_4xxx at runtime > so > > far. > > Alex suggested to put CRYPTO_DEV_QAT_4XXX as the dependency of this > > variant > > driver. > > For CONFIG_PCI_IOV, if it is disabled, this variant driver doesn't serve the > > user as > > well since no VFs will be created by QAT PF driver. To keep the consistency, > it > > might > > be right to make it as the dependency of this variant driver as Arnd pointed > > out. > > What do you think? > > > > Following this rationale then we need also make PCI_IOV a dependency > for mlx5 and hisilicon given they are for VF migration too? After more thoughts about this, I would agree with your first point that PCI_IOV should not be the dependency of the variant driver if we consider passthrough VFs in a nested virtualized environment. So decoupling PCI_IOV from migration helpers in QAT PF driver sounds a better option. Thanks, Xin
On Fri, 31 May 2024 02:56:09 +0000 "Zeng, Xin" <xin.zeng@intel.com> wrote: > On Wednesday, May 29, 2024 1:36 PM, Tian, Kevin <kevin.tian@intel.com> wrote: > > To: Zeng, Xin <xin.zeng@intel.com>; Arnd Bergmann <arnd@kernel.org>; > > Cabiddu, Giovanni <giovanni.cabiddu@intel.com>; Alex Williamson > > <alex.williamson@redhat.com>; Cao, Yahui <yahui.cao@intel.com> > > Cc: Arnd Bergmann <arnd@arndb.de>; Jason Gunthorpe <jgg@ziepe.ca>; > > Yishai Hadas <yishaih@nvidia.com>; Shameer Kolothum > > <shameerali.kolothum.thodi@huawei.com>; kvm@vger.kernel.org; qat- > > linux <qat-linux@intel.com>; linux-kernel@vger.kernel.org > > Subject: RE: [PATCH] vfio/qat: add PCI_IOV dependency > > > > > From: Zeng, Xin <xin.zeng@intel.com> > > > Sent: Wednesday, May 29, 2024 11:11 AM > > > > > > On Wednesday, May 29, 2024 10:25 AM, Tian, Kevin <kevin.tian@intel.com> > > > > To: Arnd Bergmann <arnd@kernel.org>; Zeng, Xin <xin.zeng@intel.com>; > > > > Cabiddu, Giovanni <giovanni.cabiddu@intel.com>; Alex Williamson > > > > <alex.williamson@redhat.com>; Cao, Yahui <yahui.cao@intel.com> > > > > Cc: Arnd Bergmann <arnd@arndb.de>; Jason Gunthorpe <jgg@ziepe.ca>; > > > > Yishai Hadas <yishaih@nvidia.com>; Shameer Kolothum > > > > <shameerali.kolothum.thodi@huawei.com>; kvm@vger.kernel.org; qat- > > > > linux <qat-linux@intel.com>; linux-kernel@vger.kernel.org > > > > Subject: RE: [PATCH] vfio/qat: add PCI_IOV dependency > > > > > > > > > From: Arnd Bergmann <arnd@kernel.org> > > > > > Sent: Tuesday, May 28, 2024 8:05 PM > > > > > > > > > > From: Arnd Bergmann <arnd@arndb.de> > > > > > > > > > > The newly added driver depends on the crypto driver, but it uses > > > exported > > > > > symbols that are only available when IOV is also turned on: > > > > > > > > at a glance those undefined symbols don't use any symbol under > > > > IOV. They are just wrappers to certain callbacks registered by > > > > by respective qat drivers which support migration. > > > > > > > > Probably they'd better be moved out of CONFIG_PCI_IOV in > > > > "drivers/crypto/intel/qat/qat_common/Makefile" to remove > > > > this dependency in vfio variant driver. > > > > > > > > > > Thanks, Kevin :-). This dependency is like the relationship between the QAT > > > vfio > > > variant driver and macro CRYPTO_DEV_QAT_4XXX. The variant driver > > doesn't > > > directly reference the symbols exported by module qat_4xxx which is > > > protected > > > by CRYPTO_DEV_QAT_4XXX, but requires the module qat_4xxx at runtime > > so > > > far. > > > Alex suggested to put CRYPTO_DEV_QAT_4XXX as the dependency of this > > > variant > > > driver. > > > For CONFIG_PCI_IOV, if it is disabled, this variant driver doesn't serve the > > > user as > > > well since no VFs will be created by QAT PF driver. To keep the consistency, > > it > > > might > > > be right to make it as the dependency of this variant driver as Arnd pointed > > > out. > > > What do you think? > > > > > > > Following this rationale then we need also make PCI_IOV a dependency > > for mlx5 and hisilicon given they are for VF migration too? > > After more thoughts about this, I would agree with your first point that > PCI_IOV should not be the dependency of the variant driver if we consider > passthrough VFs in a nested virtualized environment. > So decoupling PCI_IOV from migration helpers in QAT PF driver sounds a > better option. Is this then being taken care of in the QAT PF driver? Are there patches posted targeting v6.10? Is there an archive of qat-linux list somewhere? I can't find any relevant postings on linux-crypto@vger and can't find a public archive of qat-linux(??). Thanks, Alex
On Fri, Jun 07, 2024 at 03:34:06PM -0600, Alex Williamson wrote: > Is this then being taken care of in the QAT PF driver? Are there > patches posted targeting v6.10? Yes. This is being taken care in the QAT PF driver. Xin just sent a fix for it [1]. @Herbert, can this be sent to stable after the review? > Is there an archive of qat-linux list somewhere? > I can't find any relevant postings on linux-crypto@vger and > can't find a public archive of qat-linux(??). Thanks, There isn't a public archive of qat-linux. All patches to the QAT driver are sent to linux-crypto@vger.kernel.org. [1] https://lore.kernel.org/all/20240610143756.2031626-1-xin.zeng@intel.com/ Regards,
On Mon, Jun 10, 2024 at 04:26:59PM +0100, Cabiddu, Giovanni wrote: > On Fri, Jun 07, 2024 at 03:34:06PM -0600, Alex Williamson wrote: > > Is this then being taken care of in the QAT PF driver? Are there > > patches posted targeting v6.10? > Yes. This is being taken care in the QAT PF driver. Xin just sent a fix > for it [1]. > @Herbert, can this be sent to stable after the review? This patch wasn't sent to linux-crypto so I cannot apply it. In any case, is there any reason why the fix can't go through the vfio tree? Cheers,
Hi Herbert, On Tue, Jun 11, 2024 at 11:20:50AM +0800, Herbert Xu wrote: > On Mon, Jun 10, 2024 at 04:26:59PM +0100, Cabiddu, Giovanni wrote: > > On Fri, Jun 07, 2024 at 03:34:06PM -0600, Alex Williamson wrote: > > > Is this then being taken care of in the QAT PF driver? Are there > > > patches posted targeting v6.10? > > Yes. This is being taken care in the QAT PF driver. Xin just sent a fix > > for it [1]. > > @Herbert, can this be sent to stable after the review? > > This patch wasn't sent to linux-crypto so I cannot apply it. Was sent yesterday to linux-crypto: https://patchwork.kernel.org/project/linux-crypto/patch/20240610143756.2031626-1-xin.zeng@intel.com/ > In any case, is there any reason why the fix can't go through the > vfio tree? It is a change in the QAT driver. That's why I asked. Anyway, it should be fixed as part of v6.10 - I was wrong saying it should be sent to stable. Regards,
On Tue, Jun 11, 2024 at 12:27:22PM +0100, Cabiddu, Giovanni wrote: > > > In any case, is there any reason why the fix can't go through the > > vfio tree? > It is a change in the QAT driver. That's why I asked. Sorry, I got confused by the subject line. I'll get onto this patch. Thanks,
diff --git a/drivers/vfio/pci/qat/Kconfig b/drivers/vfio/pci/qat/Kconfig index bf52cfa4b595..fae9d6cb8ccb 100644 --- a/drivers/vfio/pci/qat/Kconfig +++ b/drivers/vfio/pci/qat/Kconfig @@ -1,8 +1,9 @@ # SPDX-License-Identifier: GPL-2.0-only config QAT_VFIO_PCI tristate "VFIO support for QAT VF PCI devices" - select VFIO_PCI_CORE depends on CRYPTO_DEV_QAT_4XXX + depends on PCI_IOV + select VFIO_PCI_CORE help This provides migration support for Intel(R) QAT Virtual Function using the VFIO framework.