| Message ID | 1373621543-54836-4-git-send-email-wangyijing@huawei.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
On Fri, Jul 12, 2013 at 3:32 AM, Yijing Wang <wangyijing@huawei.com> wrote: > If device was removed from slot and reinsert a new device during > suspend, pciehp can not identify the physical device change now. > So the old driver .resume() method will be called for the new device, > this is bad. If device support device serial number capability, > we can identify this by DSN. So the reasonable way is first remove > the old device, then enable the new device. > > Signed-off-by: Yijing Wang <wangyijing@huawei.com> > Cc: Paul Bolle <pebolle@tiscali.nl> > Cc: "Rafael J. Wysocki" <rjw@sisk.pl> > Cc: Oliver Neukum <oneukum@suse.de> > Cc: Gu Zheng <guz.fnst@cn.fujitsu.com> > Cc: linux-pci@vger.kernel.org > --- > drivers/pci/hotplug/pciehp_core.c | 45 +++++++++++++++++++++++++++++++++++++ > 1 files changed, 45 insertions(+), 0 deletions(-) > > diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c > index 7fe9dbd..bc47f8e 100644 > --- a/drivers/pci/hotplug/pciehp_core.c > +++ b/drivers/pci/hotplug/pciehp_core.c > @@ -296,11 +296,38 @@ static int pciehp_suspend (struct pcie_device *dev) > return 0; > } > > +/* > + * check the func 0 device serial number is changed, > + * if device does not support device serial number, > + * return false. > + */ > +static bool device_serial_number_changed(struct pci_bus *pbus) > +{ > + u64 old_dsn, new_dsn; > + struct pci_dev *pdev; > + > + pdev = pci_get_slot(pbus, PCI_DEVFN(0, 0)); > + if (!pdev) > + return false; > + > + old_dsn = pdev->sn; > + > + /* get func 0 device serial number */ > + new_dsn = pci_device_serial_number(pdev); > + pci_dev_put(pdev); > + > + if (old_dsn != new_dsn) > + return true; > + > + return false; > +} > + > static int pciehp_resume (struct pcie_device *dev) > { > struct controller *ctrl; > struct slot *slot; > struct pci_bus *pbus = dev->port->subordinate; > + int retval = 0; > u8 status; > > ctrl = get_service_data(dev); > @@ -315,6 +342,24 @@ static int pciehp_resume (struct pcie_device *dev) > if (status) { > if (list_empty(&pbus->devices)) > pciehp_enable_slot(slot); > + > + if (device_serial_number_changed(pbus)) { > + /* > + * first power off slot, avoid the old driver > + * .remove() method touch the new hardware > + */ > + if (POWER_CTRL(ctrl)) { > + retval = pciehp_power_off_slot(slot); > + if (retval) { > + ctrl_err(ctrl, > + "Issue of Slot Disable command failed\n"); > + return 0; > + } > + msleep(1000); > + pciehp_unconfigure_device(slot); > + pciehp_enable_slot(slot); > + } > + } I'm not sure this is implemented at the correct place. The idea of using the serial number to detect card swaps is not really specific to pciehp. I know the Device Serial Number capability is defined in the PCIe spec, but it doesn't *require* any PCIe functionality, and there's no reason a similar capability couldn't be defined for conventional PCI. I don't know much about it, but conventional PCI *does* in fact support the VPD capability, which can contain a serial number. I wonder if we should enhance pci_device_serial_number() to look for a VPD serial number if there's no PCIe DSN. Then we would want this check for card swap in a more generic place, e.g., somewhere in pci_scan_slot(), so all forms of hotplug would benefit from it. Also, I think it's possible to use acpiphp for ExpressCard slots, and this patch doesn't help acpiphp detect card swaps. I don't see any mention of suspend/resume in acpiphp, so I don't know if it does anything at all to detect card changes while suspended. Maybe Rafael can shed some light? I put the first two patches on a pci/yijing-dsn-v3 branch while we work out the details of this one. Bjorn > } else if (!list_empty(&pbus->devices)) { > pciehp_disable_slot(slot); > } > -- > 1.7.1 > > -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Bjorn, Thanks for your review and comments! > > I'm not sure this is implemented at the correct place. The idea of > using the serial number to detect card swaps is not really specific to > pciehp. I know the Device Serial Number capability is defined in the > PCIe spec, but it doesn't *require* any PCIe functionality, and > there's no reason a similar capability couldn't be defined for > conventional PCI. Yes, maybe implement this interface in PCI core is better. > > I don't know much about it, but conventional PCI *does* in fact > support the VPD capability, which can contain a serial number. I > wonder if we should enhance pci_device_serial_number() to look for a > VPD serial number if there's no PCIe DSN. Then we would want this > check for card swap in a more generic place, e.g., somewhere in > pci_scan_slot(), so all forms of hotplug would benefit from it. This is a good idea, I will try to enhance pci_device_serial_number to support legacy PCI device VPD capability. Then I will try to move this to a more generic place. > > Also, I think it's possible to use acpiphp for ExpressCard slots, and > this patch doesn't help acpiphp detect card swaps. I don't see any > mention of suspend/resume in acpiphp, so I don't know if it does > anything at all to detect card changes while suspended. Maybe Rafael > can shed some light? Acpiphp driver is not attached to a specific pci/pcie device, so i think there is no point to call driver->suspend/resume interface. Add cc Rafael J. Wysocki. > > I put the first two patches on a pci/yijing-dsn-v3 branch while we > work out the details of this one. Thanks! > > Bjorn > >> } else if (!list_empty(&pbus->devices)) { >> pciehp_disable_slot(slot); >> } >> -- >> 1.7.1 >> >> > > . >
> I'm not sure this is implemented at the correct place. The idea of > using the serial number to detect card swaps is not really specific to > pciehp. I know the Device Serial Number capability is defined in the > PCIe spec, but it doesn't *require* any PCIe functionality, and > there's no reason a similar capability couldn't be defined for > conventional PCI. > > I don't know much about it, but conventional PCI *does* in fact > support the VPD capability, which can contain a serial number. I > wonder if we should enhance pci_device_serial_number() to look for a > VPD serial number if there's no PCIe DSN. Then we would want this > check for card swap in a more generic place, e.g., somewhere in > pci_scan_slot(), so all forms of hotplug would benefit from it. Hi Bjorn, I'm reworking this patch, but found some strange info about vpd serial number. I have two x86 machines, they almost have the same hardware topology. But by lspci, I found two different Broadcom BCM5709 NIC in different machine have the same vpd serial number. If this is normal, vpd serial number seems to be useless for identify device change. The first machine: linux:/home/yijing # lspci -vvv -s 0000:01:00.0 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) Subsystem: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 28 Region 0: Memory at c0000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [48] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] Vital Product Data Product Name: Broadcom NetXtreme II Ethernet Controller Read-only fields: [PN] Part number: BCM95706A0 [EC] Engineering changes: 220197-2 [SN] Serial number: 0123456789 [MN] Manufacture ID: 31 34 65 34 [RV] Reserved: checksum good, 31 byte(s) reserved End Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 ..............[snip]................... The second machine: linux-suse-vmdq:~/pciutils-3.2.0 # ./lspci -vvv -s 01:00.0 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) Subsystem: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 28 Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [48] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] Vital Product Data Product Name: Broadcom NetXtreme II Ethernet Controller Read-only fields: [PN] Part number: BCM95706A0 [EC] Engineering changes: 220197-2 [SN] Serial number: 0123456789 [MN] Manufacture ID: 31 34 65 34 [RV] Reserved: checksum good, 31 byte(s) reserved End .......[snip]........ > > Also, I think it's possible to use acpiphp for ExpressCard slots, and > this patch doesn't help acpiphp detect card swaps. I don't see any > mention of suspend/resume in acpiphp, so I don't know if it does > anything at all to detect card changes while suspended. Maybe Rafael > can shed some light? > > I put the first two patches on a pci/yijing-dsn-v3 branch while we > work out the details of this one. > > Bjorn > >> } else if (!list_empty(&pbus->devices)) { >> pciehp_disable_slot(slot); >> } >> -- >> 1.7.1 >> >> > > . >
On Mon, Jul 29, 2013 at 9:46 PM, Yijing Wang <wangyijing@huawei.com> wrote: >> I'm not sure this is implemented at the correct place. The idea of >> using the serial number to detect card swaps is not really specific to >> pciehp. I know the Device Serial Number capability is defined in the >> PCIe spec, but it doesn't *require* any PCIe functionality, and >> there's no reason a similar capability couldn't be defined for >> conventional PCI. >> >> I don't know much about it, but conventional PCI *does* in fact >> support the VPD capability, which can contain a serial number. I >> wonder if we should enhance pci_device_serial_number() to look for a >> VPD serial number if there's no PCIe DSN. Then we would want this >> check for card swap in a more generic place, e.g., somewhere in >> pci_scan_slot(), so all forms of hotplug would benefit from it. > > Hi Bjorn, > I'm reworking this patch, but found some strange info about vpd serial number. > I have two x86 machines, they almost have the same hardware topology. But by lspci, > I found two different Broadcom BCM5709 NIC in different machine have the same vpd serial > number. If this is normal, vpd serial number seems to be useless for identify device change. I wouldn't say it's completely useless. If the VPD serial number changes, we can be pretty confident that the card has changed. It's just that if the VPD serial number is unchanged, we might incorrectly assume it's the same device. But we currently *always* assume it's the same device, since we don't look at serial numbers at all. If we can detect some changes, that should be an improvement over what we have today even though it's not perfect. > The first machine: > linux:/home/yijing # lspci -vvv -s 0000:01:00.0 > 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) > Subsystem: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 256 bytes > Interrupt: pin A routed to IRQ 28 > Region 0: Memory at c0000000 (64-bit, non-prefetchable) [size=32M] > Capabilities: [48] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] Vital Product Data > Product Name: Broadcom NetXtreme II Ethernet Controller > Read-only fields: > [PN] Part number: BCM95706A0 > [EC] Engineering changes: 220197-2 > [SN] Serial number: 0123456789 > [MN] Manufacture ID: 31 34 65 34 > [RV] Reserved: checksum good, 31 byte(s) reserved > End > Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ > Address: 0000000000000000 Data: 0000 > ..............[snip]................... > > The second machine: > linux-suse-vmdq:~/pciutils-3.2.0 # ./lspci -vvv -s 01:00.0 > 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) > Subsystem: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 256 bytes > Interrupt: pin A routed to IRQ 28 > Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M] > Capabilities: [48] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] Vital Product Data > Product Name: Broadcom NetXtreme II Ethernet Controller > Read-only fields: > [PN] Part number: BCM95706A0 > [EC] Engineering changes: 220197-2 > [SN] Serial number: 0123456789 > [MN] Manufacture ID: 31 34 65 34 > [RV] Reserved: checksum good, 31 byte(s) reserved > End > .......[snip]........ > > >> >> Also, I think it's possible to use acpiphp for ExpressCard slots, and >> this patch doesn't help acpiphp detect card swaps. I don't see any >> mention of suspend/resume in acpiphp, so I don't know if it does >> anything at all to detect card changes while suspended. Maybe Rafael >> can shed some light? >> >> I put the first two patches on a pci/yijing-dsn-v3 branch while we >> work out the details of this one. >> >> Bjorn >> >>> } else if (!list_empty(&pbus->devices)) { >>> pciehp_disable_slot(slot); >>> } >>> -- >>> 1.7.1 >>> >>> >> >> . >> > > > -- > Thanks! > Yijing > -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
>> >> Hi Bjorn, >> I'm reworking this patch, but found some strange info about vpd serial number. >> I have two x86 machines, they almost have the same hardware topology. But by lspci, >> I found two different Broadcom BCM5709 NIC in different machine have the same vpd serial >> number. If this is normal, vpd serial number seems to be useless for identify device change. > > I wouldn't say it's completely useless. If the VPD serial number > changes, we can be pretty confident that the card has changed. It's > just that if the VPD serial number is unchanged, we might incorrectly > assume it's the same device. > > But we currently *always* assume it's the same device, since we don't > look at serial numbers at all. If we can detect some changes, that > should be an improvement over what we have today even though it's not > perfect. Hmmm, Yes, different VPD serial number must comes from different device. Although the same serial number maybe not the same device. I will send out the new version patchset soon. Thanks! Yijing. > >> The first machine: >> linux:/home/yijing # lspci -vvv -s 0000:01:00.0 >> 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) >> Subsystem: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- >> Latency: 0, Cache Line Size: 256 bytes >> Interrupt: pin A routed to IRQ 28 >> Region 0: Memory at c0000000 (64-bit, non-prefetchable) [size=32M] >> Capabilities: [48] Power Management version 3 >> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- >> Capabilities: [50] Vital Product Data >> Product Name: Broadcom NetXtreme II Ethernet Controller >> Read-only fields: >> [PN] Part number: BCM95706A0 >> [EC] Engineering changes: 220197-2 >> [SN] Serial number: 0123456789 >> [MN] Manufacture ID: 31 34 65 34 >> [RV] Reserved: checksum good, 31 byte(s) reserved >> End >> Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ >> Address: 0000000000000000 Data: 0000 >> ..............[snip]................... >> >> The second machine: >> linux-suse-vmdq:~/pciutils-3.2.0 # ./lspci -vvv -s 01:00.0 >> 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) >> Subsystem: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- >> Latency: 0, Cache Line Size: 256 bytes >> Interrupt: pin A routed to IRQ 28 >> Region 0: Memory at f6000000 (64-bit, non-prefetchable) [size=32M] >> Capabilities: [48] Power Management version 3 >> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- >> Capabilities: [50] Vital Product Data >> Product Name: Broadcom NetXtreme II Ethernet Controller >> Read-only fields: >> [PN] Part number: BCM95706A0 >> [EC] Engineering changes: 220197-2 >> [SN] Serial number: 0123456789 >> [MN] Manufacture ID: 31 34 65 34 >> [RV] Reserved: checksum good, 31 byte(s) reserved >> End >> .......[snip]........ >> >> >>> >>> Also, I think it's possible to use acpiphp for ExpressCard slots, and >>> this patch doesn't help acpiphp detect card swaps. I don't see any >>> mention of suspend/resume in acpiphp, so I don't know if it does >>> anything at all to detect card changes while suspended. Maybe Rafael >>> can shed some light? >>> >>> I put the first two patches on a pci/yijing-dsn-v3 branch while we >>> work out the details of this one. >>> >>> Bjorn >>> >>>> } else if (!list_empty(&pbus->devices)) { >>>> pciehp_disable_slot(slot); >>>> } >>>> -- >>>> 1.7.1 >>>> >>>> >>> >>> . >>> >> >> >> -- >> Thanks! >> Yijing >> > > . >
diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c index 7fe9dbd..bc47f8e 100644 --- a/drivers/pci/hotplug/pciehp_core.c +++ b/drivers/pci/hotplug/pciehp_core.c @@ -296,11 +296,38 @@ static int pciehp_suspend (struct pcie_device *dev) return 0; } +/* + * check the func 0 device serial number is changed, + * if device does not support device serial number, + * return false. + */ +static bool device_serial_number_changed(struct pci_bus *pbus) +{ + u64 old_dsn, new_dsn; + struct pci_dev *pdev; + + pdev = pci_get_slot(pbus, PCI_DEVFN(0, 0)); + if (!pdev) + return false; + + old_dsn = pdev->sn; + + /* get func 0 device serial number */ + new_dsn = pci_device_serial_number(pdev); + pci_dev_put(pdev); + + if (old_dsn != new_dsn) + return true; + + return false; +} + static int pciehp_resume (struct pcie_device *dev) { struct controller *ctrl; struct slot *slot; struct pci_bus *pbus = dev->port->subordinate; + int retval = 0; u8 status; ctrl = get_service_data(dev); @@ -315,6 +342,24 @@ static int pciehp_resume (struct pcie_device *dev) if (status) { if (list_empty(&pbus->devices)) pciehp_enable_slot(slot); + + if (device_serial_number_changed(pbus)) { + /* + * first power off slot, avoid the old driver + * .remove() method touch the new hardware + */ + if (POWER_CTRL(ctrl)) { + retval = pciehp_power_off_slot(slot); + if (retval) { + ctrl_err(ctrl, + "Issue of Slot Disable command failed\n"); + return 0; + } + msleep(1000); + pciehp_unconfigure_device(slot); + pciehp_enable_slot(slot); + } + } } else if (!list_empty(&pbus->devices)) { pciehp_disable_slot(slot); }
If device was removed from slot and reinsert a new device during suspend, pciehp can not identify the physical device change now. So the old driver .resume() method will be called for the new device, this is bad. If device support device serial number capability, we can identify this by DSN. So the reasonable way is first remove the old device, then enable the new device. Signed-off-by: Yijing Wang <wangyijing@huawei.com> Cc: Paul Bolle <pebolle@tiscali.nl> Cc: "Rafael J. Wysocki" <rjw@sisk.pl> Cc: Oliver Neukum <oneukum@suse.de> Cc: Gu Zheng <guz.fnst@cn.fujitsu.com> Cc: linux-pci@vger.kernel.org --- drivers/pci/hotplug/pciehp_core.c | 45 +++++++++++++++++++++++++++++++++++++ 1 files changed, 45 insertions(+), 0 deletions(-)