| Message ID | 20220328230008.3587975-1-tansuresh@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Asynchronous shutdown interface and example implementation | expand |
On Mon, Mar 28, 2022 at 04:00:05PM -0700, Tanjore Suresh wrote: > Problem: > > Some of our machines are configured with many NVMe devices and > are validated for strict shutdown time requirements. Each NVMe > device plugged into the system, typicaly takes about 4.5 secs > to shutdown. A system with 16 such NVMe devices will takes > approximately 80 secs to shutdown and go through reboot. > > The current shutdown APIs as defined at bus level is defined to be > synchronous. Therefore, more devices are in the system the greater > the time it takes to shutdown. This shutdown time significantly > contributes the machine reboot time. > > Solution: > > This patch set proposes an asynchronous shutdown interface at bus level, > modifies the core driver, device shutdown routine to exploit the > new interface while maintaining backward compatibility with synchronous > implementation already existing (Patch 1 of 3) and exploits new interface > to enable all PCI-E based devices to use asynchronous interface semantics > if necessary (Patch 2 of 3). The implementation at PCI-E level also works > in a backward compatible way, to allow exiting device implementation > to work with current synchronous semantics. Only show cases an example > implementation for NVMe device to exploit this asynchronous shutdown > interface. (Patch 3 of 3). > > Tanjore Suresh (3): > driver core: Support asynchronous driver shutdown > PCI: Support asynchronous shutdown > nvme: Add async shutdown support > > drivers/base/core.c | 39 ++++++++++++++++++- > drivers/nvme/host/core.c | 28 +++++++++---- > drivers/nvme/host/nvme.h | 8 ++++ > drivers/nvme/host/pci.c | 80 ++++++++++++++++++++++++-------------- > drivers/pci/pci-driver.c | 17 ++++++-- > include/linux/device/bus.h | 10 +++++ > include/linux/pci.h | 2 + > 7 files changed, 144 insertions(+), 40 deletions(-) > > -- > 2.35.1.1021.g381101b075-goog > Hi, This is the friendly patch-bot of Greg Kroah-Hartman. You have sent him a patch that has triggered this response. He used to manually respond to these common problems, but in order to save his sanity (he kept writing the same thing over and over, yet to different people), I was created. Hopefully you will not take offence and will fix the problem in your patch and resubmit it so that it can be accepted into the Linux kernel tree. You are receiving this message because of the following common error(s) as indicated below: - This looks like a new version of a previously submitted patch, but you did not list below the --- line any changes from the previous version. Please read the section entitled "The canonical patch format" in the kernel file, Documentation/SubmittingPatches for what needs to be done here to properly describe this. If you wish to discuss this problem further, or you have questions about how to resolve this issue, please feel free to respond to this email and Greg will reply once he has dug out from the pending patches received from other developers. thanks, greg k-h's patch email bot
On Mon, Mar 28, 2022 at 04:00:05PM -0700, Tanjore Suresh wrote: > Problem: > > Some of our machines are configured with many NVMe devices and > are validated for strict shutdown time requirements. Each NVMe > device plugged into the system, typicaly takes about 4.5 secs > to shutdown. A system with 16 such NVMe devices will takes > approximately 80 secs to shutdown and go through reboot. > > The current shutdown APIs as defined at bus level is defined to be > synchronous. Therefore, more devices are in the system the greater > the time it takes to shutdown. This shutdown time significantly > contributes the machine reboot time. > > Solution: > > This patch set proposes an asynchronous shutdown interface at bus level, > modifies the core driver, device shutdown routine to exploit the > new interface while maintaining backward compatibility with synchronous > implementation already existing (Patch 1 of 3) and exploits new interface > to enable all PCI-E based devices to use asynchronous interface semantics > if necessary (Patch 2 of 3). The implementation at PCI-E level also works > in a backward compatible way, to allow exiting device implementation > to work with current synchronous semantics. Only show cases an example > implementation for NVMe device to exploit this asynchronous shutdown > interface. (Patch 3 of 3). Thanks, I agree we should improve shutdown times. I tried a while ago, but lost track to follow up at the time. Here's the reference, fwiw, though it may be out of date :): http://lists.infradead.org/pipermail/linux-nvme/2014-May/000826.html The above solution is similiar to how probe waits on an async domain. Maybe pci can schedule the async shutdown instead of relying on low-level drivers so that everyone implicitly benefits instead of just nvme? I'll double-check if that's reasonable, but I'll look through this series too.
On Tue, Mar 29, 2022 at 08:07:51PM -0600, Keith Busch wrote: > Thanks, I agree we should improve shutdown times. I tried a while ago, but > lost track to follow up at the time. Here's the reference, fwiw, though it > may be out of date :): > > http://lists.infradead.org/pipermail/linux-nvme/2014-May/000826.html > > The above solution is similiar to how probe waits on an async domain. > Maybe pci can schedule the async shutdown instead of relying on low-level > drivers so that everyone implicitly benefits instead of just nvme? I'll > double-check if that's reasonable, but I'll look through this series too. Using the async API seems much more reasonable than adding new callbacks. However I'd argue that it shouldn't be necessary to amend any drivers, this should all be doable in the driver core: Basically a device needs to wait for its children and device links consumers to shutdown, apart from that everything should be able to run asynchronously. Thanks, Lukas
On Wed, Mar 30, 2022 at 8:25 AM Lukas Wunner <lukas@wunner.de> wrote: > > On Tue, Mar 29, 2022 at 08:07:51PM -0600, Keith Busch wrote: > > Thanks, I agree we should improve shutdown times. I tried a while ago, but > > lost track to follow up at the time. Here's the reference, fwiw, though it > > may be out of date :): > > > > http://lists.infradead.org/pipermail/linux-nvme/2014-May/000826.html > > > > The above solution is similiar to how probe waits on an async domain. > > Maybe pci can schedule the async shutdown instead of relying on low-level > > drivers so that everyone implicitly benefits instead of just nvme? I'll > > double-check if that's reasonable, but I'll look through this series too. > > Using the async API seems much more reasonable than adding new callbacks. > > However I'd argue that it shouldn't be necessary to amend any drivers, > this should all be doable in the driver core: Basically a device needs > to wait for its children and device links consumers to shutdown, apart > from that everything should be able to run asynchronously. Well, this is done already in the system-wide and hibernation paths. It should be possible to implement asynchronous shutdown analogously.