| Message ID | 20191210145305.6605-1-pdurrant@amazon.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | xen-blkback: prevent premature module unload | expand |
On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote: > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem > cache. This cache is destoyed when xen-blkif is unloaded so it is > necessary to wait for the deferred free routine used for such objects to > complete. This necessity was missed in commit 14855954f636 "xen-blkback: > allow module to be cleanly unloaded". This patch fixes the problem by > taking/releasing extra module references in xen_blkif_alloc/free() > respectively. > > Signed-off-by: Paul Durrant <pdurrant@amazon.com> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com> One nit below. > --- > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > Cc: "Roger Pau Monné" <roger.pau@citrix.com> > Cc: Jens Axboe <axboe@kernel.dk> > --- > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c > index e8c5c54e1d26..59d576d27ca7 100644 > --- a/drivers/block/xen-blkback/xenbus.c > +++ b/drivers/block/xen-blkback/xenbus.c > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid) > blkif->domid = domid; > atomic_set(&blkif->refcnt, 1); > init_completion(&blkif->drain_complete); > + > + /* > + * Because freeing back to the cache may be deferred, it is not > + * safe to unload the module (and hence destroy the cache) until > + * this has completed. To prevent premature unloading, take an > + * extra module reference here and release only when the object > + * has been free back to the cache. ^ freed > + */ > + __module_get(THIS_MODULE); > INIT_WORK(&blkif->free_work, xen_blkif_deferred_free); > > return blkif; > @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif) > > /* Make sure everything is drained before shutting down */ > kmem_cache_free(xen_blkif_cachep, blkif); > + module_put(THIS_MODULE); > } > > int __init xen_blkif_interface_init(void) > -- > 2.20.1 >
> -----Original Message----- > From: Roger Pau Monné <roger.pau@citrix.com> > Sent: 11 December 2019 11:29 > To: Durrant, Paul <pdurrant@amazon.com> > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux- > kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; > Jens Axboe <axboe@kernel.dk> > Subject: Re: [PATCH] xen-blkback: prevent premature module unload > > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote: > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem > > cache. This cache is destoyed when xen-blkif is unloaded so it is > > necessary to wait for the deferred free routine used for such objects to > > complete. This necessity was missed in commit 14855954f636 "xen-blkback: > > allow module to be cleanly unloaded". This patch fixes the problem by > > taking/releasing extra module references in xen_blkif_alloc/free() > > respectively. > > > > Signed-off-by: Paul Durrant <pdurrant@amazon.com> > > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com> > > One nit below. > > > --- > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > > Cc: "Roger Pau Monné" <roger.pau@citrix.com> > > Cc: Jens Axboe <axboe@kernel.dk> > > --- > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++ > > 1 file changed, 10 insertions(+) > > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen- > blkback/xenbus.c > > index e8c5c54e1d26..59d576d27ca7 100644 > > --- a/drivers/block/xen-blkback/xenbus.c > > +++ b/drivers/block/xen-blkback/xenbus.c > > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t > domid) > > blkif->domid = domid; > > atomic_set(&blkif->refcnt, 1); > > init_completion(&blkif->drain_complete); > > + > > + /* > > + * Because freeing back to the cache may be deferred, it is not > > + * safe to unload the module (and hence destroy the cache) until > > + * this has completed. To prevent premature unloading, take an > > + * extra module reference here and release only when the object > > + * has been free back to the cache. > ^ freed Oh yes. Can this be done on commit, or would you like me to send a v2? Paul > > + */ > > + __module_get(THIS_MODULE); > > INIT_WORK(&blkif->free_work, xen_blkif_deferred_free); > > > > return blkif; > > @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif) > > > > /* Make sure everything is drained before shutting down */ > > kmem_cache_free(xen_blkif_cachep, blkif); > > + module_put(THIS_MODULE); > > } > > > > int __init xen_blkif_interface_init(void) > > -- > > 2.20.1 > >
On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote: > > -----Original Message----- > > From: Roger Pau Monné <roger.pau@citrix.com> > > Sent: 11 December 2019 11:29 > > To: Durrant, Paul <pdurrant@amazon.com> > > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux- > > kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; > > Jens Axboe <axboe@kernel.dk> > > Subject: Re: [PATCH] xen-blkback: prevent premature module unload > > > > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote: > > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem > > > cache. This cache is destoyed when xen-blkif is unloaded so it is > > > necessary to wait for the deferred free routine used for such objects to > > > complete. This necessity was missed in commit 14855954f636 "xen-blkback: > > > allow module to be cleanly unloaded". This patch fixes the problem by > > > taking/releasing extra module references in xen_blkif_alloc/free() > > > respectively. > > > > > > Signed-off-by: Paul Durrant <pdurrant@amazon.com> > > > > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com> > > > > One nit below. > > > > > --- > > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > > > Cc: "Roger Pau Monné" <roger.pau@citrix.com> > > > Cc: Jens Axboe <axboe@kernel.dk> > > > --- > > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++ > > > 1 file changed, 10 insertions(+) > > > > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen- > > blkback/xenbus.c > > > index e8c5c54e1d26..59d576d27ca7 100644 > > > --- a/drivers/block/xen-blkback/xenbus.c > > > +++ b/drivers/block/xen-blkback/xenbus.c > > > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t > > domid) > > > blkif->domid = domid; > > > atomic_set(&blkif->refcnt, 1); > > > init_completion(&blkif->drain_complete); > > > + > > > + /* > > > + * Because freeing back to the cache may be deferred, it is not > > > + * safe to unload the module (and hence destroy the cache) until > > > + * this has completed. To prevent premature unloading, take an > > > + * extra module reference here and release only when the object > > > + * has been free back to the cache. > > ^ freed > > Oh yes. Can this be done on commit, or would you like me to send a v2? Adjusting on commit would be fine for me, but it's up to Juergen since he is the one that will pick this up. IIRC the module unload patches didn't go through the block subsystem. Thanks, Roger.
On 11.12.19 14:55, Roger Pau Monné wrote: > On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote: >>> -----Original Message----- >>> From: Roger Pau Monné <roger.pau@citrix.com> >>> Sent: 11 December 2019 11:29 >>> To: Durrant, Paul <pdurrant@amazon.com> >>> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux- >>> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; >>> Jens Axboe <axboe@kernel.dk> >>> Subject: Re: [PATCH] xen-blkback: prevent premature module unload >>> >>> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote: >>>> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem >>>> cache. This cache is destoyed when xen-blkif is unloaded so it is >>>> necessary to wait for the deferred free routine used for such objects to >>>> complete. This necessity was missed in commit 14855954f636 "xen-blkback: >>>> allow module to be cleanly unloaded". This patch fixes the problem by >>>> taking/releasing extra module references in xen_blkif_alloc/free() >>>> respectively. >>>> >>>> Signed-off-by: Paul Durrant <pdurrant@amazon.com> >>> >>> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com> >>> >>> One nit below. >>> >>>> --- >>>> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> >>>> Cc: "Roger Pau Monné" <roger.pau@citrix.com> >>>> Cc: Jens Axboe <axboe@kernel.dk> >>>> --- >>>> drivers/block/xen-blkback/xenbus.c | 10 ++++++++++ >>>> 1 file changed, 10 insertions(+) >>>> >>>> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen- >>> blkback/xenbus.c >>>> index e8c5c54e1d26..59d576d27ca7 100644 >>>> --- a/drivers/block/xen-blkback/xenbus.c >>>> +++ b/drivers/block/xen-blkback/xenbus.c >>>> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t >>> domid) >>>> blkif->domid = domid; >>>> atomic_set(&blkif->refcnt, 1); >>>> init_completion(&blkif->drain_complete); >>>> + >>>> + /* >>>> + * Because freeing back to the cache may be deferred, it is not >>>> + * safe to unload the module (and hence destroy the cache) until >>>> + * this has completed. To prevent premature unloading, take an >>>> + * extra module reference here and release only when the object >>>> + * has been free back to the cache. >>> ^ freed >> >> Oh yes. Can this be done on commit, or would you like me to send a v2? > > Adjusting on commit would be fine for me, but it's up to Juergen since > he is the one that will pick this up. IIRC the module unload patches > didn't go through the block subsystem. Oh, right. Yes, will fix this when committing. Juergen
> -----Original Message----- > From: Roger Pau Monné <roger.pau@citrix.com> > Sent: 11 December 2019 13:55 > To: Durrant, Paul <pdurrant@amazon.com>; Juergen Gross <jgross@suse.com> > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux- > kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; > Jens Axboe <axboe@kernel.dk> > Subject: Re: [PATCH] xen-blkback: prevent premature module unload > > On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote: > > > -----Original Message----- > > > From: Roger Pau Monné <roger.pau@citrix.com> > > > Sent: 11 December 2019 11:29 > > > To: Durrant, Paul <pdurrant@amazon.com> > > > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; > linux- > > > kernel@vger.kernel.org; Konrad Rzeszutek Wilk > <konrad.wilk@oracle.com>; > > > Jens Axboe <axboe@kernel.dk> > > > Subject: Re: [PATCH] xen-blkback: prevent premature module unload > > > > > > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote: > > > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' > kmem > > > > cache. This cache is destoyed when xen-blkif is unloaded so it is > > > > necessary to wait for the deferred free routine used for such > objects to > > > > complete. This necessity was missed in commit 14855954f636 "xen- > blkback: > > > > allow module to be cleanly unloaded". This patch fixes the problem > by > > > > taking/releasing extra module references in xen_blkif_alloc/free() > > > > respectively. > > > > > > > > Signed-off-by: Paul Durrant <pdurrant@amazon.com> > > > > > > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com> > > > > > > One nit below. > > > > > > > --- > > > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > > > > Cc: "Roger Pau Monné" <roger.pau@citrix.com> > > > > Cc: Jens Axboe <axboe@kernel.dk> > > > > --- > > > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++ > > > > 1 file changed, 10 insertions(+) > > > > > > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen- > > > blkback/xenbus.c > > > > index e8c5c54e1d26..59d576d27ca7 100644 > > > > --- a/drivers/block/xen-blkback/xenbus.c > > > > +++ b/drivers/block/xen-blkback/xenbus.c > > > > @@ -171,6 +171,15 @@ static struct xen_blkif > *xen_blkif_alloc(domid_t > > > domid) > > > > blkif->domid = domid; > > > > atomic_set(&blkif->refcnt, 1); > > > > init_completion(&blkif->drain_complete); > > > > + > > > > + /* > > > > + * Because freeing back to the cache may be deferred, it is > not > > > > + * safe to unload the module (and hence destroy the cache) > until > > > > + * this has completed. To prevent premature unloading, take an > > > > + * extra module reference here and release only when the > object > > > > + * has been free back to the cache. > > > ^ freed > > > > Oh yes. Can this be done on commit, or would you like me to send a v2? > > Adjusting on commit would be fine for me, but it's up to Juergen since > he is the one that will pick this up. IIRC the module unload patches > didn't go through the block subsystem. True. I forgot manually add Juergen cc list. Paul > > Thanks, Roger.
On 10.12.19 15:53, Paul Durrant wrote: > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem > cache. This cache is destoyed when xen-blkif is unloaded so it is > necessary to wait for the deferred free routine used for such objects to > complete. This necessity was missed in commit 14855954f636 "xen-blkback: > allow module to be cleanly unloaded". This patch fixes the problem by > taking/releasing extra module references in xen_blkif_alloc/free() > respectively. > > Signed-off-by: Paul Durrant <pdurrant@amazon.com> Pushed to xen/tip.git for-linus-5.5b Juergen
diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c index e8c5c54e1d26..59d576d27ca7 100644 --- a/drivers/block/xen-blkback/xenbus.c +++ b/drivers/block/xen-blkback/xenbus.c @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid) blkif->domid = domid; atomic_set(&blkif->refcnt, 1); init_completion(&blkif->drain_complete); + + /* + * Because freeing back to the cache may be deferred, it is not + * safe to unload the module (and hence destroy the cache) until + * this has completed. To prevent premature unloading, take an + * extra module reference here and release only when the object + * has been free back to the cache. + */ + __module_get(THIS_MODULE); INIT_WORK(&blkif->free_work, xen_blkif_deferred_free); return blkif; @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif) /* Make sure everything is drained before shutting down */ kmem_cache_free(xen_blkif_cachep, blkif); + module_put(THIS_MODULE); } int __init xen_blkif_interface_init(void)
Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback: allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively. Signed-off-by: Paul Durrant <pdurrant@amazon.com> --- Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: "Roger Pau Monné" <roger.pau@citrix.com> Cc: Jens Axboe <axboe@kernel.dk> --- drivers/block/xen-blkback/xenbus.c | 10 ++++++++++ 1 file changed, 10 insertions(+)