| Message ID | 20171212190134.535941-2-tj@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hello tejun Sorry for missing the V2, same comment again. On 12/13/2017 03:01 AM, Tejun Heo wrote: > Currently, blk-mq protects only the issue path with RCU. This patch > puts the completion path under the same RCU protection. This will be > used to synchronize issue/completion against timeout by later patches, > which will also add the comments. > > Signed-off-by: Tejun Heo <tj@kernel.org> > --- > block/blk-mq.c | 16 ++++++++++++++-- > 1 file changed, 14 insertions(+), 2 deletions(-) > > diff --git a/block/blk-mq.c b/block/blk-mq.c > index 1109747..acf4fbb 100644 > --- a/block/blk-mq.c > +++ b/block/blk-mq.c > @@ -568,11 +568,23 @@ static void __blk_mq_complete_request(struct request *rq) > void blk_mq_complete_request(struct request *rq) > { > struct request_queue *q = rq->q; > + struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, rq->mq_ctx->cpu); > + int srcu_idx; > > if (unlikely(blk_should_fake_timeout(q))) > return; > - if (!blk_mark_rq_complete(rq)) > - __blk_mq_complete_request(rq); > + > + if (!(hctx->flags & BLK_MQ_F_BLOCKING)) { > + rcu_read_lock(); > + if (!blk_mark_rq_complete(rq)) > + __blk_mq_complete_request(rq); > + rcu_read_unlock(); > + } else { > + srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); > + if (!blk_mark_rq_complete(rq)) > + __blk_mq_complete_request(rq); > + srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); The __blk_mq_complete_request() could be executed in irq context. There should not be any sleeping operations in it. If just synchronize with the timeout path to ensure the aborted_gstate to be seen, only rcu is needed here ,as well as the blk_mq_timeout_work. > + } > } > EXPORT_SYMBOL(blk_mq_complete_request); > >
Hello, On Wed, Dec 13, 2017 at 11:30:48AM +0800, jianchao.wang wrote: > > + } else { > > + srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); > > + if (!blk_mark_rq_complete(rq)) > > + __blk_mq_complete_request(rq); > > + srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); > > The __blk_mq_complete_request() could be executed in irq context. There should not be any > sleeping operations in it. If just synchronize with the timeout path to ensure the aborted_gstate > to be seen, only rcu is needed here ,as well as the blk_mq_timeout_work. Sure, but it's just a lot cleaner to use the same to protect both issue and completion; otherwise, whoever who wants to synchronize against them have to do awkward double rcu locking. Thanks.
On 12/14/2017 12:13 AM, Tejun Heo wrote: > Hello, > > On Wed, Dec 13, 2017 at 11:30:48AM +0800, jianchao.wang wrote: >>> + } else { >>> + srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); >>> + if (!blk_mark_rq_complete(rq)) >>> + __blk_mq_complete_request(rq); >>> + srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); >> >> The __blk_mq_complete_request() could be executed in irq context. There should not be any >> sleeping operations in it. If just synchronize with the timeout path to ensure the aborted_gstate >> to be seen, only rcu is needed here ,as well as the blk_mq_timeout_work. > > Sure, but it's just a lot cleaner to use the same to protect both > issue and completion; otherwise, whoever who wants to synchronize > against them have to do awkward double rcu locking. > It's fair. Thanks for your detailed response. That's really appreciated. > Thanks. >
On Tue, 2017-12-12 at 11:01 -0800, Tejun Heo wrote: > + } else { > + srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); > + if (!blk_mark_rq_complete(rq)) > + __blk_mq_complete_request(rq); > + srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); Hello Tejun, The name queue_rq_srcu was chosen to reflect the original use of that structure, namely to protect .queue_rq() calls. Your patch series broadens the use of that srcu structure so I would appreciate it if it would be renamed, e.g. into "srcu". See also commit 6a83e74d214a ("blk-mq: Introduce blk_mq_quiesce_queue()"). Thanks, Bart.
On Thu, Dec 14, 2017 at 05:01:06PM +0000, Bart Van Assche wrote: > On Tue, 2017-12-12 at 11:01 -0800, Tejun Heo wrote: > > + } else { > > + srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); > > + if (!blk_mark_rq_complete(rq)) > > + __blk_mq_complete_request(rq); > > + srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); > > Hello Tejun, > > The name queue_rq_srcu was chosen to reflect the original use of that structure, > namely to protect .queue_rq() calls. Your patch series broadens the use of that Yeah, will add a patch to rename it. > srcu structure so I would appreciate it if it would be renamed, e.g. into "srcu". > See also commit 6a83e74d214a ("blk-mq: Introduce blk_mq_quiesce_queue()"). Ah yeah, it'd be nice to have the [s]rcu synchronize calls factored out. Thanks.
diff --git a/block/blk-mq.c b/block/blk-mq.c index 1109747..acf4fbb 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -568,11 +568,23 @@ static void __blk_mq_complete_request(struct request *rq) void blk_mq_complete_request(struct request *rq) { struct request_queue *q = rq->q; + struct blk_mq_hw_ctx *hctx = blk_mq_map_queue(q, rq->mq_ctx->cpu); + int srcu_idx; if (unlikely(blk_should_fake_timeout(q))) return; - if (!blk_mark_rq_complete(rq)) - __blk_mq_complete_request(rq); + + if (!(hctx->flags & BLK_MQ_F_BLOCKING)) { + rcu_read_lock(); + if (!blk_mark_rq_complete(rq)) + __blk_mq_complete_request(rq); + rcu_read_unlock(); + } else { + srcu_idx = srcu_read_lock(hctx->queue_rq_srcu); + if (!blk_mark_rq_complete(rq)) + __blk_mq_complete_request(rq); + srcu_read_unlock(hctx->queue_rq_srcu, srcu_idx); + } } EXPORT_SYMBOL(blk_mq_complete_request);
Currently, blk-mq protects only the issue path with RCU. This patch puts the completion path under the same RCU protection. This will be used to synchronize issue/completion against timeout by later patches, which will also add the comments. Signed-off-by: Tejun Heo <tj@kernel.org> --- block/blk-mq.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)