From patchwork Fri Oct 16 05:02:32 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kiyoshi Ueda X-Patchwork-Id: 54153 X-Patchwork-Delegate: agk@redhat.com Received: from hormel.redhat.com (hormel1.redhat.com [209.132.177.33]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n9G53Cmd031255 for ; Fri, 16 Oct 2009 05:03:13 GMT Received: from listman.util.phx.redhat.com (listman.util.phx.redhat.com [10.8.4.110]) by hormel.redhat.com (Postfix) with ESMTP id 70EFB61A18F; Fri, 16 Oct 2009 01:03:12 -0400 (EDT) Received: from int-mx03.intmail.prod.int.phx2.redhat.com (nat-pool.util.phx.redhat.com [10.8.5.200]) by listman.util.phx.redhat.com (8.13.1/8.13.1) with ESMTP id n9G53B2w018628 for ; Fri, 16 Oct 2009 01:03:11 -0400 Received: from mx1.redhat.com (ext-mx04.extmail.prod.ext.phx2.redhat.com [10.5.110.8]) by int-mx03.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id n9G53BxI017426; Fri, 16 Oct 2009 01:03:11 -0400 Received: from tyo201.gate.nec.co.jp (TYO201.gate.nec.co.jp [202.32.8.193]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id n9G530Hq012880; Fri, 16 Oct 2009 01:03:01 -0400 Received: from mailgate3.nec.co.jp ([10.7.69.197]) by tyo201.gate.nec.co.jp (8.13.8/8.13.4) with ESMTP id n9G530Ql008017; Fri, 16 Oct 2009 14:03:00 +0900 (JST) Received: (from root@localhost) by mailgate3.nec.co.jp (8.11.7/3.7W-MAILGATE-NEC) id n9G52xu18884; Fri, 16 Oct 2009 14:02:59 +0900 (JST) Received: from mail03.kamome.nec.co.jp (mail03.kamome.nec.co.jp [10.25.43.7]) by mailsv4.nec.co.jp (8.13.8/8.13.4) with ESMTP id n9G52xkj003346; Fri, 16 Oct 2009 14:02:59 +0900 (JST) Received: from saigo.jp.nec.com ([10.26.220.6] [10.26.220.6]) by mail02.kamome.nec.co.jp with ESMTP id BT-MMP-2613314; Fri, 16 Oct 2009 14:02:33 +0900 Received: from elcondor.linux.bs1.fc.nec.co.jp ([10.34.125.195] [10.34.125.195]) by mail.jp.nec.com with ESMTP; Fri, 16 Oct 2009 14:02:32 +0900 Message-ID: <4AD7FE68.30508@ct.jp.nec.com> Date: Fri, 16 Oct 2009 14:02:32 +0900 From: Kiyoshi Ueda User-Agent: Thunderbird 2.0.0.23 (X11/20090825) MIME-Version: 1.0 To: Alasdair Kergon References: <4AD7FC11.7030009@ct.jp.nec.com> In-Reply-To: <4AD7FC11.7030009@ct.jp.nec.com> X-RedHat-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.67 on 10.5.11.16 X-Scanned-By: MIMEDefang 2.67 on 10.5.110.8 X-loop: dm-devel@redhat.com Cc: device-mapper development Subject: [dm-devel] [PATCH 6/9] rqdm core: use md->pending for in_flight I/O counting X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.5 Precedence: junk Reply-To: device-mapper development List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com Index: 2.6.32-rc4/drivers/md/dm.c =================================================================== --- 2.6.32-rc4.orig/drivers/md/dm.c +++ 2.6.32-rc4/drivers/md/dm.c @@ -725,23 +725,16 @@ static void end_clone_bio(struct bio *cl * the md may be freed in dm_put() at the end of this function. * Or do dm_get() before calling this function and dm_put() later. */ -static void rq_completed(struct mapped_device *md, int run_queue) +static void rq_completed(struct mapped_device *md, int rw, int run_queue) { - int wakeup_waiters = 0; - struct request_queue *q = md->queue; - unsigned long flags; - - spin_lock_irqsave(q->queue_lock, flags); - if (!queue_in_flight(q)) - wakeup_waiters = 1; - spin_unlock_irqrestore(q->queue_lock, flags); + atomic_dec(&md->pending[rw]); /* nudge anyone waiting on suspend queue */ - if (wakeup_waiters) + if (!md_in_flight(md)) wake_up(&md->wait); if (run_queue) - blk_run_queue(q); + blk_run_queue(md->queue); /* * dm_put() must be at the end of this function. See the comment above @@ -772,6 +765,7 @@ static void dm_unprep_request(struct req */ void dm_requeue_unmapped_request(struct request *clone) { + int rw = rq_data_dir(clone); struct dm_rq_target_io *tio = clone->end_io_data; struct mapped_device *md = tio->md; struct request *rq = tio->orig; @@ -786,7 +780,7 @@ void dm_requeue_unmapped_request(struct blk_requeue_request(q, rq); spin_unlock_irqrestore(q->queue_lock, flags); - rq_completed(md, 0); + rq_completed(md, rw, 0); } EXPORT_SYMBOL_GPL(dm_requeue_unmapped_request); @@ -825,6 +819,7 @@ static void start_queue(struct request_q */ static void dm_end_request(struct request *clone, int error) { + int rw = rq_data_dir(clone); struct dm_rq_target_io *tio = clone->end_io_data; struct mapped_device *md = tio->md; struct request *rq = tio->orig; @@ -846,7 +841,7 @@ static void dm_end_request(struct reques blk_end_request_all(rq, error); - rq_completed(md, 1); + rq_completed(md, rw, 1); } /* @@ -1541,12 +1536,12 @@ static void dm_request_fn(struct request struct mapped_device *md = q->queuedata; struct dm_table *map = dm_get_table(md); struct dm_target *ti; - struct request *rq; + struct request *rq, *clone; /* - * For suspend, check blk_queue_stopped() not to increment - * the number of in-flight I/Os after the queue is stopped - * in dm_suspend(). + * For suspend, check blk_queue_stopped() and increment ->pending + * within a single queue_lock not to increment the number of + * in-flight I/Os after the queue is stopped in dm_suspend(). */ while (!blk_queue_plugged(q) && !blk_queue_stopped(q)) { rq = blk_peek_request(q); @@ -1558,8 +1553,11 @@ static void dm_request_fn(struct request goto plug_and_out; blk_start_request(rq); + clone = rq->special; + atomic_inc(&md->pending[rq_data_dir(clone)]); + spin_unlock(q->queue_lock); - map_request(ti, rq->special, md); + map_request(ti, clone, md); spin_lock_irq(q->queue_lock); } @@ -2071,8 +2069,6 @@ static int dm_wait_for_completion(struct { int r = 0; DECLARE_WAITQUEUE(wait, current); - struct request_queue *q = md->queue; - unsigned long flags; dm_unplug_all(md->queue); @@ -2082,14 +2078,7 @@ static int dm_wait_for_completion(struct set_current_state(interruptible); smp_mb(); - if (dm_request_based(md)) { - spin_lock_irqsave(q->queue_lock, flags); - if (!queue_in_flight(q)) { - spin_unlock_irqrestore(q->queue_lock, flags); - break; - } - spin_unlock_irqrestore(q->queue_lock, flags); - } else if (!md_in_flight(md)) + if (!md_in_flight(md)) break; if (interruptible == TASK_INTERRUPTIBLE &&