From patchwork Thu Jul 21 19:34:21 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Rees X-Patchwork-Id: 996962 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter2.kernel.org (8.14.4/8.14.4) with ESMTP id p6LJYV0s015518 for ; Thu, 21 Jul 2011 19:34:52 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753752Ab1GUTeu (ORCPT ); Thu, 21 Jul 2011 15:34:50 -0400 Received: from merit-proxy02.merit.edu ([207.75.116.194]:40417 "EHLO merit-proxy02.merit.edu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753745Ab1GUTet (ORCPT ); Thu, 21 Jul 2011 15:34:49 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by merit-proxy02.merit.edu (Postfix) with ESMTP id 2D2652039B0A; Thu, 21 Jul 2011 15:34:49 -0400 (EDT) X-Virus-Scanned: amavisd-new at merit-proxy02.merit.edu Received: from merit-proxy02.merit.edu ([127.0.0.1]) by localhost (merit-proxy02.merit.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XsNi2TlBqESK; Thu, 21 Jul 2011 15:34:48 -0400 (EDT) Received: from merit.edu (castelmezzano.citi.umich.edu [141.212.112.22]) by merit-proxy02.merit.edu (Postfix) with ESMTPSA id 4314B2039B0B; Thu, 21 Jul 2011 15:34:48 -0400 (EDT) From: Jim Rees To: Trond Myklebust Cc: linux-nfs@vger.kernel.org, peter honeyman Subject: [PATCH v2 21/25] pnfsblock: bl_write_pagelist Date: Thu, 21 Jul 2011 15:34:21 -0400 Message-Id: <1311276865-29484-22-git-send-email-rees@umich.edu> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1311276865-29484-1-git-send-email-rees@umich.edu> References: <1311276865-29484-1-git-send-email-rees@umich.edu> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter2.kernel.org [140.211.167.43]); Thu, 21 Jul 2011 19:34:52 +0000 (UTC) From: Fred Isaman Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly. When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at. [pnfsblock: bl_write_pagelist support functions] [pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS] Signed-off-by: Fred Isaman [pnfsblock: handle errors when read or write pagelist.] Signed-off-by: Zhang Jingwang [pnfs-block: use new write_pagelist api] Signed-off-by: Benny Halevy Signed-off-by: Benny Halevy [SQUASHME: pnfsblock: mds_offset is set in the generic layer] Signed-off-by: Boaz Harrosh Signed-off-by: Benny Halevy [pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED] Signed-off-by: Peng Tao [pnfsblock: SQUASHME: adjust to API change] Signed-off-by: Fred Isaman [pnfsblock: fixup blksize alignment in bl_setup_layoutcommit] Signed-off-by: Benny Halevy Signed-off-by: Benny Halevy [pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS] Signed-off-by: Fred Isaman [pnfsblock: handle errors when read or write pagelist.] Signed-off-by: Zhang Jingwang [pnfs-block: use new write_pagelist api] Signed-off-by: Benny Halevy Signed-off-by: Benny Halevy --- fs/nfs/blocklayout/blocklayout.c | 137 +++++++++++++++++++++++++++++++++++++- 1 files changed, 134 insertions(+), 3 deletions(-) diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c index 764096c..6d6ac0e 100644 --- a/fs/nfs/blocklayout/blocklayout.c +++ b/fs/nfs/blocklayout/blocklayout.c @@ -74,6 +74,19 @@ static int is_hole(struct pnfs_block_extent *be, sector_t isect) return !is_sector_initialized(be->be_inval, isect); } +/* Given the be associated with isect, determine if page data can be + * written to disk. + */ +static int is_writable(struct pnfs_block_extent *be, sector_t isect) +{ + if (be->be_state == PNFS_BLOCK_READWRITE_DATA) + return 1; + else if (be->be_state != PNFS_BLOCK_INVALID_DATA) + return 0; + else + return is_sector_initialized(be->be_inval, isect); +} + static int dont_like_caller(struct nfs_page *req) { @@ -333,11 +346,129 @@ out: return PNFS_NOT_ATTEMPTED; } +/* This is basically copied from mpage_end_io_read */ +static void bl_end_io_write(struct bio *bio, int err) +{ + struct parallel_io *par = bio->bi_private; + const int uptodate = test_bit(BIO_UPTODATE, &bio->bi_flags); + struct nfs_write_data *wdata = (struct nfs_write_data *)par->data; + + if (!uptodate) { + if (!wdata->pnfs_error) + wdata->pnfs_error = -EIO; + bl_set_lo_fail(wdata->lseg); + } + bio_put(bio); + put_parallel(par); +} + +/* Function scheduled for call during bl_end_par_io_write, + * it marks sectors as written and extends the commitlist. + */ +static void bl_write_cleanup(struct work_struct *work) +{ + struct rpc_task *task; + struct nfs_write_data *wdata; + dprintk("%s enter\n", __func__); + task = container_of(work, struct rpc_task, u.tk_work); + wdata = container_of(task, struct nfs_write_data, task); + pnfs_ld_write_done(wdata); +} + +/* Called when last of bios associated with a bl_write_pagelist call finishes */ +static void +bl_end_par_io_write(void *data) +{ + struct nfs_write_data *wdata = data; + + /* STUB - ignoring error handling */ + wdata->task.tk_status = 0; + wdata->verf.committed = NFS_FILE_SYNC; + INIT_WORK(&wdata->task.u.tk_work, bl_write_cleanup); + schedule_work(&wdata->task.u.tk_work); +} + static enum pnfs_try_status -bl_write_pagelist(struct nfs_write_data *wdata, - int sync) +bl_write_pagelist(struct nfs_write_data *wdata, int sync) { - return PNFS_NOT_ATTEMPTED; + int i; + struct bio *bio = NULL; + struct pnfs_block_extent *be = NULL; + sector_t isect, extent_length = 0; + struct parallel_io *par; + loff_t offset = wdata->args.offset; + size_t count = wdata->args.count; + struct page **pages = wdata->args.pages; + int pg_index = wdata->args.pgbase >> PAGE_CACHE_SHIFT; + + dprintk("%s enter, %Zu@%lld\n", __func__, count, offset); + if (!wdata->lseg) { + dprintk("%s no lseg, falling back to MDS\n", __func__); + return PNFS_NOT_ATTEMPTED; + } + if (dont_like_caller(wdata->req)) { + dprintk("%s dont_like_caller failed\n", __func__); + return PNFS_NOT_ATTEMPTED; + } + /* At this point, wdata->pages is a (sequential) list of nfs_pages. + * We want to write each, and if there is an error remove it from + * list and call + * nfs_retry_request(req) to have it redone using nfs. + * QUEST? Do as block or per req? Think have to do per block + * as part of end_bio + */ + par = alloc_parallel(wdata); + if (!par) + return PNFS_NOT_ATTEMPTED; + par->call_ops = *wdata->mds_ops; + par->call_ops.rpc_call_done = bl_rpc_do_nothing; + par->pnfs_callback = bl_end_par_io_write; + /* At this point, have to be more careful with error handling */ + + isect = (sector_t) ((offset & (long)PAGE_CACHE_MASK) >> SECTOR_SHIFT); + for (i = pg_index; i < wdata->npages ; i++) { + if (!extent_length) { + /* We've used up the previous extent */ + bl_put_extent(be); + bio = bl_submit_bio(WRITE, bio); + /* Get the next one */ + be = bl_find_get_extent(BLK_LSEG2EXT(wdata->lseg), + isect, NULL); + if (!be || !is_writable(be, isect)) { + wdata->pnfs_error = -ENOMEM; + goto out; + } + extent_length = be->be_length - + (isect - be->be_f_offset); + } + for (;;) { + if (!bio) { + bio = bio_alloc(GFP_NOIO, wdata->npages - i); + if (!bio) { + wdata->pnfs_error = -ENOMEM; + goto out; + } + bio->bi_sector = isect - be->be_f_offset + + be->be_v_offset; + bio->bi_bdev = be->be_mdev; + bio->bi_end_io = bl_end_io_write; + bio->bi_private = par; + } + if (bio_add_page(bio, pages[i], PAGE_SIZE, 0)) + break; + bio = bl_submit_bio(WRITE, bio); + } + isect += PAGE_CACHE_SECTORS; + extent_length -= PAGE_CACHE_SECTORS; + } + wdata->res.count = (isect << SECTOR_SHIFT) - (offset); + if (count < wdata->res.count) + wdata->res.count = count; +out: + bl_put_extent(be); + bl_submit_bio(WRITE, bio); + put_parallel(par); + return PNFS_ATTEMPTED; } /* FIXME - range ignored */