From patchwork Tue Feb 7 12:28:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 9559755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 850EA602B1 for ; Tue, 7 Feb 2017 12:28:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 766AA2522B for ; Tue, 7 Feb 2017 12:28:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B4E8280DE; Tue, 7 Feb 2017 12:28:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E726E2522B for ; Tue, 7 Feb 2017 12:28:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753927AbdBGM2e (ORCPT ); Tue, 7 Feb 2017 07:28:34 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34446 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753829AbdBGM2a (ORCPT ); Tue, 7 Feb 2017 07:28:30 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 555C761BBA; Tue, 7 Feb 2017 12:28:31 +0000 (UTC) Received: from tleilax.poochiereds.net (ovpn-120-47.rdu2.redhat.com [10.10.120.47] (may be forged)) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v17CSTZT008114; Tue, 7 Feb 2017 07:28:30 -0500 From: Jeff Layton To: ceph-devel@vger.kernel.org Cc: zyan@redhat.com, sage@redhat.com, idryomov@gmail.com, jspray@redhat.com Subject: [PATCH v3 1/5] libceph: allow requests to return immediately on full conditions if caller wishes Date: Tue, 7 Feb 2017 07:28:24 -0500 Message-Id: <20170207122828.5550-2-jlayton@redhat.com> In-Reply-To: <20170207122828.5550-1-jlayton@redhat.com> References: <20170207122828.5550-1-jlayton@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 07 Feb 2017 12:28:31 +0000 (UTC) Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Right now, cephfs will cancel any in-flight OSD write operations when a new map comes in that shows the OSD or pool as full, but nothing prevents new requests from stalling out after that point. If the caller knows that it will want an immediate error return instead of blocking on a full or at-quota error condition then allow it to set a flag to request that behavior. Cephfs write requests will always set that flag. Signed-off-by: Jeff Layton --- fs/ceph/addr.c | 4 ++++ fs/ceph/file.c | 4 ++++ include/linux/ceph/osd_client.h | 1 + net/ceph/osd_client.c | 6 ++++++ 4 files changed, 15 insertions(+) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 4547bbf80e4f..308787eeee2c 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -1040,6 +1040,7 @@ static int ceph_writepages_start(struct address_space *mapping, req->r_callback = writepages_finish; req->r_inode = inode; + req->r_abort_on_full = true; /* Format the osd request message and submit the write */ len = 0; @@ -1689,6 +1690,7 @@ int ceph_uninline_data(struct file *filp, struct page *locked_page) } req->r_mtime = inode->i_mtime; + req->r_abort_on_full = true; err = ceph_osdc_start_request(&fsc->client->osdc, req, false); if (!err) err = ceph_osdc_wait_request(&fsc->client->osdc, req); @@ -1732,6 +1734,7 @@ int ceph_uninline_data(struct file *filp, struct page *locked_page) } req->r_mtime = inode->i_mtime; + req->r_abort_on_full = true; err = ceph_osdc_start_request(&fsc->client->osdc, req, false); if (!err) err = ceph_osdc_wait_request(&fsc->client->osdc, req); @@ -1893,6 +1896,7 @@ static int __ceph_pool_perm_get(struct ceph_inode_info *ci, err = ceph_osdc_start_request(&fsc->client->osdc, rd_req, false); wr_req->r_mtime = ci->vfs_inode.i_mtime; + wr_req->r_abort_on_full = true; err2 = ceph_osdc_start_request(&fsc->client->osdc, wr_req, false); if (!err) diff --git a/fs/ceph/file.c b/fs/ceph/file.c index a91a4f1fc837..987dcb9b566f 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -714,6 +714,7 @@ static void ceph_aio_retry_work(struct work_struct *work) req->r_callback = ceph_aio_complete_req; req->r_inode = inode; req->r_priv = aio_req; + req->r_abort_on_full = true; ret = ceph_osdc_start_request(req->r_osdc, req, false); out: @@ -912,6 +913,7 @@ ceph_direct_read_write(struct kiocb *iocb, struct iov_iter *iter, osd_req_op_init(req, 1, CEPH_OSD_OP_STARTSYNC, 0); req->r_mtime = mtime; + req->r_abort_on_full = true; } osd_req_op_extent_osd_data_pages(req, 0, pages, len, start, @@ -1105,6 +1107,7 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, false, true); req->r_mtime = mtime; + req->r_abort_on_full = true; ret = ceph_osdc_start_request(&fsc->client->osdc, req, false); if (!ret) ret = ceph_osdc_wait_request(&fsc->client->osdc, req); @@ -1557,6 +1560,7 @@ static int ceph_zero_partial_object(struct inode *inode, } req->r_mtime = inode->i_mtime; + req->r_abort_on_full = true; ret = ceph_osdc_start_request(&fsc->client->osdc, req, false); if (!ret) { ret = ceph_osdc_wait_request(&fsc->client->osdc, req); diff --git a/include/linux/ceph/osd_client.h b/include/linux/ceph/osd_client.h index 03a6653d329a..5da666cc5891 100644 --- a/include/linux/ceph/osd_client.h +++ b/include/linux/ceph/osd_client.h @@ -171,6 +171,7 @@ struct ceph_osd_request { int r_result; bool r_got_reply; + bool r_abort_on_full; /* return ENOSPC when full */ struct ceph_osd_client *r_osdc; struct kref r_kref; diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index 3a2417bb6ff0..f68bb42da240 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -49,6 +49,7 @@ static void link_linger(struct ceph_osd *osd, struct ceph_osd_linger_request *lreq); static void unlink_linger(struct ceph_osd *osd, struct ceph_osd_linger_request *lreq); +static void complete_request(struct ceph_osd_request *req, int err); #if 1 static inline bool rwsem_is_wrlocked(struct rw_semaphore *sem) @@ -1636,6 +1637,7 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) enum calc_target_result ct_res; bool need_send = false; bool promoted = false; + int ret = 0; WARN_ON(req->r_tid || req->r_got_reply); dout("%s req %p wrlocked %d\n", __func__, req, wrlocked); @@ -1670,6 +1672,8 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) pr_warn_ratelimited("FULL or reached pool quota\n"); req->r_t.paused = true; maybe_request_map(osdc); + if (req->r_abort_on_full) + ret = -ENOSPC; } else if (!osd_homeless(osd)) { need_send = true; } else { @@ -1686,6 +1690,8 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) link_request(osd, req); if (need_send) send_request(req); + else if (ret) + complete_request(req, ret); mutex_unlock(&osd->lock); if (ct_res == CALC_TARGET_POOL_DNE)