From patchwork Thu Jan 31 15:45:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10790837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34EB8184E for ; Thu, 31 Jan 2019 15:45:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 281A3290D2 for ; Thu, 31 Jan 2019 15:45:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C38E30FCE; Thu, 31 Jan 2019 15:45:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C28C1290D2 for ; Thu, 31 Jan 2019 15:45:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387621AbfAaPpa (ORCPT ); Thu, 31 Jan 2019 10:45:30 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:43777 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729342AbfAaPpa (ORCPT ); Thu, 31 Jan 2019 10:45:30 -0500 Received: by mail-wr1-f65.google.com with SMTP id r10so3791805wrs.10 for ; Thu, 31 Jan 2019 07:45:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=E4MKslIV05xWh4l1wKj2vgGSS530K1EBt3sGfeVP3ow=; b=dD2FMbgvqAF5xFApR5UqN0fAZE0Y+Nk8IrSn2B8JSDEgpiIGbV8sVruxsd9m3js6eh 9dEli0cHu4W04UwOABoeVSMTmZ/ppFOejQ3oAuVCjmL1PjhVZXMBTfDYLFMZG8m1N6ts ep8fC6pbfHW3V3B6si0sCm2gxvsiVMA5SMsKKvp5gDCuRunK3Vj8gTcfUIaxBWZesmYO I1fQwWTx65liLifgZ3qcjRp5P4dlKzdreR3XqY5m+liOY1Tp0vq2WJbpk8FZ5v4NsaGK gX3N8qiC803nlAlrgpTbxeu6oMU3hTQwHAgqJpFxw7JZs765+Qss4h3O8rrszDhaTU+D gLwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=E4MKslIV05xWh4l1wKj2vgGSS530K1EBt3sGfeVP3ow=; b=dTpN8FYpYPWlNn+vhS04Hf9VYLDmm6KT7N2U361axQif4vmoaLrLSNOoWmzq/EBVsL oUxwIYqvCV+CNTr//DZBfiusFFEZQhQ7FbogxgBlLmnvQG/X62cskJozdZAVdxfeC5ER mVcuq6YeGaqxeS9Crc2OMCdb8uiY07aI6jsbZEIrQsovadv0WVhH6ryDkBpqE3MAOilZ a2LaCO0gHqew76jJM6Gh0QxrPQ7ie9MDMxneArsTgsVsLnZ+dC8zcDheSfs3YidIBnlh gMlKH6PYT+Ohmb19FyGITSMaKIFCkMEcJ+B8DECXaGNv7xC2CAh6W9dswg46rLyxVaVB iVtg== X-Gm-Message-State: AJcUukc3BlI2o/xSNjRLOoSB2b9aONcF5bJqFzFw/q0Fkn3t21/R9Fks 09yYbrIoVnF+bbOiAI+bsBcNQUiw X-Google-Smtp-Source: ALg8bN5K4g/FwuyZba3BkubORSJPUk5eTIbEJSksyfvgL9O/nk5PsabImVm4FPCbpdQDZMWugRo9tg== X-Received: by 2002:adf:ee46:: with SMTP id w6mr35772259wro.261.1548949528552; Thu, 31 Jan 2019 07:45:28 -0800 (PST) Received: from orange.redhat.com (ovpn-brq.redhat.com. [213.175.37.11]) by smtp.gmail.com with ESMTPSA id w16sm5736168wrp.1.2019.01.31.07.45.27 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 31 Jan 2019 07:45:27 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH v2 1/3] rbd: get rid of obj_req->obj_request_count Date: Thu, 31 Jan 2019 16:45:06 +0100 Message-Id: <20190131154508.26050-2-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20190131154508.26050-1-idryomov@gmail.com> References: <20190131154508.26050-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It is effectively unused. Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 2f91dee0ab5f..7f78b814d57f 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -291,7 +291,6 @@ struct rbd_img_request { int result; /* first nonzero obj_request result */ struct list_head object_extents; /* obj_req.ex structs */ - u32 obj_request_count; u32 pending_count; struct kref kref; @@ -1345,7 +1344,6 @@ static inline void rbd_img_obj_request_add(struct rbd_img_request *img_request, /* Image request now owns object's original reference */ obj_request->img_request = img_request; - img_request->obj_request_count++; img_request->pending_count++; dout("%s: img %p obj %p\n", __func__, img_request, obj_request); } @@ -1355,8 +1353,6 @@ static inline void rbd_img_obj_request_del(struct rbd_img_request *img_request, { dout("%s: img %p obj %p\n", __func__, img_request, obj_request); list_del(&obj_request->ex.oe_item); - rbd_assert(img_request->obj_request_count > 0); - img_request->obj_request_count--; rbd_assert(obj_request->img_request == img_request); rbd_obj_request_put(obj_request); } @@ -1672,7 +1668,6 @@ static void rbd_img_request_destroy(struct kref *kref) for_each_obj_request_safe(img_request, obj_request, next_obj_request) rbd_img_obj_request_del(img_request, obj_request); - rbd_assert(img_request->obj_request_count == 0); if (img_request_layered_test(img_request)) { img_request_layered_clear(img_request); From patchwork Thu Jan 31 15:45:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10790841 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC87514E1 for ; Thu, 31 Jan 2019 15:45:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0FD2290D2 for ; Thu, 31 Jan 2019 15:45:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4E9430FCE; Thu, 31 Jan 2019 15:45:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3071D290D2 for ; Thu, 31 Jan 2019 15:45:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387692AbfAaPpd (ORCPT ); Thu, 31 Jan 2019 10:45:33 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:35229 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729342AbfAaPpc (ORCPT ); Thu, 31 Jan 2019 10:45:32 -0500 Received: by mail-wr1-f65.google.com with SMTP id 96so3868067wrb.2 for ; Thu, 31 Jan 2019 07:45:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=3AL2m8djKPyWpc9adBvsz7yXQfP/7lZ+7oaE315WrP8=; b=mhxIyjLKGePXTfC+HAg/cwK8Y7v1FM53HNcVvaU2YynwV+DJf+Ao4d3G5mQS/81khn /oiHtH0tBzErMV/4uCnDJh40GbUaM7TDZbYhWHsHkIf6XYHAR/JQkcs7RD95+ZsnBL+5 +lbKeJh/Z5Y8panW3VN/L4eH0cq0Fm4Megj5+CaceOQc/iVyEVAw15mb068oXJXcVHKS fBMV/L0liCG2OADhsdH5R0fdmLIubBCr0TtFXbz1aeYwq9jYHEE0XQN/M1QZjcltd/Bl FsIGs6YJDPlPBvjzpPjNvaiad5xYvxZql+Waac24YNKeUj/AXxuLmMoI2CiFaFfmkR3h wNIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=3AL2m8djKPyWpc9adBvsz7yXQfP/7lZ+7oaE315WrP8=; b=MKnx7JR6xUpMJEOIaOIEhSEUotQhARvmFjelfmc9QwTdzJ2MiBh/2Xp01iaQcCnwsA g4AN6g379iqkTXBjhXgUKnpfCRiGq7XWQtG8rRp+ZSUsOO3nAfhgjh+MgmkK40Sm9lkJ +S/5Ndsc9hP5SgL9Rm71vzIK21y0QKRL+ZvA3ii2KRbespvDXRVZV/S1w56aHNwzYJW5 XmVbMOmt1+b/onFxKuwBZbFDBjJBdMQyuk0Z/FT7L6DWgKrLn8bCyZWwRaVjHajAWBfb oJ9nICaU28DfCyg9hmxfcvSgGfl5ePfKXmTS4kc4ba63uImGR0m2cXx3DTzGo8cOBSQ/ qF9w== X-Gm-Message-State: AHQUAuZ8CXKiimMLsxMS+hHdVfbWzA96IZPKwgMBl7fJtayOHYqt2UOb jHPm+gU7og+JeW1rpPrZn3CbdPli X-Google-Smtp-Source: AHgI3IYuQjaPMCc4kpjP1VfhfrL1CBYVVHnxpHq9Dymt7T8abVKJZlSu7fiHCDpdAxji/ONq0W1vFA== X-Received: by 2002:adf:f401:: with SMTP id g1mr14437619wro.103.1548949529776; Thu, 31 Jan 2019 07:45:29 -0800 (PST) Received: from orange.redhat.com (ovpn-brq.redhat.com. [213.175.37.11]) by smtp.gmail.com with ESMTPSA id w16sm5736168wrp.1.2019.01.31.07.45.28 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 31 Jan 2019 07:45:28 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH v2 2/3] rbd: handle DISCARD and WRITE_ZEROES separately Date: Thu, 31 Jan 2019 16:45:07 +0100 Message-Id: <20190131154508.26050-3-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20190131154508.26050-1-idryomov@gmail.com> References: <20190131154508.26050-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With discard_zeroes_data gone in commit 48920ff2a5a9 ("block: remove the discard_zeroes_data flag"), continuing to provide this guarantee is pointless: applications can't query it and discards can only be used for deallocating. Add OBJ_OP_ZEROOUT and move the existing logic under it. As the first step to divorcing OBJ_OP_DISCARD, stop worrying about copyups but keep special casing whole-object layered discards. Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 61 ++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 7f78b814d57f..24c6d48e8fe5 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -214,6 +214,7 @@ enum obj_operation_type { OBJ_OP_READ = 1, OBJ_OP_WRITE, OBJ_OP_DISCARD, + OBJ_OP_ZEROOUT, }; /* @@ -857,6 +858,8 @@ static char* obj_op_name(enum obj_operation_type op_type) return "write"; case OBJ_OP_DISCARD: return "discard"; + case OBJ_OP_ZEROOUT: + return "zeroout"; default: return "???"; } @@ -1419,6 +1422,7 @@ static bool rbd_img_is_write(struct rbd_img_request *img_req) return false; case OBJ_OP_WRITE: case OBJ_OP_DISCARD: + case OBJ_OP_ZEROOUT: return true; default: BUG(); @@ -1841,7 +1845,40 @@ static int rbd_obj_setup_write(struct rbd_obj_request *obj_req) return 0; } -static void __rbd_obj_setup_discard(struct rbd_obj_request *obj_req, +static u16 truncate_or_zero_opcode(struct rbd_obj_request *obj_req) +{ + return rbd_obj_is_tail(obj_req) ? CEPH_OSD_OP_TRUNCATE : + CEPH_OSD_OP_ZERO; +} + +static int rbd_obj_setup_discard(struct rbd_obj_request *obj_req) +{ + int ret; + + /* reverse map the entire object onto the parent */ + ret = rbd_obj_calc_img_extents(obj_req, true); + if (ret) + return ret; + + obj_req->osd_req = rbd_osd_req_create(obj_req, 1); + if (!obj_req->osd_req) + return -ENOMEM; + + if (rbd_obj_is_entire(obj_req) && !obj_req->num_img_extents) { + osd_req_op_init(obj_req->osd_req, 0, CEPH_OSD_OP_DELETE, 0); + } else { + osd_req_op_extent_init(obj_req->osd_req, 0, + truncate_or_zero_opcode(obj_req), + obj_req->ex.oe_off, obj_req->ex.oe_len, + 0, 0); + } + + obj_req->write_state = RBD_OBJ_WRITE_FLAT; + rbd_osd_req_format_write(obj_req); + return 0; +} + +static void __rbd_obj_setup_zeroout(struct rbd_obj_request *obj_req, unsigned int which) { u16 opcode; @@ -1856,10 +1893,8 @@ static void __rbd_obj_setup_discard(struct rbd_obj_request *obj_req, CEPH_OSD_OP_DELETE, 0); opcode = 0; } - } else if (rbd_obj_is_tail(obj_req)) { - opcode = CEPH_OSD_OP_TRUNCATE; } else { - opcode = CEPH_OSD_OP_ZERO; + opcode = truncate_or_zero_opcode(obj_req); } if (opcode) @@ -1871,7 +1906,7 @@ static void __rbd_obj_setup_discard(struct rbd_obj_request *obj_req, rbd_osd_req_format_write(obj_req); } -static int rbd_obj_setup_discard(struct rbd_obj_request *obj_req) +static int rbd_obj_setup_zeroout(struct rbd_obj_request *obj_req) { unsigned int num_osd_ops, which = 0; int ret; @@ -1907,7 +1942,7 @@ static int rbd_obj_setup_discard(struct rbd_obj_request *obj_req) return ret; } - __rbd_obj_setup_discard(obj_req, which); + __rbd_obj_setup_zeroout(obj_req, which); return 0; } @@ -1932,6 +1967,9 @@ static int __rbd_img_fill_request(struct rbd_img_request *img_req) case OBJ_OP_DISCARD: ret = rbd_obj_setup_discard(obj_req); break; + case OBJ_OP_ZEROOUT: + ret = rbd_obj_setup_zeroout(obj_req); + break; default: rbd_assert(0); } @@ -2392,9 +2430,9 @@ static int rbd_obj_issue_copyup(struct rbd_obj_request *obj_req, u32 bytes) case OBJ_OP_WRITE: __rbd_obj_setup_write(obj_req, 1); break; - case OBJ_OP_DISCARD: + case OBJ_OP_ZEROOUT: rbd_assert(!rbd_obj_is_entire(obj_req)); - __rbd_obj_setup_discard(obj_req, 1); + __rbd_obj_setup_zeroout(obj_req, 1); break; default: rbd_assert(0); @@ -2524,6 +2562,7 @@ static bool __rbd_obj_handle_request(struct rbd_obj_request *obj_req) case OBJ_OP_WRITE: return rbd_obj_handle_write(obj_req); case OBJ_OP_DISCARD: + case OBJ_OP_ZEROOUT: if (rbd_obj_handle_write(obj_req)) { /* * Hide -ENOENT from delete/truncate/zero -- discarding @@ -3636,9 +3675,11 @@ static void rbd_queue_workfn(struct work_struct *work) switch (req_op(rq)) { case REQ_OP_DISCARD: - case REQ_OP_WRITE_ZEROES: op_type = OBJ_OP_DISCARD; break; + case REQ_OP_WRITE_ZEROES: + op_type = OBJ_OP_ZEROOUT; + break; case REQ_OP_WRITE: op_type = OBJ_OP_WRITE; break; @@ -3718,7 +3759,7 @@ static void rbd_queue_workfn(struct work_struct *work) img_request->rq = rq; snapc = NULL; /* img_request consumes a ref */ - if (op_type == OBJ_OP_DISCARD) + if (op_type == OBJ_OP_DISCARD || op_type == OBJ_OP_ZEROOUT) result = rbd_img_fill_nodata(img_request, offset, length); else result = rbd_img_fill_from_bio(img_request, offset, length, From patchwork Thu Jan 31 15:45:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10790839 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E5B76139A for ; Thu, 31 Jan 2019 15:45:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D9D4F290D2 for ; Thu, 31 Jan 2019 15:45:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDF6730FCE; Thu, 31 Jan 2019 15:45:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43EB4290D2 for ; Thu, 31 Jan 2019 15:45:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387728AbfAaPpd (ORCPT ); Thu, 31 Jan 2019 10:45:33 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:42847 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387662AbfAaPpc (ORCPT ); Thu, 31 Jan 2019 10:45:32 -0500 Received: by mail-wr1-f68.google.com with SMTP id q18so3797740wrx.9 for ; Thu, 31 Jan 2019 07:45:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=+AHLZNSbyP+AaKJWsPd+igjwlgsBMvdjORjMvx4b/RY=; b=FA2hTzDhKpMUr+K8NYhSsre4QF22YyGG/qZkIBuRG8Dej5U0YWuR/QK2AscN028N2n rA3DAeBDeCc+1yUjjwF0ARg5mNThoiRir3YnSJkpVEOrDl+vZUIAZV4Ht9pqULDHMEza nHfq74H3A1BNndQmk2ZVm62M7TSQGSfehCuHWO+wlNQGebvmxnLkvGu5RjBcgi/4RbDl gKzrTwG49r++EWshI33xDWUsWflLSO/pwQY9osYEZFkGlnGQXgo7fRFNymk9h1gZZNfi +fesvZIbWf0saKhkR3W+EB7MnmJ1yS2RpQmQ+DqdytZDf/X7IXPvSzomdjFNCPs5NRO6 ltHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+AHLZNSbyP+AaKJWsPd+igjwlgsBMvdjORjMvx4b/RY=; b=JYR3Hcg45/VHBpTdueADBoaxLf6ZGz8Sv70IzP9j64HZ+FM/4lQrEEjyK9joeCxPLK IgmmfwoY54xS1f8KNVa74Fm4ZrzaGTIWe1coQUQi/eP24UFj6TVZmjkU1I5ZS1NCgQxh HWP3mnqWw7BQDlGrnrdFK589+/fKw+PdBC6eWL5fk1CcsIN38fVIEu6WZJI5QBZH4LmY oUVN4PJNZ0QdYtwuHYzuTMFR4/50eVJjuHq4Xs4uyJvBDoTPtxMZ1ABhqp5evNv6jCg8 rgzOO1eKIBTvvoXosBNS7VenjJRrmARdGFML5hebZBPCAResolUKsFgrXjF2J9dMpY7Y TphA== X-Gm-Message-State: AJcUukcQq4GZiohSjW83eN3nIqANvBsnwYXYXb7EGeWnH4JDTfcqj17+ ELa21TFMisM/JfPBQXyEo1y8oMli X-Google-Smtp-Source: ALg8bN4iX68S5wpmh0Gtnb1dDImd4UU9icZedVUEDMOciROkXfUd2TlvMnLmMHAYSj/ionvftEjFDw== X-Received: by 2002:adf:e3c8:: with SMTP id k8mr34945469wrm.83.1548949530911; Thu, 31 Jan 2019 07:45:30 -0800 (PST) Received: from orange.redhat.com (ovpn-brq.redhat.com. [213.175.37.11]) by smtp.gmail.com with ESMTPSA id w16sm5736168wrp.1.2019.01.31.07.45.29 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 31 Jan 2019 07:45:30 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH v2 3/3] rbd: round off and ignore discards that are too small Date: Thu, 31 Jan 2019 16:45:08 +0100 Message-Id: <20190131154508.26050-4-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20190131154508.26050-1-idryomov@gmail.com> References: <20190131154508.26050-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If, after rounding off, the discard request is smaller than alloc_size, drop it on the floor in __rbd_img_fill_request(). Default alloc_size to 64k. This should cover both HDD and SSD based bluestore OSDs and somewhat improve things for filestore. For OSDs on filestore with filestore_punch_hole = false, alloc_size is best set to object size in order to allow deletes and truncates and disallow zero op. Signed-off-by: Ilya Dryomov --- drivers/block/rbd.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 55 insertions(+), 6 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 24c6d48e8fe5..14418d28db60 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -733,6 +733,7 @@ static struct rbd_client *rbd_client_find(struct ceph_options *ceph_opts) */ enum { Opt_queue_depth, + Opt_alloc_size, Opt_lock_timeout, Opt_last_int, /* int args above */ @@ -749,6 +750,7 @@ enum { static match_table_t rbd_opts_tokens = { {Opt_queue_depth, "queue_depth=%d"}, + {Opt_alloc_size, "alloc_size=%d"}, {Opt_lock_timeout, "lock_timeout=%d"}, /* int args above */ {Opt_pool_ns, "_pool_ns=%s"}, @@ -765,6 +767,7 @@ static match_table_t rbd_opts_tokens = { struct rbd_options { int queue_depth; + int alloc_size; unsigned long lock_timeout; bool read_only; bool lock_on_read; @@ -773,6 +776,7 @@ struct rbd_options { }; #define RBD_QUEUE_DEPTH_DEFAULT BLKDEV_MAX_RQ +#define RBD_ALLOC_SIZE_DEFAULT (64 * 1024) #define RBD_LOCK_TIMEOUT_DEFAULT 0 /* no timeout */ #define RBD_READ_ONLY_DEFAULT false #define RBD_LOCK_ON_READ_DEFAULT false @@ -812,6 +816,17 @@ static int parse_rbd_opts_token(char *c, void *private) } pctx->opts->queue_depth = intval; break; + case Opt_alloc_size: + if (intval < 1) { + pr_err("alloc_size out of range\n"); + return -EINVAL; + } + if (!is_power_of_2(intval)) { + pr_err("alloc_size must be a power of 2\n"); + return -EINVAL; + } + pctx->opts->alloc_size = intval; + break; case Opt_lock_timeout: /* 0 is "wait forever" (i.e. infinite timeout) */ if (intval < 0 || intval > INT_MAX / 1000) { @@ -1853,8 +1868,27 @@ static u16 truncate_or_zero_opcode(struct rbd_obj_request *obj_req) static int rbd_obj_setup_discard(struct rbd_obj_request *obj_req) { + struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev; + u64 off = obj_req->ex.oe_off; + u64 next_off = obj_req->ex.oe_off + obj_req->ex.oe_len; int ret; + /* + * Align the range to alloc_size boundary and punt on discards + * that are too small to free up any space. + * + * alloc_size == object_size && is_tail() is a special case for + * filestore with filestore_punch_hole = false, needed to allow + * truncate (in addition to delete). + */ + if (rbd_dev->opts->alloc_size != rbd_dev->layout.object_size || + !rbd_obj_is_tail(obj_req)) { + off = round_up(off, rbd_dev->opts->alloc_size); + next_off = round_down(next_off, rbd_dev->opts->alloc_size); + if (off >= next_off) + return 1; + } + /* reverse map the entire object onto the parent */ ret = rbd_obj_calc_img_extents(obj_req, true); if (ret) @@ -1867,10 +1901,12 @@ static int rbd_obj_setup_discard(struct rbd_obj_request *obj_req) if (rbd_obj_is_entire(obj_req) && !obj_req->num_img_extents) { osd_req_op_init(obj_req->osd_req, 0, CEPH_OSD_OP_DELETE, 0); } else { + dout("%s %p %llu~%llu -> %llu~%llu\n", __func__, + obj_req, obj_req->ex.oe_off, obj_req->ex.oe_len, + off, next_off - off); osd_req_op_extent_init(obj_req->osd_req, 0, truncate_or_zero_opcode(obj_req), - obj_req->ex.oe_off, obj_req->ex.oe_len, - 0, 0); + off, next_off - off, 0, 0); } obj_req->write_state = RBD_OBJ_WRITE_FLAT; @@ -1953,10 +1989,10 @@ static int rbd_obj_setup_zeroout(struct rbd_obj_request *obj_req) */ static int __rbd_img_fill_request(struct rbd_img_request *img_req) { - struct rbd_obj_request *obj_req; + struct rbd_obj_request *obj_req, *next_obj_req; int ret; - for_each_obj_request(img_req, obj_req) { + for_each_obj_request_safe(img_req, obj_req, next_obj_req) { switch (img_req->op_type) { case OBJ_OP_READ: ret = rbd_obj_setup_read(obj_req); @@ -1973,8 +2009,14 @@ static int __rbd_img_fill_request(struct rbd_img_request *img_req) default: rbd_assert(0); } - if (ret) + if (ret < 0) return ret; + if (ret > 0) { + img_req->xferred += obj_req->ex.oe_len; + img_req->pending_count--; + rbd_img_obj_request_del(img_req, obj_req); + continue; + } ret = ceph_osdc_alloc_messages(obj_req->osd_req, GFP_NOIO); if (ret) @@ -3764,7 +3806,7 @@ static void rbd_queue_workfn(struct work_struct *work) else result = rbd_img_fill_from_bio(img_request, offset, length, rq->bio); - if (result) + if (result || !img_request->pending_count) goto err_img_request; rbd_img_request_submit(img_request); @@ -5425,6 +5467,7 @@ static int rbd_add_parse_args(const char *buf, pctx.opts->read_only = RBD_READ_ONLY_DEFAULT; pctx.opts->queue_depth = RBD_QUEUE_DEPTH_DEFAULT; + pctx.opts->alloc_size = RBD_ALLOC_SIZE_DEFAULT; pctx.opts->lock_timeout = RBD_LOCK_TIMEOUT_DEFAULT; pctx.opts->lock_on_read = RBD_LOCK_ON_READ_DEFAULT; pctx.opts->exclusive = RBD_EXCLUSIVE_DEFAULT; @@ -5922,6 +5965,12 @@ static ssize_t do_rbd_add(struct bus_type *bus, if (rbd_dev->spec->snap_id != CEPH_NOSNAP) rbd_dev->opts->read_only = true; + if (rbd_dev->opts->alloc_size > rbd_dev->layout.object_size) { + rbd_warn(rbd_dev, "alloc_size adjusted to %u", + rbd_dev->layout.object_size); + rbd_dev->opts->alloc_size = rbd_dev->layout.object_size; + } + rc = rbd_dev_device_setup(rbd_dev); if (rc) goto err_out_image_probe;