From patchwork Wed Sep 13 16:44:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev\" via" X-Patchwork-Id: 9952033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A725F60245 for ; Wed, 13 Sep 2017 20:19:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99EA628C65 for ; Wed, 13 Sep 2017 20:19:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8CEE628C6E; Wed, 13 Sep 2017 20:19:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BEC4F28C65 for ; Wed, 13 Sep 2017 20:19:53 +0000 (UTC) Received: from localhost ([::1]:44423 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsE8W-0004Cv-5t for patchwork-qemu-devel@patchwork.kernel.org; Wed, 13 Sep 2017 16:19:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44077) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dsAn9-0007vV-Bp for qemu-devel@nongnu.org; Wed, 13 Sep 2017 12:45:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dsAn6-0004qZ-5s for qemu-devel@nongnu.org; Wed, 13 Sep 2017 12:45:35 -0400 Received: from mail-it0-x234.google.com ([2607:f8b0:4001:c0b::234]:49059) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dsAn6-0004pO-0c for qemu-devel@nongnu.org; Wed, 13 Sep 2017 12:45:32 -0400 Received: by mail-it0-x234.google.com with SMTP id c195so4453717itb.1 for ; Wed, 13 Sep 2017 09:45:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=digitalocean.com; s=google; h=from:to:cc:subject:date:message-id; bh=4XrU61y5bq9ghRIMgyhSm9lI5ISA4o1bKOqXmQ5BR8Y=; b=L1WAPMu0Ke6K9t1ZqT4Zcvg9UKeS5r51+zA1GZElQ15cDbwbJPJCYyDxi/rSFKvVPN ND6MS9LamELNX6N7BUtblRJ6jFBGJH4iqRSdQ2dUHPBNb6NRqx5gO/e3+Fn4qGNmEYPN dz/H509+U1G+B+15FCq1PZMv0frniSQOM0/Gs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=4XrU61y5bq9ghRIMgyhSm9lI5ISA4o1bKOqXmQ5BR8Y=; b=IrFIKwEz/EwNOS5wui1FNbwU5C4dVhVCbKekLXmBTSEtAN3hH4BYF8fX3V2/YMWF// ndcdA8xQc9ME6BhVTKa2UOQezZ2clp8g2eKKwa2cNKJsV31J9tnzAaa4x0vuiRu8PW30 MlJhLC3q+h29u1mxoVT8pY3J8Z/tpou5RbDUf/Zi3WZVQRB1fDwkPK0/47wl9RmT7LgY LjxLMIGZXJHMRTMxdwaEc6oTToxqgeA4Xe7zTwkaxydPSX5kQ4fAI6g+krELWin8ZkcP UmkQbEM+YBHBnp1GFECIFWi023osNzBBrUieQgUZtEUKrA5aI0rlnYqb5BADEvs74NDO o9Bw== X-Gm-Message-State: AHPjjUhxwtoyjBX0A0zUmHSMcWyKQT7HU5PPm/NFxG6U+Hlcl2rg/LgI dSvffMaCHVWkbbWy X-Google-Smtp-Source: AOwi7QDd2mTyOrug3AiJdn5dmJt6iuLykit8LmY12vQnirHYwDMEDsOD7vs641/ik/Pzfk1Hkojjfw== X-Received: by 10.36.31.200 with SMTP id d191mr5584553itd.81.1505321129961; Wed, 13 Sep 2017 09:45:29 -0700 (PDT) Received: from kigali.local (d172-219-57-182.abhsia.telus.net. [172.219.57.182]) by smtp.gmail.com with ESMTPSA id l82sm7782755ioi.68.2017.09.13.09.45.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Sep 2017 09:45:28 -0700 (PDT) Received: by kigali.local (Postfix, from userid 501) id 1EDF2F1656A; Wed, 13 Sep 2017 10:45:26 -0600 (MDT) To: qemu-block@nongnu.org Date: Wed, 13 Sep 2017 10:44:24 -0600 Message-Id: <20170913164424.32129-1-awg@digitalocean.com> X-Mailer: git-send-email 2.14.1 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4001:c0b::234 X-Mailman-Approved-At: Wed, 13 Sep 2017 16:19:01 -0400 Subject: [Qemu-devel] [PATCH] rbd: Detect rbd image resizes and propagate them X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adam Wolfe Gordon via Qemu-devel From: "Denis V. Lunev\" via" Reply-To: Adam Wolfe Gordon Cc: Kevin Wolf , Josh Durgin , Adam Wolfe Gordon , "open list:All patches CC here" , Max Reitz , Jeff Cody Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Register a watcher with rbd so that we get notified when an image is resized. Propagate resizes to parent block devices so that guest devices get resized without user intervention. Signed-off-by: Adam Wolfe Gordon --- Hello! We are using this patch in production at DigitalOcean and have tested it fairly extensively. However, we use the same block device configuration everywhere, so I'd be interested to get an answer to the question I've left in the code: > /* NOTE: This assumes there's only one layer between us and the > block-backend. Is this always true? */ If that isn't true, this patch will need a bit of work to traverse up the block device tree and find the block-backend. Of course, any other feedback would be welcome too - this is my first foray into the qemu code. Regards, Adam block/rbd.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/block/rbd.c b/block/rbd.c index 144f350e1f..1c9fcbec1f 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -96,6 +96,8 @@ typedef struct BDRVRBDState { rbd_image_t image; char *image_name; char *snap; + uint64_t watcher_handle; + uint64_t size_bytes; } BDRVRBDState; static char *qemu_rbd_next_tok(char *src, char delim, char **p) @@ -540,6 +542,67 @@ out: return rados_str; } +/* BH for when rbd notifies us of an update. */ +static void qemu_rbd_update_bh(void *arg) +{ + BlockDriverState *parent, *bs = arg; + BDRVRBDState *s = bs->opaque; + bool was_variable_length; + uint64_t new_size_bytes; + int64_t new_parent_len; + BdrvChild *c; + int r; + + r = rbd_get_size(s->image, &new_size_bytes); + if (r < 0) { + error_report("error reading size for %s: %s", s->name, strerror(-r)); + return; + } + + /* Avoid no-op resizes on non-resize notifications. */ + if (new_size_bytes == s->size_bytes) { + error_printf("skipping non-resize rbd cb\n"); + return; + } + + /* NOTE: This assumes there's only one layer between us and the + block-backend. Is this always true? */ + parent = bs->inherits_from; + if (parent == NULL) { + error_report("bs %s does not have parent", bdrv_get_device_or_node_name(bs)); + return; + } + + /* Force parents to re-read our size. */ + was_variable_length = bs->drv->has_variable_length; + bs->drv->has_variable_length = true; + new_parent_len = bdrv_getlength(parent); + if (new_parent_len < 0) { + error_report("getlength failed on parent %s", bdrv_get_device_or_node_name(parent)); + bs->drv->has_variable_length = was_variable_length; + return; + } + bs->drv->has_variable_length = was_variable_length; + + /* Notify block backends that that we have resized. + Copied from bdrv_parent_cb_resize. */ + QLIST_FOREACH(c, &parent->parents, next_parent) { + if (c->role->resize) { + c->role->resize(c); + } + } + + s->size_bytes = new_size_bytes; +} + +/* Called from non-qemu thread - careful! */ +static void qemu_rbd_update_cb(void *arg) +{ + BlockDriverState *bs = arg; + + aio_bh_schedule_oneshot(bdrv_get_aio_context(bs), qemu_rbd_update_bh, bs); +} + static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, Error **errp) { @@ -672,9 +735,25 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } } + r = rbd_update_watch(s->image, &s->watcher_handle, qemu_rbd_update_cb, bs); + if (r < 0) { + error_setg_errno(errp, -r, "error registering watcher on %s", s->name); + goto failed_watch; + } + + r = rbd_get_size(s->image, &s->size_bytes); + if (r < 0) { + error_setg_errno(errp, -r, "error reading size for %s", s->name); + goto failed_sz; + } + qemu_opts_del(opts); return 0; +failed_sz: + rbd_update_unwatch(s->image, s->watcher_handle); +failed_watch: + rbd_close(s->image); failed_open: rados_ioctx_destroy(s->io_ctx); failed_shutdown: @@ -712,6 +791,7 @@ static void qemu_rbd_close(BlockDriverState *bs) { BDRVRBDState *s = bs->opaque; + rbd_update_unwatch(s->image, s->watcher_handle); rbd_close(s->image); rados_ioctx_destroy(s->io_ctx); g_free(s->snap);