From patchwork Sun Dec 27 16:42:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D587EC433E0 for ; Sun, 27 Dec 2020 16:56:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2C1E322512 for ; Sun, 27 Dec 2020 16:56:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2C1E322512 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49638 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZLF-00046Y-9Z for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:56:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44882) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZIt-0002VW-3M for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:53:59 -0500 Received: from kerio.kamp.de ([195.62.97.192]:34018) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZIq-0007ZP-Kd for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:53:58 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22368 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 5536513D855; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 1/7] block/rbd: bump librbd requirement to luminous release Date: Sun, 27 Dec 2020 17:42:30 +0100 Message-Id: <20201227164236.10143-2-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" even luminous (version 12.2) is unmaintained for over 3 years now. Bump the requirement to get rid of the ifdef'ry in the code. Signed-off-by: Peter Lieven --- block/rbd.c | 120 ++++------------------------------------------------ configure | 7 +-- 2 files changed, 12 insertions(+), 115 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 9bd2bce716..650e27c351 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -55,24 +55,10 @@ * leading "\". */ -/* rbd_aio_discard added in 0.1.2 */ -#if LIBRBD_VERSION_CODE >= LIBRBD_VERSION(0, 1, 2) -#define LIBRBD_SUPPORTS_DISCARD -#else -#undef LIBRBD_SUPPORTS_DISCARD -#endif - #define OBJ_MAX_SIZE (1UL << OBJ_DEFAULT_OBJ_ORDER) #define RBD_MAX_SNAPS 100 -/* The LIBRBD_SUPPORTS_IOVEC is defined in librbd.h */ -#ifdef LIBRBD_SUPPORTS_IOVEC -#define LIBRBD_USE_IOVEC 1 -#else -#define LIBRBD_USE_IOVEC 0 -#endif - typedef enum { RBD_AIO_READ, RBD_AIO_WRITE, @@ -84,7 +70,6 @@ typedef struct RBDAIOCB { BlockAIOCB common; int64_t ret; QEMUIOVector *qiov; - char *bounce; RBDAIOCmd cmd; int error; struct BDRVRBDState *s; @@ -94,7 +79,6 @@ typedef struct RADOSCB { RBDAIOCB *acb; struct BDRVRBDState *s; int64_t size; - char *buf; int64_t ret; } RADOSCB; @@ -332,13 +316,9 @@ static int qemu_rbd_set_keypairs(rados_t cluster, const char *keypairs_json, static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs) { - if (LIBRBD_USE_IOVEC) { - RBDAIOCB *acb = rcb->acb; - iov_memset(acb->qiov->iov, acb->qiov->niov, offs, 0, - acb->qiov->size - offs); - } else { - memset(rcb->buf + offs, 0, rcb->size - offs); - } + RBDAIOCB *acb = rcb->acb; + iov_memset(acb->qiov->iov, acb->qiov->niov, offs, 0, + acb->qiov->size - offs); } /* FIXME Deprecate and remove keypairs or make it available in QMP. */ @@ -493,13 +473,6 @@ static void qemu_rbd_complete_aio(RADOSCB *rcb) g_free(rcb); - if (!LIBRBD_USE_IOVEC) { - if (acb->cmd == RBD_AIO_READ) { - qemu_iovec_from_buf(acb->qiov, 0, acb->bounce, acb->qiov->size); - } - qemu_vfree(acb->bounce); - } - acb->common.cb(acb->common.opaque, (acb->ret > 0 ? 0 : acb->ret)); qemu_aio_unref(acb); @@ -866,28 +839,6 @@ static void rbd_finish_aiocb(rbd_completion_t c, RADOSCB *rcb) rbd_finish_bh, rcb); } -static int rbd_aio_discard_wrapper(rbd_image_t image, - uint64_t off, - uint64_t len, - rbd_completion_t comp) -{ -#ifdef LIBRBD_SUPPORTS_DISCARD - return rbd_aio_discard(image, off, len, comp); -#else - return -ENOTSUP; -#endif -} - -static int rbd_aio_flush_wrapper(rbd_image_t image, - rbd_completion_t comp) -{ -#ifdef LIBRBD_SUPPORTS_AIO_FLUSH - return rbd_aio_flush(image, comp); -#else - return -ENOTSUP; -#endif -} - static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, int64_t off, QEMUIOVector *qiov, @@ -910,21 +861,6 @@ static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, rcb = g_new(RADOSCB, 1); - if (!LIBRBD_USE_IOVEC) { - if (cmd == RBD_AIO_DISCARD || cmd == RBD_AIO_FLUSH) { - acb->bounce = NULL; - } else { - acb->bounce = qemu_try_blockalign(bs, qiov->size); - if (acb->bounce == NULL) { - goto failed; - } - } - if (cmd == RBD_AIO_WRITE) { - qemu_iovec_to_buf(acb->qiov, 0, acb->bounce, qiov->size); - } - rcb->buf = acb->bounce; - } - acb->ret = 0; acb->error = 0; acb->s = s; @@ -938,7 +874,7 @@ static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, } switch (cmd) { - case RBD_AIO_WRITE: { + case RBD_AIO_WRITE: /* * RBD APIs don't allow us to write more than actual size, so in order * to support growing images, we resize the image before write @@ -950,25 +886,16 @@ static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, goto failed_completion; } } -#ifdef LIBRBD_SUPPORTS_IOVEC - r = rbd_aio_writev(s->image, qiov->iov, qiov->niov, off, c); -#else - r = rbd_aio_write(s->image, off, size, rcb->buf, c); -#endif + r = rbd_aio_writev(s->image, qiov->iov, qiov->niov, off, c); break; - } case RBD_AIO_READ: -#ifdef LIBRBD_SUPPORTS_IOVEC - r = rbd_aio_readv(s->image, qiov->iov, qiov->niov, off, c); -#else - r = rbd_aio_read(s->image, off, size, rcb->buf, c); -#endif + r = rbd_aio_readv(s->image, qiov->iov, qiov->niov, off, c); break; case RBD_AIO_DISCARD: - r = rbd_aio_discard_wrapper(s->image, off, size, c); + r = rbd_aio_discard(s->image, off, size, c); break; case RBD_AIO_FLUSH: - r = rbd_aio_flush_wrapper(s->image, c); + r = rbd_aio_flush(s->image, c); break; default: r = -EINVAL; @@ -983,9 +910,6 @@ failed_completion: rbd_aio_release(c); failed: g_free(rcb); - if (!LIBRBD_USE_IOVEC) { - qemu_vfree(acb->bounce); - } qemu_aio_unref(acb); return NULL; @@ -1011,7 +935,6 @@ static BlockAIOCB *qemu_rbd_aio_pwritev(BlockDriverState *bs, RBD_AIO_WRITE); } -#ifdef LIBRBD_SUPPORTS_AIO_FLUSH static BlockAIOCB *qemu_rbd_aio_flush(BlockDriverState *bs, BlockCompletionFunc *cb, void *opaque) @@ -1019,20 +942,6 @@ static BlockAIOCB *qemu_rbd_aio_flush(BlockDriverState *bs, return rbd_start_aio(bs, 0, NULL, 0, cb, opaque, RBD_AIO_FLUSH); } -#else - -static int qemu_rbd_co_flush(BlockDriverState *bs) -{ -#if LIBRBD_VERSION_CODE >= LIBRBD_VERSION(0, 1, 1) - /* rbd_flush added in 0.1.1 */ - BDRVRBDState *s = bs->opaque; - return rbd_flush(s->image); -#else - return 0; -#endif -} -#endif - static int qemu_rbd_getinfo(BlockDriverState *bs, BlockDriverInfo *bdi) { BDRVRBDState *s = bs->opaque; @@ -1198,7 +1107,6 @@ static int qemu_rbd_snap_list(BlockDriverState *bs, return snap_count; } -#ifdef LIBRBD_SUPPORTS_DISCARD static BlockAIOCB *qemu_rbd_aio_pdiscard(BlockDriverState *bs, int64_t offset, int bytes, @@ -1208,9 +1116,7 @@ static BlockAIOCB *qemu_rbd_aio_pdiscard(BlockDriverState *bs, return rbd_start_aio(bs, offset, NULL, bytes, cb, opaque, RBD_AIO_DISCARD); } -#endif -#ifdef LIBRBD_SUPPORTS_INVALIDATE static void coroutine_fn qemu_rbd_co_invalidate_cache(BlockDriverState *bs, Error **errp) { @@ -1220,7 +1126,6 @@ static void coroutine_fn qemu_rbd_co_invalidate_cache(BlockDriverState *bs, error_setg_errno(errp, -r, "Failed to invalidate the cache"); } } -#endif static QemuOptsList qemu_rbd_create_opts = { .name = "rbd-create-opts", @@ -1278,23 +1183,14 @@ static BlockDriver bdrv_rbd = { .bdrv_aio_preadv = qemu_rbd_aio_preadv, .bdrv_aio_pwritev = qemu_rbd_aio_pwritev, -#ifdef LIBRBD_SUPPORTS_AIO_FLUSH .bdrv_aio_flush = qemu_rbd_aio_flush, -#else - .bdrv_co_flush_to_disk = qemu_rbd_co_flush, -#endif - -#ifdef LIBRBD_SUPPORTS_DISCARD .bdrv_aio_pdiscard = qemu_rbd_aio_pdiscard, -#endif .bdrv_snapshot_create = qemu_rbd_snap_create, .bdrv_snapshot_delete = qemu_rbd_snap_remove, .bdrv_snapshot_list = qemu_rbd_snap_list, .bdrv_snapshot_goto = qemu_rbd_snap_rollback, -#ifdef LIBRBD_SUPPORTS_INVALIDATE .bdrv_co_invalidate_cache = qemu_rbd_co_invalidate_cache, -#endif .strong_runtime_opts = qemu_rbd_strong_runtime_opts, }; diff --git a/configure b/configure index c228f7c21e..e07f80cab4 100755 --- a/configure +++ b/configure @@ -3663,8 +3663,9 @@ if test "$rbd" != "no" ; then #include #include int main(void) { - rados_t cluster; - rados_create(&cluster, NULL); +#if LIBRBD_VERSION_CODE < LIBRBD_VERSION(1, 12, 0) +#error +#endif return 0; } EOF @@ -3673,7 +3674,7 @@ EOF rbd=yes else if test "$rbd" = "yes" ; then - feature_not_found "rados block device" "Install librbd/ceph devel" + feature_not_found "rados block device" "Install librbd/ceph devel (>= 12.0.0)" fi rbd=no fi From patchwork Sun Dec 27 16:42:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72C63C433DB for ; Sun, 27 Dec 2020 16:48:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C567020857 for ; Sun, 27 Dec 2020 16:48:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C567020857 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZDA-0006do-Va for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:48:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43276) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8G-0001R3-At for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:43:00 -0500 Received: from kerio.kamp.de ([195.62.97.192]:33896) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ89-0003wv-Pm for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:43:00 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22369 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 596F513DD23; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 2/7] block/rbd: store object_size in BDRVRBDState Date: Sun, 27 Dec 2020 17:42:31 +0100 Message-Id: <20201227164236.10143-3-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Lieven --- block/rbd.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 650e27c351..bc8cf8af9b 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -90,6 +90,7 @@ typedef struct BDRVRBDState { char *snap; char *namespace; uint64_t image_size; + uint64_t object_size; } BDRVRBDState; static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, @@ -663,6 +664,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, const QDictEntry *e; Error *local_err = NULL; char *keypairs, *secretid; + rbd_image_info_t info; int r; keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs")); @@ -727,13 +729,15 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, goto failed_open; } - r = rbd_get_size(s->image, &s->image_size); + r = rbd_stat(s->image, &info, sizeof(info)); if (r < 0) { - error_setg_errno(errp, -r, "error getting image size from %s", + error_setg_errno(errp, -r, "error getting image info from %s", s->image_name); rbd_close(s->image); goto failed_open; } + s->image_size = info.size; + s->object_size = info.obj_size; /* If we are using an rbd snapshot, we must be r/o, otherwise * leave as-is */ @@ -945,15 +949,7 @@ static BlockAIOCB *qemu_rbd_aio_flush(BlockDriverState *bs, static int qemu_rbd_getinfo(BlockDriverState *bs, BlockDriverInfo *bdi) { BDRVRBDState *s = bs->opaque; - rbd_image_info_t info; - int r; - - r = rbd_stat(s->image, &info, sizeof(info)); - if (r < 0) { - return r; - } - - bdi->cluster_size = info.obj_size; + bdi->cluster_size = s->object_size; return 0; } From patchwork Sun Dec 27 16:42:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F112C433DB for ; Sun, 27 Dec 2020 16:44:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5BE7207B2 for ; Sun, 27 Dec 2020 16:44:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5BE7207B2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZA0-0003oC-Rp for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:44:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43358) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8J-0001X8-Rc for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:43:03 -0500 Received: from kerio.kamp.de ([195.62.97.192]:33923) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8H-0003yv-U2 for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:43:03 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22357 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 5C95D13DD24; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 3/7] block/rbd: use stored image_size in qemu_rbd_getlength Date: Sun, 27 Dec 2020 17:42:32 +0100 Message-Id: <20201227164236.10143-4-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Lieven --- block/rbd.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index bc8cf8af9b..a2da70e37f 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -956,15 +956,7 @@ static int qemu_rbd_getinfo(BlockDriverState *bs, BlockDriverInfo *bdi) static int64_t qemu_rbd_getlength(BlockDriverState *bs) { BDRVRBDState *s = bs->opaque; - rbd_image_info_t info; - int r; - - r = rbd_stat(s->image, &info, sizeof(info)); - if (r < 0) { - return r; - } - - return info.size; + return s->image_size; } static int coroutine_fn qemu_rbd_co_truncate(BlockDriverState *bs, From patchwork Sun Dec 27 16:42:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CED1AC433E0 for ; Sun, 27 Dec 2020 16:56:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61D0122512 for ; Sun, 27 Dec 2020 16:56:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61D0122512 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZL7-00042B-GG for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:56:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZIt-0002VZ-Nm for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:53:59 -0500 Received: from kerio.kamp.de ([195.62.97.192]:34019) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZIq-0007ZR-Oi for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:53:59 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22370 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 5FB2B13DD25; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 4/7] block/rbd: add bdrv_{attach,detach}_aio_context Date: Sun, 27 Dec 2020 17:42:33 +0100 Message-Id: <20201227164236.10143-5-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Lieven --- block/rbd.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index a2da70e37f..27b232f4d8 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -91,6 +91,7 @@ typedef struct BDRVRBDState { char *namespace; uint64_t image_size; uint64_t object_size; + AioContext *aio_context; } BDRVRBDState; static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, @@ -749,6 +750,8 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } } + s->aio_context = bdrv_get_aio_context(bs); + /* When extending regular files, we get zeros from the OS */ bs->supported_truncate_flags = BDRV_REQ_ZERO_WRITE; @@ -839,8 +842,7 @@ static void rbd_finish_aiocb(rbd_completion_t c, RADOSCB *rcb) rcb->ret = rbd_aio_get_return_value(c); rbd_aio_release(c); - replay_bh_schedule_oneshot_event(bdrv_get_aio_context(acb->common.bs), - rbd_finish_bh, rcb); + replay_bh_schedule_oneshot_event(acb->s->aio_context, rbd_finish_bh, rcb); } static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, @@ -1151,6 +1153,18 @@ static const char *const qemu_rbd_strong_runtime_opts[] = { NULL }; +static void qemu_rbd_attach_aio_context(BlockDriverState *bs, + AioContext *new_context) +{ + BDRVRBDState *s = bs->opaque; + s->aio_context = new_context; +} + +static void qemu_rbd_detach_aio_context(BlockDriverState *bs) +{ + +} + static BlockDriver bdrv_rbd = { .format_name = "rbd", .instance_size = sizeof(BDRVRBDState), @@ -1180,6 +1194,9 @@ static BlockDriver bdrv_rbd = { .bdrv_snapshot_goto = qemu_rbd_snap_rollback, .bdrv_co_invalidate_cache = qemu_rbd_co_invalidate_cache, + .bdrv_attach_aio_context = qemu_rbd_attach_aio_context, + .bdrv_detach_aio_context = qemu_rbd_detach_aio_context, + .strong_runtime_opts = qemu_rbd_strong_runtime_opts, }; From patchwork Sun Dec 27 16:42:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AA9BC433DB for ; Sun, 27 Dec 2020 16:49:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1A33C2250F for ; Sun, 27 Dec 2020 16:49:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1A33C2250F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZEk-0007cs-7K for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:49:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43322) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8I-0001VD-Mg for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:43:02 -0500 Received: from kerio.kamp.de ([195.62.97.192]:33908) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8G-0003ya-DK for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:43:02 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22358 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 6251313DD26; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 5/7] block/rbd: migrate from aio to coroutines Date: Sun, 27 Dec 2020 17:42:34 +0100 Message-Id: <20201227164236.10143-6-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Lieven --- block/rbd.c | 247 ++++++++++++++++++---------------------------------- 1 file changed, 84 insertions(+), 163 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 27b232f4d8..2d77d0007f 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -66,22 +66,6 @@ typedef enum { RBD_AIO_FLUSH } RBDAIOCmd; -typedef struct RBDAIOCB { - BlockAIOCB common; - int64_t ret; - QEMUIOVector *qiov; - RBDAIOCmd cmd; - int error; - struct BDRVRBDState *s; -} RBDAIOCB; - -typedef struct RADOSCB { - RBDAIOCB *acb; - struct BDRVRBDState *s; - int64_t size; - int64_t ret; -} RADOSCB; - typedef struct BDRVRBDState { rados_t cluster; rados_ioctx_t io_ctx; @@ -94,6 +78,13 @@ typedef struct BDRVRBDState { AioContext *aio_context; } BDRVRBDState; +typedef struct RBDTask { + BDRVRBDState *s; + Coroutine *co; + bool complete; + int64_t ret; +} RBDTask; + static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, BlockdevOptionsRbd *opts, bool cache, const char *keypairs, const char *secretid, @@ -316,13 +307,6 @@ static int qemu_rbd_set_keypairs(rados_t cluster, const char *keypairs_json, return ret; } -static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs) -{ - RBDAIOCB *acb = rcb->acb; - iov_memset(acb->qiov->iov, acb->qiov->niov, offs, 0, - acb->qiov->size - offs); -} - /* FIXME Deprecate and remove keypairs or make it available in QMP. */ static int qemu_rbd_do_create(BlockdevCreateOptions *options, const char *keypairs, const char *password_secret, @@ -440,46 +424,6 @@ exit: return ret; } -/* - * This aio completion is being called from rbd_finish_bh() and runs in qemu - * BH context. - */ -static void qemu_rbd_complete_aio(RADOSCB *rcb) -{ - RBDAIOCB *acb = rcb->acb; - int64_t r; - - r = rcb->ret; - - if (acb->cmd != RBD_AIO_READ) { - if (r < 0) { - acb->ret = r; - acb->error = 1; - } else if (!acb->error) { - acb->ret = rcb->size; - } - } else { - if (r < 0) { - qemu_rbd_memset(rcb, 0); - acb->ret = r; - acb->error = 1; - } else if (r < rcb->size) { - qemu_rbd_memset(rcb, r); - if (!acb->error) { - acb->ret = rcb->size; - } - } else if (!acb->error) { - acb->ret = r; - } - } - - g_free(rcb); - - acb->common.cb(acb->common.opaque, (acb->ret > 0 ? 0 : acb->ret)); - - qemu_aio_unref(acb); -} - static char *qemu_rbd_mon_host(BlockdevOptionsRbd *opts, Error **errp) { const char **vals; @@ -817,88 +761,49 @@ static int qemu_rbd_resize(BlockDriverState *bs, uint64_t size) return 0; } -static const AIOCBInfo rbd_aiocb_info = { - .aiocb_size = sizeof(RBDAIOCB), -}; - -static void rbd_finish_bh(void *opaque) +static void qemu_rbd_finish_bh(void *opaque) { - RADOSCB *rcb = opaque; - qemu_rbd_complete_aio(rcb); + RBDTask *task = opaque; + task->complete = 1; + aio_co_wake(task->co); } -/* - * This is the callback function for rbd_aio_read and _write - * - * Note: this function is being called from a non qemu thread so - * we need to be careful about what we do here. Generally we only - * schedule a BH, and do the rest of the io completion handling - * from rbd_finish_bh() which runs in a qemu context. - */ -static void rbd_finish_aiocb(rbd_completion_t c, RADOSCB *rcb) +static void qemu_rbd_completion_cb(rbd_completion_t c, RBDTask *task) { - RBDAIOCB *acb = rcb->acb; - - rcb->ret = rbd_aio_get_return_value(c); + task->ret = rbd_aio_get_return_value(c); rbd_aio_release(c); - - replay_bh_schedule_oneshot_event(acb->s->aio_context, rbd_finish_bh, rcb); + aio_bh_schedule_oneshot(task->s->aio_context, qemu_rbd_finish_bh, task); } -static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, - int64_t off, - QEMUIOVector *qiov, - int64_t size, - BlockCompletionFunc *cb, - void *opaque, - RBDAIOCmd cmd) +static int coroutine_fn qemu_rbd_start_co(BlockDriverState *bs, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + int flags, + RBDAIOCmd cmd) { - RBDAIOCB *acb; - RADOSCB *rcb = NULL; - rbd_completion_t c; - int r; - BDRVRBDState *s = bs->opaque; + RBDTask task = { .s = s, .co = qemu_coroutine_self() }; + rbd_completion_t c; + int r, ret = -EIO; - acb = qemu_aio_get(&rbd_aiocb_info, bs, cb, opaque); - acb->cmd = cmd; - acb->qiov = qiov; - assert(!qiov || qiov->size == size); - - rcb = g_new(RADOSCB, 1); - - acb->ret = 0; - acb->error = 0; - acb->s = s; + assert(!qiov || qiov->size == bytes); - rcb->acb = acb; - rcb->s = acb->s; - rcb->size = size; - r = rbd_aio_create_completion(rcb, (rbd_callback_t) rbd_finish_aiocb, &c); + r = rbd_aio_create_completion(&task, + (rbd_callback_t) qemu_rbd_completion_cb, &c); if (r < 0) { goto failed; } switch (cmd) { - case RBD_AIO_WRITE: - /* - * RBD APIs don't allow us to write more than actual size, so in order - * to support growing images, we resize the image before write - * operations that exceed the current size. - */ - if (off + size > s->image_size) { - r = qemu_rbd_resize(bs, off + size); - if (r < 0) { - goto failed_completion; - } - } - r = rbd_aio_writev(s->image, qiov->iov, qiov->niov, off, c); - break; case RBD_AIO_READ: - r = rbd_aio_readv(s->image, qiov->iov, qiov->niov, off, c); + r = rbd_aio_readv(s->image, qiov->iov, qiov->niov, offset, c); + break; + case RBD_AIO_WRITE: + r = rbd_aio_writev(s->image, qiov->iov, qiov->niov, offset, c); break; case RBD_AIO_DISCARD: - r = rbd_aio_discard(s->image, off, size, c); + r = rbd_aio_discard(s->image, offset, bytes, c); break; case RBD_AIO_FLUSH: r = rbd_aio_flush(s->image, c); @@ -908,44 +813,71 @@ static BlockAIOCB *rbd_start_aio(BlockDriverState *bs, } if (r < 0) { + task.complete = 1; + task.ret = r; + } + + while (!task.complete) { + qemu_coroutine_yield(); + } + + if (task.ret < 0) { + error_report("rbd request failed: cmd %d offset %" PRIu64 " bytes %" + PRIu64 " flags %d task.ret %" PRIi64 " (%s)", cmd, offset, + bytes, flags, task.ret, strerror(-task.ret)); goto failed_completion; } - return &acb->common; + + /* zero pad short reads */ + if (cmd == RBD_AIO_READ && task.ret < qiov->size) { + qemu_iovec_memset(qiov, task.ret, 0, qiov->size - task.ret); + } + + return 0; failed_completion: rbd_aio_release(c); failed: - g_free(rcb); + return ret; +} - qemu_aio_unref(acb); - return NULL; +static int +coroutine_fn qemu_rbd_co_preadv(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, + int flags) +{ + return qemu_rbd_start_co(bs, offset, bytes, qiov, flags, RBD_AIO_READ); } -static BlockAIOCB *qemu_rbd_aio_preadv(BlockDriverState *bs, - uint64_t offset, uint64_t bytes, - QEMUIOVector *qiov, int flags, - BlockCompletionFunc *cb, - void *opaque) +static int +coroutine_fn qemu_rbd_co_pwritev(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, + int flags) { - return rbd_start_aio(bs, offset, qiov, bytes, cb, opaque, - RBD_AIO_READ); + BDRVRBDState *s = bs->opaque; + /* + * RBD APIs don't allow us to write more than actual size, so in order + * to support growing images, we resize the image before write + * operations that exceed the current size. + */ + if (offset + bytes > s->image_size) { + int r = qemu_rbd_resize(bs, offset + bytes); + if (r < 0) { + return r; + } + } + return qemu_rbd_start_co(bs, offset, bytes, qiov, flags, RBD_AIO_WRITE); } -static BlockAIOCB *qemu_rbd_aio_pwritev(BlockDriverState *bs, - uint64_t offset, uint64_t bytes, - QEMUIOVector *qiov, int flags, - BlockCompletionFunc *cb, - void *opaque) +static int coroutine_fn qemu_rbd_co_flush(BlockDriverState *bs) { - return rbd_start_aio(bs, offset, qiov, bytes, cb, opaque, - RBD_AIO_WRITE); + return qemu_rbd_start_co(bs, 0, 0, NULL, 0, RBD_AIO_FLUSH); } -static BlockAIOCB *qemu_rbd_aio_flush(BlockDriverState *bs, - BlockCompletionFunc *cb, - void *opaque) +static int coroutine_fn qemu_rbd_co_pdiscard(BlockDriverState *bs, + int64_t offset, int count) { - return rbd_start_aio(bs, 0, NULL, 0, cb, opaque, RBD_AIO_FLUSH); + return qemu_rbd_start_co(bs, offset, count, NULL, 0, RBD_AIO_DISCARD); } static int qemu_rbd_getinfo(BlockDriverState *bs, BlockDriverInfo *bdi) @@ -1097,16 +1029,6 @@ static int qemu_rbd_snap_list(BlockDriverState *bs, return snap_count; } -static BlockAIOCB *qemu_rbd_aio_pdiscard(BlockDriverState *bs, - int64_t offset, - int bytes, - BlockCompletionFunc *cb, - void *opaque) -{ - return rbd_start_aio(bs, offset, NULL, bytes, cb, opaque, - RBD_AIO_DISCARD); -} - static void coroutine_fn qemu_rbd_co_invalidate_cache(BlockDriverState *bs, Error **errp) { @@ -1182,11 +1104,10 @@ static BlockDriver bdrv_rbd = { .bdrv_co_truncate = qemu_rbd_co_truncate, .protocol_name = "rbd", - .bdrv_aio_preadv = qemu_rbd_aio_preadv, - .bdrv_aio_pwritev = qemu_rbd_aio_pwritev, - - .bdrv_aio_flush = qemu_rbd_aio_flush, - .bdrv_aio_pdiscard = qemu_rbd_aio_pdiscard, + .bdrv_co_preadv = qemu_rbd_co_preadv, + .bdrv_co_pwritev = qemu_rbd_co_pwritev, + .bdrv_co_flush_to_disk = qemu_rbd_co_flush, + .bdrv_co_pdiscard = qemu_rbd_co_pdiscard, .bdrv_snapshot_create = qemu_rbd_snap_create, .bdrv_snapshot_delete = qemu_rbd_snap_remove, From patchwork Sun Dec 27 16:42:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B930AC433E0 for ; Sun, 27 Dec 2020 16:46:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 610E9207B2 for ; Sun, 27 Dec 2020 16:46:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 610E9207B2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34352 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZC1-0005ky-Fd for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:46:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43268) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8D-0001OZ-Gr for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:42:57 -0500 Received: from kerio.kamp.de ([195.62.97.192]:33892) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ89-0003wr-PK for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:42:57 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22366 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 64E4E13DD27; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 6/7] block/rbd: add write zeroes support Date: Sun, 27 Dec 2020 17:42:35 +0100 Message-Id: <20201227164236.10143-7-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Peter Lieven --- block/rbd.c | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/block/rbd.c b/block/rbd.c index 2d77d0007f..27b4404adf 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -63,7 +63,8 @@ typedef enum { RBD_AIO_READ, RBD_AIO_WRITE, RBD_AIO_DISCARD, - RBD_AIO_FLUSH + RBD_AIO_FLUSH, + RBD_AIO_WRITE_ZEROES } RBDAIOCmd; typedef struct BDRVRBDState { @@ -221,8 +222,12 @@ done: static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) { + BDRVRBDState *s = bs->opaque; /* XXX Does RBD support AIO on less than 512-byte alignment? */ bs->bl.request_alignment = 512; +#ifdef LIBRBD_SUPPORTS_WRITE_ZEROES + bs->bl.pwrite_zeroes_alignment = s->object_size; +#endif } @@ -695,6 +700,9 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, } s->aio_context = bdrv_get_aio_context(bs); +#ifdef LIBRBD_SUPPORTS_WRITE_ZEROES + bs->supported_zero_flags = BDRV_REQ_MAY_UNMAP; +#endif /* When extending regular files, we get zeros from the OS */ bs->supported_truncate_flags = BDRV_REQ_ZERO_WRITE; @@ -808,6 +816,11 @@ static int coroutine_fn qemu_rbd_start_co(BlockDriverState *bs, case RBD_AIO_FLUSH: r = rbd_aio_flush(s->image, c); break; +#ifdef LIBRBD_SUPPORTS_WRITE_ZEROES + case RBD_AIO_WRITE_ZEROES: + r = rbd_aio_write_zeroes(s->image, offset, bytes, c, 0, 0); + break; +#endif default: r = -EINVAL; } @@ -880,6 +893,19 @@ static int coroutine_fn qemu_rbd_co_pdiscard(BlockDriverState *bs, return qemu_rbd_start_co(bs, offset, count, NULL, 0, RBD_AIO_DISCARD); } +#ifdef LIBRBD_SUPPORTS_WRITE_ZEROES +static int +coroutine_fn qemu_rbd_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset, + int count, BdrvRequestFlags flags) +{ + if (!(flags & BDRV_REQ_MAY_UNMAP)) { + return -ENOTSUP; + } + return qemu_rbd_start_co(bs, offset, count, NULL, flags, + RBD_AIO_WRITE_ZEROES); +} +#endif + static int qemu_rbd_getinfo(BlockDriverState *bs, BlockDriverInfo *bdi) { BDRVRBDState *s = bs->opaque; @@ -1108,6 +1134,9 @@ static BlockDriver bdrv_rbd = { .bdrv_co_pwritev = qemu_rbd_co_pwritev, .bdrv_co_flush_to_disk = qemu_rbd_co_flush, .bdrv_co_pdiscard = qemu_rbd_co_pdiscard, +#ifdef LIBRBD_SUPPORTS_WRITE_ZEROES + .bdrv_co_pwrite_zeroes = qemu_rbd_co_pwrite_zeroes, +#endif .bdrv_snapshot_create = qemu_rbd_snap_create, .bdrv_snapshot_delete = qemu_rbd_snap_remove, From patchwork Sun Dec 27 16:42:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 11990635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FFFEC433E0 for ; Sun, 27 Dec 2020 16:44:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1A145207B2 for ; Sun, 27 Dec 2020 16:44:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1A145207B2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kamp.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56282 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktZ9f-0002xs-85 for qemu-devel@archiver.kernel.org; Sun, 27 Dec 2020 11:44:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ8C-0001OV-35 for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:42:57 -0500 Received: from kerio.kamp.de ([195.62.97.192]:33894) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktZ89-0003wt-Pk for qemu-devel@nongnu.org; Sun, 27 Dec 2020 11:42:55 -0500 X-Footer: a2FtcC5kZQ== Received: from submission.kamp.de ([195.62.97.28]) by kerio.kamp.de with ESMTPS (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) for qemu-devel@nongnu.org; Sun, 27 Dec 2020 17:42:42 +0100 Received: (qmail 22374 invoked from network); 27 Dec 2020 16:42:44 -0000 Received: from lieven-pc.kamp-intra.net (HELO lieven-pc) (relay@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPA; 27 Dec 2020 16:42:44 -0000 Received: by lieven-pc (Postfix, from userid 1060) id 6772013DD28; Sun, 27 Dec 2020 17:42:44 +0100 (CET) From: Peter Lieven To: qemu-block@nongnu.org Subject: [PATCH 7/7] block/rbd: change request alignment to 1 byte Date: Sun, 27 Dec 2020 17:42:36 +0100 Message-Id: <20201227164236.10143-8-pl@kamp.de> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201227164236.10143-1-pl@kamp.de> References: <20201227164236.10143-1-pl@kamp.de> Received-SPF: pass client-ip=195.62.97.192; envelope-from=pl@kamp.de; helo=kerio.kamp.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, Peter Lieven , qemu-devel@nongnu.org, ct@flyingcircus.io, mreitz@redhat.com, dillaman@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" since we implement byte interfaces and librbd supports aio on byte granularity we can lift the 512 byte alignment. Signed-off-by: Peter Lieven --- block/rbd.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 27b4404adf..8673e8f553 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -223,8 +223,6 @@ done: static void qemu_rbd_refresh_limits(BlockDriverState *bs, Error **errp) { BDRVRBDState *s = bs->opaque; - /* XXX Does RBD support AIO on less than 512-byte alignment? */ - bs->bl.request_alignment = 512; #ifdef LIBRBD_SUPPORTS_WRITE_ZEROES bs->bl.pwrite_zeroes_alignment = s->object_size; #endif