From patchwork Thu Dec 6 06:35:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10715309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD91A18A7 for ; Thu, 6 Dec 2018 06:38:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B488E29305 for ; Thu, 6 Dec 2018 06:38:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4746291BF; Thu, 6 Dec 2018 06:38:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B8E902852B for ; Thu, 6 Dec 2018 06:38:36 +0000 (UTC) Received: from localhost ([::1]:39096 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gUnIx-0002pV-UH for patchwork-qemu-devel@patchwork.kernel.org; Thu, 06 Dec 2018 01:38:35 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52880) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gUnHR-0001Xn-Aj for qemu-devel@nongnu.org; Thu, 06 Dec 2018 01:37:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gUnHP-0003bB-9J for qemu-devel@nongnu.org; Thu, 06 Dec 2018 01:37:01 -0500 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:35574) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gUnHL-0003YS-7O for qemu-devel@nongnu.org; Thu, 06 Dec 2018 01:36:57 -0500 Received: by mail-pl1-x641.google.com with SMTP id p8so11325221plo.2 for ; Wed, 05 Dec 2018 22:36:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dU/eUAUX7IkjZsgttuh1Gp6IYyyjhsKSGjoHIekrBvA=; b=VthW2pEyM0HU8hwrtNwVAdGkXDDAHPEjosPZzZHZzGymPlWjr7id4xwOVj7VFRnovJ NuhvpkVnDKfQyzskO5Rc4vXZHt1kPVfcor3XAwbW22I5/DUDRrbfePvGbh9WAGO/FqQN 0jTp8D5KPRZ2dxjUgX+apXkA+Ahq9ymKXDpvTfSAS2ex4MqOogFWpwt6leXjGEXkoGMF 4BTwCU8rGQzzYk2B6jUmBJKpy6wX+M3DKiKrh4i55YQ+PffX2pCHBxOTB08qDOSVpRg7 k7EuKS227C3b9mcy7VBFALj8mSw2Kj0GUU3086LFrY7hTJOtKTdvGgNfwEnWY2CaEa14 3neA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dU/eUAUX7IkjZsgttuh1Gp6IYyyjhsKSGjoHIekrBvA=; b=UYj4mKTCu6nN3q4FGc+jpnDh9+dJ9AvL83pQujOTGaeqNTV+45Wk5vDy/8LZTXF2n+ WnjlcCQPayr90cAj0Q7je1EBY5ph04PijBiVnrfOo4++aZ1m2/Duq4ykhlJo+Q/m6FZL 2yjVTirDGWJumAoCHXPP/Nel7WvheNrunQx5AIdMacI4J2UbCJQYuSLWm/GqLdqLJoB+ VXiSvyJo1cI9OtqyOmIbljEmt21GXbx6ittgDpOLDeS+3iUvukK2PKjW8ZwNy8P4+bHj kQM4i0uaug26VqHuWv1iLw7P9SSc0M24eppVMLsvRp2YpYxYzpLVzV877mPA3q+er663 nS1w== X-Gm-Message-State: AA+aEWbrFUa+v+O8mVi3c4wESUrhCSsaDUDRcvfzQS5WyovT9qgEsKO+ Q24TNzuU2AgRhv1WuZTpwrk= X-Google-Smtp-Source: AFSGD/VE8Q/0UEil6tIKqP0WYh6e/SIvp65U1LF8YpECkLRTCHqDeJJmaQJ0C2/vj95v5SFZQqYweg== X-Received: by 2002:a17:902:724a:: with SMTP id c10mr27826816pll.51.1544078213098; Wed, 05 Dec 2018 22:36:53 -0800 (PST) Received: from yongji-Ubuntu.internal.baidu.com ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id x186sm26175249pfb.59.2018.12.05.22.36.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Dec 2018 22:36:52 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com Date: Thu, 6 Dec 2018 14:35:51 +0800 Message-Id: <20181206063552.6701-6-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181206063552.6701-1-xieyongji@baidu.com> References: <20181206063552.6701-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::641 Subject: [Qemu-devel] [PATCH for-4.0 5/6] vhost-user-blk: Add support for reconnecting backend X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji Since the new message VHOST_USER_SET_VRING_INFLIGHT, the backend is able to restart safely. This patch allow qemu to reconnect the backend after connection closed. Signed-off-by: Xie Yongji Signed-off-by: Ni Xun Signed-off-by: Zhang Yu --- hw/block/vhost-user-blk.c | 169 +++++++++++++++++++++++++++-- include/hw/virtio/vhost-user-blk.h | 4 + 2 files changed, 161 insertions(+), 12 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 1451940845..663e91bcf6 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -101,7 +101,7 @@ const VhostDevConfigOps blk_ops = { .vhost_dev_config_notifier = vhost_user_blk_handle_config_change, }; -static void vhost_user_blk_start(VirtIODevice *vdev) +static int vhost_user_blk_start(VirtIODevice *vdev) { VHostUserBlk *s = VHOST_USER_BLK(vdev); BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); @@ -110,13 +110,13 @@ static void vhost_user_blk_start(VirtIODevice *vdev) if (!k->set_guest_notifiers) { error_report("binding does not support guest notifiers"); - return; + return -ENOSYS; } ret = vhost_dev_enable_notifiers(&s->dev, vdev); if (ret < 0) { error_report("Error enabling host notifiers: %d", -ret); - return; + return ret; } ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, true); @@ -140,12 +140,13 @@ static void vhost_user_blk_start(VirtIODevice *vdev) vhost_virtqueue_mask(&s->dev, vdev, i, false); } - return; + return ret; err_guest_notifiers: k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&s->dev, vdev); + return ret; } static void vhost_user_blk_stop(VirtIODevice *vdev) @@ -164,7 +165,6 @@ static void vhost_user_blk_stop(VirtIODevice *vdev) ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); - return; } vhost_dev_disable_notifiers(&s->dev, vdev); @@ -174,21 +174,39 @@ static void vhost_user_blk_set_status(VirtIODevice *vdev, uint8_t status) { VHostUserBlk *s = VHOST_USER_BLK(vdev); bool should_start = status & VIRTIO_CONFIG_S_DRIVER_OK; + int ret; if (!vdev->vm_running) { should_start = false; } - if (s->dev.started == should_start) { + if (s->should_start == should_start) { + return; + } + + if (!s->connected || s->dev.started == should_start) { + s->should_start = should_start; return; } if (should_start) { - vhost_user_blk_start(vdev); + s->should_start = true; + /* make sure we ignore fake guest kick by + * vhost_dev_enable_notifiers() */ + barrier(); + ret = vhost_user_blk_start(vdev); + if (ret < 0) { + error_report("vhost-user-blk: vhost start failed: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&s->chardev); + } } else { vhost_user_blk_stop(vdev); + /* make sure we ignore fake guest kick by + * vhost_dev_disable_notifiers() */ + barrier(); + s->should_start = false; } - } static uint64_t vhost_user_blk_get_features(VirtIODevice *vdev, @@ -218,13 +236,22 @@ static uint64_t vhost_user_blk_get_features(VirtIODevice *vdev, static void vhost_user_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) { VHostUserBlk *s = VHOST_USER_BLK(vdev); - int i; + int i, ret; if (!(virtio_host_has_feature(vdev, VIRTIO_F_VERSION_1) && !virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1))) { return; } + if (s->should_start) { + return; + } + s->should_start = true; + + if (!s->connected) { + return; + } + if (s->dev.started) { return; } @@ -232,7 +259,13 @@ static void vhost_user_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) /* Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start * vhost here instead of waiting for .set_status(). */ - vhost_user_blk_start(vdev); + ret = vhost_user_blk_start(vdev); + if (ret < 0) { + error_report("vhost-user-blk: vhost start failed: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&s->chardev); + return; + } /* Kick right away to begin processing requests already in vring */ for (i = 0; i < s->dev.nvqs; i++) { @@ -245,6 +278,106 @@ static void vhost_user_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) } } +static void vhost_user_blk_reset(VirtIODevice *vdev) +{ + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + if (s->vhost_user) { + vhost_user_inflight_reset(s->vhost_user); + } +} + +static int vhost_user_blk_connect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + int ret = 0; + + if (s->connected) { + return 0; + } + s->connected = true; + + s->dev.nvqs = s->num_queues; + s->dev.vqs = s->vqs; + s->dev.vq_index = 0; + s->dev.backend_features = 0; + + vhost_dev_set_config_notifier(&s->dev, &blk_ops); + + ret = vhost_dev_init(&s->dev, s->vhost_user, VHOST_BACKEND_TYPE_USER, 0); + if (ret < 0) { + error_report("vhost-user-blk: vhost initialization failed: %s", + strerror(-ret)); + return ret; + } + + if (s->should_start && !s->dev.started) { + ret = vhost_user_blk_start(vdev); + if (ret < 0) { + error_report("vhost-user-blk: vhost start failed: %s", + strerror(-ret)); + return ret; + } + } + + return 0; +} + +static void vhost_user_blk_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + if (!s->connected) { + return; + } + s->connected = false; + + if (s->dev.started) { + vhost_user_blk_stop(vdev); + } + + vhost_dev_cleanup(&s->dev); +} + +static gboolean vhost_user_blk_watch(GIOChannel *chan, GIOCondition cond, + void *opaque) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + qemu_chr_fe_disconnect(&s->chardev); + + return true; +} + +static void vhost_user_blk_event(void *opaque, int event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_blk_connect(dev) < 0) { + qemu_chr_fe_disconnect(&s->chardev); + return; + } + s->watch = qemu_chr_fe_add_watch(&s->chardev, G_IO_HUP, + vhost_user_blk_watch, dev); + break; + case CHR_EVENT_CLOSED: + vhost_user_blk_disconnect(dev); + if (s->watch) { + g_source_remove(s->watch); + s->watch = 0; + } + break; + } +} + static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) { VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -284,8 +417,13 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) vhost_user_blk_handle_output); } + s->vqs = g_new(struct vhost_virtqueue, s->num_queues); + s->watch = 0; + s->should_start = false; + s->connected = true; + s->dev.nvqs = s->num_queues; - s->dev.vqs = g_new(struct vhost_virtqueue, s->dev.nvqs); + s->dev.vqs = s->vqs; s->dev.vq_index = 0; s->dev.backend_features = 0; @@ -309,6 +447,9 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) s->blkcfg.num_queues = s->num_queues; } + qemu_chr_fe_set_handlers(&s->chardev, NULL, NULL, vhost_user_blk_event, + NULL, (void *)dev, NULL, true); + return; vhost_err: @@ -328,8 +469,11 @@ static void vhost_user_blk_device_unrealize(DeviceState *dev, Error **errp) VHostUserBlk *s = VHOST_USER_BLK(dev); vhost_user_blk_set_status(vdev, 0); + qemu_chr_fe_set_handlers(&s->chardev, NULL, NULL, NULL, + NULL, NULL, NULL, false); vhost_dev_cleanup(&s->dev); - g_free(s->dev.vqs); + + g_free(s->vqs); virtio_cleanup(vdev); if (s->vhost_user) { @@ -379,6 +523,7 @@ static void vhost_user_blk_class_init(ObjectClass *klass, void *data) vdc->set_config = vhost_user_blk_set_config; vdc->get_features = vhost_user_blk_get_features; vdc->set_status = vhost_user_blk_set_status; + vdc->reset = vhost_user_blk_reset; } static const TypeInfo vhost_user_blk_info = { diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h index d52944aeeb..560bb21459 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -37,6 +37,10 @@ typedef struct VHostUserBlk { uint32_t config_wce; struct vhost_dev dev; VhostUserState *vhost_user; + struct vhost_virtqueue *vqs; + guint watch; + bool should_start; + bool connected; } VHostUserBlk; #endif