From patchwork Thu Jan 3 10:18:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747179 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D78646C5 for ; Thu, 3 Jan 2019 10:23:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C54DB28563 for ; Thu, 3 Jan 2019 10:23:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B83FF28623; Thu, 3 Jan 2019 10:23:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2B51428563 for ; Thu, 3 Jan 2019 10:23:42 +0000 (UTC) Received: from localhost ([127.0.0.1]:50443 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf0A9-0000yO-AH for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:23:41 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44044) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf05k-00042M-ET for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf05i-0002ir-De for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:08 -0500 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:42173) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf05g-0002gu-CH for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:06 -0500 Received: by mail-pf1-x444.google.com with SMTP id 64so16489607pfr.9 for ; Thu, 03 Jan 2019 02:19:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aAATJahSzpb7CqJ/MNyIYQ/06vLxFUTaW2AexnRpbuc=; b=YcUtBNOD1giqNxRd2+D2yF+WqfQqorq5xPHS57TD6Wp06z5XMplmcENR2387lESiCH FG/JlCeqdZitGrOBQUaZMHT1Ckwplo0U+ZTh5566B7/BxBsIG2jo8SnBU+T5eTkw42+X n1Aroze/cjI290ZshHYn6tvpJh+Lr1SNQ6S+Wg7uPGPUG4tcql+jOIUtQ5cnMJDn2OV7 3BX5KDOp2yUePOajNEuHzNjg7GSCOpXo12O3H3l8LYFBH2GFo/gPBnCi2mb0EePCaJbw D6bsrr8F9HHCpXYFNC559fParzYq2VapnEDCg984TjtQ6ck9Nqmik5jS81IaEjjKHtaI nnUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aAATJahSzpb7CqJ/MNyIYQ/06vLxFUTaW2AexnRpbuc=; b=cgKuXS5mx86ehYOrpbZ2zOgte69iihddFqIqOGW12U4ODfYSdqvSkK1UttxPjL4JP2 glPjqomFxK3bx3L+haagDdNifQzfyAKnVmPz6/iLHzOjZq/2dptvVcxiLr+y6WfR3hjY DNGMec409p1wQ4voSEtzXSeFUTI3XAD4zag3lsckHivGzkmiOfq4sRYmRimyABuivo+t Y9Y0oH++H8bq3o+AODFDBUS9Hx1pTJ/n7uKUMajhsfYBq6UEOi0jPPs8AhfXuX4J/5Hq tZqT27Y5pahQFceHJVIwmZIwbexUuF+wGRLmCfjyBzyeQzXBBs1L+jsLxM7e69VjesRY zUQg== X-Gm-Message-State: AA+aEWbtAqx6tEnqpUvXS6TayE+WTOznBoDaPzZo+V6ifRC4cDqr7Wc0 08JKXlcZYW8IeOSZdR8VJF4= X-Google-Smtp-Source: AFSGD/Xvr4VvoDqse4ItzgxKvTBeMqp/C2SUPAtC0RGmKexCSX09mcm8snD3wm3nxAeGEC/wJtYn0g== X-Received: by 2002:aa7:810c:: with SMTP id b12mr47376726pfi.44.1546510743125; Thu, 03 Jan 2019 02:19:03 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id w185sm83109728pfb.135.2019.01.03.02.19.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:02 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:13 +0800 Message-Id: <20190103101819.7418-2-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::444 Subject: [Qemu-devel] [PATCH v3 for-4.0 1/7] char-socket: Enable "nowait" option on client sockets X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji Enable "nowait" option to make QEMU not do a connect on client sockets during initialization of the chardev. Then we can use qemu_chr_fe_wait_connected() to connect when necessary. Now it would be used for unix domain socket of vhost-user-blk device to support reconnect. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu --- chardev/char-socket.c | 56 +++++++++++++++++++++---------------------- qapi/char.json | 3 +-- qemu-options.hx | 9 ++++--- 3 files changed, 35 insertions(+), 33 deletions(-) diff --git a/chardev/char-socket.c b/chardev/char-socket.c index eaa8e8b68f..f803f4f7d3 100644 --- a/chardev/char-socket.c +++ b/chardev/char-socket.c @@ -1072,37 +1072,37 @@ static void qmp_chardev_open_socket(Chardev *chr, s->reconnect_time = reconnect; } - if (s->reconnect_time) { - tcp_chr_connect_async(chr); - } else { - if (s->is_listen) { - char *name; - s->listener = qio_net_listener_new(); + if (s->is_listen) { + char *name; + s->listener = qio_net_listener_new(); - name = g_strdup_printf("chardev-tcp-listener-%s", chr->label); - qio_net_listener_set_name(s->listener, name); - g_free(name); + name = g_strdup_printf("chardev-tcp-listener-%s", chr->label); + qio_net_listener_set_name(s->listener, name); + g_free(name); - if (qio_net_listener_open_sync(s->listener, s->addr, errp) < 0) { - object_unref(OBJECT(s->listener)); - s->listener = NULL; - goto error; - } + if (qio_net_listener_open_sync(s->listener, s->addr, errp) < 0) { + object_unref(OBJECT(s->listener)); + s->listener = NULL; + goto error; + } - qapi_free_SocketAddress(s->addr); - s->addr = socket_local_address(s->listener->sioc[0]->fd, errp); - update_disconnected_filename(s); + qapi_free_SocketAddress(s->addr); + s->addr = socket_local_address(s->listener->sioc[0]->fd, errp); + update_disconnected_filename(s); - if (is_waitconnect && - qemu_chr_wait_connected(chr, errp) < 0) { - return; - } - if (!s->ioc) { - qio_net_listener_set_client_func_full(s->listener, - tcp_chr_accept, - chr, NULL, - chr->gcontext); - } + if (is_waitconnect && + qemu_chr_wait_connected(chr, errp) < 0) { + return; + } + if (!s->ioc) { + qio_net_listener_set_client_func_full(s->listener, + tcp_chr_accept, + chr, NULL, + chr->gcontext); + } + } else if (is_waitconnect) { + if (s->reconnect_time) { + tcp_chr_connect_async(chr); } else if (qemu_chr_wait_connected(chr, errp) < 0) { goto error; } @@ -1120,7 +1120,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend, Error **errp) { bool is_listen = qemu_opt_get_bool(opts, "server", false); - bool is_waitconnect = is_listen && qemu_opt_get_bool(opts, "wait", true); + bool is_waitconnect = qemu_opt_get_bool(opts, "wait", true); bool is_telnet = qemu_opt_get_bool(opts, "telnet", false); bool is_tn3270 = qemu_opt_get_bool(opts, "tn3270", false); bool is_websock = qemu_opt_get_bool(opts, "websocket", false); diff --git a/qapi/char.json b/qapi/char.json index 77ed847972..6a3b5bcd71 100644 --- a/qapi/char.json +++ b/qapi/char.json @@ -249,8 +249,7 @@ # or connect to (server=false) # @tls-creds: the ID of the TLS credentials object (since 2.6) # @server: create server socket (default: true) -# @wait: wait for incoming connection on server -# sockets (default: false). +# @wait: wait for being connected or connecting to (default: false) # @nodelay: set TCP_NODELAY socket option (default: false) # @telnet: enable telnet protocol on server # sockets (default: false) diff --git a/qemu-options.hx b/qemu-options.hx index df42116ecc..66d99c6e83 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -2556,8 +2556,9 @@ undefined if TCP options are specified for a unix socket. @option{server} specifies that the socket shall be a listening socket. -@option{nowait} specifies that QEMU should not block waiting for a client to -connect to a listening socket. +@option{nowait} specifies that QEMU should not wait for being connected on +server sockets or try to do a sync/async connect on client sockets during +initialization of the chardev. @option{telnet} specifies that traffic on the socket should interpret telnet escape sequences. @@ -3093,7 +3094,9 @@ I/O to a location or wait for a connection from a location. By default the TCP Net Console is sent to @var{host} at the @var{port}. If you use the @var{server} option QEMU will wait for a client socket application to connect to the port before continuing, unless the @code{nowait} -option was specified. The @code{nodelay} option disables the Nagle buffering +option was specified. And the @code{nowait} option could also be +used when @var{noserver} is set to disallow QEMU to connect during +initialization. The @code{nodelay} option disables the Nagle buffering algorithm. The @code{reconnect} option only applies if @var{noserver} is set, if the connection goes down it will attempt to reconnect at the given interval. If @var{host} is omitted, 0.0.0.0 is assumed. Only From patchwork Thu Jan 3 10:18:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747169 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84EA014E5 for ; Thu, 3 Jan 2019 10:20:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 670E728563 for ; Thu, 3 Jan 2019 10:20:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5952C28620; Thu, 3 Jan 2019 10:20:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E48BD28564 for ; Thu, 3 Jan 2019 10:20:41 +0000 (UTC) Received: from localhost ([127.0.0.1]:50415 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf07F-0005qh-4q for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:20:41 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44107) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf05m-000449-KV for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf05k-0002kB-M3 for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:10 -0500 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:36140) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf05k-0002j9-CZ for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:08 -0500 Received: by mail-pg1-x541.google.com with SMTP id n2so15833961pgm.3 for ; Thu, 03 Jan 2019 02:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jKelcYITJnIk9qRUb14wDKa6xQutGzaYjKb25tJaYMA=; b=eXYKJtmNJqoQl4sR40HJ18N55uRmwdhzKl7rVSte8ta3Cm+CKCo1RJdqY58gRpnjr9 DUgBLF7tPsYeZJvy30OH/Adcj5lV7KlgXPyhaKQYWK+u8ZjDCApanl4+4zpdo1z8Hq1Z BFYAgENnagOSN2UL78iVooBgA8k9ptMTl5WJPSk9iHVCmFRqi2rsSn4UpXbQW275Vn5b /Bf+EWOioUIsDpCz17tw2qMGIdFVRBan8ZOxKMaifNRG/RHDFt8JjpPvWR4sndUyADjd lDo/Upvc71E0KuNWbYtx9ZXVrw73FGf2+ngoCET8WE43vMtJDskntSD6Xl8QR9zuwdbw wAAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jKelcYITJnIk9qRUb14wDKa6xQutGzaYjKb25tJaYMA=; b=o/kcFuDvyqScQWSAaAxMWhbW2vEA29XDWhkc8QNWH7RXKop4KZiCq43t3oZVBtOgPN ozbzMCqIaytGM1AW/h5C7+ENdYtvpce4fA2OvjU/cv+ni05gejUyvnH/FUbxv/YxPwYt 2A9QYAOVdISjpgGYn6qlovIZGosn7h8sAfv+uVvuh3eAGuaGrgKdeS8eQ29zb8pPmR5L 4wEvUW88eqhywd9NP2fnk4BUhVbeOefgIrCPUBJSysr0+loDRe/ZoM9++x4Y4nzIiwMi kfIhsKcfN0HmzOmzok2D/bd8nErnZ0TYZk+yvcOUDwt87UkN+n/ywL1Yi+MU0LCwdDbu Xy9w== X-Gm-Message-State: AA+aEWZIbXtd0yVitukJXjuRXJZf9mKUKUzSOpPNVprAn0AjfEVRQWv7 GvOZlFgHD9JmJ5WC8DyGTT8= X-Google-Smtp-Source: AFSGD/Xpn2KXxX5rdunfgOo5lv6qHEOvfMqkPp/C1iv/TcjnVEpp86SLUW+LM8LyYO2ghOR75E18YA== X-Received: by 2002:a62:5716:: with SMTP id l22mr48915114pfb.16.1546510746408; Thu, 03 Jan 2019 02:19:06 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id c4sm112439902pfm.151.2019.01.03.02.19.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:05 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:14 +0800 Message-Id: <20190103101819.7418-3-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::541 Subject: [Qemu-devel] [PATCH v3 for-4.0 2/7] vhost-user: Support providing shared memory to backend X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji This patch introduces two new messages VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD to support providing shared memory to backend. Firstly, qemu uses VHOST_USER_GET_SHM_SIZE to get the required size of shared memory from backend. Then, qemu allocates memory and sends them back to backend through VHOST_USER_SET_SHM_FD. Note that the shared memory should be used to record inflight I/O by backend. Qemu will clear it when vm reset. Signed-off-by: Xie Yongji Signed-off-by: Chai Wen Signed-off-by: Zhang Yu --- docs/interop/vhost-user.txt | 41 +++++++++++ hw/virtio/vhost-user.c | 86 ++++++++++++++++++++++ hw/virtio/vhost.c | 117 ++++++++++++++++++++++++++++++ include/hw/virtio/vhost-backend.h | 9 +++ include/hw/virtio/vhost.h | 19 +++++ 5 files changed, 272 insertions(+) diff --git a/docs/interop/vhost-user.txt b/docs/interop/vhost-user.txt index c2194711d9..5ee9c28ab0 100644 --- a/docs/interop/vhost-user.txt +++ b/docs/interop/vhost-user.txt @@ -142,6 +142,19 @@ Depending on the request type, payload can be: Offset: a 64-bit offset of this area from the start of the supplied file descriptor + * Shm description + ----------------------------------- + | mmap_size | mmap_offset | dev_size | vq_size | align | version | + ----------------------------------- + + Mmap_size: a 64-bit size of the shared memory + Mmap_offset: a 64-bit offset of the shared memory from the start + of the supplied file descriptor + Dev_size: a 32-bit size of device region in shared memory + Vq_size: a 32-bit size of each virtqueue region in shared memory + Align: a 32-bit align of each region in shared memory + Version: a 32-bit version of this shared memory + In QEMU the vhost-user message is implemented with the following struct: typedef struct VhostUserMsg { @@ -157,6 +170,7 @@ typedef struct VhostUserMsg { struct vhost_iotlb_msg iotlb; VhostUserConfig config; VhostUserVringArea area; + VhostUserShm shm; }; } QEMU_PACKED VhostUserMsg; @@ -175,6 +189,7 @@ the ones that do: * VHOST_USER_GET_PROTOCOL_FEATURES * VHOST_USER_GET_VRING_BASE * VHOST_USER_SET_LOG_BASE (if VHOST_USER_PROTOCOL_F_LOG_SHMFD) + * VHOST_USER_GET_SHM_SIZE (if VHOST_USER_PROTOCOL_F_SLAVE_SHMFD) [ Also see the section on REPLY_ACK protocol extension. ] @@ -188,6 +203,7 @@ in the ancillary data: * VHOST_USER_SET_VRING_CALL * VHOST_USER_SET_VRING_ERR * VHOST_USER_SET_SLAVE_REQ_FD + * VHOST_USER_SET_SHM_FD (if VHOST_USER_PROTOCOL_F_SLAVE_SHMFD) If Master is unable to send the full message or receives a wrong reply it will close the connection. An optional reconnection mechanism can be implemented. @@ -397,6 +413,7 @@ Protocol features #define VHOST_USER_PROTOCOL_F_CONFIG 9 #define VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD 10 #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER 11 +#define VHOST_USER_PROTOCOL_F_SLAVE_SHMFD 12 Master message types -------------------- @@ -761,6 +778,30 @@ Master message types was previously sent. The value returned is an error indication; 0 is success. + * VHOST_USER_GET_SHM_SIZE + Id: 31 + Equivalent ioctl: N/A + Master payload: shm description + + When VHOST_USER_PROTOCOL_F_SLAVE_SHMFD protocol feature has been + successfully negotiated, master need to provide a shared memory to + slave. This message is used by master to get required size from slave. + The shared memory contains one region for device and several regions + for virtqueue. The size of those two kinds of regions is specified + by dev_size field and vq_size filed. The align field specify the alignment + of those regions. + + * VHOST_USER_SET_SHM_FD + Id: 32 + Equivalent ioctl: N/A + Master payload: shm description + + When VHOST_USER_PROTOCOL_F_SLAVE_SHMFD protocol feature has been + successfully negotiated, master uses this message to set shared memory + for slave. The memory fd is passed in the ancillary data. The shared + memory should be used to record inflight I/O by slave. And master will + clear it when vm reset. + Slave message types ------------------- diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index e09bed0e4a..8cdf3b5121 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -52,6 +52,7 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_CONFIG = 9, VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD = 10, VHOST_USER_PROTOCOL_F_HOST_NOTIFIER = 11, + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD = 12, VHOST_USER_PROTOCOL_F_MAX }; @@ -89,6 +90,8 @@ typedef enum VhostUserRequest { VHOST_USER_POSTCOPY_ADVISE = 28, VHOST_USER_POSTCOPY_LISTEN = 29, VHOST_USER_POSTCOPY_END = 30, + VHOST_USER_GET_SHM_SIZE = 31, + VHOST_USER_SET_SHM_FD = 32, VHOST_USER_MAX } VhostUserRequest; @@ -147,6 +150,15 @@ typedef struct VhostUserVringArea { uint64_t offset; } VhostUserVringArea; +typedef struct VhostUserShm { + uint64_t mmap_size; + uint64_t mmap_offset; + uint32_t dev_size; + uint32_t vq_size; + uint32_t align; + uint32_t version; +} VhostUserShm; + typedef struct { VhostUserRequest request; @@ -169,6 +181,7 @@ typedef union { VhostUserConfig config; VhostUserCryptoSession session; VhostUserVringArea area; + VhostUserShm shm; } VhostUserPayload; typedef struct VhostUserMsg { @@ -1739,6 +1752,77 @@ static bool vhost_user_mem_section_filter(struct vhost_dev *dev, return result; } +static int vhost_user_get_shm_size(struct vhost_dev *dev, + struct vhost_shm *shm) +{ + VhostUserMsg msg = { + .hdr.request = VHOST_USER_GET_SHM_SIZE, + .hdr.flags = VHOST_USER_VERSION, + .hdr.size = sizeof(msg.payload.shm), + }; + + if (!virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD)) { + shm->dev_size = 0; + shm->vq_size = 0; + return 0; + } + + if (vhost_user_write(dev, &msg, NULL, 0) < 0) { + return -1; + } + + if (vhost_user_read(dev, &msg) < 0) { + return -1; + } + + if (msg.hdr.request != VHOST_USER_GET_SHM_SIZE) { + error_report("Received unexpected msg type. " + "Expected %d received %d", + VHOST_USER_GET_SHM_SIZE, msg.hdr.request); + return -1; + } + + if (msg.hdr.size != sizeof(msg.payload.shm)) { + error_report("Received bad msg size."); + return -1; + } + + shm->dev_size = msg.payload.shm.dev_size; + shm->vq_size = msg.payload.shm.vq_size; + shm->align = msg.payload.shm.align; + shm->version = msg.payload.shm.version; + + return 0; +} + +static int vhost_user_set_shm_fd(struct vhost_dev *dev, + struct vhost_shm *shm) +{ + VhostUserMsg msg = { + .hdr.request = VHOST_USER_SET_SHM_FD, + .hdr.flags = VHOST_USER_VERSION, + .payload.shm.mmap_size = shm->mmap_size, + .payload.shm.mmap_offset = 0, + .payload.shm.dev_size = shm->dev_size, + .payload.shm.vq_size = shm->vq_size, + .payload.shm.align = shm->align, + .payload.shm.version = shm->version, + .hdr.size = sizeof(msg.payload.shm), + }; + + if (!virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD)) { + return 0; + } + + if (vhost_user_write(dev, &msg, &shm->fd, 1) < 0) { + return -1; + } + + return 0; +} + VhostUserState *vhost_user_init(void) { VhostUserState *user = g_new0(struct VhostUserState, 1); @@ -1790,4 +1874,6 @@ const VhostOps user_ops = { .vhost_crypto_create_session = vhost_user_crypto_create_session, .vhost_crypto_close_session = vhost_user_crypto_close_session, .vhost_backend_mem_section_filter = vhost_user_mem_section_filter, + .vhost_get_shm_size = vhost_user_get_shm_size, + .vhost_set_shm_fd = vhost_user_set_shm_fd, }; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 569c4053ea..7a38fed50f 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1481,6 +1481,123 @@ void vhost_dev_set_config_notifier(struct vhost_dev *hdev, hdev->config_ops = ops; } +void vhost_dev_reset_shm(struct vhost_shm *shm) +{ + if (shm->addr) { + memset(shm->addr, 0, shm->mmap_size); + } +} + +void vhost_dev_free_shm(struct vhost_shm *shm) +{ + if (shm->addr) { + qemu_memfd_free(shm->addr, shm->mmap_size, shm->fd); + shm->addr = NULL; + shm->fd = -1; + } +} + +int vhost_dev_alloc_shm(struct vhost_shm *shm) +{ + Error *err = NULL; + int fd = -1; + void *addr = qemu_memfd_alloc("vhost-shm", shm->mmap_size, + F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL, + &fd, &err); + if (err) { + error_report_err(err); + return -1; + } + + shm->addr = addr; + shm->fd = fd; + + return 0; +} + +void vhost_dev_save_shm(struct vhost_shm *shm, QEMUFile *f) +{ + if (shm->addr) { + qemu_put_be64(f, shm->mmap_size); + qemu_put_be32(f, shm->dev_size); + qemu_put_be32(f, shm->vq_size); + qemu_put_be32(f, shm->align); + qemu_put_be32(f, shm->version); + qemu_put_buffer(f, shm->addr, shm->mmap_size); + } else { + qemu_put_be64(f, 0); + } +} + +int vhost_dev_load_shm(struct vhost_shm *shm, QEMUFile *f) +{ + uint64_t mmap_size; + + mmap_size = qemu_get_be64(f); + if (!mmap_size) { + return 0; + } + + vhost_dev_free_shm(shm); + + shm->mmap_size = mmap_size; + shm->dev_size = qemu_get_be32(f); + shm->vq_size = qemu_get_be32(f); + shm->align = qemu_get_be32(f); + shm->version = qemu_get_be32(f); + + if (vhost_dev_alloc_shm(shm)) { + return -ENOMEM; + } + + qemu_get_buffer(f, shm->addr, mmap_size); + + return 0; +} + +int vhost_dev_set_shm(struct vhost_dev *dev, struct vhost_shm *shm) +{ + int r; + + if (dev->vhost_ops->vhost_set_shm_fd && shm->addr) { + r = dev->vhost_ops->vhost_set_shm_fd(dev, shm); + if (r) { + VHOST_OPS_DEBUG("vhost_set_vring_shm_fd failed"); + return -errno; + } + } + + return 0; +} + +int vhost_dev_init_shm(struct vhost_dev *dev, struct vhost_shm *shm) +{ + int r; + + if (dev->vhost_ops->vhost_get_shm_size) { + r = dev->vhost_ops->vhost_get_shm_size(dev, shm); + if (r) { + VHOST_OPS_DEBUG("vhost_get_vring_shm_size failed"); + return -errno; + } + + if (!shm->dev_size && !shm->vq_size) { + return 0; + } + + shm->mmap_size = QEMU_ALIGN_UP(shm->dev_size, shm->align) + + dev->nvqs * QEMU_ALIGN_UP(shm->vq_size, shm->align); + + if (vhost_dev_alloc_shm(shm)) { + return -ENOMEM; + } + + vhost_dev_reset_shm(shm); + } + + return 0; +} + /* Host notifiers must be enabled at this point. */ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev) { diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h index 81283ec50f..4e7f13c9e9 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -25,6 +25,7 @@ typedef enum VhostSetConfigType { VHOST_SET_CONFIG_TYPE_MIGRATION = 1, } VhostSetConfigType; +struct vhost_shm; struct vhost_dev; struct vhost_log; struct vhost_memory; @@ -104,6 +105,12 @@ typedef int (*vhost_crypto_close_session_op)(struct vhost_dev *dev, typedef bool (*vhost_backend_mem_section_filter_op)(struct vhost_dev *dev, MemoryRegionSection *section); +typedef int (*vhost_get_shm_size_op)(struct vhost_dev *dev, + struct vhost_shm *shm); + +typedef int (*vhost_set_shm_fd_op)(struct vhost_dev *dev, + struct vhost_shm *shm); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init; @@ -142,6 +149,8 @@ typedef struct VhostOps { vhost_crypto_create_session_op vhost_crypto_create_session; vhost_crypto_close_session_op vhost_crypto_close_session; vhost_backend_mem_section_filter_op vhost_backend_mem_section_filter; + vhost_get_shm_size_op vhost_get_shm_size; + vhost_set_shm_fd_op vhost_set_shm_fd; } VhostOps; extern const VhostOps user_ops; diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index a7f449fa87..b6e3d6ab56 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -7,6 +7,17 @@ #include "exec/memory.h" /* Generic structures common for any vhost based device. */ + +struct vhost_shm { + void *addr; + uint64_t mmap_size; + uint32_t dev_size; + uint32_t vq_size; + uint32_t align; + uint32_t version; + int fd; +}; + struct vhost_virtqueue { int kick; int call; @@ -120,4 +131,12 @@ int vhost_dev_set_config(struct vhost_dev *dev, const uint8_t *data, */ void vhost_dev_set_config_notifier(struct vhost_dev *dev, const VhostDevConfigOps *ops); + +void vhost_dev_reset_shm(struct vhost_shm *shm); +void vhost_dev_free_shm(struct vhost_shm *shm); +int vhost_dev_alloc_shm(struct vhost_shm *shm); +void vhost_dev_save_shm(struct vhost_shm *shm, QEMUFile *f); +int vhost_dev_load_shm(struct vhost_shm *shm, QEMUFile *f); +int vhost_dev_set_shm(struct vhost_dev *dev, struct vhost_shm *shm); +int vhost_dev_init_shm(struct vhost_dev *dev, struct vhost_shm *shm); #endif From patchwork Thu Jan 3 10:18:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747175 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0085D6C5 for ; Thu, 3 Jan 2019 10:21:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3D8428563 for ; Thu, 3 Jan 2019 10:21:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D7E0028623; Thu, 3 Jan 2019 10:21:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5B30F28563 for ; Thu, 3 Jan 2019 10:21:13 +0000 (UTC) Received: from localhost ([127.0.0.1]:50422 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf07j-0006Rb-QR for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:21:11 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44124) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf05n-00045x-W4 for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf05m-0002ml-T7 for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:11 -0500 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:42058) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf05m-0002lq-Ks for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:10 -0500 Received: by mail-pl1-x643.google.com with SMTP id y1so15726219plp.9 for ; Thu, 03 Jan 2019 02:19:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GKc3qU8HujCH0kBZ1CDODJ6Og5HhIYG0TFOCATmd5UE=; b=R+d/jB5dtUoFNpDd6Xb27kjj3eHGou6C0G9GWrjKASC3FyBoed9GrkbTq/j4akeg2g jFdyMPYhESoVqiVUz4vzIx5WHLEQQRrKupQzhb7QqPdMEDhtdfwjcijHbnRwbkOpLUzt IXwWyAacSPvNVFbMg9ouEs5haAL5c4mB89S3zxerJyEo8ftB9/ACXb2khJ4qrwtZkP0d soE4QinfxNBS/A1564aRtZicq135D4HFZym5DLl89sAvZd9QwUquJGMNqF23GMgmHmgM CpyFchH8JSXqMlRpY6AwNSYFsQjj0ZMlQ1OheBpwNGfqJV3xcTi5d7D6NmT3v/FxO08O ZQig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GKc3qU8HujCH0kBZ1CDODJ6Og5HhIYG0TFOCATmd5UE=; b=WNNqujVvFLxliKo3tuM9mF9jE7i+dc8nKbeUZH7kkvN1Rbamkk5P8FyIkDVhCrdQ8m zUnoV+MT0UJMAZZTkawGNyyqLrzKUxlRwr5+KTLCym13Q+G50fKDyVPzwsEbATTKniPt pxTPVjaZwmUopZE0fO3lbn9ljAF2kG32wxRB9XaRjBdnyUv+h6Dywa+7Fovxaq8qzP/R UDY26ucA1KC36VXIB29pOJ3DjBQG3blBHnCSiGLO/6AeNNaCwnjL8YhKkh+Rr6pabFsH aA89YPqrvUhDJYqnF4nSlWj5Wm5LReAjr/WdRdTuVhRU4UGejStMSzYs9NIeFkBSLiQL yigQ== X-Gm-Message-State: AJcUukcx6bHmIujcoIwW4LbD+idwhuWKAgBPnNPAaFZ5RvegfwxH2x9h gSWVFZte3dovNDiuv7QejMg= X-Google-Smtp-Source: ALg8bN6HCg1BcKeP/hbt5yjXTTN3ltoK7J27lhNwKo4giuN2s1OVwoiGCv7S33Jmm+m9cIBiUZHGeQ== X-Received: by 2002:a17:902:227:: with SMTP id 36mr46619276plc.140.1546510749734; Thu, 03 Jan 2019 02:19:09 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id 134sm65593331pgb.78.2019.01.03.02.19.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:09 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:15 +0800 Message-Id: <20190103101819.7418-4-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::643 Subject: [Qemu-devel] [PATCH v3 for-4.0 3/7] libvhost-user: Introduce vu_queue_map_desc() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji Introduce vu_queue_map_desc() which should be independent with vu_queue_pop(); Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu Reviewed-by: Marc-André Lureau --- contrib/libvhost-user/libvhost-user.c | 88 ++++++++++++++++----------- 1 file changed, 51 insertions(+), 37 deletions(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost-user.c index a6b46cdc03..23bd52264c 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -1853,49 +1853,20 @@ virtqueue_alloc_element(size_t sz, return elem; } -void * -vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) +static void * +vu_queue_map_desc(VuDev *dev, VuVirtq *vq, unsigned int idx, size_t sz) { - unsigned int i, head, max, desc_len; + struct vring_desc *desc = vq->vring.desc; uint64_t desc_addr, read_len; + unsigned int desc_len; + unsigned int max = vq->vring.num; + unsigned int i = idx; VuVirtqElement *elem; - unsigned out_num, in_num; + unsigned int out_num = 0, in_num = 0; struct iovec iov[VIRTQUEUE_MAX_SIZE]; struct vring_desc desc_buf[VIRTQUEUE_MAX_SIZE]; - struct vring_desc *desc; int rc; - if (unlikely(dev->broken) || - unlikely(!vq->vring.avail)) { - return NULL; - } - - if (vu_queue_empty(dev, vq)) { - return NULL; - } - /* Needed after virtio_queue_empty(), see comment in - * virtqueue_num_heads(). */ - smp_rmb(); - - /* When we start there are none of either input nor output. */ - out_num = in_num = 0; - - max = vq->vring.num; - if (vq->inuse >= vq->vring.num) { - vu_panic(dev, "Virtqueue size exceeded"); - return NULL; - } - - if (!virtqueue_get_head(dev, vq, vq->last_avail_idx++, &head)) { - return NULL; - } - - if (vu_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) { - vring_set_avail_event(vq, vq->last_avail_idx); - } - - i = head; - desc = vq->vring.desc; if (desc[i].flags & VRING_DESC_F_INDIRECT) { if (desc[i].len % sizeof(struct vring_desc)) { vu_panic(dev, "Invalid size for indirect buffer table"); @@ -1947,12 +1918,13 @@ vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) } while (rc == VIRTQUEUE_READ_DESC_MORE); if (rc == VIRTQUEUE_READ_DESC_ERROR) { + vu_panic(dev, "read descriptor error"); return NULL; } /* Now copy what we have collected and mapped */ elem = virtqueue_alloc_element(sz, out_num, in_num); - elem->index = head; + elem->index = idx; for (i = 0; i < out_num; i++) { elem->out_sg[i] = iov[i]; } @@ -1960,6 +1932,48 @@ vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) elem->in_sg[i] = iov[out_num + i]; } + return elem; +} + +void * +vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) +{ + unsigned int head; + VuVirtqElement *elem; + + if (unlikely(dev->broken) || + unlikely(!vq->vring.avail)) { + return NULL; + } + + if (vu_queue_empty(dev, vq)) { + return NULL; + } + /* + * Needed after virtio_queue_empty(), see comment in + * virtqueue_num_heads(). + */ + smp_rmb(); + + if (vq->inuse >= vq->vring.num) { + vu_panic(dev, "Virtqueue size exceeded"); + return NULL; + } + + if (!virtqueue_get_head(dev, vq, vq->last_avail_idx++, &head)) { + return NULL; + } + + if (vu_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) { + vring_set_avail_event(vq, vq->last_avail_idx); + } + + elem = vu_queue_map_desc(dev, vq, head, sz); + + if (!elem) { + return NULL; + } + vq->inuse++; return elem; From patchwork Thu Jan 3 10:18:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747171 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6BF3714E5 for ; Thu, 3 Jan 2019 10:20:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5470528563 for ; Thu, 3 Jan 2019 10:20:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 48B1728620; Thu, 3 Jan 2019 10:20:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4EF8C28563 for ; Thu, 3 Jan 2019 10:20:56 +0000 (UTC) Received: from localhost ([127.0.0.1]:50417 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf07T-00067r-6h for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:20:55 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44171) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf05x-0004IG-DC for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf05t-0002si-Iu for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:21 -0500 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:34053) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf05r-0002r0-JR for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:17 -0500 Received: by mail-pl1-x643.google.com with SMTP id w4so15749981plz.1 for ; Thu, 03 Jan 2019 02:19:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ku389Ea7iJJNi12slF2hwtTC0PlyQNn6AYfSdp+9lEM=; b=rn1qehwJm/byG7FC3w/v5bt11paltRfs/qdiW5ypkkiEyDNiVGx2aHh2OCEJoiJYx6 3B8RKqHJmwakcO372IB36C8YMOCcrmHnUd+rIJ6/4Esv/8Kp0aECEQMH2z0VAl3qf9yF Jj3LhK+dMyKIGQ7qerkuAobgvPlbNuq0sDBV8WxY1z0duz1zOu9eJxGjBKJmee4CY8ZJ 7CxV/g/dM5b9pxJWg0l6MuA/eBvDI6UVh+fZ9BNhSWioVGgAxMYFHKgB/oXJpB9NCr3u Q7ITNQFonR8RKSwVvYFYckSOz8fjPfuCJBuTzC9+XeDKAhRrcW1nTBOpZ8iEQ89t8tpL RwQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ku389Ea7iJJNi12slF2hwtTC0PlyQNn6AYfSdp+9lEM=; b=QmzH9ds9ySa1FgzHxq4W8pz8vx+uiAqbThbhU0WCel1gi5A6rBlSHd/r5Jo6unvZzP CkVZ8DGImV0kyaRH7eOW/qaTIb8lMTR0HL2xMXoFuY1TYWvqZ/lNls5w9tlu6vwKOGPw ruQszN2zRUVWreHrKMMhxXwCuyt1vqUtnMHrnZdCVMypwGlxkegRPfhpktOlVmBGYQ/+ 5IhSiwNl7cK1h6t+9gzW0EgWc5HvXTPpOVHaKbwPCNkUBGBUOYMZ3NhoDRl9Ao1VJKAU RH7/LuxY4H4JaeMVzOqmiaNToWc+VjpCwET5kkC4cPTvLjq2tMui55In8YrsqREA16Z4 vQrg== X-Gm-Message-State: AJcUukdYspGcXsiFNKvu9hB0dPsmoSh/+g38S9jFavPEFufCgLPkoM5O 4wF5JTG3Pj632k9D2WbYiHVJsXa9sVI= X-Google-Smtp-Source: ALg8bN40NtzF6R07YkK5oPXQ/Hz/G7An6l9ss7nA1oCCGQlgOf9zf1AZbDcrpqbcVfSF5zAQ0uYPxw== X-Received: by 2002:a17:902:7848:: with SMTP id e8mr47415043pln.100.1546510753852; Thu, 03 Jan 2019 02:19:13 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id d13sm82021324pfd.58.2019.01.03.02.19.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:13 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:16 +0800 Message-Id: <20190103101819.7418-5-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::643 Subject: [Qemu-devel] [PATCH v3 for-4.0 4/7] libvhost-user: Support recording inflight I/O in shared memory X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji This patch adds support for VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD message to get shared memory from qemu. Then we maintain a "bitmap" of all descriptors in the shared memory for each queue to record inflight I/O. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu --- contrib/libvhost-user/libvhost-user.c | 221 +++++++++++++++++++++++++- contrib/libvhost-user/libvhost-user.h | 33 ++++ 2 files changed, 248 insertions(+), 6 deletions(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost-user.c index 23bd52264c..f18f5e6e62 100644 --- a/contrib/libvhost-user/libvhost-user.c +++ b/contrib/libvhost-user/libvhost-user.c @@ -53,6 +53,18 @@ _min1 < _min2 ? _min1 : _min2; }) #endif +/* Round number down to multiple */ +#define ALIGN_DOWN(n, m) ((n) / (m) * (m)) + +/* Round number up to multiple */ +#define ALIGN_UP(n, m) ALIGN_DOWN((n) + (m) - 1, (m)) + +/* Align each region to cache line size in shared memory */ +#define SHM_ALIGNMENT 64 + +/* The version of shared memory */ +#define SHM_VERSION 1 + #define VHOST_USER_HDR_SIZE offsetof(VhostUserMsg, payload.u64) /* The version of the protocol we support */ @@ -100,6 +112,8 @@ vu_request_to_string(unsigned int req) REQ(VHOST_USER_POSTCOPY_ADVISE), REQ(VHOST_USER_POSTCOPY_LISTEN), REQ(VHOST_USER_POSTCOPY_END), + REQ(VHOST_USER_GET_SHM_SIZE), + REQ(VHOST_USER_SET_SHM_FD), REQ(VHOST_USER_MAX), }; #undef REQ @@ -890,6 +904,41 @@ vu_check_queue_msg_file(VuDev *dev, VhostUserMsg *vmsg) return true; } +static int +vu_check_queue_inflights(VuDev *dev, VuVirtq *vq) +{ + int i = 0; + + if ((dev->protocol_features & + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD) == 0) { + return 0; + } + + if (unlikely(!vq->shm)) { + return -1; + } + + vq->used_idx = vq->vring.used->idx; + vq->inflight_num = 0; + for (i = 0; i < vq->vring.num; i++) { + if (vq->shm->inflight[i] == 0) { + continue; + } + + vq->inflight_desc[vq->inflight_num++] = i; + vq->inuse++; + } + vq->shadow_avail_idx = vq->last_avail_idx = vq->inuse + vq->used_idx; + + /* in case of I/O hang after reconnecting */ + if (eventfd_write(vq->kick_fd, 1) || + eventfd_write(vq->call_fd, 1)) { + return -1; + } + + return 0; +} + static bool vu_set_vring_kick_exec(VuDev *dev, VhostUserMsg *vmsg) { @@ -925,6 +974,10 @@ vu_set_vring_kick_exec(VuDev *dev, VhostUserMsg *vmsg) dev->vq[index].kick_fd, index); } + if (vu_check_queue_inflights(dev, &dev->vq[index])) { + vu_panic(dev, "Failed to check inflights for vq: %d\n", index); + } + return false; } @@ -1215,6 +1268,115 @@ vu_set_postcopy_end(VuDev *dev, VhostUserMsg *vmsg) return true; } +static int +vu_setup_shm(VuDev *dev) +{ + int i; + char *addr = (char *)dev->shm_info.addr; + uint64_t size = 0; + uint32_t vq_size = ALIGN_UP(dev->shm_info.vq_size, dev->shm_info.align); + + if (dev->shm_info.version != SHM_VERSION) { + DPRINT("Invalid version for shm: %d", dev->shm_info.version); + return -1; + } + + if (dev->shm_info.dev_size != 0) { + DPRINT("Invalid dev_size for shm: %d", dev->shm_info.dev_size); + return -1; + } + + if (dev->shm_info.vq_size != sizeof(VuVirtqShm)) { + DPRINT("Invalid vq_size for shm: %d", dev->shm_info.vq_size); + return -1; + } + + for (i = 0; i < VHOST_MAX_NR_VIRTQUEUE; i++) { + size += vq_size; + if (size > dev->shm_info.mmap_size) { + break; + } + dev->vq[i].shm = (VuVirtqShm *)addr; + addr += vq_size; + } + + return 0; +} + +static bool +vu_get_shm_size(VuDev *dev, VhostUserMsg *vmsg) +{ + if (vmsg->size != sizeof(vmsg->payload.shm)) { + vu_panic(dev, "Invalid get_shm_size message:%d", vmsg->size); + vmsg->size = 0; + return true; + } + + vmsg->payload.shm.dev_size = 0; + vmsg->payload.shm.vq_size = sizeof(VuVirtqShm); + vmsg->payload.shm.align = SHM_ALIGNMENT; + vmsg->payload.shm.version = SHM_VERSION; + + DPRINT("send shm dev_size: %"PRId32"\n", vmsg->payload.shm.dev_size); + DPRINT("send shm vq_size: %"PRId32"\n", vmsg->payload.shm.vq_size); + DPRINT("send shm align: %"PRId32"\n", vmsg->payload.shm.align); + DPRINT("send shm version: %"PRId32"\n", vmsg->payload.shm.version); + + return true; +} + +static bool +vu_set_shm_fd(VuDev *dev, VhostUserMsg *vmsg) +{ + int fd; + uint64_t mmap_size, mmap_offset; + void *rc; + + if (vmsg->fd_num != 1 || + vmsg->size != sizeof(vmsg->payload.shm)) { + vu_panic(dev, "Invalid set_shm_fd message size:%d fds:%d", + vmsg->size, vmsg->fd_num); + return false; + } + + fd = vmsg->fds[0]; + mmap_size = vmsg->payload.shm.mmap_size; + mmap_offset = vmsg->payload.shm.mmap_offset; + DPRINT("set_shm_fd mmap_size: %"PRId64"\n", mmap_size); + DPRINT("set_shm_fd mmap_offset: %"PRId64"\n", mmap_offset); + DPRINT("set_shm_fd dev_size: %"PRId32"\n", vmsg->payload.shm.dev_size); + DPRINT("set_shm_fd vq_size: %"PRId32"\n", vmsg->payload.shm.vq_size); + DPRINT("set_shm_fd align: %"PRId32"\n", vmsg->payload.shm.align); + DPRINT("set_shm_fd version: %"PRId32"\n", vmsg->payload.shm.version); + + rc = mmap(0, mmap_size, PROT_READ | PROT_WRITE, MAP_SHARED, + fd, mmap_offset); + + close(fd); + + if (rc == MAP_FAILED) { + vu_panic(dev, "set_shm_fd mmap error: %s", strerror(errno)); + return false; + } + + if (dev->shm_info.addr) { + munmap(dev->shm_info.addr, dev->shm_info.mmap_size); + } + dev->shm_info.addr = rc; + dev->shm_info.mmap_size = mmap_size; + dev->shm_info.dev_size = vmsg->payload.shm.dev_size; + dev->shm_info.vq_size = vmsg->payload.shm.vq_size; + dev->shm_info.align = vmsg->payload.shm.align; + dev->shm_info.version = vmsg->payload.shm.version; + + if (vu_setup_shm(dev)) { + vu_panic(dev, "setup shm failed"); + return false; + } + + return false; +} + static bool vu_process_message(VuDev *dev, VhostUserMsg *vmsg) { @@ -1292,6 +1454,10 @@ vu_process_message(VuDev *dev, VhostUserMsg *vmsg) return vu_set_postcopy_listen(dev, vmsg); case VHOST_USER_POSTCOPY_END: return vu_set_postcopy_end(dev, vmsg); + case VHOST_USER_GET_SHM_SIZE: + return vu_get_shm_size(dev, vmsg); + case VHOST_USER_SET_SHM_FD: + return vu_set_shm_fd(dev, vmsg); default: vmsg_close_fds(vmsg); vu_panic(dev, "Unhandled request: %d", vmsg->request); @@ -1359,8 +1525,13 @@ vu_deinit(VuDev *dev) close(vq->err_fd); vq->err_fd = -1; } + vq->shm = NULL; } + if (dev->shm_info.addr) { + munmap(dev->shm_info.addr, dev->shm_info.mmap_size); + dev->shm_info.addr = NULL; + } vu_close_log(dev); if (dev->slave_fd != -1) { @@ -1829,12 +2000,6 @@ virtqueue_map_desc(VuDev *dev, *p_num_sg = num_sg; } -/* Round number down to multiple */ -#define ALIGN_DOWN(n, m) ((n) / (m) * (m)) - -/* Round number up to multiple */ -#define ALIGN_UP(n, m) ALIGN_DOWN((n) + (m) - 1, (m)) - static void * virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num) @@ -1935,9 +2100,44 @@ vu_queue_map_desc(VuDev *dev, VuVirtq *vq, unsigned int idx, size_t sz) return elem; } +static int +vu_queue_inflight_get(VuDev *dev, VuVirtq *vq, int desc_idx) +{ + if ((dev->protocol_features & + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD) == 0) { + return 0; + } + + if (unlikely(!vq->shm)) { + return -1; + } + + vq->shm->inflight[desc_idx] = 1; + + return 0; +} + +static int +vu_queue_inflight_put(VuDev *dev, VuVirtq *vq, int desc_idx) +{ + if ((dev->protocol_features & + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD) == 0) { + return 0; + } + + if (unlikely(!vq->shm)) { + return -1; + } + + vq->shm->inflight[desc_idx] = 0; + + return 0; +} + void * vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) { + int i; unsigned int head; VuVirtqElement *elem; @@ -1946,6 +2146,12 @@ vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) return NULL; } + if (unlikely(vq->inflight_num > 0)) { + i = (--vq->inflight_num); + elem = vu_queue_map_desc(dev, vq, vq->inflight_desc[i], sz); + return elem; + } + if (vu_queue_empty(dev, vq)) { return NULL; } @@ -1976,6 +2182,8 @@ vu_queue_pop(VuDev *dev, VuVirtq *vq, size_t sz) vq->inuse++; + vu_queue_inflight_get(dev, vq, head); + return elem; } @@ -2121,4 +2329,5 @@ vu_queue_push(VuDev *dev, VuVirtq *vq, { vu_queue_fill(dev, vq, elem, len, 0); vu_queue_flush(dev, vq, 1); + vu_queue_inflight_put(dev, vq, elem->index); } diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/libvhost-user.h index 4aa55b4d2d..fdfda688d2 100644 --- a/contrib/libvhost-user/libvhost-user.h +++ b/contrib/libvhost-user/libvhost-user.h @@ -53,6 +53,7 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_CONFIG = 9, VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD = 10, VHOST_USER_PROTOCOL_F_HOST_NOTIFIER = 11, + VHOST_USER_PROTOCOL_F_SLAVE_SHMFD = 12, VHOST_USER_PROTOCOL_F_MAX }; @@ -91,6 +92,8 @@ typedef enum VhostUserRequest { VHOST_USER_POSTCOPY_ADVISE = 28, VHOST_USER_POSTCOPY_LISTEN = 29, VHOST_USER_POSTCOPY_END = 30, + VHOST_USER_GET_SHM_SIZE = 31, + VHOST_USER_SET_SHM_FD = 32, VHOST_USER_MAX } VhostUserRequest; @@ -138,6 +141,15 @@ typedef struct VhostUserVringArea { uint64_t offset; } VhostUserVringArea; +typedef struct VhostUserShm { + uint64_t mmap_size; + uint64_t mmap_offset; + uint32_t dev_size; + uint32_t vq_size; + uint32_t align; + uint32_t version; +} VhostUserShm; + #if defined(_WIN32) # define VU_PACKED __attribute__((gcc_struct, packed)) #else @@ -163,6 +175,7 @@ typedef struct VhostUserMsg { VhostUserLog log; VhostUserConfig config; VhostUserVringArea area; + VhostUserShm shm; } payload; int fds[VHOST_MEMORY_MAX_NREGIONS]; @@ -234,9 +247,19 @@ typedef struct VuRing { uint32_t flags; } VuRing; +typedef struct VuVirtqShm { + char inflight[VIRTQUEUE_MAX_SIZE]; +} VuVirtqShm; + typedef struct VuVirtq { VuRing vring; + VuVirtqShm *shm; + + uint16_t inflight_desc[VIRTQUEUE_MAX_SIZE]; + + uint16_t inflight_num; + /* Next head to pop */ uint16_t last_avail_idx; @@ -279,11 +302,21 @@ typedef void (*vu_set_watch_cb) (VuDev *dev, int fd, int condition, vu_watch_cb cb, void *data); typedef void (*vu_remove_watch_cb) (VuDev *dev, int fd); +typedef struct VuDevShmInfo { + void *addr; + uint64_t mmap_size; + uint32_t dev_size; + uint32_t vq_size; + uint32_t align; + uint32_t version; +} VuDevShmInfo; + struct VuDev { int sock; uint32_t nregions; VuDevRegion regions[VHOST_MEMORY_MAX_NREGIONS]; VuVirtq vq[VHOST_MAX_NR_VIRTQUEUE]; + VuDevShmInfo shm_info; int log_call_fd; int slave_fd; uint64_t log_size; From patchwork Thu Jan 3 10:18:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747177 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F9B814E5 for ; Thu, 3 Jan 2019 10:23:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D92728563 for ; Thu, 3 Jan 2019 10:23:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 11A5B28623; Thu, 3 Jan 2019 10:23:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A560028563 for ; Thu, 3 Jan 2019 10:23:26 +0000 (UTC) Received: from localhost ([127.0.0.1]:50441 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf09t-0000hf-Ta for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:23:25 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44182) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf05z-0004Iq-Bs for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf05x-0002uJ-DS for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:23 -0500 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:33839) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf05v-0002sG-EE for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:20 -0500 Received: by mail-pf1-x444.google.com with SMTP id h3so16508060pfg.1 for ; Thu, 03 Jan 2019 02:19:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=a7DLZuow3M9M2aEACsQREQCdzbl9zrjyaeO9vV3x0Ng=; b=XK5Ae/GZ8SSNhkn/irCZIyK6Kh+DkP+UgoG8+W+zqs9PWwlxPERgDEE+rR2/D9fo2V retT2V2wB/DF+2aNUje2zUBUPzedbBB64y27BIz0FR7vxZnB5a+LqcrOzfPC8ncjYMHz h4JCeR5T7ryTCmaUOp67jrMYaFHOv9zB65b1a7OzJOjXdeErXe2yCI89KoiGZsBLONfb E7n7V2hw4spKjcwKfaWb93aZ3EfXKKm/sc0W0vSAb7LLNvg9r5MVDEZIU2ovTJfJMnZy rtEE0AAuwCkupHFcSenMYH4bVwTKyOsGUSNrVCKq5IC06mpwOVgPNzuD6JTC13xsMvaZ H78g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=a7DLZuow3M9M2aEACsQREQCdzbl9zrjyaeO9vV3x0Ng=; b=gG+jGfeaE4i5tcEfwighq8CI6616nim0mA2W2YaXUa7DHM5/SzCgMYlouln23T/XrV j+HsbNpvVqvAaD+TrTOXqTjMKGKsGIHnRgHuBXTj+/h/zKHmYc9bCT42sj762UpM/5+R NBfF0jszv7WTJMHYGa5kd8w1k3gfXWanLj4ksdp3cmK/ep3M+bnl/wjDr222d045AK8X eXw6yJfl7vKgo7flprGW8V3kSEWUKg3myTouwLp2xJf3c/osMucCBRv10mvqYQpBHYsc sNUnhdx3MhATx/mN/JGIiXZOZg34sZMlNVn2IALHJjDiXonos9ICHUGcmrbEutbPP1pW YLIA== X-Gm-Message-State: AJcUukf4owsYnQncnjmgASsXBIP+YSNs42G8i5ZPxMkGC8LPYDRfG0zo koCZHEv+teKz+SbOZBBKTBU= X-Google-Smtp-Source: ALg8bN7bU530c85iQVYunKZjybV+maBoJXmx1HQKnRCXFyGFlHUoPdhOxXNUDeAaFN2RqxxnefltTA== X-Received: by 2002:a63:8742:: with SMTP id i63mr16333735pge.298.1546510757012; Thu, 03 Jan 2019 02:19:17 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id p7sm102550278pfj.72.2019.01.03.02.19.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:16 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:17 +0800 Message-Id: <20190103101819.7418-6-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::444 Subject: [Qemu-devel] [PATCH v3 for-4.0 5/7] vhost-user-blk: Add support to provide shared memory to backend X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji This patch add supports for vhost-user-blk device to provide shared memory to backend. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu --- hw/block/vhost-user-blk.c | 26 ++++++++++++++++++++++++++ include/hw/virtio/vhost-user-blk.h | 1 + 2 files changed, 27 insertions(+) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 1451940845..27028cf996 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -126,6 +126,13 @@ static void vhost_user_blk_start(VirtIODevice *vdev) } s->dev.acked_features = vdev->guest_features; + + ret = vhost_dev_set_shm(&s->dev, s->shm); + if (ret < 0) { + error_report("Error set shared memory: %d", -ret); + goto err_guest_notifiers; + } + ret = vhost_dev_start(&s->dev, vdev); if (ret < 0) { error_report("Error starting vhost: %d", -ret); @@ -245,6 +252,13 @@ static void vhost_user_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) } } +static void vhost_user_blk_reset(VirtIODevice *vdev) +{ + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + vhost_dev_reset_shm(s->shm); +} + static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) { VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -284,6 +298,8 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) vhost_user_blk_handle_output); } + s->shm = g_new0(struct vhost_shm, 1); + s->dev.nvqs = s->num_queues; s->dev.vqs = g_new(struct vhost_virtqueue, s->dev.nvqs); s->dev.vq_index = 0; @@ -309,12 +325,19 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) s->blkcfg.num_queues = s->num_queues; } + ret = vhost_dev_init_shm(&s->dev, s->shm); + if (ret < 0) { + error_setg(errp, "vhost-user-blk: init shared memory failed"); + goto vhost_err; + } + return; vhost_err: vhost_dev_cleanup(&s->dev); virtio_err: g_free(s->dev.vqs); + g_free(s->shm); virtio_cleanup(vdev); vhost_user_cleanup(user); @@ -329,7 +352,9 @@ static void vhost_user_blk_device_unrealize(DeviceState *dev, Error **errp) vhost_user_blk_set_status(vdev, 0); vhost_dev_cleanup(&s->dev); + vhost_dev_free_shm(s->shm); g_free(s->dev.vqs); + g_free(s->shm); virtio_cleanup(vdev); if (s->vhost_user) { @@ -379,6 +404,7 @@ static void vhost_user_blk_class_init(ObjectClass *klass, void *data) vdc->set_config = vhost_user_blk_set_config; vdc->get_features = vhost_user_blk_get_features; vdc->set_status = vhost_user_blk_set_status; + vdc->reset = vhost_user_blk_reset; } static const TypeInfo vhost_user_blk_info = { diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h index d52944aeeb..bb706d70b3 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -36,6 +36,7 @@ typedef struct VHostUserBlk { uint32_t queue_size; uint32_t config_wce; struct vhost_dev dev; + struct vhost_shm *shm; VhostUserState *vhost_user; } VHostUserBlk; From patchwork Thu Jan 3 10:18:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747183 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9339614E5 for ; Thu, 3 Jan 2019 10:25:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E32528623 for ; Thu, 3 Jan 2019 10:25:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6CDBF2862D; Thu, 3 Jan 2019 10:25:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A223D28623 for ; Thu, 3 Jan 2019 10:25:16 +0000 (UTC) Received: from localhost ([127.0.0.1]:50459 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf0Bf-0002jd-U6 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:25:15 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44198) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf061-0004Kr-DR for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf05z-0002vc-EC for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:25 -0500 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:46268) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf05x-0002tw-Eu for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:23 -0500 Received: by mail-pl1-x643.google.com with SMTP id t13so15719711ply.13 for ; Thu, 03 Jan 2019 02:19:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ONpocrNvfvxOJrLwZpGloZARxJC8GSGpmRfms8KcDDA=; b=ilqvUKu6wJmRKdw5bXECL5s+ZEtynqu5quYdIiMkiZeu9eLc1n+FMDxeGP6BLUN0Hv GwKZamuk2s5+bIld7U8aF/BANJXkoafRSlNuYbfbAqBl/B+lcl2WlcE+b7MPtc05h9GZ wIVkKaR5wYEUu8SNwpielyUlZXpMAoM0lz6P72rgQDivpNox7NR8HRyYQt8JrhwXbvj4 FYEsyPvH3nI/musYIfsZ6Tjg3VfJ7YghtOlnsC/4SAOk77aASU28MGxSBxAoT+V2J42t AdlxnsUSU/4XIO7XAiS9ZnCw/e7BHTBdkrzL/a/lPBMD9FWnOshLh07Y1dfIErSgoFYt 0wxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ONpocrNvfvxOJrLwZpGloZARxJC8GSGpmRfms8KcDDA=; b=GiyqKJcQDFG39gpy2NeEFCU4pC/MbJAh090r4FZXKNeCsG9//WBMfVK/H0W1QBzeqy edoaIMtt6WIjFlwsI13oJoHcPo3+F3hF9UJD3eRTkaSeDHc3XkVvNLos1MlkFHV8Xv0A AQMGS+0wf/zybufYYjyQFzHF4d8ItW3hAyjabblFUj/V3yeLPjLA1uyDb0F9npoSLHlj DSuduAQhbGlYTFnImqhqe6Vv/ITPvqU9ziM94X+yXhyaLqes9leHy6KMfxHcuqJnh0+K 6QO89TTmkZ3H+Zs137nXyR5SzybcRIRltSGWl8CN/QKvPChsGFy6vQ3ZXY+pM853hVvc gzuQ== X-Gm-Message-State: AJcUukdN6DD7FeJvudiAUbKr7msLHgz1/K4H2YEWWqXU4L2Xjrenup2J GrxTFF7dqEdc1DRpzGJjxHqN4BvKJWg= X-Google-Smtp-Source: ALg8bN48poJ4PTdL9eq9aLD8qIg3Db0iqMLI71OKVid8BNTLlg9bKYwQlpg16iIXveYbBdsXpHpULQ== X-Received: by 2002:a17:902:bd86:: with SMTP id q6mr45168261pls.16.1546510760223; Thu, 03 Jan 2019 02:19:20 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id y5sm92388899pge.49.2019.01.03.02.19.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:19 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:18 +0800 Message-Id: <20190103101819.7418-7-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::643 Subject: [Qemu-devel] [PATCH v3 for-4.0 6/7] vhost-user-blk: Add support to reconnect backend X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji Since we now support the message VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD. The backend is able to restart safely because it can record inflight I/O in shared memory. This patch allows qemu to reconnect the backend after connection closed. Signed-off-by: Xie Yongji Signed-off-by: Ni Xun Signed-off-by: Zhang Yu --- hw/block/vhost-user-blk.c | 205 +++++++++++++++++++++++------ include/hw/virtio/vhost-user-blk.h | 4 + 2 files changed, 168 insertions(+), 41 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 27028cf996..c0725e8e4a 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -101,7 +101,7 @@ const VhostDevConfigOps blk_ops = { .vhost_dev_config_notifier = vhost_user_blk_handle_config_change, }; -static void vhost_user_blk_start(VirtIODevice *vdev) +static int vhost_user_blk_start(VirtIODevice *vdev) { VHostUserBlk *s = VHOST_USER_BLK(vdev); BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); @@ -110,13 +110,13 @@ static void vhost_user_blk_start(VirtIODevice *vdev) if (!k->set_guest_notifiers) { error_report("binding does not support guest notifiers"); - return; + return -ENOSYS; } ret = vhost_dev_enable_notifiers(&s->dev, vdev); if (ret < 0) { error_report("Error enabling host notifiers: %d", -ret); - return; + return ret; } ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, true); @@ -147,12 +147,13 @@ static void vhost_user_blk_start(VirtIODevice *vdev) vhost_virtqueue_mask(&s->dev, vdev, i, false); } - return; + return ret; err_guest_notifiers: k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&s->dev, vdev); + return ret; } static void vhost_user_blk_stop(VirtIODevice *vdev) @@ -171,7 +172,6 @@ static void vhost_user_blk_stop(VirtIODevice *vdev) ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); - return; } vhost_dev_disable_notifiers(&s->dev, vdev); @@ -181,21 +181,43 @@ static void vhost_user_blk_set_status(VirtIODevice *vdev, uint8_t status) { VHostUserBlk *s = VHOST_USER_BLK(vdev); bool should_start = status & VIRTIO_CONFIG_S_DRIVER_OK; + int ret; if (!vdev->vm_running) { should_start = false; } - if (s->dev.started == should_start) { + if (s->should_start == should_start) { + return; + } + + if (!s->connected || s->dev.started == should_start) { + s->should_start = should_start; return; } if (should_start) { - vhost_user_blk_start(vdev); + s->should_start = true; + /* + * make sure vhost_user_blk_handle_output() ignores fake + * guest kick by vhost_dev_enable_notifiers() + */ + barrier(); + ret = vhost_user_blk_start(vdev); + if (ret < 0) { + error_report("vhost-user-blk: vhost start failed: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&s->chardev); + } } else { vhost_user_blk_stop(vdev); + /* + * make sure vhost_user_blk_handle_output() ignore fake + * guest kick by vhost_dev_disable_notifiers() + */ + barrier(); + s->should_start = false; } - } static uint64_t vhost_user_blk_get_features(VirtIODevice *vdev, @@ -225,13 +247,22 @@ static uint64_t vhost_user_blk_get_features(VirtIODevice *vdev, static void vhost_user_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) { VHostUserBlk *s = VHOST_USER_BLK(vdev); - int i; + int i, ret; if (!(virtio_host_has_feature(vdev, VIRTIO_F_VERSION_1) && !virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1))) { return; } + if (s->should_start) { + return; + } + s->should_start = true; + + if (!s->connected) { + return; + } + if (s->dev.started) { return; } @@ -239,7 +270,13 @@ static void vhost_user_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) /* Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start * vhost here instead of waiting for .set_status(). */ - vhost_user_blk_start(vdev); + ret = vhost_user_blk_start(vdev); + if (ret < 0) { + error_report("vhost-user-blk: vhost start failed: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&s->chardev); + return; + } /* Kick right away to begin processing requests already in vring */ for (i = 0; i < s->dev.nvqs; i++) { @@ -259,12 +296,105 @@ static void vhost_user_blk_reset(VirtIODevice *vdev) vhost_dev_reset_shm(s->shm); } +static int vhost_user_blk_connect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + int ret = 0; + + if (s->connected) { + return 0; + } + s->connected = true; + + s->dev.nvqs = s->num_queues; + s->dev.vqs = s->vqs; + s->dev.vq_index = 0; + s->dev.backend_features = 0; + + vhost_dev_set_config_notifier(&s->dev, &blk_ops); + + ret = vhost_dev_init(&s->dev, s->vhost_user, VHOST_BACKEND_TYPE_USER, 0); + if (ret < 0) { + error_report("vhost-user-blk: vhost initialization failed: %s", + strerror(-ret)); + return ret; + } + + /* restore vhost state */ + if (s->should_start) { + ret = vhost_user_blk_start(vdev); + if (ret < 0) { + error_report("vhost-user-blk: vhost start failed: %s", + strerror(-ret)); + return ret; + } + } + + return 0; +} + +static void vhost_user_blk_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + if (!s->connected) { + return; + } + s->connected = false; + + if (s->dev.started) { + vhost_user_blk_stop(vdev); + } + + vhost_dev_cleanup(&s->dev); +} + +static gboolean vhost_user_blk_watch(GIOChannel *chan, GIOCondition cond, + void *opaque) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + qemu_chr_fe_disconnect(&s->chardev); + + return true; +} + +static void vhost_user_blk_event(void *opaque, int event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_blk_connect(dev) < 0) { + qemu_chr_fe_disconnect(&s->chardev); + return; + } + s->watch = qemu_chr_fe_add_watch(&s->chardev, G_IO_HUP, + vhost_user_blk_watch, dev); + break; + case CHR_EVENT_CLOSED: + vhost_user_blk_disconnect(dev); + if (s->watch) { + g_source_remove(s->watch); + s->watch = 0; + } + break; + } +} + static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) { VirtIODevice *vdev = VIRTIO_DEVICE(dev); VHostUserBlk *s = VHOST_USER_BLK(vdev); VhostUserState *user; int i, ret; + Error *err = NULL; if (!s->chardev.chr) { error_setg(errp, "vhost-user-blk: chardev is mandatory"); @@ -299,26 +429,28 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) } s->shm = g_new0(struct vhost_shm, 1); - - s->dev.nvqs = s->num_queues; - s->dev.vqs = g_new(struct vhost_virtqueue, s->dev.nvqs); - s->dev.vq_index = 0; - s->dev.backend_features = 0; - - vhost_dev_set_config_notifier(&s->dev, &blk_ops); - - ret = vhost_dev_init(&s->dev, s->vhost_user, VHOST_BACKEND_TYPE_USER, 0); - if (ret < 0) { - error_setg(errp, "vhost-user-blk: vhost initialization failed: %s", - strerror(-ret)); - goto virtio_err; - } + s->vqs = g_new(struct vhost_virtqueue, s->num_queues); + s->watch = 0; + s->should_start = false; + s->connected = false; + + qemu_chr_fe_set_handlers(&s->chardev, NULL, NULL, vhost_user_blk_event, + NULL, (void *)dev, NULL, true); + +reconnect: + do { + if (qemu_chr_fe_wait_connected(&s->chardev, &err) < 0) { + error_report_err(err); + err = NULL; + sleep(1); + } + } while (!s->connected); ret = vhost_dev_get_config(&s->dev, (uint8_t *)&s->blkcfg, - sizeof(struct virtio_blk_config)); + sizeof(struct virtio_blk_config)); if (ret < 0) { - error_setg(errp, "vhost-user-blk: get block config failed"); - goto vhost_err; + error_report("vhost-user-blk: get block config failed"); + goto reconnect; } if (s->blkcfg.num_queues != s->num_queues) { @@ -327,22 +459,11 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) ret = vhost_dev_init_shm(&s->dev, s->shm); if (ret < 0) { - error_setg(errp, "vhost-user-blk: init shared memory failed"); - goto vhost_err; + error_report("vhost-user-blk: init shared memory failed"); + goto reconnect; } return; - -vhost_err: - vhost_dev_cleanup(&s->dev); -virtio_err: - g_free(s->dev.vqs); - g_free(s->shm); - virtio_cleanup(vdev); - - vhost_user_cleanup(user); - g_free(user); - s->vhost_user = NULL; } static void vhost_user_blk_device_unrealize(DeviceState *dev, Error **errp) @@ -351,9 +472,11 @@ static void vhost_user_blk_device_unrealize(DeviceState *dev, Error **errp) VHostUserBlk *s = VHOST_USER_BLK(dev); vhost_user_blk_set_status(vdev, 0); + qemu_chr_fe_set_handlers(&s->chardev, NULL, NULL, NULL, + NULL, NULL, NULL, false); vhost_dev_cleanup(&s->dev); vhost_dev_free_shm(s->shm); - g_free(s->dev.vqs); + g_free(s->vqs); g_free(s->shm); virtio_cleanup(vdev); diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h index bb706d70b3..c17d47402b 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -38,6 +38,10 @@ typedef struct VHostUserBlk { struct vhost_dev dev; struct vhost_shm *shm; VhostUserState *vhost_user; + struct vhost_virtqueue *vqs; + guint watch; + bool should_start; + bool connected; } VHostUserBlk; #endif From patchwork Thu Jan 3 10:18:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10747181 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0FA2B6C5 for ; Thu, 3 Jan 2019 10:23:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F12E027F94 for ; Thu, 3 Jan 2019 10:23:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E278028178; Thu, 3 Jan 2019 10:23:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9A2A527F94 for ; Thu, 3 Jan 2019 10:23:49 +0000 (UTC) Received: from localhost ([127.0.0.1]:50436 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf0AG-0008I2-R3 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 03 Jan 2019 05:23:48 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44221) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gf062-0004M3-AU for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gf061-0002wv-Hx for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:26 -0500 Received: from mail-pg1-x543.google.com ([2607:f8b0:4864:20::543]:46740) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gf061-0002vo-A8 for qemu-devel@nongnu.org; Thu, 03 Jan 2019 05:19:25 -0500 Received: by mail-pg1-x543.google.com with SMTP id w7so15809128pgp.13 for ; Thu, 03 Jan 2019 02:19:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XwarkgJl5t9GVipPy4GZ/zOWp92+r2vb3CtyKc7ZAAc=; b=o+oUPBLi+DOkW6z5nZDDgQcEoKeHVTciy9FFFTT0m8CSIHzqFdq0VvUfm6doj2Nm9k ikn4SsqXiE+2qryvCC9PWDJdBIc79/e1riYNH6CgGJJ45hSBjyttE51dg8ZkFe5nlbDw tKLmQARP/SQdFnFos5kvcAh+ZyMyFpkve4eZP+v9HebmtVNwcDGRP2/dQ23VOmS+FBEH F6M0G4wbdxpZLsIzAGH/yo9W4UKSfX5h44HAJ84/5WIpqBU8/QMIIH2r86yNAbchxqCW /J2rp7D555EYSmBvXLknfoL9gO0Vo+5pxfhFpoqRiXWtNdCln+4P2vWoQ6dHoD6cyvt3 1tHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XwarkgJl5t9GVipPy4GZ/zOWp92+r2vb3CtyKc7ZAAc=; b=DL+NLHVxQrnvVPSJ3YkYM0svF0S+5/ACvVLJc0NDnopTESM/8bI4mfQ5mqLX/xXmWw TUc8YsBpw6DqJaqoKtKr/5IlLHaAApvlTbdb8Lha2Pl66TdyYxzORvr0t8d05/ScdESE 5px3ztwDMLadxrJQ+/BcPp7MYBHiAhwUBqPbKUduNCUP0kEBNjUqkgzh6dJ8YHiZ5sO2 UxpAoQGA31X51Q63+x3MtkkfB8fXbLLkpDv84akKEudnXEE5ICoVOXqfzj4fwQCxBKZt +8Yagl1gfvIcREfpltpqDBQrroQD+vm3JLbLWbWKSZEdWX3FLquNYxqQRlZe7B131lkz o1RA== X-Gm-Message-State: AJcUukfV1BP52idwNtqj8LIUgxd0IU+FVfRMB4gECC3S8AQSIHB2yoFf 4/KoLaPGlFOudT+qKNIDRSpU9tN/ZfM= X-Google-Smtp-Source: ALg8bN5K6QEYFuMuI9rNc9G0XbVtyUofwHs0mFl8/cFguVOmHNtamOoHjnnOI5GcOMLWrgTI9rRQ7A== X-Received: by 2002:a65:5c02:: with SMTP id u2mr16369842pgr.13.1546510763380; Thu, 03 Jan 2019 02:19:23 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id u186sm83560870pfu.51.2019.01.03.02.19.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 03 Jan 2019 02:19:23 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Thu, 3 Jan 2019 18:18:19 +0800 Message-Id: <20190103101819.7418-8-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190103101819.7418-1-xieyongji@baidu.com> References: <20190103101819.7418-1-xieyongji@baidu.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::543 Subject: [Qemu-devel] [PATCH v3 for-4.0 7/7] contrib/vhost-user-blk: enable inflight I/O recording X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji This patch enables inflight I/O recording for vhost-user-blk backend so that we could restart it safely. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu --- contrib/vhost-user-blk/vhost-user-blk.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/vhost-user-blk/vhost-user-blk.c b/contrib/vhost-user-blk/vhost-user-blk.c index 858221ad95..6d9a417f4a 100644 --- a/contrib/vhost-user-blk/vhost-user-blk.c +++ b/contrib/vhost-user-blk/vhost-user-blk.c @@ -327,7 +327,8 @@ vub_get_features(VuDev *dev) static uint64_t vub_get_protocol_features(VuDev *dev) { - return 1ull << VHOST_USER_PROTOCOL_F_CONFIG; + return 1ull << VHOST_USER_PROTOCOL_F_CONFIG | + 1ull << VHOST_USER_PROTOCOL_F_SLAVE_SHMFD; } static int