From patchwork Tue Aug 4 08:11:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lukas Straub X-Patchwork-Id: 11699953 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26B4E913 for ; Tue, 4 Aug 2020 08:19:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 21D952076C for ; Tue, 4 Aug 2020 08:19:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=web.de header.i=@web.de header.b="WVmTnNZX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21D952076C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=web.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:52020 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k2sA1-0008VS-2O for patchwork-qemu-devel@patchwork.kernel.org; Tue, 04 Aug 2020 04:19:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k2s31-0003Wa-TY; Tue, 04 Aug 2020 04:11:47 -0400 Received: from mout.web.de ([212.227.17.12]:41563) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k2s2z-0006ZP-KY; Tue, 04 Aug 2020 04:11:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1596528699; bh=gYLkaLVifVVybkD4dZzByIAONcymChjA5zGr5nxBC4k=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=WVmTnNZXS8YWJsXUGRNZWZiYhu/JXFhETNum2gjkbkv8xO4j12BCbqSMidLnSc5Y3 ddLUU/arITmvXb3RQ4wNTYVikmH/S89nMjZDlof+JmHdR7eNl6Bc9d7czh2/En6yM2 kmgl/6InD4b/AGAWVXCbc8sm4tjDNtbEqncwyTO8= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from luklap ([89.247.255.220]) by smtp.web.de (mrweb102 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MeSKR-1kO5x62UOd-00Q8VG; Tue, 04 Aug 2020 10:11:39 +0200 Date: Tue, 4 Aug 2020 10:11:37 +0200 From: Lukas Straub To: qemu-devel Subject: [PATCH v7 2/8] block/nbd.c: Add yank feature Message-ID: <331864b96d198a6cfabd050581e94bc3452362aa.1596528468.git.lukasstraub2@web.de> In-Reply-To: References: MIME-Version: 1.0 X-Provags-ID: V03:K1:RQvrVnOEw5VqZQ61FbfDNTJgN/jY2h+db6zwFQ//gDqUK2YzBWs YVZaJnsYcKB5t0/QyjhWufZqUGemmlIrnZwLFARs6ZXgahhwlu5bRvXBH22g2AlfNNNxWVK Ggpodglbql1NpU5jsbozVwjysIv4KR531m1fKKLjvf1ysl4cCs7aAKDw/54Por9DE7ms7c2 m35ByH5z7b912LZYsZRcg== X-UI-Out-Filterresults: notjunk:1;V03:K0:pCJd6LLGNZI=:Q5pHYFxlu6tHPNN09Nz05+ tw/mQSICLiuhiGdJBxo4y2qkJ592ArNfUKCWubC6ftcp3x3NKF8NtE/FQFqvJ/t9wULeh43oV dQMbXOvcX1D6bmFf7PADgj+zH8Xkj7yWEviMoNxyXBWNR3/EOd16jxx93bOLn0PaP1LZRcZzF KRof6d0+znSrCBx5hDRiT84ftDZWzv6ZkqxxWi7NmWGmshtpOO/TuS2kuM1oTt5rQrB8XGwAz uIUy8vLTWplafjOpkrscpISOVHXMpKOZ+PKpcdG2/WgPHP8VEtCTsdQbHmVanw0CgNPuNYaQe wbPMeXmVNglkTX5pTI50pVKmxWk+gwEiYKUAe1q2xZPhRtco2z4EmHYLZHqkPGTG8xuHH6t5R WynG3gGOiWCMQu/UrV3MDD+sYXBaQNAtOnexNsSJELlBKv18vJUbGTbazH404R3pvWy7dqQPu KKhFaoPC1bEF8W8zGbPs6sgZ8nMmfvc5I1ltRbZ/TdoZxv0dFPE4yNE7avv5S5OwqfZnN9fdY Jp1vAOT/PxFsf19g/jYAPIrPpFkyTf8J5yHzuptIbucmBP32KPhkZeNdKf+YyDstaR7iI/Pmz 9Ih2vB+RQmc0aOofTvbnBKSMYIoQkbeVmeHezfMmX46CG15ntO8NNjGX7WaVuMqPeLvxT08J5 oPE+3Xs6fsfoOtiqmf4ArQpz3IVCjnwythcHY0uWJHW4lOWl7AABKiqPS09izfq77pINhdPx1 5KDivcoL20cxl59od6nfeYBphWzrRG0lUWEwJqjVS/zhCSFZ+cDKhal6ePXjTwLSZ//jSelmX Bf/R0tEmPsPJCJOeXqoirnUqPUaRlV915iNpEkKvtmOPrAdoNKsluOKvT+awpGtjm1SeyOv1c gIBE8oDpe9VnpbmOZgNeR2/mXjascCOQxcVvovuIm+zTL7xBPMV5qgl5hquXvSYRGgK59Oi3s mptuJUS2NlDXPveaVsYCtB/VlmD0m2JAQW/eZQFsoWyaCWCKr5HGbP567zFP7ycUN2QYGKCdI Cf87sgi4/absZJRpieM10bloiPaVaHGZo+xUpcrlaYCZPnhzK0hhlHIyGCnuHj7o8+r1f1i/D +cukqPpm6j92gpldSG+QeWIE6a6gpE8OVH1jngoQ3LbV6VxIIn8pQyX78za06VZDALZlk2GL3 VWbO2euPhHO0zH9Tjn9BapoQ3QDpbmrz+ESeTI7kkLIoffcwLP7eq+oB3JLX15uQefTvaiE3S GcXjwB7qIgnDHMcToiPm13pApKeV+q7TneyDwuA== Received-SPF: pass client-ip=212.227.17.12; envelope-from=lukasstraub2@web.de; helo=mout.web.de X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/04 04:11:44 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , "Daniel P. =?utf-8?b?QmVycmFuZ8Op?= " , qemu-block , Juan Quintela , "Dr. David Alan Gilbert" , Max Reitz , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9?= Lureau Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Register a yank function which shuts down the socket and sets s->state = NBD_CLIENT_QUIT. This is the same behaviour as if an error occured. Signed-off-by: Lukas Straub Acked-by: Stefan Hajnoczi Reviewed-by: Daniel P. Berrangé --- block/nbd.c | 129 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 80 insertions(+), 49 deletions(-) -- 2.20.1 diff --git a/block/nbd.c b/block/nbd.c index 7bb881fef4..8632cf5340 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -35,6 +35,7 @@ #include "qemu/option.h" #include "qemu/cutils.h" #include "qemu/main-loop.h" +#include "qemu/atomic.h" #include "qapi/qapi-visit-sockets.h" #include "qapi/qmp/qstring.h" @@ -43,6 +44,8 @@ #include "block/nbd.h" #include "block/block_int.h" +#include "qemu/yank.h" + #define EN_OPTSTR ":exportname=" #define MAX_NBD_REQUESTS 16 @@ -84,6 +87,8 @@ typedef struct BDRVNBDState { NBDReply reply; BlockDriverState *bs; + char *yank_name; + /* Connection parameters */ uint32_t reconnect_delay; SocketAddress *saddr; @@ -93,10 +98,10 @@ typedef struct BDRVNBDState { char *x_dirty_bitmap; } BDRVNBDState; -static QIOChannelSocket *nbd_establish_connection(SocketAddress *saddr, - Error **errp); -static int nbd_client_handshake(BlockDriverState *bs, QIOChannelSocket *sioc, - Error **errp); +static int nbd_establish_connection(BlockDriverState *bs, SocketAddress *saddr, + Error **errp); +static int nbd_client_handshake(BlockDriverState *bs, Error **errp); +static void nbd_yank(void *opaque); static void nbd_clear_bdrvstate(BDRVNBDState *s) { @@ -109,17 +114,19 @@ static void nbd_clear_bdrvstate(BDRVNBDState *s) s->tlscredsid = NULL; g_free(s->x_dirty_bitmap); s->x_dirty_bitmap = NULL; + g_free(s->yank_name); + s->yank_name = NULL; } static void nbd_channel_error(BDRVNBDState *s, int ret) { if (ret == -EIO) { - if (s->state == NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTED) { s->state = s->reconnect_delay ? NBD_CLIENT_CONNECTING_WAIT : NBD_CLIENT_CONNECTING_NOWAIT; } } else { - if (s->state == NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTED) { qio_channel_shutdown(s->ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); } s->state = NBD_CLIENT_QUIT; @@ -170,7 +177,7 @@ static void nbd_client_attach_aio_context(BlockDriverState *bs, * s->connection_co is either yielded from nbd_receive_reply or from * nbd_co_reconnect_loop() */ - if (s->state == NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTED) { qio_channel_attach_aio_context(QIO_CHANNEL(s->ioc), new_context); } @@ -237,20 +244,20 @@ static void nbd_teardown_connection(BlockDriverState *bs) static bool nbd_client_connecting(BDRVNBDState *s) { - return s->state == NBD_CLIENT_CONNECTING_WAIT || - s->state == NBD_CLIENT_CONNECTING_NOWAIT; + NBDClientState state = atomic_load_acquire(&s->state); + return state == NBD_CLIENT_CONNECTING_WAIT || + state == NBD_CLIENT_CONNECTING_NOWAIT; } static bool nbd_client_connecting_wait(BDRVNBDState *s) { - return s->state == NBD_CLIENT_CONNECTING_WAIT; + return atomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTING_WAIT; } static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) { int ret; Error *local_err = NULL; - QIOChannelSocket *sioc; if (!nbd_client_connecting(s)) { return; @@ -283,21 +290,21 @@ static coroutine_fn void nbd_reconnect_attempt(BDRVNBDState *s) /* Finalize previous connection if any */ if (s->ioc) { nbd_client_detach_aio_context(s->bs); + yank_unregister_function(s->yank_name, nbd_yank, s->bs); object_unref(OBJECT(s->sioc)); s->sioc = NULL; object_unref(OBJECT(s->ioc)); s->ioc = NULL; } - sioc = nbd_establish_connection(s->saddr, &local_err); - if (!sioc) { + if (nbd_establish_connection(s->bs, s->saddr, &local_err) < 0) { ret = -ECONNREFUSED; goto out; } bdrv_dec_in_flight(s->bs); - ret = nbd_client_handshake(s->bs, sioc, &local_err); + ret = nbd_client_handshake(s->bs, &local_err); if (s->drained) { s->wait_drained_end = true; @@ -334,7 +341,7 @@ static coroutine_fn void nbd_co_reconnect_loop(BDRVNBDState *s) nbd_reconnect_attempt(s); while (nbd_client_connecting(s)) { - if (s->state == NBD_CLIENT_CONNECTING_WAIT && + if (atomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTING_WAIT && qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_time_ns > delay_ns) { s->state = NBD_CLIENT_CONNECTING_NOWAIT; @@ -371,7 +378,7 @@ static coroutine_fn void nbd_connection_entry(void *opaque) int ret = 0; Error *local_err = NULL; - while (s->state != NBD_CLIENT_QUIT) { + while (atomic_load_acquire(&s->state) != NBD_CLIENT_QUIT) { /* * The NBD client can only really be considered idle when it has * yielded from qio_channel_readv_all_eof(), waiting for data. This is @@ -386,7 +393,7 @@ static coroutine_fn void nbd_connection_entry(void *opaque) nbd_co_reconnect_loop(s); } - if (s->state != NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) != NBD_CLIENT_CONNECTED) { continue; } @@ -441,6 +448,7 @@ static coroutine_fn void nbd_connection_entry(void *opaque) s->connection_co = NULL; if (s->ioc) { nbd_client_detach_aio_context(s->bs); + yank_unregister_function(s->yank_name, nbd_yank, s->bs); object_unref(OBJECT(s->sioc)); s->sioc = NULL; object_unref(OBJECT(s->ioc)); @@ -465,7 +473,7 @@ static int nbd_co_send_request(BlockDriverState *bs, qemu_co_queue_wait(&s->free_sema, &s->send_mutex); } - if (s->state != NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) != NBD_CLIENT_CONNECTED) { rc = -EIO; goto err; } @@ -492,7 +500,7 @@ static int nbd_co_send_request(BlockDriverState *bs, if (qiov) { qio_channel_set_cork(s->ioc, true); rc = nbd_send_request(s->ioc, request); - if (rc >= 0 && s->state == NBD_CLIENT_CONNECTED) { + if (rc >= 0 && atomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTED) { if (qio_channel_writev_all(s->ioc, qiov->iov, qiov->niov, NULL) < 0) { rc = -EIO; @@ -807,7 +815,7 @@ static coroutine_fn int nbd_co_do_receive_one_chunk( s->requests[i].receiving = true; qemu_coroutine_yield(); s->requests[i].receiving = false; - if (s->state != NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) != NBD_CLIENT_CONNECTED) { error_setg(errp, "Connection closed"); return -EIO; } @@ -966,7 +974,7 @@ static bool nbd_reply_chunk_iter_receive(BDRVNBDState *s, NBDReply local_reply; NBDStructuredReplyChunk *chunk; Error *local_err = NULL; - if (s->state != NBD_CLIENT_CONNECTED) { + if (atomic_load_acquire(&s->state) != NBD_CLIENT_CONNECTED) { error_setg(&local_err, "Connection closed"); nbd_iter_channel_error(iter, -EIO, &local_err); goto break_loop; @@ -991,7 +999,8 @@ static bool nbd_reply_chunk_iter_receive(BDRVNBDState *s, } /* Do not execute the body of NBD_FOREACH_REPLY_CHUNK for simple reply. */ - if (nbd_reply_is_simple(reply) || s->state != NBD_CLIENT_CONNECTED) { + if (nbd_reply_is_simple(reply) || + atomic_load_acquire(&s->state) != NBD_CLIENT_CONNECTED) { goto break_loop; } @@ -1423,6 +1432,15 @@ static int nbd_client_reopen_prepare(BDRVReopenState *state, return 0; } +static void nbd_yank(void *opaque) +{ + BlockDriverState *bs = opaque; + BDRVNBDState *s = (BDRVNBDState *)bs->opaque; + + atomic_store_release(&s->state, NBD_CLIENT_QUIT); + qio_channel_shutdown(QIO_CHANNEL(s->sioc), QIO_CHANNEL_SHUTDOWN_BOTH, NULL); +} + static void nbd_client_close(BlockDriverState *bs) { BDRVNBDState *s = (BDRVNBDState *)bs->opaque; @@ -1435,52 +1453,53 @@ static void nbd_client_close(BlockDriverState *bs) nbd_teardown_connection(bs); } -static QIOChannelSocket *nbd_establish_connection(SocketAddress *saddr, - Error **errp) +static int nbd_establish_connection(BlockDriverState *bs, + SocketAddress *saddr, + Error **errp) { ERRP_GUARD(); - QIOChannelSocket *sioc; + BDRVNBDState *s = (BDRVNBDState *)bs->opaque; - sioc = qio_channel_socket_new(); - qio_channel_set_name(QIO_CHANNEL(sioc), "nbd-client"); + s->sioc = qio_channel_socket_new(); + qio_channel_set_name(QIO_CHANNEL(s->sioc), "nbd-client"); + yank_register_function(s->yank_name, nbd_yank, bs); - qio_channel_socket_connect_sync(sioc, saddr, errp); + qio_channel_socket_connect_sync(s->sioc, saddr, errp); if (*errp) { - object_unref(OBJECT(sioc)); - return NULL; + yank_unregister_function(s->yank_name, nbd_yank, bs); + object_unref(OBJECT(s->sioc)); + s->sioc = NULL; + return -1; } - qio_channel_set_delay(QIO_CHANNEL(sioc), false); + qio_channel_set_delay(QIO_CHANNEL(s->sioc), false); - return sioc; + return 0; } -/* nbd_client_handshake takes ownership on sioc. On failure it is unref'ed. */ -static int nbd_client_handshake(BlockDriverState *bs, QIOChannelSocket *sioc, - Error **errp) +/* nbd_client_handshake takes ownership on s->sioc. On failure it's unref'ed. */ +static int nbd_client_handshake(BlockDriverState *bs, Error **errp) { BDRVNBDState *s = (BDRVNBDState *)bs->opaque; AioContext *aio_context = bdrv_get_aio_context(bs); int ret; trace_nbd_client_handshake(s->export); - - s->sioc = sioc; - - qio_channel_set_blocking(QIO_CHANNEL(sioc), false, NULL); - qio_channel_attach_aio_context(QIO_CHANNEL(sioc), aio_context); + qio_channel_set_blocking(QIO_CHANNEL(s->sioc), false, NULL); + qio_channel_attach_aio_context(QIO_CHANNEL(s->sioc), aio_context); s->info.request_sizes = true; s->info.structured_reply = true; s->info.base_allocation = true; s->info.x_dirty_bitmap = g_strdup(s->x_dirty_bitmap); s->info.name = g_strdup(s->export ?: ""); - ret = nbd_receive_negotiate(aio_context, QIO_CHANNEL(sioc), s->tlscreds, + ret = nbd_receive_negotiate(aio_context, QIO_CHANNEL(s->sioc), s->tlscreds, s->hostname, &s->ioc, &s->info, errp); g_free(s->info.x_dirty_bitmap); g_free(s->info.name); if (ret < 0) { - object_unref(OBJECT(sioc)); + yank_unregister_function(s->yank_name, nbd_yank, bs); + object_unref(OBJECT(s->sioc)); s->sioc = NULL; return ret; } @@ -1508,7 +1527,7 @@ static int nbd_client_handshake(BlockDriverState *bs, QIOChannelSocket *sioc, } if (!s->ioc) { - s->ioc = QIO_CHANNEL(sioc); + s->ioc = QIO_CHANNEL(s->sioc); object_ref(OBJECT(s->ioc)); } @@ -1524,9 +1543,10 @@ static int nbd_client_handshake(BlockDriverState *bs, QIOChannelSocket *sioc, { NBDRequest request = { .type = NBD_CMD_DISC }; - nbd_send_request(s->ioc ?: QIO_CHANNEL(sioc), &request); + nbd_send_request(s->ioc ?: QIO_CHANNEL(s->sioc), &request); - object_unref(OBJECT(sioc)); + yank_unregister_function(s->yank_name, nbd_yank, bs); + object_unref(OBJECT(s->sioc)); s->sioc = NULL; return ret; @@ -1918,7 +1938,6 @@ static int nbd_open(BlockDriverState *bs, QDict *options, int flags, { int ret; BDRVNBDState *s = (BDRVNBDState *)bs->opaque; - QIOChannelSocket *sioc; ret = nbd_process_options(bs, options, errp); if (ret < 0) { @@ -1929,17 +1948,28 @@ static int nbd_open(BlockDriverState *bs, QDict *options, int flags, qemu_co_mutex_init(&s->send_mutex); qemu_co_queue_init(&s->free_sema); + s->yank_name = g_strconcat("blockdev:", bs->node_name, NULL); + yank_register_instance(s->yank_name, errp); + if (*errp) { + g_free(s->yank_name); + s->yank_name = NULL; + return -EEXIST; + } + /* * establish TCP connection, return error if it fails * TODO: Configurable retry-until-timeout behaviour. */ - sioc = nbd_establish_connection(s->saddr, errp); - if (!sioc) { + if (nbd_establish_connection(bs, s->saddr, errp) < 0) { + yank_unregister_instance(s->yank_name); + g_free(s->yank_name); + s->yank_name = NULL; return -ECONNREFUSED; } - ret = nbd_client_handshake(bs, sioc, errp); + ret = nbd_client_handshake(bs, errp); if (ret < 0) { + yank_unregister_instance(s->yank_name); nbd_clear_bdrvstate(s); return ret; } @@ -1997,6 +2027,7 @@ static void nbd_close(BlockDriverState *bs) BDRVNBDState *s = bs->opaque; nbd_client_close(bs); + yank_unregister_instance(s->yank_name); nbd_clear_bdrvstate(s); }