From patchwork Thu Apr 6 21:02:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 9668425 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6EECB60364 for ; Thu, 6 Apr 2017 21:02:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 778D1285DE for ; Thu, 6 Apr 2017 21:02:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6C759285E3; Thu, 6 Apr 2017 21:02:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC24B285DE for ; Thu, 6 Apr 2017 21:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755317AbdDFVC3 (ORCPT ); Thu, 6 Apr 2017 17:02:29 -0400 Received: from mail-qk0-f194.google.com ([209.85.220.194]:35688 "EHLO mail-qk0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754281AbdDFVCU (ORCPT ); Thu, 6 Apr 2017 17:02:20 -0400 Received: by mail-qk0-f194.google.com with SMTP id k139so7622467qke.2 for ; Thu, 06 Apr 2017 14:02:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=Q0p2uzP8dLUsw+SO/Z+HyFbFZhT3afN4GlXa8zQ4jZI=; b=ErEUUhJp/sf4B0sGNiNMrNgA9hHl+eJbpTi5I4duuToI07RJQVweVfaRJ3CETqd3Y3 8kKca1Yl9t/dQugz1Akj7VMGsromhpQNN0G6NbfkAWXnBuY4c8PYkQiHA5AcLE/W9j48 xZfbXpsbXD/a11uLxQferG9GJOjITeeKFrNLvay22bKuad64bgeWZENCHCY0V1fj/ykd P7r0xxxqLtolWbt9NI3IidfSEUGrH9CvO7wgO/Ko8dfpgazY2JG/QYNm0v3K9fhVlcZp x05yEyTZTHJyxDfomDs3238gbwi8LkFLMk+/XjqZHCehiYDTUDCts+RuX4rEFRcEBS7C X7fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Q0p2uzP8dLUsw+SO/Z+HyFbFZhT3afN4GlXa8zQ4jZI=; b=sNcKGQvZm8SqmqojZAJ98FI8g9X9Dkj9pUsCm0V4HtRP4RVWTYDcF5Qs1aVM/uY7j9 SOJsG8jILU6tCPLGpgm/eyqS4f9TofaB5NlnJFQi2SgoxDL0CKQwoXYsHkeUTYxMkjcA Lj8drug4CGJAD0bUuQvfpEwuCMSHqovyr8oWHoCzxnTpNPG+WQTj2gftlgtNkJss2RF9 gKMa0YJ1Xg5Kq5L+Ey4TxKszL+9HZ2vww4OSppIsFfae23g3OxntXzl9eHHuRagTX3S1 Bg9sEQlUBFJxZNv9kTPnrabwv06sz33C6g84NhWV/Ir5lUuhMj9Mq9qRetjzCDCNHokI XjJg== X-Gm-Message-State: AN3rC/6Cv/u+p1zPOkWbBAlYZgbr+wxN/SSRQmhFz72ssrGuR39kSRorJmdKszh3eDvsaw== X-Received: by 10.55.166.213 with SMTP id p204mr2919305qke.22.1491512538080; Thu, 06 Apr 2017 14:02:18 -0700 (PDT) Received: from localhost (cpe-2606-A000-4381-1201-225-22FF-FEB3-E51A.dyn6.twc.com. [2606:a000:4381:1201:225:22ff:feb3:e51a]) by smtp.gmail.com with ESMTPSA id z196sm1715430qkb.42.2017.04.06.14.02.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Apr 2017 14:02:17 -0700 (PDT) From: Josef Bacik X-Google-Original-From: Josef Bacik To: axboe@kernel.dk, nbd-general@lists.sourceforge.net, linux-block@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 06/12] nbd: add a reconfigure netlink command Date: Thu, 6 Apr 2017 17:02:01 -0400 Message-Id: <1491512527-4286-7-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1491512527-4286-1-git-send-email-jbacik@fb.com> References: <1491512527-4286-1-git-send-email-jbacik@fb.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We want to be able to reconnect dead connections to existing block devices, so add a reconfigure netlink command. We will also allow users to change their timeout on the fly, but everything else will require a disconnect and reconnect. You won't be able to add more connections either, simply replace dead connections with new more lively connections. Signed-off-by: Josef Bacik --- drivers/block/nbd.c | 141 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/nbd-netlink.h | 1 + 2 files changed, 142 insertions(+) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index b8764e1..27958c3 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -785,6 +785,59 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg, return 0; } +static int nbd_reconnect_socket(struct nbd_device *nbd, unsigned long arg) +{ + struct nbd_config *config = nbd->config; + struct socket *sock, *old; + struct recv_thread_args *args; + int i; + int err; + + sock = sockfd_lookup(arg, &err); + if (!sock) + return err; + + args = kzalloc(sizeof(*args), GFP_KERNEL); + if (!args) { + sockfd_put(sock); + return -ENOMEM; + } + + for (i = 0; i < config->num_connections; i++) { + struct nbd_sock *nsock = config->socks[i]; + + if (!nsock->dead) + continue; + + mutex_lock(&nsock->tx_lock); + if (!nsock->dead) { + mutex_unlock(&nsock->tx_lock); + continue; + } + sk_set_memalloc(sock->sk); + atomic_inc(&config->recv_threads); + refcount_inc(&nbd->config_refs); + old = nsock->sock; + nsock->fallback_index = -1; + nsock->sock = sock; + nsock->dead = false; + INIT_WORK(&args->work, recv_work); + args->index = i; + args->nbd = nbd; + mutex_unlock(&nsock->tx_lock); + sockfd_put(old); + + /* We take the tx_mutex in an error path in the recv_work, so we + * need to queue_work outside of the tx_mutex. + */ + queue_work(recv_workqueue, &args->work); + return 0; + } + sockfd_put(sock); + kfree(args); + return -ENOSPC; +} + /* Reset all properties of an NBD device */ static void nbd_reset(struct nbd_device *nbd) { @@ -1529,6 +1582,89 @@ static int nbd_genl_disconnect(struct sk_buff *skb, struct genl_info *info) return 0; } +static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info) +{ + struct nbd_device *nbd = NULL; + struct nbd_config *config; + int index; + int ret = -EINVAL; + + if (!netlink_capable(skb, CAP_SYS_ADMIN)) + return -EPERM; + + if (!info->attrs[NBD_ATTR_INDEX]) { + printk(KERN_ERR "nbd: must specify a device to reconfigure\n"); + return -EINVAL; + } + index = nla_get_u32(info->attrs[NBD_ATTR_INDEX]); + mutex_lock(&nbd_index_mutex); + nbd = idr_find(&nbd_index_idr, index); + mutex_unlock(&nbd_index_mutex); + if (!nbd) { + printk(KERN_ERR "nbd: couldn't find a device at index %d\n", + index); + return -EINVAL; + } + + if (!refcount_inc_not_zero(&nbd->config_refs)) { + dev_err(nbd_to_dev(nbd), + "not configured, cannot reconfigure\n"); + return -EINVAL; + } + + mutex_lock(&nbd->config_lock); + config = nbd->config; + if (!test_bit(NBD_BOUND, &config->runtime_flags) || + !nbd->task_recv) { + dev_err(nbd_to_dev(nbd), + "not configured, cannot reconfigure\n"); + goto out; + } + + if (info->attrs[NBD_ATTR_TIMEOUT]) { + u64 timeout = nla_get_u64(info->attrs[NBD_ATTR_TIMEOUT]); + nbd->tag_set.timeout = timeout * HZ; + blk_queue_rq_timeout(nbd->disk->queue, timeout * HZ); + } + + if (info->attrs[NBD_ATTR_SOCKETS]) { + struct nlattr *attr; + int rem, fd; + + nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS], + rem) { + struct nlattr *socks[NBD_SOCK_MAX+1]; + + if (nla_type(attr) != NBD_SOCK_ITEM) { + printk(KERN_ERR "nbd: socks must be embedded in a SOCK_ITEM attr\n"); + ret = -EINVAL; + goto out; + } + ret = nla_parse_nested(socks, NBD_SOCK_MAX, attr, + nbd_sock_policy); + if (ret != 0) { + printk(KERN_ERR "nbd: error processing sock list\n"); + ret = -EINVAL; + goto out; + } + if (!socks[NBD_SOCK_FD]) + continue; + fd = (int)nla_get_u32(socks[NBD_SOCK_FD]); + ret = nbd_reconnect_socket(nbd, fd); + if (ret) { + if (ret == -ENOSPC) + ret = 0; + goto out; + } + dev_info(nbd_to_dev(nbd), "reconnected socket\n"); + } + } +out: + mutex_unlock(&nbd->config_lock); + nbd_config_put(nbd); + return ret; +} + static const struct genl_ops nbd_connect_genl_ops[] = { { .cmd = NBD_CMD_CONNECT, @@ -1540,6 +1676,11 @@ static const struct genl_ops nbd_connect_genl_ops[] = { .policy = nbd_attr_policy, .doit = nbd_genl_disconnect, }, + { + .cmd = NBD_CMD_RECONFIGURE, + .policy = nbd_attr_policy, + .doit = nbd_genl_reconfigure, + }, }; static struct genl_family nbd_genl_family __ro_after_init = { diff --git a/include/uapi/linux/nbd-netlink.h b/include/uapi/linux/nbd-netlink.h index fd0f4e4..f932f96 100644 --- a/include/uapi/linux/nbd-netlink.h +++ b/include/uapi/linux/nbd-netlink.h @@ -62,6 +62,7 @@ enum { NBD_CMD_UNSPEC, NBD_CMD_CONNECT, NBD_CMD_DISCONNECT, + NBD_CMD_RECONFIGURE, __NBD_CMD_MAX, }; #define NBD_CMD_MAX (__NBD_CMD_MAX - 1)