From patchwork Tue Dec 10 11:33:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11281925 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9CF014E3 for ; Tue, 10 Dec 2019 11:35:18 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5F7E20663 for ; Tue, 10 Dec 2019 11:35:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="nNV+M+f+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5F7E20663 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmT-00064a-1t; Tue, 10 Dec 2019 11:34:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmR-00064F-Ez for xen-devel@lists.xenproject.org; Tue, 10 Dec 2019 11:34:15 +0000 X-Inumbo-ID: fb94f6eb-1b40-11ea-8925-12813bfff9fa Received: from smtp-fw-6002.amazon.com (unknown [52.95.49.90]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id fb94f6eb-1b40-11ea-8925-12813bfff9fa; Tue, 10 Dec 2019 11:34:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1575977652; x=1607513652; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DPziW0dLjY9DvgtbJPoGFRZkja9CKYaPD9IgY+xvU3Y=; b=nNV+M+f+7P7DhBBc7DQRyuzg3Ci+nv76fyF98cwWUpd04fBIE8CPHJJL L+aUGOX0NkMNsfRlSA+ISbaAfwvMJb2hs4giFwszoiERyTWOb1CRuBxAh XZecleR4fQ+yQfivqgTp+L5iZHRsfbZxj29whkSK1qMhRA5OYERkLRqQ/ 8=; IronPort-SDR: d8F7m+ntFLm4JeurEShiAWdrMfjuZ+jeZaDYybmliHDcYNkXbr19qt8xlZkstIdGYOkZnI90U0 r3YDWGCO4HUQ== X-IronPort-AV: E=Sophos;i="5.69,299,1571702400"; d="scan'208";a="6964413" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2c-6f38efd9.us-west-2.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 10 Dec 2019 11:34:11 +0000 Received: from EX13MTAUEA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2c-6f38efd9.us-west-2.amazon.com (Postfix) with ESMTPS id 3CBD1A1C16; Tue, 10 Dec 2019 11:34:09 +0000 (UTC) Received: from EX13D32EUB001.ant.amazon.com (10.43.166.125) by EX13MTAUEA001.ant.amazon.com (10.43.61.243) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:08 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D32EUB001.ant.amazon.com (10.43.166.125) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:07 +0000 Received: from u2f063a87eabd5f.cbg10.amazon.com (10.125.106.135) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 10 Dec 2019 11:34:06 +0000 From: Paul Durrant To: , Date: Tue, 10 Dec 2019 11:33:44 +0000 Message-ID: <20191210113347.3404-2-pdurrant@amazon.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191210113347.3404-1-pdurrant@amazon.com> References: <20191210113347.3404-1-pdurrant@amazon.com> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v2 1/4] xenbus: move xenbus_dev_shutdown() into frontend code... X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Paul Durrant , Stefano Stabellini , Boris Ostrovsky Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" ...and make it static xenbus_dev_shutdown() is seemingly intended to cause clean shutdown of PV frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to xenbus_frontend_closed(). This patch removes the shutdown() method from backends and moves xenbus_dev_shutdown() from xenbus_probe.c into xenbus_probe_frontend.c, renaming it appropriately and making it static. NOTE: In the case where the backend is running in a driver domain, the toolstack should have already terminated any frontends that may be using it (since Xen does not support re-startable PV driver domains) so xenbus_dev_shutdown() should never be called. Signed-off-by: Paul Durrant Reviewed-by: Juergen Gross --- Cc: Boris Ostrovsky Cc: Stefano Stabellini --- drivers/xen/xenbus/xenbus.h | 2 -- drivers/xen/xenbus/xenbus_probe.c | 23 --------------------- drivers/xen/xenbus/xenbus_probe_backend.c | 1 - drivers/xen/xenbus/xenbus_probe_frontend.c | 24 +++++++++++++++++++++- 4 files changed, 23 insertions(+), 27 deletions(-) diff --git a/drivers/xen/xenbus/xenbus.h b/drivers/xen/xenbus/xenbus.h index d75a2385b37c..5f5b8a7d5b80 100644 --- a/drivers/xen/xenbus/xenbus.h +++ b/drivers/xen/xenbus/xenbus.h @@ -116,8 +116,6 @@ int xenbus_probe_devices(struct xen_bus_type *bus); void xenbus_dev_changed(const char *node, struct xen_bus_type *bus); -void xenbus_dev_shutdown(struct device *_dev); - int xenbus_dev_suspend(struct device *dev); int xenbus_dev_resume(struct device *dev); int xenbus_dev_cancel(struct device *dev); diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 4461f4583476..a10311c348b9 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -281,29 +281,6 @@ int xenbus_dev_remove(struct device *_dev) } EXPORT_SYMBOL_GPL(xenbus_dev_remove); -void xenbus_dev_shutdown(struct device *_dev) -{ - struct xenbus_device *dev = to_xenbus_device(_dev); - unsigned long timeout = 5*HZ; - - DPRINTK("%s", dev->nodename); - - get_device(&dev->dev); - if (dev->state != XenbusStateConnected) { - pr_info("%s: %s: %s != Connected, skipping\n", - __func__, dev->nodename, xenbus_strstate(dev->state)); - goto out; - } - xenbus_switch_state(dev, XenbusStateClosing); - timeout = wait_for_completion_timeout(&dev->down, timeout); - if (!timeout) - pr_info("%s: %s timeout closing device\n", - __func__, dev->nodename); - out: - put_device(&dev->dev); -} -EXPORT_SYMBOL_GPL(xenbus_dev_shutdown); - int xenbus_register_driver_common(struct xenbus_driver *drv, struct xen_bus_type *bus, struct module *owner, const char *mod_name) diff --git a/drivers/xen/xenbus/xenbus_probe_backend.c b/drivers/xen/xenbus/xenbus_probe_backend.c index b0bed4faf44c..14876faff3b0 100644 --- a/drivers/xen/xenbus/xenbus_probe_backend.c +++ b/drivers/xen/xenbus/xenbus_probe_backend.c @@ -198,7 +198,6 @@ static struct xen_bus_type xenbus_backend = { .uevent = xenbus_uevent_backend, .probe = xenbus_dev_probe, .remove = xenbus_dev_remove, - .shutdown = xenbus_dev_shutdown, .dev_groups = xenbus_dev_groups, }, }; diff --git a/drivers/xen/xenbus/xenbus_probe_frontend.c b/drivers/xen/xenbus/xenbus_probe_frontend.c index a7d90a719cea..8a1650bbe18f 100644 --- a/drivers/xen/xenbus/xenbus_probe_frontend.c +++ b/drivers/xen/xenbus/xenbus_probe_frontend.c @@ -126,6 +126,28 @@ static int xenbus_frontend_dev_probe(struct device *dev) return xenbus_dev_probe(dev); } +static void xenbus_frontend_dev_shutdown(struct device *_dev) +{ + struct xenbus_device *dev = to_xenbus_device(_dev); + unsigned long timeout = 5*HZ; + + DPRINTK("%s", dev->nodename); + + get_device(&dev->dev); + if (dev->state != XenbusStateConnected) { + pr_info("%s: %s: %s != Connected, skipping\n", + __func__, dev->nodename, xenbus_strstate(dev->state)); + goto out; + } + xenbus_switch_state(dev, XenbusStateClosing); + timeout = wait_for_completion_timeout(&dev->down, timeout); + if (!timeout) + pr_info("%s: %s timeout closing device\n", + __func__, dev->nodename); + out: + put_device(&dev->dev); +} + static const struct dev_pm_ops xenbus_pm_ops = { .suspend = xenbus_dev_suspend, .resume = xenbus_frontend_dev_resume, @@ -146,7 +168,7 @@ static struct xen_bus_type xenbus_frontend = { .uevent = xenbus_uevent_frontend, .probe = xenbus_frontend_dev_probe, .remove = xenbus_dev_remove, - .shutdown = xenbus_dev_shutdown, + .shutdown = xenbus_frontend_dev_shutdown, .dev_groups = xenbus_dev_groups, .pm = &xenbus_pm_ops, From patchwork Tue Dec 10 11:33:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11281927 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1D1414BD for ; Tue, 10 Dec 2019 11:35:31 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BE46E20663 for ; Tue, 10 Dec 2019 11:35:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="TOCRvsH6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE46E20663 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmj-00069c-HN; Tue, 10 Dec 2019 11:34:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmi-00069N-I4 for xen-devel@lists.xenproject.org; Tue, 10 Dec 2019 11:34:32 +0000 X-Inumbo-ID: 08799988-1b41-11ea-8925-12813bfff9fa Received: from smtp-fw-9102.amazon.com (unknown [207.171.184.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 08799988-1b41-11ea-8925-12813bfff9fa; Tue, 10 Dec 2019 11:34:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1575977673; x=1607513673; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nJN0wSFtdOZDYjIZyP04UHN4GCv/sSLr82yiruvNA8s=; b=TOCRvsH6ZQWZbwyQfDzP2HkHbzfmCbHAOgwgCpX5yAdoC3j1QbaqwTR2 vhOQUzc3/gP9y4SchK/+1pTGA2kiI9cEEoAwiaxdY9cv/tjEpEgb7dw0P TPmcmEBXegZQfTmxlcPZyG1ERsp/zp0CGSs1YIzm/d3BreKD8D6emsx1c 0=; IronPort-SDR: v9HAsxUvYmEiTpVmKIHL6RlIhHRXGh0KROqBNNZAN+n39/I6tRV+5fSJgtDts5HUSYayq9NVAY 7jAdk/ztpExw== X-IronPort-AV: E=Sophos;i="5.69,299,1571702400"; d="scan'208";a="12635017" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2c-4e7c8266.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 10 Dec 2019 11:34:13 +0000 Received: from EX13MTAUEA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2c-4e7c8266.us-west-2.amazon.com (Postfix) with ESMTPS id 05FA3A1F60; Tue, 10 Dec 2019 11:34:10 +0000 (UTC) Received: from EX13D32EUB002.ant.amazon.com (10.43.166.114) by EX13MTAUEA001.ant.amazon.com (10.43.61.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:10 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D32EUB002.ant.amazon.com (10.43.166.114) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:09 +0000 Received: from u2f063a87eabd5f.cbg10.amazon.com (10.125.106.135) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 10 Dec 2019 11:34:08 +0000 From: Paul Durrant To: , Date: Tue, 10 Dec 2019 11:33:45 +0000 Message-ID: <20191210113347.3404-3-pdurrant@amazon.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191210113347.3404-1-pdurrant@amazon.com> References: <20191210113347.3404-1-pdurrant@amazon.com> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v2 2/4] xenbus: limit when state is forced to closed X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Paul Durrant , Stefano Stabellini , Boris Ostrovsky Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If a driver probe() fails then leave the xenstore state alone. There is no reason to modify it as the failure may be due to transient resource allocation issues and hence a subsequent probe() may succeed. If the driver supports re-binding then only force state to closed during remove() only in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal. NOTE: Re-bind support is indicated by new boolean in struct xenbus_driver, which defaults to false. Subsequent patches will add support to some backend drivers. Signed-off-by: Paul Durrant --- Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini v2: - Introduce the 'allow_rebind' flag - Expand the commit comment --- drivers/xen/xenbus/xenbus_probe.c | 12 ++++++++++-- include/xen/xenbus.h | 1 + 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index a10311c348b9..9303ff35b2bd 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -255,7 +255,6 @@ int xenbus_dev_probe(struct device *_dev) module_put(drv->driver.owner); fail: xenbus_dev_error(dev, err, "xenbus_dev_probe on %s", dev->nodename); - xenbus_switch_state(dev, XenbusStateClosed); return err; } EXPORT_SYMBOL_GPL(xenbus_dev_probe); @@ -276,7 +275,16 @@ int xenbus_dev_remove(struct device *_dev) free_otherend_details(dev); - xenbus_switch_state(dev, XenbusStateClosed); + /* + * If the toolstack has forced the device state to closing then set + * the state to closed now to allow it to be cleaned up. + * Similarly, if the driver does not support re-bind, set the + * closed. + */ + if (!drv->allow_rebind || + xenbus_read_driver_state(dev->nodename) == XenbusStateClosing) + xenbus_switch_state(dev, XenbusStateClosed); + return 0; } EXPORT_SYMBOL_GPL(xenbus_dev_remove); diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 869c816d5f8c..24228a102141 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -93,6 +93,7 @@ struct xenbus_device_id struct xenbus_driver { const char *name; /* defaults to ids[0].devicetype */ const struct xenbus_device_id *ids; + bool allow_rebind; /* avoid setting xenstore closed during remove */ int (*probe)(struct xenbus_device *dev, const struct xenbus_device_id *id); void (*otherend_changed)(struct xenbus_device *dev, From patchwork Tue Dec 10 11:33:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11281929 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9550314BD for ; Tue, 10 Dec 2019 11:35:33 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 721BF20726 for ; Tue, 10 Dec 2019 11:35:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="dtYrwZZk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 721BF20726 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmn-0006B5-Rw; Tue, 10 Dec 2019 11:34:37 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmm-0006An-Lx for xen-devel@lists.xenproject.org; Tue, 10 Dec 2019 11:34:36 +0000 X-Inumbo-ID: 0876b100-1b41-11ea-88e7-bc764e2007e4 Received: from smtp-fw-33001.amazon.com (unknown [207.171.190.10]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 0876b100-1b41-11ea-88e7-bc764e2007e4; Tue, 10 Dec 2019 11:34:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1575977672; x=1607513672; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SKfQoX2A8rZ0EnP2XkyIWyJjMF/So/eVFwUYzJEpgTw=; b=dtYrwZZkF/0xOh/4jeA7vHw3mTCAqC9RmbNcNXqhi1bAUHt5+ylULzw+ zGA8JghD+nq+kwj7JtoZE1ofrwryd5CxM5d2eMhVq4mvJ8lwb30LwHWy7 dq+wz+S35R2kRMGAfyhsxYxQUVdpX/PzzSqXMFybDHmKEpDY5fNGhYy+L Y=; IronPort-SDR: vewG7bUB+/omNwoB9Ygkc4A38bNyqW2e/VrOCzOaaGd4TY8FDadyuilEJ/ZdAM7WPMm0aG8e+l FNzqIDT2rUYQ== X-IronPort-AV: E=Sophos;i="5.69,299,1571702400"; d="scan'208";a="14008117" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-90c42d1d.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 10 Dec 2019 11:34:13 +0000 Received: from EX13MTAUEA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2a-90c42d1d.us-west-2.amazon.com (Postfix) with ESMTPS id EC26DA11A3; Tue, 10 Dec 2019 11:34:12 +0000 (UTC) Received: from EX13D32EUC001.ant.amazon.com (10.43.164.159) by EX13MTAUEA001.ant.amazon.com (10.43.61.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:12 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D32EUC001.ant.amazon.com (10.43.164.159) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:11 +0000 Received: from u2f063a87eabd5f.cbg10.amazon.com (10.125.106.135) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 10 Dec 2019 11:34:09 +0000 From: Paul Durrant To: , Date: Tue, 10 Dec 2019 11:33:46 +0000 Message-ID: <20191210113347.3404-4-pdurrant@amazon.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191210113347.3404-1-pdurrant@amazon.com> References: <20191210113347.3404-1-pdurrant@amazon.com> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v2 3/4] xen/interface: re-define FRONT/BACK_RING_ATTACH() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Paul Durrant , Stefano Stabellini , Boris Ostrovsky Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Currently these macros are defined to re-initialize a front/back ring (respectively) to values read from the shared ring in such a way that any requests/responses that are added to the shared ring whilst the front/back is detached will be skipped over. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed. Since the macros are currently unused, take this opportunity to re-define them to re-initialize a front/back ring using specified values. This also allows FRONT/BACK_RING_INIT() to be re-defined in terms of FRONT/BACK_RING_ATTACH() using a specified value of 0. NOTE: BACK_RING_ATTACH() will be used directly in a subsequent patch. Signed-off-by: Paul Durrant --- Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini A patch to add the FRONT/BACK_RING_ATTACH() macros to the canonical ring.h in xen.git will be sent once the definitions have been agreed. v2: - change definitions to take explicit initial index values --- include/xen/interface/io/ring.h | 29 +++++++++-------------------- 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/include/xen/interface/io/ring.h b/include/xen/interface/io/ring.h index 3f40501fc60b..2af7a1cd6658 100644 --- a/include/xen/interface/io/ring.h +++ b/include/xen/interface/io/ring.h @@ -125,35 +125,24 @@ struct __name##_back_ring { \ memset((_s)->pad, 0, sizeof((_s)->pad)); \ } while(0) -#define FRONT_RING_INIT(_r, _s, __size) do { \ - (_r)->req_prod_pvt = 0; \ - (_r)->rsp_cons = 0; \ +#define FRONT_RING_ATTACH(_r, _s, _i, __size) do { \ + (_r)->req_prod_pvt = (_i); \ + (_r)->rsp_cons = (_i); \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ (_r)->sring = (_s); \ } while (0) -#define BACK_RING_INIT(_r, _s, __size) do { \ - (_r)->rsp_prod_pvt = 0; \ - (_r)->req_cons = 0; \ - (_r)->nr_ents = __RING_SIZE(_s, __size); \ - (_r)->sring = (_s); \ -} while (0) +#define FRONT_RING_INIT(_r, _s, __size) FRONT_RING_ATTACH(_r, _s, 0, __size) -/* Initialize to existing shared indexes -- for recovery */ -#define FRONT_RING_ATTACH(_r, _s, __size) do { \ - (_r)->sring = (_s); \ - (_r)->req_prod_pvt = (_s)->req_prod; \ - (_r)->rsp_cons = (_s)->rsp_prod; \ +#define BACK_RING_ATTACH(_r, _s, _i, __size) do { \ + (_r)->rsp_prod_pvt = (_i); \ + (_r)->req_cons = (_i); \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ -} while (0) - -#define BACK_RING_ATTACH(_r, _s, __size) do { \ (_r)->sring = (_s); \ - (_r)->rsp_prod_pvt = (_s)->rsp_prod; \ - (_r)->req_cons = (_s)->req_prod; \ - (_r)->nr_ents = __RING_SIZE(_s, __size); \ } while (0) +#define BACK_RING_INIT(_r, _s, __size) BACK_RING_ATTACH(_r, _s, 0, __size) + /* How big is this ring? */ #define RING_SIZE(_r) \ ((_r)->nr_ents) From patchwork Tue Dec 10 11:33:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11281931 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C850014BD for ; Tue, 10 Dec 2019 11:35:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A454920663 for ; Tue, 10 Dec 2019 11:35:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="OUkxD0Co" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A454920663 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmv-0006Eb-7B; Tue, 10 Dec 2019 11:34:45 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iedmt-0006Dj-1W for xen-devel@lists.xenproject.org; Tue, 10 Dec 2019 11:34:43 +0000 X-Inumbo-ID: 0e92781c-1b41-11ea-8925-12813bfff9fa Received: from smtp-fw-9102.amazon.com (unknown [207.171.184.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0e92781c-1b41-11ea-8925-12813bfff9fa; Tue, 10 Dec 2019 11:34:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1575977683; x=1607513683; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IKPqIf9U+emKxRr0L9uELmQeeXO75kuyRnRZTLa/FT0=; b=OUkxD0CoFOJRRuC6Y+jnKJZmeTAzHK3VCmMrxQRGOVLC1gBHKlBtTRw6 WbyL2lSI0y40XxeFpzr8aFJXbTyCEp5XiH1JSGCTpq/5IbMYzyGma7HWP RFoew50XTr05mBhO8/mJVBnUazE0gqRh1E1csrMPBzjp6MDrKeNf37PNx 0=; IronPort-SDR: pAKF5MoAr76J9iO89BZo3uc8b1Jq1EE0fQSdRsliBQFEAmOwujQiv6pHuhrAy/NjYWjuvaYCSe xQd6Q8GGdWnQ== X-IronPort-AV: E=Sophos;i="5.69,299,1571702400"; d="scan'208";a="12635074" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-90c42d1d.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 10 Dec 2019 11:34:42 +0000 Received: from EX13MTAUEA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2a-90c42d1d.us-west-2.amazon.com (Postfix) with ESMTPS id 26633A192C; Tue, 10 Dec 2019 11:34:41 +0000 (UTC) Received: from EX13D32EUC001.ant.amazon.com (10.43.164.159) by EX13MTAUEA001.ant.amazon.com (10.43.61.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:13 +0000 Received: from EX13MTAUEE001.ant.amazon.com (10.43.62.200) by EX13D32EUC001.ant.amazon.com (10.43.164.159) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Tue, 10 Dec 2019 11:34:12 +0000 Received: from u2f063a87eabd5f.cbg10.amazon.com (10.125.106.135) by mail-relay.amazon.com (10.43.62.226) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Tue, 10 Dec 2019 11:34:11 +0000 From: Paul Durrant To: , Date: Tue, 10 Dec 2019 11:33:47 +0000 Message-ID: <20191210113347.3404-5-pdurrant@amazon.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191210113347.3404-1-pdurrant@amazon.com> References: <20191210113347.3404-1-pdurrant@amazon.com> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v2 4/4] xen-blkback: support dynamic unbind/bind X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Jens Axboe , Juergen Gross , Stefano Stabellini , Konrad Rzeszutek Wilk , Paul Durrant , Boris Ostrovsky , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" By simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest. This has been tested by running: while true; do fio --name=randwrite --ioengine=libaio --iodepth=16 \ --rw=randwrite --bs=4k --direct=1 --size=1G --verify=crc32; done in a PV guest whilst running: while true; do echo vbd-$DOMID-$VBD >unbind; echo unbound; sleep 5; echo vbd-$DOMID-$VBD >bind; echo bound; sleep 3; done in dom0 from /sys/bus/xen-backend/drivers/vbd to continuously unbind and re-bind its system disk image. This is a highly useful feature for a backend module as it allows it to be unloaded and re-loaded (i.e. updated) without requiring domUs to be halted. This was also tested by running: while true; do echo vbd-$DOMID-$VBD >unbind; echo unbound; sleep 5; rmmod xen-blkback; echo unloaded; sleep 1; modprobe xen-blkback; echo bound; cd $(pwd); sleep 3; done in dom0 whilst running the same loop as above in the (single) PV guest. Some (less stressful) testing has also been done using a Windows HVM guest with the latest 9.0 PV drivers installed. Signed-off-by: Paul Durrant --- Cc: Konrad Rzeszutek Wilk Cc: "Roger Pau Monné" Cc: Jens Axboe Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini v2: - Apply a sanity check to the value of rsp_prod and fail the re-attach if it is implausible - Set allow_rebind to prevent ring from being closed on unbind - Update test workload from dd to fio (with verification) --- drivers/block/xen-blkback/xenbus.c | 59 +++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c index e8c5c54e1d26..13d09630b237 100644 --- a/drivers/block/xen-blkback/xenbus.c +++ b/drivers/block/xen-blkback/xenbus.c @@ -181,6 +181,8 @@ static int xen_blkif_map(struct xen_blkif_ring *ring, grant_ref_t *gref, { int err; struct xen_blkif *blkif = ring->blkif; + struct blkif_common_sring *sring_common; + RING_IDX rsp_prod, req_prod; /* Already connected through? */ if (ring->irq) @@ -191,46 +193,66 @@ static int xen_blkif_map(struct xen_blkif_ring *ring, grant_ref_t *gref, if (err < 0) return err; + sring_common = (struct blkif_common_sring *)ring->blk_ring; + rsp_prod = READ_ONCE(sring_common->rsp_prod); + req_prod = READ_ONCE(sring_common->req_prod); + switch (blkif->blk_protocol) { case BLKIF_PROTOCOL_NATIVE: { - struct blkif_sring *sring; - sring = (struct blkif_sring *)ring->blk_ring; - BACK_RING_INIT(&ring->blk_rings.native, sring, - XEN_PAGE_SIZE * nr_grefs); + struct blkif_sring *sring_native = + (struct blkif_sring *)ring->blk_ring; + unsigned int size = __RING_SIZE(sring_native, + XEN_PAGE_SIZE * nr_grefs); + + BACK_RING_ATTACH(&ring->blk_rings.native, sring_native, + rsp_prod, XEN_PAGE_SIZE * nr_grefs); + err = (req_prod - rsp_prod > size) ? -EIO : 0; break; } case BLKIF_PROTOCOL_X86_32: { - struct blkif_x86_32_sring *sring_x86_32; - sring_x86_32 = (struct blkif_x86_32_sring *)ring->blk_ring; - BACK_RING_INIT(&ring->blk_rings.x86_32, sring_x86_32, - XEN_PAGE_SIZE * nr_grefs); + struct blkif_x86_32_sring *sring_x86_32 = + (struct blkif_x86_32_sring *)ring->blk_ring; + unsigned int size = __RING_SIZE(sring_x86_32, + XEN_PAGE_SIZE * nr_grefs); + + BACK_RING_ATTACH(&ring->blk_rings.x86_32, sring_x86_32, + rsp_prod, XEN_PAGE_SIZE * nr_grefs); + err = (req_prod - rsp_prod > size) ? -EIO : 0; break; } case BLKIF_PROTOCOL_X86_64: { - struct blkif_x86_64_sring *sring_x86_64; - sring_x86_64 = (struct blkif_x86_64_sring *)ring->blk_ring; - BACK_RING_INIT(&ring->blk_rings.x86_64, sring_x86_64, - XEN_PAGE_SIZE * nr_grefs); + struct blkif_x86_64_sring *sring_x86_64 = + (struct blkif_x86_64_sring *)ring->blk_ring; + unsigned int size = __RING_SIZE(sring_x86_64, + XEN_PAGE_SIZE * nr_grefs); + + BACK_RING_ATTACH(&ring->blk_rings.x86_64, sring_x86_64, + rsp_prod, XEN_PAGE_SIZE * nr_grefs); + err = (req_prod - rsp_prod > size) ? -EIO : 0; break; } default: BUG(); } + if (err < 0) + goto fail; err = bind_interdomain_evtchn_to_irqhandler(blkif->domid, evtchn, xen_blkif_be_int, 0, "blkif-backend", ring); - if (err < 0) { - xenbus_unmap_ring_vfree(blkif->be->dev, ring->blk_ring); - ring->blk_rings.common.sring = NULL; - return err; - } + if (err < 0) + goto fail; ring->irq = err; return 0; + +fail: + xenbus_unmap_ring_vfree(blkif->be->dev, ring->blk_ring); + ring->blk_rings.common.sring = NULL; + return err; } static int xen_blkif_disconnect(struct xen_blkif *blkif) @@ -1121,7 +1143,8 @@ static struct xenbus_driver xen_blkbk_driver = { .ids = xen_blkbk_ids, .probe = xen_blkbk_probe, .remove = xen_blkbk_remove, - .otherend_changed = frontend_changed + .otherend_changed = frontend_changed, + .allow_rebind = true, }; int xen_blkif_xenbus_init(void)