From patchwork Thu Dec 24 11:53:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11989523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9B90C433E0 for ; Thu, 24 Dec 2020 11:54:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6F50222273 for ; Thu, 24 Dec 2020 11:54:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F50222273 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.58704.103445 (Exim 4.92) (envelope-from ) id 1ksPC1-0002jH-6p; Thu, 24 Dec 2020 11:54:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 58704.103445; Thu, 24 Dec 2020 11:54:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPC1-0002j9-2n; Thu, 24 Dec 2020 11:54:05 +0000 Received: by outflank-mailman (input) for mailman id 58704; Thu, 24 Dec 2020 11:54:03 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBz-0002WC-9I for xen-devel@lists.xenproject.org; Thu, 24 Dec 2020 11:54:03 +0000 Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 95dbbb1f-fdab-4be3-9ab5-a456c7902189; Thu, 24 Dec 2020 11:53:39 +0000 (UTC) Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ksPBN-0001IK-T1; Thu, 24 Dec 2020 11:53:26 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1ksPBN-00Er3j-FX; Thu, 24 Dec 2020 11:53:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list X-Inumbo-ID: 95dbbb1f-fdab-4be3-9ab5-a456c7902189 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=NXk0UWcV3/VkmO3V390JOuM5MMOhtqzPdQBiicIGjhg=; b=ZCj4mb12ayQWaU4XH37Q5Jcpbr 0BKSUCmI8vbB1mT7TBXQ6s1xJAlfks1KlKrZT/vvOfFJJnZqo16qp7FvtSaaz96opYQzuMbdGAIfv ENyzO0U4QAxzdPbZv/kjVcmb1oQolb76DO6tUXVDfAlHvny1vvpf0hMyymJYZXiexxNZHNqXts7mb 7jrRwsoc/Ki7FYjfZAXb1hIHfkb3NcGldJFo1rwcZBi7kmQ6i3NG8nwJp9RUt0j9pHynjrQF8AOm5 sSetl+1RANLkfykAtab1EGgb5Ns9qx6Fz9p5BFYfK8PevRI9bXIFiju9amlAGFAaNTJ8LtHaRV/kG WbKuXUJw==; From: David Woodhouse To: x86@kernel.org Cc: Stefano Stabellini , Boris Ostrovsky , Juergen Gross , Paul Durrant , jgrall@amazon.com, karahmed@amazon.de, xen-devel Subject: [PATCH 1/5] xen: Fix event channel callback via INTX/GSI Date: Thu, 24 Dec 2020 11:53:19 +0000 Message-Id: <20201224115323.3540130-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201224115323.3540130-1-dwmw2@infradead.org> References: <20201224115323.3540130-1-dwmw2@infradead.org> MIME-Version: 1.0 Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse For a while, event channel notification via the PCI platform device has been broken, because we attempt to communicate with xenstore before we even have notifications working, with the xs_reset_watches() call in xs_init(). We tend to get away with this on Xen versions below 4.0 because we avoid calling xs_reset_watches() anyway, because xenstore might not cope with reading a non-existent key. And newer Xen *does* have the vector callback support, so we rarely fall back to INTX/GSI delivery. To fix it, clean up a bit of the mess of xs_init() and xenbus_probe() startup. Call xs_init() directly from xenbus_init() only in the !XS_HVM case, deferring it to be called from xenbus_probe() in the XS_HVM case instead. Then fix up the invocation of xenbus_probe() to happen either from its device_initcall if the callback is available early enough, or when the callback is finally set up. This means that the hack of calling xenbus_probe() from a workqueue after the first interrupt, or directly from the PCI platform device setup, is no longer needed. Signed-off-by: David Woodhouse Reviewed-by: Boris Ostrovsky --- arch/arm/xen/enlighten.c | 2 +- drivers/xen/events/events_base.c | 10 ----- drivers/xen/platform-pci.c | 1 - drivers/xen/xenbus/xenbus.h | 1 + drivers/xen/xenbus/xenbus_comms.c | 8 ---- drivers/xen/xenbus/xenbus_probe.c | 68 ++++++++++++++++++++++++------- include/xen/xenbus.h | 2 +- 7 files changed, 57 insertions(+), 35 deletions(-) diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c index 60e901cd0de6..5a957a9a0984 100644 --- a/arch/arm/xen/enlighten.c +++ b/arch/arm/xen/enlighten.c @@ -371,7 +371,7 @@ static int __init xen_guest_init(void) } gnttab_init(); if (!xen_initial_domain()) - xenbus_probe(NULL); + xenbus_probe(); /* * Making sure board specific code will not set up ops for diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 6038c4c35db5..bbebe248b726 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -2010,16 +2010,6 @@ static struct irq_chip xen_percpu_chip __read_mostly = { .irq_ack = ack_dynirq, }; -int xen_set_callback_via(uint64_t via) -{ - struct xen_hvm_param a; - a.domid = DOMID_SELF; - a.index = HVM_PARAM_CALLBACK_IRQ; - a.value = via; - return HYPERVISOR_hvm_op(HVMOP_set_param, &a); -} -EXPORT_SYMBOL_GPL(xen_set_callback_via); - #ifdef CONFIG_XEN_PVHVM /* Vector callbacks are better than PCI interrupts to receive event * channel notifications because we can receive vector callbacks on any diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index dd911e1ff782..9db557b76511 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -149,7 +149,6 @@ static int platform_pci_probe(struct pci_dev *pdev, ret = gnttab_init(); if (ret) goto grant_out; - xenbus_probe(NULL); return 0; grant_out: gnttab_free_auto_xlat_frames(); diff --git a/drivers/xen/xenbus/xenbus.h b/drivers/xen/xenbus/xenbus.h index 5f5b8a7d5b80..05bbda51103f 100644 --- a/drivers/xen/xenbus/xenbus.h +++ b/drivers/xen/xenbus/xenbus.h @@ -113,6 +113,7 @@ int xenbus_probe_node(struct xen_bus_type *bus, const char *type, const char *nodename); int xenbus_probe_devices(struct xen_bus_type *bus); +void xenbus_probe(void); void xenbus_dev_changed(const char *node, struct xen_bus_type *bus); diff --git a/drivers/xen/xenbus/xenbus_comms.c b/drivers/xen/xenbus/xenbus_comms.c index eb5151fc8efa..e5fda0256feb 100644 --- a/drivers/xen/xenbus/xenbus_comms.c +++ b/drivers/xen/xenbus/xenbus_comms.c @@ -57,16 +57,8 @@ DEFINE_MUTEX(xs_response_mutex); static int xenbus_irq; static struct task_struct *xenbus_task; -static DECLARE_WORK(probe_work, xenbus_probe); - - static irqreturn_t wake_waiting(int irq, void *unused) { - if (unlikely(xenstored_ready == 0)) { - xenstored_ready = 1; - schedule_work(&probe_work); - } - wake_up(&xb_waitq); return IRQ_HANDLED; } diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 38725d97d909..876f381b100a 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -682,29 +682,63 @@ void unregister_xenstore_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(unregister_xenstore_notifier); -void xenbus_probe(struct work_struct *unused) +void xenbus_probe(void) { xenstored_ready = 1; + /* + * In the HVM case, xenbus_init() deferred its call to + * xs_init() in case callbacks were not operational yet. + * So do it now. + */ + if (xen_store_domain_type == XS_HVM) + xs_init(); + /* Notify others that xenstore is up */ blocking_notifier_call_chain(&xenstore_chain, 0, NULL); } -EXPORT_SYMBOL_GPL(xenbus_probe); static int __init xenbus_probe_initcall(void) { - if (!xen_domain()) - return -ENODEV; - - if (xen_initial_domain() || xen_hvm_domain()) - return 0; + /* + * Probe XenBus here in the XS_PV case, and also XS_HVM unless we + * need to wait for the platform PCI device to come up, which is + * the (XEN_PVPVM && !xen_have_vector_callback) case. + */ + if (xen_store_domain_type == XS_PV || + (xen_store_domain_type == XS_HVM && + (!IS_ENABLED(CONFIG_XEN_PVHVM) || xen_have_vector_callback))) + xenbus_probe(); - xenbus_probe(NULL); return 0; } - device_initcall(xenbus_probe_initcall); +int xen_set_callback_via(uint64_t via) +{ + struct xen_hvm_param a; + int ret; + + a.domid = DOMID_SELF; + a.index = HVM_PARAM_CALLBACK_IRQ; + a.value = via; + + ret = HYPERVISOR_hvm_op(HVMOP_set_param, &a); + if (ret) + return ret; + + /* + * If xenbus_probe_initcall() deferred the xenbus_probe() + * due to the callback not functioning yet, we can do it now. + */ + if (!xenstored_ready && xen_store_domain_type == XS_HVM && + IS_ENABLED(CONFIG_XEN_PVHVM) && !xen_have_vector_callback) + xenbus_probe(); + + return ret; +} +EXPORT_SYMBOL_GPL(xen_set_callback_via); + /* Set up event channel for xenstored which is run as a local process * (this is normally used only in dom0) */ @@ -817,11 +851,17 @@ static int __init xenbus_init(void) break; } - /* Initialize the interface to xenstore. */ - err = xs_init(); - if (err) { - pr_warn("Error initializing xenstore comms: %i\n", err); - goto out_error; + /* + * HVM domains may not have a functional callback yet. In that + * case let xs_init() be called from xenbus_probe(), which will + * get invoked at an appropriate time. + */ + if (xen_store_domain_type != XS_HVM) { + err = xs_init(); + if (err) { + pr_warn("Error initializing xenstore comms: %i\n", err); + goto out_error; + } } if ((xen_store_domain_type != XS_LOCAL) && diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 5a8315e6d8a6..61202c83d560 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -183,7 +183,7 @@ void xs_suspend_cancel(void); struct work_struct; -void xenbus_probe(struct work_struct *); +void xenbus_probe(void); #define XENBUS_IS_ERR_READ(str) ({ \ if (!IS_ERR(str) && strlen(str) == 0) { \ From patchwork Thu Dec 24 11:53:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11989517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA875C433E0 for ; Thu, 24 Dec 2020 11:54:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5E2922283 for ; Thu, 24 Dec 2020 11:54:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5E2922283 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.58702.103421 (Exim 4.92) (envelope-from ) id 1ksPBq-0002aY-KN; Thu, 24 Dec 2020 11:53:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 58702.103421; Thu, 24 Dec 2020 11:53:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBq-0002aR-HD; Thu, 24 Dec 2020 11:53:54 +0000 Received: by outflank-mailman (input) for mailman id 58702; Thu, 24 Dec 2020 11:53:53 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBp-0002WC-8w for xen-devel@lists.xenproject.org; Thu, 24 Dec 2020 11:53:53 +0000 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id b450edf5-14d2-46e0-beb6-64d5732f1582; Thu, 24 Dec 2020 11:53:38 +0000 (UTC) Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ksPBO-0002az-Hj; Thu, 24 Dec 2020 11:53:26 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1ksPBN-00Er3m-GF; Thu, 24 Dec 2020 11:53:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list X-Inumbo-ID: b450edf5-14d2-46e0-beb6-64d5732f1582 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=zk4UYemafseG/MJkQtPNYJOGxL13WedQ005wS02nvsk=; b=YOiTNNDVcCi71gJnkmi5I3nAWT /Q+4A66AfywOS5VtbWG0k9kR9csEvksh+dtmcviONDQRV0FOeJYD0jhbhteng8kgZBlfOz0TeQ55F BaqqZZQltTwzcRvUPQx0i5SQLoxdMUDqQ9RnrSOsWQisrlZv9688B7ZvgZXKzf2Qt408OkxqGbFkU T2u91UjRw3E7CxBwaYQVqKoBlEFVFUlAVnvunwhcfs011ctXe/fiRuOZNy1av1UuV9dwQZFscuAOi srq+n/nKV5lmXdVWisRMJS/KxR50shTeTftbnTFSu9YUf08ajZl0Og9nI7nsHQzk0ZAp/WX3YjPip PqCvHiZA==; From: David Woodhouse To: x86@kernel.org Cc: Stefano Stabellini , Boris Ostrovsky , Juergen Gross , Paul Durrant , jgrall@amazon.com, karahmed@amazon.de, xen-devel Subject: [PATCH 2/5] xen: Set platform PCI device INTX affinity to CPU0 Date: Thu, 24 Dec 2020 11:53:20 +0000 Message-Id: <20201224115323.3540130-3-dwmw2@infradead.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201224115323.3540130-1-dwmw2@infradead.org> References: <20201224115323.3540130-1-dwmw2@infradead.org> MIME-Version: 1.0 Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse With INTX or GSI delivery, Xen uses the event channel structures of CPU0. If the interrupt gets handled by Linux on a different CPU, then no events are seen as pending. Rather than introducing locking to allow other CPUs to process CPU0's events, just ensure that the PCI interrupts happens only on CPU0. Signed-off-by: David Woodhouse Reviewed-by: Boris Ostrovsky --- drivers/xen/platform-pci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c index 9db557b76511..18f0ed8b1f93 100644 --- a/drivers/xen/platform-pci.c +++ b/drivers/xen/platform-pci.c @@ -132,6 +132,13 @@ static int platform_pci_probe(struct pci_dev *pdev, dev_warn(&pdev->dev, "request_irq failed err=%d\n", ret); goto out; } + /* + * It doesn't strictly *have* to run on CPU0 but it sure + * as hell better process the event channel ports delivered + * to CPU0. + */ + irq_set_affinity(pdev->irq, cpumask_of(0)); + callback_via = get_callback_via(pdev); ret = xen_set_callback_via(callback_via); if (ret) { From patchwork Thu Dec 24 11:53:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11989521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5318EC43381 for ; Thu, 24 Dec 2020 11:54:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A14822283 for ; Thu, 24 Dec 2020 11:54:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A14822283 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.58700.103397 (Exim 4.92) (envelope-from ) id 1ksPBh-0002WO-4c; Thu, 24 Dec 2020 11:53:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 58700.103397; Thu, 24 Dec 2020 11:53:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBh-0002WH-1G; Thu, 24 Dec 2020 11:53:45 +0000 Received: by outflank-mailman (input) for mailman id 58700; Thu, 24 Dec 2020 11:53:44 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBf-0002WC-Du for xen-devel@lists.xenproject.org; Thu, 24 Dec 2020 11:53:44 +0000 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 186c498a-2b40-4594-ae2b-41b1da3537ec; Thu, 24 Dec 2020 11:53:39 +0000 (UTC) Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ksPBO-0002b0-ID; Thu, 24 Dec 2020 11:53:26 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1ksPBN-00Er3p-Gl; Thu, 24 Dec 2020 11:53:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list X-Inumbo-ID: 186c498a-2b40-4594-ae2b-41b1da3537ec DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=O3AtkiW5GTt6yxuiaTLnSJ1UIhL9jkFSRWxqdPyBQOk=; b=sWR9MZNWda/BBmFcp64YqtZ00E 4V4+DwxHPnL8bLzOe6a3GelxX58UVJamvHqE4ViZl592hb49tYRUtDHGLmC0PWs4U8KE1lF5tcD5c GaIDIwo+VVocX3BnTEyDrDW/VTvbKOTNpdixlLNYcphIxUag+PhA7rk9gR5ZAsSSHi7/WorD358J5 o3Thps3ZSEYhqopE+5aikVYvbBKDnWz0Q0amjd8fKrveC2GXrKIzPEsDsVfDvxpWQ4soIAdp2M2oP GXKiQ7WScUa7BXhYpA5ve8mXjGmNeNdN+LL3DFvuKPeVAyDGUywsFpW520hofBBpmIjpIM36JyQdX wVXk806Q==; From: David Woodhouse To: x86@kernel.org Cc: Stefano Stabellini , Boris Ostrovsky , Juergen Gross , Paul Durrant , jgrall@amazon.com, karahmed@amazon.de, xen-devel Subject: [PATCH 3/5] x86/xen: Add a no_vector_callback option to test PCI INTX delivery Date: Thu, 24 Dec 2020 11:53:21 +0000 Message-Id: <20201224115323.3540130-4-dwmw2@infradead.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201224115323.3540130-1-dwmw2@infradead.org> References: <20201224115323.3540130-1-dwmw2@infradead.org> MIME-Version: 1.0 Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse It's useful to be able to test non-vector event channel delivery, to make sure Linux will work properly on older Xen which doesn't have it. It's also useful for those working on Xen and Xen-compatible hypervisors, because there are guest kernels still in active use which use PCI INTX even when vector delivery is available. Signed-off-by: David Woodhouse --- arch/x86/xen/enlighten_hvm.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index 9e87ab010c82..a1c07e0c888e 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -188,6 +188,8 @@ static int xen_cpu_dead_hvm(unsigned int cpu) return 0; } +static bool no_vector_callback __initdata; + static void __init xen_hvm_guest_init(void) { if (xen_pv_domain()) @@ -207,7 +209,7 @@ static void __init xen_hvm_guest_init(void) xen_panic_handler_init(); - if (xen_feature(XENFEAT_hvm_callback_vector)) + if (!no_vector_callback && xen_feature(XENFEAT_hvm_callback_vector)) xen_have_vector_callback = 1; xen_hvm_smp_init(); @@ -233,6 +235,13 @@ static __init int xen_parse_nopv(char *arg) } early_param("xen_nopv", xen_parse_nopv); +static __init int xen_parse_no_vector_callback(char *arg) +{ + no_vector_callback = true; + return 0; +} +early_param("no_vector_callback", xen_parse_no_vector_callback); + bool __init xen_hvm_need_lapic(void) { if (xen_pv_domain()) From patchwork Thu Dec 24 11:53:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11989515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C27AC433DB for ; Thu, 24 Dec 2020 11:54:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BDE022273 for ; Thu, 24 Dec 2020 11:54:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BDE022273 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.58703.103433 (Exim 4.92) (envelope-from ) id 1ksPBv-0002eg-UZ; Thu, 24 Dec 2020 11:53:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 58703.103433; Thu, 24 Dec 2020 11:53:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBv-0002eX-Qg; Thu, 24 Dec 2020 11:53:59 +0000 Received: by outflank-mailman (input) for mailman id 58703; Thu, 24 Dec 2020 11:53:58 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBu-0002WC-9F for xen-devel@lists.xenproject.org; Thu, 24 Dec 2020 11:53:58 +0000 Received: from casper.infradead.org (unknown [2001:8b0:10b:1236::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a9c3c9f7-f28f-4b01-8d0b-17da43b60027; Thu, 24 Dec 2020 11:53:39 +0000 (UTC) Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ksPBN-0001IL-UT; Thu, 24 Dec 2020 11:53:26 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1ksPBN-00Er3t-HI; Thu, 24 Dec 2020 11:53:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list X-Inumbo-ID: a9c3c9f7-f28f-4b01-8d0b-17da43b60027 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=IRb1VunvIiJxLPUFQ4utXydsXUgkW5BhNr6SXAIAjkE=; b=Z63QCpTsmbNseE3AFM3QFE8rHW 4QIp3TdV9LwRqaoJgFRzIRJ1v2wP0BNaTdOOYVAZvxn7gnbES94Ygsdy4hhWEPpkmm12F7KSgl0+e F2W6ndobGrou3JlpyCQLKsawu19yJSEJTXYLEzp5CKoJOyzdqCjnkZeNiXm+2rHhh7gDGf9rLmdjW 3SFLvjf/pfKIFmh+17m2ffIMXgZ6+cnpHnKT6TEevoUipz+MSixmf3u6HxZz16859Yba1pHa1l8ML 4kWgCP5YlertuYHd5iDaf8vQp2UmDbaMfrTjUMBQqgrmGpUC3WOwqFQsCQESlsNhwteM5PgeCJpPt BAJsowfQ==; From: David Woodhouse To: x86@kernel.org Cc: Stefano Stabellini , Boris Ostrovsky , Juergen Gross , Paul Durrant , jgrall@amazon.com, karahmed@amazon.de, xen-devel Subject: [PATCH 4/5] x86/xen: Don't register Xen IPIs when they aren't going to be used Date: Thu, 24 Dec 2020 11:53:22 +0000 Message-Id: <20201224115323.3540130-5-dwmw2@infradead.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201224115323.3540130-1-dwmw2@infradead.org> References: <20201224115323.3540130-1-dwmw2@infradead.org> MIME-Version: 1.0 Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse In the case where xen_have_vector_callback is false, we still register the IPI vectors in xen_smp_intr_init() for the secondary CPUs even though they aren't going to be used. Stop doing that. Signed-off-by: David Woodhouse --- arch/x86/xen/enlighten_hvm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index a1c07e0c888e..7a6ef517e81a 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -164,10 +164,10 @@ static int xen_cpu_up_prepare_hvm(unsigned int cpu) else per_cpu(xen_vcpu_id, cpu) = cpu; rc = xen_vcpu_setup(cpu); - if (rc) + if (rc || !xen_have_vector_callback) return rc; - if (xen_have_vector_callback && xen_feature(XENFEAT_hvm_safe_pvclock)) + if (xen_feature(XENFEAT_hvm_safe_pvclock)) xen_setup_timer(cpu); rc = xen_smp_intr_init(cpu); From patchwork Thu Dec 24 11:53:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 11989519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08B7CC433E9 for ; Thu, 24 Dec 2020 11:54:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B2A4F22273 for ; Thu, 24 Dec 2020 11:54:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B2A4F22273 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.58701.103409 (Exim 4.92) (envelope-from ) id 1ksPBl-0002Xj-CU; Thu, 24 Dec 2020 11:53:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 58701.103409; Thu, 24 Dec 2020 11:53:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBl-0002Xc-9I; Thu, 24 Dec 2020 11:53:49 +0000 Received: by outflank-mailman (input) for mailman id 58701; Thu, 24 Dec 2020 11:53:48 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ksPBk-0002WC-8u for xen-devel@lists.xenproject.org; Thu, 24 Dec 2020 11:53:48 +0000 Received: from merlin.infradead.org (unknown [2001:8b0:10b:1231::1]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 839ae6fa-46d6-4ec2-9691-18c61f485f5d; Thu, 24 Dec 2020 11:53:38 +0000 (UTC) Received: from i7.infradead.org ([2001:8b0:10b:1:21e:67ff:fecb:7a92]) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1ksPBO-0002b1-JE; Thu, 24 Dec 2020 11:53:26 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1ksPBN-00Er3w-Hp; Thu, 24 Dec 2020 11:53:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list X-Inumbo-ID: 839ae6fa-46d6-4ec2-9691-18c61f485f5d DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=529q7/1gaAJwNhvRKFVZNAtSN1/LIWxT+b3mgEbuAUg=; b=NICvtL4896QMhOF1HTfxSmrS72 p8d8PA2yocHzLDIM3daetiJEmh4/NglWvaNG3qsGW+4SHQmC4x7CZmZ6IoaALTuRJ7QntYlY52FEc H/XwQaYtBOtDdYi6anterI6y7VZ3B3ASgDnJBVoIrClNAH40YAF/sGJ1FRnUmGQozDD3l51AX9i+n /KQtzoyhrjDO8DFESvLXDx9r1ViHK8C9XaUhLshO7d7Etxv6iG65UXfxjPpIS/gumxmpZ43ttID3p VWxOJ/vZIHR+NZl5OSv+teiaf8k3DXJKDX9n+ivm5NEfLLD0cr/6114tLiqhjAAmoIkWLBMAXhdRt YAM0hMjg==; From: David Woodhouse To: x86@kernel.org Cc: Stefano Stabellini , Boris Ostrovsky , Juergen Gross , Paul Durrant , jgrall@amazon.com, karahmed@amazon.de, xen-devel Subject: [PATCH 5/5] x86/xen: Don't register PV spinlock IPI when it isn't going to be used Date: Thu, 24 Dec 2020 11:53:23 +0000 Message-Id: <20201224115323.3540130-6-dwmw2@infradead.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201224115323.3540130-1-dwmw2@infradead.org> References: <20201224115323.3540130-1-dwmw2@infradead.org> MIME-Version: 1.0 Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by merlin.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse When xen_have_vector_callback is false, we still register the PV spinlock kicker IPI on the secondary CPUs. Stop doing that. Signed-off-by: David Woodhouse Signed-off-by: David Woodhouse --- arch/x86/xen/spinlock.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c index 799f4eba0a62..b240ea483e63 100644 --- a/arch/x86/xen/spinlock.c +++ b/arch/x86/xen/spinlock.c @@ -68,7 +68,7 @@ void xen_init_lock_cpu(int cpu) int irq; char *name; - if (!xen_pvspin) + if (!xen_pvspin || !xen_have_vector_callback) return; WARN(per_cpu(lock_kicker_irq, cpu) >= 0, "spinlock on CPU%d exists on IRQ%d!\n", @@ -93,7 +93,7 @@ void xen_init_lock_cpu(int cpu) void xen_uninit_lock_cpu(int cpu) { - if (!xen_pvspin) + if (!xen_pvspin || !xen_have_vector_callback) return; unbind_from_irqhandler(per_cpu(lock_kicker_irq, cpu), NULL); @@ -115,7 +115,7 @@ PV_CALLEE_SAVE_REGS_THUNK(xen_vcpu_stolen); void __init xen_init_spinlocks(void) { /* Don't need to use pvqspinlock code if there is only 1 vCPU. */ - if (num_possible_cpus() == 1 || nopvspin) + if (num_possible_cpus() == 1 || nopvspin || !xen_have_vector_callback) xen_pvspin = false; if (!xen_pvspin) {