From patchwork Mon Aug 22 15:27:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0055CC3F6B0 for ; Mon, 22 Aug 2022 15:27:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391469.629318 (Exim 4.92) (envelope-from ) id 1oQ9Kv-0003TU-TN; Mon, 22 Aug 2022 15:27:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391469.629318; Mon, 22 Aug 2022 15:27:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9Kv-0003TN-Qb; Mon, 22 Aug 2022 15:27:33 +0000 Received: by outflank-mailman (input) for mailman id 391469; Mon, 22 Aug 2022 15:27:32 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9Kt-0003Dm-VS for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:32 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ee1133b3-222e-11ed-bd2e-47488cf2e6aa; Mon, 22 Aug 2022 17:27:28 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 7D1DA320099F; Mon, 22 Aug 2022 11:27:26 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 22 Aug 2022 11:27:27 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:24 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ee1133b3-222e-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182046; x=1661268446; bh=sEQP7E6J6N sdiYDQ6i1IvqeCjtQRZXD8ynCmZHPXkuc=; b=nxvfjo0HIRZhWc4LS+bDDzczYM QBG1lXo2KdJM1WUBFKAR/Qli52fbGcqujbr1WA5uEsN/QzomxkbbGbqFQbAYvJx7 BPsy+JEUTmf59hsQYCBOmhvlZkXJbAGLexv0vkYYRY2UywF3kLy38k6iOfHdnFYc Je16lTKIvAbY5TUY3eBQU7j4oVA8SLyThcqshkakBnCP+yzXDwCgwL3g8oieODfp d1nxHw3N25ZsFNvINBJ2Bqfkbl0z2RCvqibUt96MYhboMwbqu+zh67/v4DvwryXv +aZSIIZqTmqskqQedPY6jitVtkVVNvO8ad6hlBclS2hTk1kMZ07fOYGg9sgA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182046; x= 1661268446; bh=sEQP7E6J6NsdiYDQ6i1IvqeCjtQRZXD8ynCmZHPXkuc=; b=L N0CRPxm5qcATgIdFxsAS/KBvSnBjajZcqRvnqtf1zQMJFq17ihjaljKT0mjjGZm9 Jk4TMay7TfKmDtv/dmNEyW4gY6vFI51RSNEx3YHMB/D5LU0yqWk7HAwNx6BFslpY sim4VFZHH3VA7gTH4Q6vhTi1y+4psVy+m7Fe7xIp0OdqXhzWHdQSsOZDpR7fKGCc mPm74eFmTr+WUKoUuxPjK5HQs2JWgt1meGLNgA1rvQIE8cZHeAQ4DWdigQSMrW2s Ge69Qzd8d6lQjrkfCTZm5eeHV7GyMCxnlJbHdtIkxnMRmZ9CmT+K9DxIkrctSgpy KXrDKjyshMwxt1N3gT9+g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgkeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Jan Beulich , Andrew Cooper , George Dunlap , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v5 1/9] drivers/char: separate dbgp=xhci to dbc=xhci option Date: Mon, 22 Aug 2022 17:27:01 +0200 Message-Id: X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 This allows configuring EHCI and XHCI consoles separately, simultaneously. Suggested-by: Jan Beulich Signed-off-by: Marek Marczykowski-Górecki --- new in v5 --- docs/misc/xen-command-line.pandoc | 18 ++++++++++++------ xen/drivers/char/serial.c | 6 ++++++ xen/drivers/char/xhci-dbc.c | 20 ++++++++++---------- xen/include/xen/serial.h | 1 + 4 files changed, 29 insertions(+), 16 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 9a79385a3712..0d07f0c75990 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -409,7 +409,7 @@ The following are examples of correct specifications: Specify the size of the console ring buffer. ### console -> `= List of [ vga | com1[H,L] | com2[H,L] | pv | dbgp | none ]` +> `= List of [ vga | com1[H,L] | com2[H,L] | pv | dbgp | dbc | none ]` > Default: `console=com1,vga` @@ -428,7 +428,9 @@ cleared. This allows a single port to be shared by two subsystems `pv` indicates that Xen should use Xen's PV console. This option is only available when used together with `pv-in-pvh`. -`dbgp` indicates that Xen should use a USB debug port. +`dbgp` indicates that Xen should use a USB2 debug port. + +`dbc` indicates that Xen should use a USB3 debug port. `none` indicates that Xen should not use a console. This option only makes sense on its own. @@ -721,14 +723,18 @@ Available alternatives, with their meaning, are: ### dbgp > `= ehci[ | @pci:. ]` + +Specify the USB controller to use, either by instance number (when going +over the PCI busses sequentially) or by PCI device (must be on segment 0). + +### dbc > `= xhci[ | @pci:. ]` Specify the USB controller to use, either by instance number (when going over the PCI busses sequentially) or by PCI device (must be on segment 0). -Use `ehci` for EHCI debug port, use `xhci` for XHCI debug capability (output -only). XHCI driver will wait indefinitely for the debug host to connect - make -sure the cable is connected. +Output only console. XHCI driver will wait indefinitely for the debug host to +connect - make sure the cable is connected. ### debug_stack_lines > `= ` @@ -1174,7 +1180,7 @@ virtualization, to allow the L1 hypervisor to use EPT even if the L0 hypervisor does not provide `VM_ENTRY_LOAD_GUEST_PAT`. ### gdb -> `= com1[H,L] | com2[H,L] | dbgp` +> `= com1[H,L] | com2[H,L] | dbgp | dbc` > Default: `` diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c index 47899222cef8..7daaa61361bb 100644 --- a/xen/drivers/char/serial.c +++ b/xen/drivers/char/serial.c @@ -311,6 +311,12 @@ int __init serial_parse_handle(const char *conf) goto common; } + if ( !strncmp(conf, "dbc", 3) && (!conf[3] || conf[3] == ',') ) + { + handle = SERHND_DBC; + goto common; + } + if ( !strncmp(conf, "dtuart", 6) ) { handle = SERHND_DTUART; diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c index ca7d4a62139e..eb35e3a2ee4f 100644 --- a/xen/drivers/char/xhci-dbc.c +++ b/xen/drivers/char/xhci-dbc.c @@ -1058,9 +1058,9 @@ static struct xhci_dbc_ctx ctx __aligned(16); static uint8_t out_wrk_buf[DBC_WORK_RING_CAP]; static struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT]; -static char __initdata opt_dbgp[30]; +static char __initdata opt_dbc[30]; -string_param("dbgp", opt_dbgp); +string_param("dbc", opt_dbc); void __init xhci_dbc_uart_init(void) { @@ -1068,25 +1068,25 @@ void __init xhci_dbc_uart_init(void) struct dbc *dbc = &uart->dbc; const char *e; - if ( strncmp(opt_dbgp, "xhci", 4) ) + if ( strncmp(opt_dbc, "xhci", 4) ) return; memset(dbc, 0, sizeof(*dbc)); - if ( isdigit(opt_dbgp[4]) ) + if ( isdigit(opt_dbc[4]) ) { - dbc->xhc_num = simple_strtoul(opt_dbgp + 4, &e, 10); + dbc->xhc_num = simple_strtoul(opt_dbc + 4, &e, 10); } - else if ( strncmp(opt_dbgp + 4, "@pci", 4) == 0 ) + else if ( strncmp(opt_dbc + 4, "@pci", 4) == 0 ) { unsigned int bus, slot, func; - e = parse_pci(opt_dbgp + 8, NULL, &bus, &slot, &func); + e = parse_pci(opt_dbc + 8, NULL, &bus, &slot, &func); if ( !e || *e ) { printk(XENLOG_ERR - "Invalid dbgp= PCI device spec: '%s'\n", - opt_dbgp + 8); + "Invalid dbc= PCI device spec: '%s'\n", + opt_dbc + 8); return; } @@ -1102,7 +1102,7 @@ void __init xhci_dbc_uart_init(void) dbc->dbc_str = str_buf; if ( dbc_open(dbc) ) - serial_register_uart(SERHND_DBGP, &dbc_uart_driver, &dbc_uart); + serial_register_uart(SERHND_DBC, &dbc_uart_driver, &dbc_uart); } #ifdef DBC_DEBUG diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h index 4cd4ae5e6f1c..186afbed9c92 100644 --- a/xen/include/xen/serial.h +++ b/xen/include/xen/serial.h @@ -95,6 +95,7 @@ struct uart_driver { # define SERHND_COM1 (0<<0) # define SERHND_COM2 (1<<0) # define SERHND_DBGP (2<<0) +# define SERHND_DBC (3<<0) # define SERHND_DTUART (0<<0) /* Steal SERHND_COM1 value */ #define SERHND_HI (1<<2) /* Mux/demux each transferred char by MSB. */ #define SERHND_LO (1<<3) /* Ditto, except that the MSB is cleared. */ From patchwork Mon Aug 22 15:27:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 393C5C32772 for ; Mon, 22 Aug 2022 15:27:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391470.629329 (Exim 4.92) (envelope-from ) id 1oQ9Kx-0003j8-5y; Mon, 22 Aug 2022 15:27:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391470.629329; Mon, 22 Aug 2022 15:27:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9Kx-0003j1-34; Mon, 22 Aug 2022 15:27:35 +0000 Received: by outflank-mailman (input) for mailman id 391470; Mon, 22 Aug 2022 15:27:33 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9Kv-0003Dm-1x for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:33 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id effe4189-222e-11ed-bd2e-47488cf2e6aa; Mon, 22 Aug 2022 17:27:32 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id B59603200A3C; Mon, 22 Aug 2022 11:27:29 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 22 Aug 2022 11:27:30 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:27 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: effe4189-222e-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182049; x=1661268449; bh=PKFBeEd03f Bi/cJsuUHCHAaB+wlrlisP7arcB1aTEUU=; b=LeoqTdmBACcyPXRwPfzGxyl4wK XpkS5EIaFaOwl83iyV0RyuWJ3OmEgTx7acnQ0NHbjlShNKLGukWIoXyR/fmAoNl7 Ql0yCGnkrOWD6uTTLoxg7aP504G6PQv72yI6BfRuIAyLGtDm/CritssWRXTTqxZ6 2GBbulwk2doL6F4hBrtLEe+TbqapOvJ9m6ivyIjjZXuaOsakO6gZCXY1N4sOxMC+ nDBgyuZmQN9sHLg0P3ZvZdNx3ueunxmsvzizRpvHBb4KHtCfr2BNPYcwJQU/5DZ/ okRNlRjA/7l1OB4Pu/8/v86X99pcWS+EcTth7kSXol+ocIldQzHDuCThP70g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182049; x= 1661268449; bh=PKFBeEd03fBi/cJsuUHCHAaB+wlrlisP7arcB1aTEUU=; b=x BoHA13gJCQwgzaQanMAo8HoUAQcOxLRH9GwMVYob68HfhPnxe+arDGdvsG2Act8j unV2QYiX76IBR35nDm18ZqVghlyqMcuGzHAHGA44iSOOSXJOv50d8MQV0qIRt2dN 65ThL4geysZjA+igENp8KRx2JFYgfU7rdinlECwqd6ZWhn5PvWgnZIbACmjWwmRY zX1995IR5sbZm8tdZWdwsHyVfYE02FBMrASfHIKxXuAEYjosdHJThzChcAqYbQKR APoiZtFxhddnW1v+z8Xo7fi1MqHN+rZEOSrSXWhNiSbnryf9vlkRpNtNkP+ziR5+ pQ/SakYbgDmamdHjnMRxA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v5 2/9] console: support multiple serial console simultaneously Date: Mon, 22 Aug 2022 17:27:02 +0200 Message-Id: X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 Previously only one serial console was supported at the same time. Using console=com1,dbgp,vga silently ignored all but last serial console (in this case: only dbgp and vga were active). Fix this by storing not a single sercon_handle, but an array of them, up to MAX_SERCONS entries. The value of MAX_SERCONS can be chosen in kconfig, the default (4) is arbitrary, inspired by the number of SERHND_IDX values. Make console_steal() aware of multiple consoles too. It can now either steal output from specific console (for gdbstub), or from all of them at once (for console suspend). Signed-off-by: Marek Marczykowski-Górecki --- Changes in v4: - use unsigned int for loop counters - other minor changes Changes in v3: - adjust console_steal() for multiple consoles too - add MAX_SERCONS to kconfig - add warning about sync_console impact - add warning if too many consoles are configured - log issue with PCI spec parsing --- docs/misc/xen-command-line.pandoc | 4 +- xen/drivers/char/Kconfig | 11 ++++- xen/drivers/char/console.c | 98 ++++++++++++++++++++++++-------- xen/include/xen/serial.h | 1 +- 4 files changed, 92 insertions(+), 22 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index 0d07f0c75990..e31300ea3408 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -435,6 +435,9 @@ only available when used together with `pv-in-pvh`. `none` indicates that Xen should not use a console. This option only makes sense on its own. +Specifying more than one serial console will increase console latency, +especially when `sync_console` option is used. + ### console_timestamps > `= none | date | datems | boot | raw` @@ -2409,6 +2412,7 @@ vulnerabilities. Flag to force synchronous console output. Useful for debugging, but not suitable for production environments due to incurred overhead. +If multiple consoles are configured, the incurred overhead is even bigger. ### tboot (x86) > `= 0x` diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig index 06350c387371..7b5ff0c414ec 100644 --- a/xen/drivers/char/Kconfig +++ b/xen/drivers/char/Kconfig @@ -85,6 +85,17 @@ config SERIAL_TX_BUFSIZE Default value is 16384 (16kiB). +config MAX_SERCONS + int "Maximum number of serial consoles active at once" + default 4 + help + Controls how many serial consoles can be active at once. Configuring more + using `console=` parameter will be ignored. + When multiple consoles are configured, overhead of `sync_console` option + is even bigger. + + Default value is 4. + config XHCI bool "XHCI DbC UART driver" depends on X86 diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c index e8468c121ad0..60d42284f606 100644 --- a/xen/drivers/char/console.c +++ b/xen/drivers/char/console.c @@ -113,7 +113,9 @@ static char *__read_mostly conring = _conring; static uint32_t __read_mostly conring_size = _CONRING_SIZE; static uint32_t conringc, conringp; -static int __read_mostly sercon_handle = -1; +#define MAX_SERCONS CONFIG_MAX_SERCONS +static int __read_mostly sercon_handle[MAX_SERCONS]; +static unsigned int __read_mostly nr_sercon_handle = 0; #ifdef CONFIG_X86 /* Tristate: 0 disabled, 1 user enabled, -1 default enabled */ @@ -393,32 +395,61 @@ long read_console_ring(struct xen_sysctl_readconsole *op) static char serial_rx_ring[SERIAL_RX_SIZE]; static unsigned int serial_rx_cons, serial_rx_prod; -static void (*serial_steal_fn)(const char *, size_t nr) = early_puts; +/* The last entry means "steal from all consoles" */ +static void (*serial_steal_fn[])(const char *, size_t nr) = { + [MAX_SERCONS] = early_puts, +}; +/* + * Redirect console *handle* output to *fn*. Use SERHND_STEAL_ALL as *handle* to + * redirect all the consoles. + */ int console_steal(int handle, void (*fn)(const char *, size_t nr)) { - if ( (handle == -1) || (handle != sercon_handle) ) - return 0; + unsigned int i; + + if ( handle == -1 ) + return -ENOENT; + if ( serial_steal_fn[MAX_SERCONS] != NULL ) + return -EBUSY; + if ( handle == SERHND_STEAL_ALL ) + { + serial_steal_fn[MAX_SERCONS] = fn; + return MAX_SERCONS; + } + for ( i = 0; i < nr_sercon_handle; i++ ) + if ( handle == sercon_handle[i] ) + break; + if ( i == nr_sercon_handle ) + return -ENOENT; - if ( serial_steal_fn != NULL ) + if ( serial_steal_fn[i] != NULL ) return -EBUSY; - serial_steal_fn = fn; - return 1; + serial_steal_fn[i] = fn; + return i; } void console_giveback(int id) { - if ( id == 1 ) - serial_steal_fn = NULL; + if ( id >= 0 && id <= MAX_SERCONS ) + serial_steal_fn[id] = NULL; } void console_serial_puts(const char *s, size_t nr) { - if ( serial_steal_fn != NULL ) - serial_steal_fn(s, nr); + unsigned int i; + + if ( serial_steal_fn[MAX_SERCONS] != NULL ) + serial_steal_fn[MAX_SERCONS](s, nr); else - serial_puts(sercon_handle, s, nr); + for ( i = 0; i < nr_sercon_handle; i++ ) + { + if ( serial_steal_fn[i] != NULL ) + serial_steal_fn[i](s, nr); + else + serial_puts(sercon_handle[i], s, nr); + } /* Copy all serial output into PV console */ pv_console_puts(s, nr); @@ -957,6 +988,7 @@ void __init console_init_preirq(void) { char *p; int sh; + unsigned int i; serial_init_preirq(); @@ -977,8 +1009,12 @@ void __init console_init_preirq(void) continue; else if ( (sh = serial_parse_handle(p)) >= 0 ) { - sercon_handle = sh; - serial_steal_fn = NULL; + if ( nr_sercon_handle < MAX_SERCONS ) + sercon_handle[nr_sercon_handle++] = sh; + else + printk("Too many consoles (max %d), ignoring '%s'\n", + MAX_SERCONS, p); + serial_steal_fn[MAX_SERCONS] = NULL; } else { @@ -996,7 +1032,8 @@ void __init console_init_preirq(void) opt_console_xen = 0; #endif - serial_set_rx_handler(sercon_handle, serial_rx); + for ( i = 0; i < nr_sercon_handle; i++ ) + serial_set_rx_handler(sercon_handle[i], serial_rx); pv_console_set_rx_handler(serial_rx); /* HELLO WORLD --- start-of-day banner text. */ @@ -1014,7 +1051,8 @@ void __init console_init_preirq(void) if ( opt_sync_console ) { - serial_start_sync(sercon_handle); + for ( i = 0; i < nr_sercon_handle; i++ ) + serial_start_sync(sercon_handle[i]); add_taint(TAINT_SYNC_CONSOLE); printk("Console output is synchronous.\n"); warning_add(warning_sync_console); @@ -1121,13 +1159,19 @@ int __init console_has(const char *device) void console_start_log_everything(void) { - serial_start_log_everything(sercon_handle); + unsigned int i; + + for ( i = 0; i < nr_sercon_handle; i++ ) + serial_start_log_everything(sercon_handle[i]); atomic_inc(&print_everything); } void console_end_log_everything(void) { - serial_end_log_everything(sercon_handle); + unsigned int i; + + for ( i = 0; i < nr_sercon_handle; i++ ) + serial_end_log_everything(sercon_handle[i]); atomic_dec(&print_everything); } @@ -1149,23 +1193,32 @@ void console_unlock_recursive_irqrestore(unsigned long flags) void console_force_unlock(void) { + unsigned int i; + watchdog_disable(); spin_debug_disable(); spin_lock_init(&console_lock); - serial_force_unlock(sercon_handle); + for ( i = 0 ; i < nr_sercon_handle ; i++ ) + serial_force_unlock(sercon_handle[i]); console_locks_busted = 1; console_start_sync(); } void console_start_sync(void) { + unsigned int i; + atomic_inc(&print_everything); - serial_start_sync(sercon_handle); + for ( i = 0 ; i < nr_sercon_handle ; i++ ) + serial_start_sync(sercon_handle[i]); } void console_end_sync(void) { - serial_end_sync(sercon_handle); + unsigned int i; + + for ( i = 0; i < nr_sercon_handle; i++ ) + serial_end_sync(sercon_handle[i]); atomic_dec(&print_everything); } @@ -1291,7 +1344,8 @@ static int suspend_steal_id; int console_suspend(void) { - suspend_steal_id = console_steal(sercon_handle, suspend_steal_fn); + if ( nr_sercon_handle ) + suspend_steal_id = console_steal(SERHND_STEAL_ALL, suspend_steal_fn); serial_suspend(); return 0; } diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h index 186afbed9c92..31c825edb052 100644 --- a/xen/include/xen/serial.h +++ b/xen/include/xen/serial.h @@ -100,6 +100,7 @@ struct uart_driver { #define SERHND_HI (1<<2) /* Mux/demux each transferred char by MSB. */ #define SERHND_LO (1<<3) /* Ditto, except that the MSB is cleared. */ #define SERHND_COOKED (1<<4) /* Newline/carriage-return translation? */ +#define SERHND_STEAL_ALL 0xff /* Synthetic handle used in console_steal() */ /* Three-stage initialisation (before/during/after IRQ-subsystem setup). */ void serial_init_preirq(void); From patchwork Mon Aug 22 15:27:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3CD71C32789 for ; Mon, 22 Aug 2022 15:27:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391471.629345 (Exim 4.92) (envelope-from ) id 1oQ9L0-00046Z-VR; Mon, 22 Aug 2022 15:27:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391471.629345; Mon, 22 Aug 2022 15:27:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L0-00045r-Pm; Mon, 22 Aug 2022 15:27:38 +0000 Received: by outflank-mailman (input) for mailman id 391471; Mon, 22 Aug 2022 15:27:37 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9Kz-0003iu-Ao for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:37 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f1599435-222e-11ed-9250-1f966e50362f; Mon, 22 Aug 2022 17:27:34 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 4190D3200A50; Mon, 22 Aug 2022 11:27:32 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 22 Aug 2022 11:27:32 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:30 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f1599435-222e-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182051; x=1661268451; bh=MP94SeiKov HRK4tURowdKY0S3+uktUUWGY/VTPo7IQM=; b=rVpDZoD2EgVX+qb2vU7p3HE/9m XcT4UVqyjRZuGxqiSWPtxz68mY1eKv10MIdNIhUEXNv15BJsbJE63PrD1008ZN62 wI4RCJ+6hrscsRfkSomTmKqYB3Rqfu6htE4e09Gaq2SkXZmRepVIP9LEG9QLFQwv 7bK8hoqnIkmQa5JJgxyNKI37xLzxhdr5VxhkHvKJG8eqAveK6NEQ9tOdlpPXF1tP +MAAPlwvZdPWT1zE+jVzXwXbdB0J8DG6eWZED8P7TlwXBMZPsAUfbnOUnf7x7uyd QIt2a5FFGOGVbE2XrPcuG8zGknhQfl/He9Yd2uDSJ+vQgA2zRAmrVltUPB2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182051; x= 1661268451; bh=MP94SeiKovHRK4tURowdKY0S3+uktUUWGY/VTPo7IQM=; b=v YYp6GZrJaYSmsY8/pPoywMyt2ZxSMjwhaI4DQ53g+gmUhjrIddgEvuQtXtF4SGsm QWGaqBTZM/UoeVWGLfU5XOIGW4XpB2a24o5epCKqbsuGPDfO6L71HiLKjZNkUi7R YaQXnFvJeJXhiNszjcCNuRlZzSs2WxffSzG3aIKOGHhvW8eENsRSRdEF281XQN+T 65TGENKlPbU03R9vP0lCq/vGIglxVjwhMvk396+HsLDuSRJti6CMIbt2sPie2DEv muI92oOA2j2BgZeN2TA0Nr9nyYGAcYn9WLnpx1ELBNAIRVRj8P3Bq7V2nBsYmEQP GYQrykhvtNqlcmT9Zxs6g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Jan Beulich , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v5 3/9] IOMMU: add common API for device reserved memory Date: Mon, 22 Aug 2022 17:27:03 +0200 Message-Id: <3ee5187c066ad12e3717df6fed41de99172271b0.1661181584.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 Add API similar to rmrr= and ivmd= arguments, but in a common code. This will allow drivers to register reserved memory regardless of the IOMMU vendor. The direct reason for this API is xhci-dbc console driver (aka xue), that needs to use DMA. But future change may unify command line arguments for user-supplied reserved memory, and it may be useful for other drivers in the future too. This commit just introduces an API, subsequent patches will plug it in appropriate places. The reserved memory ranges needs to be saved locally, because at the point when they are collected, Xen doesn't know yet which IOMMU driver will be used. Signed-off-by: Marek Marczykowski-Górecki Acked-by: Jan Beulich --- Changes in v5: - fix indentation, keep full "reserved_device_memory" for consistency with iommu_get_reserved_device_memory Changes in v4: - mark functions as __init - use pci_sbdf_t type Changes in v3: - adjust code style --- xen/drivers/passthrough/iommu.c | 46 ++++++++++++++++++++++++++++++++++- xen/include/xen/iommu.h | 14 ++++++++++- 2 files changed, 60 insertions(+) diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c index 134cdb47e0dc..5e2a720d29b9 100644 --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -669,6 +669,52 @@ bool_t iommu_has_feature(struct domain *d, enum iommu_feature feature) return is_iommu_enabled(d) && test_bit(feature, dom_iommu(d)->features); } +#define MAX_EXTRA_RESERVED_RANGES 20 +struct extra_reserved_range { + unsigned long start; + unsigned long nr; + pci_sbdf_t sbdf; +}; +static unsigned int __initdata nr_extra_reserved_ranges; +static struct extra_reserved_range __initdata + extra_reserved_ranges[MAX_EXTRA_RESERVED_RANGES]; + +int __init iommu_add_extra_reserved_device_memory(unsigned long start, + unsigned long nr, + pci_sbdf_t sbdf) +{ + unsigned int idx; + + if ( nr_extra_reserved_ranges >= MAX_EXTRA_RESERVED_RANGES ) + return -ENOMEM; + + idx = nr_extra_reserved_ranges++; + extra_reserved_ranges[idx].start = start; + extra_reserved_ranges[idx].nr = nr; + extra_reserved_ranges[idx].sbdf = sbdf; + + return 0; +} + +int __init iommu_get_extra_reserved_device_memory(iommu_grdm_t *func, + void *ctxt) +{ + unsigned int idx; + int ret; + + for ( idx = 0; idx < nr_extra_reserved_ranges; idx++ ) + { + ret = func(extra_reserved_ranges[idx].start, + extra_reserved_ranges[idx].nr, + extra_reserved_ranges[idx].sbdf.sbdf, + ctxt); + if ( ret < 0 ) + return ret; + } + + return 0; +} + /* * Local variables: * mode: C diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h index 1240d7762d99..4f22fc1bed55 100644 --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -304,6 +304,20 @@ struct iommu_ops { #endif }; +/* + * To be called by Xen internally, to register extra RMRR/IVMD ranges. + * Needs to be called before IOMMU initialization. + */ +extern int iommu_add_extra_reserved_device_memory(unsigned long start, + unsigned long nr, + pci_sbdf_t sbdf); +/* + * To be called by specific IOMMU driver during initialization, + * to fetch ranges registered with iommu_add_extra_reserved_device_memory(). + */ +extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func, + void *ctxt); + #include #ifndef iommu_call From patchwork Mon Aug 22 15:27:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C65D7C28D13 for ; Mon, 22 Aug 2022 15:27:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391472.629340 (Exim 4.92) (envelope-from ) id 1oQ9L0-00043J-Jo; Mon, 22 Aug 2022 15:27:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391472.629340; Mon, 22 Aug 2022 15:27:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L0-000435-G8; Mon, 22 Aug 2022 15:27:38 +0000 Received: by outflank-mailman (input) for mailman id 391472; Mon, 22 Aug 2022 15:27:37 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9Kz-0003Dm-7m for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:37 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f28729ce-222e-11ed-bd2e-47488cf2e6aa; Mon, 22 Aug 2022 17:27:36 +0200 (CEST) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 69A0C3200A53; Mon, 22 Aug 2022 11:27:34 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Mon, 22 Aug 2022 11:27:34 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:33 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f28729ce-222e-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182053; x=1661268453; bh=CFew235jgq 1Bpcgaw8qOFpTlwBPLlNuYoSmc12fF7k4=; b=o/NyVVyxQsKSvsIwwHbm86kFAI JuFzawp8i1NDqsNOPB/+wFkaojCVyqdtGvZnAT1YYh1yHoZfiYBxJn/8hPrSpdi9 cP6yd4BE3c7Za7w1w676D2jE7yHFUYs8ug2x4DHX0csgbMpfsNp9W7K4LG7tDHhq iUX9gvU/ZBKiTPsA5X77WavXzf4khTexvjzPlX0NEgjGBxytB3W0lzrHTi0HdeQ/ WAhyNVwAAUi2W/JT3oq6pbMgS66oOmMA1N7f6fIQzDsR1c5RqzaJ8BpPKGZ2M0f+ HgpD9AMK4u1DaOT1gQwZXjLHVCndJRozr/HH9Ikp0EBfDZ8XU23joDQJUEqw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182053; x= 1661268453; bh=CFew235jgq1Bpcgaw8qOFpTlwBPLlNuYoSmc12fF7k4=; b=n 1FeZ1iIpaZ5G+/yaA+86taF2ZTger+q0FMxQ4/8AiHpTb21Qrc1ve6jpL6+Fnkbw 2I4It26jx+xyK2BJC/5A5ioY7ierIntrZCyEu6YJGDkbeQQhKFTpo1uSil0rfS98 pT177fw6pqrexAkb4FuA4H9zz1ZMaqEV7YhwV/oDfgVBTIOtywHM/4X0W2KfMHdu dyRC148Jj7Q8qzFlcZPd5lYl/v0gXL5vHJoV/VW1suZeSwD78vZfQKEu846AKGMs Qdcx/RHJI+V40/U0ZMoRNITNgJWNKcZAWmm+t5jnTUSEumCAcF+rUhhDptatLw/a 4TD/XVRATPnS4qVrCUnHA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Kevin Tian Subject: [PATCH v5 4/9] IOMMU/VT-d: wire common device reserved memory API Date: Mon, 22 Aug 2022 17:27:04 +0200 Message-Id: <85675b8ddc09661b92f089459431f59081b4b1bc.1661181584.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 Re-use rmrr= parameter handling code to handle common device reserved memory. Signed-off-by: Marek Marczykowski-Górecki --- Changes in v3: - make MAX_USER_RMRR_PAGES applicable only to user-configured RMRR --- xen/drivers/passthrough/vtd/dmar.c | 201 +++++++++++++++++------------- 1 file changed, 119 insertions(+), 82 deletions(-) diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c index 367304c8739c..3df5f6b69719 100644 --- a/xen/drivers/passthrough/vtd/dmar.c +++ b/xen/drivers/passthrough/vtd/dmar.c @@ -861,111 +861,139 @@ static struct user_rmrr __initdata user_rmrrs[MAX_USER_RMRR]; /* Macro for RMRR inclusive range formatting. */ #define ERMRRU_FMT "[%lx-%lx]" -#define ERMRRU_ARG(eru) eru.base_pfn, eru.end_pfn +#define ERMRRU_ARG base_pfn, end_pfn + +static int __init add_one_user_rmrr(unsigned long base_pfn, + unsigned long end_pfn, + unsigned int dev_count, + uint32_t *sbdf); static int __init add_user_rmrr(void) { + unsigned int i; + int ret; + + for ( i = 0; i < nr_rmrr; i++ ) + { + ret = add_one_user_rmrr(user_rmrrs[i].base_pfn, + user_rmrrs[i].end_pfn, + user_rmrrs[i].dev_count, + user_rmrrs[i].sbdf); + if ( ret < 0 ) + return ret; + } + return 0; +} + +/* Returns 1 on success, 0 when ignoring and < 0 on error. */ +static int __init add_one_user_rmrr(unsigned long base_pfn, + unsigned long end_pfn, + unsigned int dev_count, + uint32_t *sbdf) +{ struct acpi_rmrr_unit *rmrr, *rmrru; - unsigned int idx, seg, i; - unsigned long base, end; + unsigned int idx, seg; + unsigned long base_iter; bool overlap; - for ( i = 0; i < nr_rmrr; i++ ) + if ( iommu_verbose ) + printk(XENLOG_DEBUG VTDPREFIX + "Adding RMRR for %d device ([0]: %#x) range "ERMRRU_FMT"\n", + dev_count, sbdf[0], ERMRRU_ARG); + + if ( base_pfn > end_pfn ) { - base = user_rmrrs[i].base_pfn; - end = user_rmrrs[i].end_pfn; + printk(XENLOG_ERR VTDPREFIX + "Invalid RMRR Range "ERMRRU_FMT"\n", + ERMRRU_ARG); + return 0; + } - if ( base > end ) + overlap = false; + list_for_each_entry(rmrru, &acpi_rmrr_units, list) + { + if ( pfn_to_paddr(base_pfn) <= rmrru->end_address && + rmrru->base_address <= pfn_to_paddr(end_pfn) ) { printk(XENLOG_ERR VTDPREFIX - "Invalid RMRR Range "ERMRRU_FMT"\n", - ERMRRU_ARG(user_rmrrs[i])); - continue; + "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n", + ERMRRU_ARG, + paddr_to_pfn(rmrru->base_address), + paddr_to_pfn(rmrru->end_address)); + overlap = true; + break; } + } + /* Don't add overlapping RMRR. */ + if ( overlap ) + return 0; - if ( (end - base) >= MAX_USER_RMRR_PAGES ) + base_iter = base_pfn; + do + { + if ( !mfn_valid(_mfn(base_iter)) ) { printk(XENLOG_ERR VTDPREFIX - "RMRR range "ERMRRU_FMT" exceeds "\ - __stringify(MAX_USER_RMRR_PAGES)" pages\n", - ERMRRU_ARG(user_rmrrs[i])); - continue; + "Invalid pfn in RMRR range "ERMRRU_FMT"\n", + ERMRRU_ARG); + break; } + } while ( base_iter++ < end_pfn ); - overlap = false; - list_for_each_entry(rmrru, &acpi_rmrr_units, list) - { - if ( pfn_to_paddr(base) <= rmrru->end_address && - rmrru->base_address <= pfn_to_paddr(end) ) - { - printk(XENLOG_ERR VTDPREFIX - "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n", - ERMRRU_ARG(user_rmrrs[i]), - paddr_to_pfn(rmrru->base_address), - paddr_to_pfn(rmrru->end_address)); - overlap = true; - break; - } - } - /* Don't add overlapping RMRR. */ - if ( overlap ) - continue; + /* Invalid pfn in range as the loop ended before end_pfn was reached. */ + if ( base_iter <= end_pfn ) + return 0; - do - { - if ( !mfn_valid(_mfn(base)) ) - { - printk(XENLOG_ERR VTDPREFIX - "Invalid pfn in RMRR range "ERMRRU_FMT"\n", - ERMRRU_ARG(user_rmrrs[i])); - break; - } - } while ( base++ < end ); + rmrr = xzalloc(struct acpi_rmrr_unit); + if ( !rmrr ) + return -ENOMEM; - /* Invalid pfn in range as the loop ended before end_pfn was reached. */ - if ( base <= end ) - continue; + rmrr->scope.devices = xmalloc_array(u16, dev_count); + if ( !rmrr->scope.devices ) + { + xfree(rmrr); + return -ENOMEM; + } - rmrr = xzalloc(struct acpi_rmrr_unit); - if ( !rmrr ) - return -ENOMEM; + seg = 0; + for ( idx = 0; idx < dev_count; idx++ ) + { + rmrr->scope.devices[idx] = sbdf[idx]; + seg |= PCI_SEG(sbdf[idx]); + } + if ( seg != PCI_SEG(sbdf[0]) ) + { + printk(XENLOG_ERR VTDPREFIX + "Segments are not equal for RMRR range "ERMRRU_FMT"\n", + ERMRRU_ARG); + scope_devices_free(&rmrr->scope); + xfree(rmrr); + return 0; + } - rmrr->scope.devices = xmalloc_array(u16, user_rmrrs[i].dev_count); - if ( !rmrr->scope.devices ) - { - xfree(rmrr); - return -ENOMEM; - } + rmrr->segment = seg; + rmrr->base_address = pfn_to_paddr(base_pfn); + /* Align the end_address to the end of the page */ + rmrr->end_address = pfn_to_paddr(end_pfn) | ~PAGE_MASK; + rmrr->scope.devices_cnt = dev_count; - seg = 0; - for ( idx = 0; idx < user_rmrrs[i].dev_count; idx++ ) - { - rmrr->scope.devices[idx] = user_rmrrs[i].sbdf[idx]; - seg |= PCI_SEG(user_rmrrs[i].sbdf[idx]); - } - if ( seg != PCI_SEG(user_rmrrs[i].sbdf[0]) ) - { - printk(XENLOG_ERR VTDPREFIX - "Segments are not equal for RMRR range "ERMRRU_FMT"\n", - ERMRRU_ARG(user_rmrrs[i])); - scope_devices_free(&rmrr->scope); - xfree(rmrr); - continue; - } + if ( register_one_rmrr(rmrr) ) + printk(XENLOG_ERR VTDPREFIX + "Could not register RMMR range "ERMRRU_FMT"\n", + ERMRRU_ARG); - rmrr->segment = seg; - rmrr->base_address = pfn_to_paddr(user_rmrrs[i].base_pfn); - /* Align the end_address to the end of the page */ - rmrr->end_address = pfn_to_paddr(user_rmrrs[i].end_pfn) | ~PAGE_MASK; - rmrr->scope.devices_cnt = user_rmrrs[i].dev_count; + return 1; +} - if ( register_one_rmrr(rmrr) ) - printk(XENLOG_ERR VTDPREFIX - "Could not register RMMR range "ERMRRU_FMT"\n", - ERMRRU_ARG(user_rmrrs[i])); - } +static int __init cf_check add_one_extra_rmrr(xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt) +{ + u32 sbdf_array[] = { id }; + return add_one_user_rmrr(start, start+nr, 1, sbdf_array); +} - return 0; +static int __init add_extra_rmrr(void) +{ + return iommu_get_extra_reserved_device_memory(add_one_extra_rmrr, NULL); } #include @@ -1010,7 +1038,7 @@ int __init acpi_dmar_init(void) { iommu_init_ops = &intel_iommu_init_ops; - return add_user_rmrr(); + return add_user_rmrr() || add_extra_rmrr(); } return ret; @@ -1108,6 +1136,15 @@ static int __init cf_check parse_rmrr_param(const char *str) else end = start; + if ( (end - start) >= MAX_USER_RMRR_PAGES ) + { + printk(XENLOG_ERR VTDPREFIX + "RMRR range "ERMRRU_FMT" exceeds "\ + __stringify(MAX_USER_RMRR_PAGES)" pages\n", + start, end); + return -E2BIG; + } + user_rmrrs[nr_rmrr].base_pfn = start; user_rmrrs[nr_rmrr].end_pfn = end; From patchwork Mon Aug 22 15:27:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 87B07C32792 for ; Mon, 22 Aug 2022 15:27:49 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391473.629362 (Exim 4.92) (envelope-from ) id 1oQ9L3-0004d0-5m; Mon, 22 Aug 2022 15:27:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391473.629362; Mon, 22 Aug 2022 15:27:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L3-0004ce-22; Mon, 22 Aug 2022 15:27:41 +0000 Received: by outflank-mailman (input) for mailman id 391473; Mon, 22 Aug 2022 15:27:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L1-0003iu-6A for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:39 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f3c97865-222e-11ed-9250-1f966e50362f; Mon, 22 Aug 2022 17:27:38 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 678203200A71; Mon, 22 Aug 2022 11:27:36 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 22 Aug 2022 11:27:36 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:35 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f3c97865-222e-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182055; x=1661268455; bh=/pD4K+B9L0 ba8VyhoGCQP8h4YWebGtRYWp1NDk3Q7sk=; b=cgOWQej6yYZdOw6e1uLcBlA9jX VtPx8obQkof4ie+8oTH9GXL4yGl0MwfkC+0FXdyDOq0sKafaZ2kUlXvidJfFcWrK MDSwFoL/SZX9Kf1+D7DkjyF88zGsgmoRb6gPGmiO+m2a8qouFCpxt/DVpQSvlpLD JzZ2+KWBFfIlonsEkK0W+NcGyh3mF22+bp33vgAkCYP+hA3K5g4CauIEE9a+n9Ku rRQmKaIHotD9itNLCPzt15HaJpzRsJ+0pJU5t5WqoGop2vUWJ7MgfYnL6EudrlJ2 KxPJHOO02rxlvqTh53CxUIR1nAfF+9IhlirXkzdjOz+bvjJEdLWS+asa8DKg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182055; x= 1661268455; bh=/pD4K+B9L0ba8VyhoGCQP8h4YWebGtRYWp1NDk3Q7sk=; b=0 DRb/8cbJHaCze5PP+YEFv7WHH23TTnZgUb5otB9yxLpZdOHq5klj/s3Apo4zN4dL EVy41M/EUQo/MY2BJC/3JkF0ARDVWf5Plu47w0Lq0/ISre2JRaz8inkpKgtdfYdz mNHu4dmvrmOiv7tvdTc+tMXE4pGkxaUQVIQt2KK94LZkLg4LTdg55Ztyx+UuIeoo zNpRwoH/mtt2aOl5ofUW+f2xwCea2KgVGCFBoY3Fc/awR72tDqCcd/i/8q5Azeqf lpAV/RxkJxxko8TXS5Dssnj8QnyxzQI6rZMCveSuKWWALsLkYSqZAJsBNKqUScAt GTETI8v9eDcpTLavhuuXg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Jan Beulich , Andrew Cooper Subject: [PATCH v5 5/9] IOMMU/AMD: wire common device reserved memory API Date: Mon, 22 Aug 2022 17:27:05 +0200 Message-Id: <725db95e13a7006d55ead9bfb584dce206eaa86c.1661181584.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 Register common device reserved memory similar to how ivmd= parameter is handled. Signed-off-by: Marek Marczykowski-Górecki Acked-by: Jan Beulich --- Changes in v3: - use variable initializer - use pfn_to_paddr() --- xen/drivers/passthrough/amd/iommu_acpi.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c index ac6835225bae..3b577c9b390c 100644 --- a/xen/drivers/passthrough/amd/iommu_acpi.c +++ b/xen/drivers/passthrough/amd/iommu_acpi.c @@ -1078,6 +1078,25 @@ static inline bool_t is_ivmd_block(u8 type) type == ACPI_IVRS_TYPE_MEMORY_IOMMU); } +static int __init cf_check add_one_extra_ivmd(unsigned long start, + unsigned long nr, + uint32_t id, void *ctxt) +{ + struct acpi_ivrs_memory ivmd = { + .header = { + .length = sizeof(ivmd), + .flags = ACPI_IVMD_UNITY | ACPI_IVMD_READ | ACPI_IVMD_WRITE, + .device_id = id, + .type = ACPI_IVRS_TYPE_MEMORY_ONE, + }, + }; + + ivmd.start_address = pfn_to_paddr(start); + ivmd.memory_length = pfn_to_paddr(nr); + + return parse_ivmd_block(&ivmd); +} + static int __init cf_check parse_ivrs_table(struct acpi_table_header *table) { const struct acpi_ivrs_header *ivrs_block; @@ -1121,6 +1140,8 @@ static int __init cf_check parse_ivrs_table(struct acpi_table_header *table) AMD_IOMMU_DEBUG("IVMD: %u command line provided entries\n", nr_ivmd); for ( i = 0; !error && i < nr_ivmd; ++i ) error = parse_ivmd_block(user_ivmds + i); + if ( !error ) + error = iommu_get_extra_reserved_device_memory(add_one_extra_ivmd, NULL); /* Each IO-APIC must have been mentioned in the table. */ for ( apic = 0; !error && iommu_intremap && apic < nr_ioapics; ++apic ) From patchwork Mon Aug 22 15:27:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8E326C28D13 for ; Mon, 22 Aug 2022 15:27:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391474.629373 (Exim 4.92) (envelope-from ) id 1oQ9L6-0004zT-GG; Mon, 22 Aug 2022 15:27:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391474.629373; Mon, 22 Aug 2022 15:27:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L6-0004z9-CT; Mon, 22 Aug 2022 15:27:44 +0000 Received: by outflank-mailman (input) for mailman id 391474; Mon, 22 Aug 2022 15:27:42 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L4-0003Dm-7a for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:42 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f58f1b07-222e-11ed-bd2e-47488cf2e6aa; Mon, 22 Aug 2022 17:27:41 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 0FC4C3200A7C; Mon, 22 Aug 2022 11:27:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 22 Aug 2022 11:27:39 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:37 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f58f1b07-222e-11ed-bd2e-47488cf2e6aa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182058; x=1661268458; bh=Pn0LnI5h21 OlMhhNf4aZSkh56wy/E8EA+91CFTzjTe4=; b=sQoAuQiUsytXAe6aZHkD3+nj/X xURp8nmsI7ApvsEWjNBjeApnAbI4pWcrS9PnwrbQqWkpZyGhE2cyn4oodoEdrdry s7FlHAUaxVpc6q2ScG4DMuRu3CbRAD084xqypbz/l+HA6LKm+K3nLPy8f6Tj6qPE ZDa/2BV6igTN9e1tlmCv91a3y+7/PDHhxhuL6sgGzspYq1JgeY3IBEYJfPW+ktkT K1oAaT3aVJe0GYzlrPWnDiIoSybk+jW/GYmawFjkmDoFAMXLIxPN5MTnSFqqSQjr dguBqCXJ4S2wvTz3eCa+ZvhZ0CA+zyclnaAZUhYWuxRdqqZrwYESeBv6i3ag== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182058; x= 1661268458; bh=Pn0LnI5h21OlMhhNf4aZSkh56wy/E8EA+91CFTzjTe4=; b=i ZYZUoPMkCUWowlR78BN8JB51gue35CmgOobDn5H2tBjnJT/0WGwC7Op6Fm6eUijM MgjNETPMAr2Lvw1fqZlvl3COiFzlPJSJ9ckAh8r/tU7Bi/SkYumFZx65LdTDvcTc spUbh5BtE1mhFsc23TfWa20T6Ag9eCyXAd22zTZppB2ugRQM+CjCT0ko2dp8JJaQ f5Ow9QXbizvng4juHYkCMoy/7HCqu3z5YMo1frJXC2E8uQP26OzS861C1Cj0oylj E675Nw9jpoihACjNxid9KjIIq1bc4wDUMg8WcPaecF6qPN95C7CzlqLBxxFm8FtM t2Ryg1L3Wy+TLvbwSEsMA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v5 6/9] drivers/char: mark DMA buffers as reserved for the XHCI Date: Mon, 22 Aug 2022 17:27:06 +0200 Message-Id: <58476e4d0ce2b819050393f137603d2a9294d40a.1661181584.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 The important part is to include those buffers in IOMMU page table relevant for the USB controller. Otherwise, DbC will stop working as soon as IOMMU is enabled, regardless of to which domain device assigned (be it xen or dom0). If the device is passed through to dom0 or other domain (see later patches), that domain will effectively have access to those buffers too. It does give such domain yet another way to DoS the system (as is the case when having PCI device assigned already), but also possibly steal the console ring content. Thus, such domain should be a trusted one. In any case, prevent anything else being placed on those pages by adding artificial padding. Signed-off-by: Marek Marczykowski-Górecki Acked-by: Jan Beulich --- Changes in v5: - add missing alignment Changes in v3: - adjust for xhci-dbc rename - do not raise MAX_USER_RMRR_PAGES - adjust alignment of DMA buffers --- xen/drivers/char/xhci-dbc.c | 43 +++++++++++++++++++++++++------------- 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c index eb35e3a2ee4f..32e9efeb0f77 100644 --- a/xen/drivers/char/xhci-dbc.c +++ b/xen/drivers/char/xhci-dbc.c @@ -20,6 +20,7 @@ */ #include +#include #include #include #include @@ -1050,13 +1051,21 @@ static struct uart_driver dbc_uart_driver = { }; /* Those are accessed via DMA. */ -static struct xhci_trb evt_trb[DBC_TRB_RING_CAP]; -static struct xhci_trb out_trb[DBC_TRB_RING_CAP]; -static struct xhci_trb in_trb[DBC_TRB_RING_CAP]; -static struct xhci_erst_segment erst __aligned(16); -static struct xhci_dbc_ctx ctx __aligned(16); -static uint8_t out_wrk_buf[DBC_WORK_RING_CAP]; -static struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT]; +struct dbc_dma_bufs { + struct xhci_trb evt_trb[DBC_TRB_RING_CAP]; + struct xhci_trb out_trb[DBC_TRB_RING_CAP]; + struct xhci_trb in_trb[DBC_TRB_RING_CAP]; + uint8_t out_wrk_buf[DBC_WORK_RING_CAP]; + struct xhci_erst_segment erst __aligned(16); + struct xhci_dbc_ctx ctx __aligned(16); + struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT]; + /* + * Don't place anything else on this page - it will be + * DMA-reachable by the USB controller. + */ +}; +static struct dbc_dma_bufs __section(".bss.page_aligned") __aligned(PAGE_SIZE) + dbc_dma_bufs; static char __initdata opt_dbc[30]; @@ -1093,16 +1102,22 @@ void __init xhci_dbc_uart_init(void) dbc->sbdf = PCI_SBDF(0, bus, slot, func); } - dbc->dbc_ctx = &ctx; - dbc->dbc_erst = &erst; - dbc->dbc_ering.trb = evt_trb; - dbc->dbc_oring.trb = out_trb; - dbc->dbc_iring.trb = in_trb; - dbc->dbc_owork.buf = out_wrk_buf; - dbc->dbc_str = str_buf; + dbc->dbc_ctx = &dbc_dma_bufs.ctx; + dbc->dbc_erst = &dbc_dma_bufs.erst; + dbc->dbc_ering.trb = dbc_dma_bufs.evt_trb; + dbc->dbc_oring.trb = dbc_dma_bufs.out_trb; + dbc->dbc_iring.trb = dbc_dma_bufs.in_trb; + dbc->dbc_owork.buf = dbc_dma_bufs.out_wrk_buf; + dbc->dbc_str = dbc_dma_bufs.str_buf; if ( dbc_open(dbc) ) + { + iommu_add_extra_reserved_device_memory( + PFN_DOWN(virt_to_maddr(&dbc_dma_bufs)), + PFN_UP(sizeof(dbc_dma_bufs)), + uart->dbc.sbdf); serial_register_uart(SERHND_DBC, &dbc_uart_driver, &dbc_uart); + } } #ifdef DBC_DEBUG From patchwork Mon Aug 22 15:27:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA9C2C32772 for ; Mon, 22 Aug 2022 15:27:55 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391475.629384 (Exim 4.92) (envelope-from ) id 1oQ9L9-0005OG-1i; Mon, 22 Aug 2022 15:27:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391475.629384; Mon, 22 Aug 2022 15:27:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L8-0005Nv-TA; Mon, 22 Aug 2022 15:27:46 +0000 Received: by outflank-mailman (input) for mailman id 391475; Mon, 22 Aug 2022 15:27:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9L7-0003iu-9b for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:45 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f7526ffd-222e-11ed-9250-1f966e50362f; Mon, 22 Aug 2022 17:27:44 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 05709320099F; Mon, 22 Aug 2022 11:27:41 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 22 Aug 2022 11:27:42 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:40 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f7526ffd-222e-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182061; x=1661268461; bh=Ds5W69sMUj ID7EvzJTjybVIdCA4RPqmY+2ZVAuvq13Y=; b=YFm1zyQCx1oUm0WFUxW6XGY+Sq /26HGIXHuZaHoVm0qYHt/Tnp7OZ5Em14VOT/o3UMqEvU4R3fQvZ5rv5SC8Sk2OCC 2OE1c8rFGxfgnsIudD0ab3NdWZqlWZGnCOk+yG9qSJPFkEecEgV0F2E/neJW8dq7 lmhT4ODtmmgsk4bldU76A/T1BO1R0QslDeJ6S/ExPRCyLjma0SnEXN11CYPIkK56 t6XwhmfecUjvi+HIHrchEUTXBW9tOdFUIhSnTSDJdsv6/bIlhzS6ZUrj+YRFw+YM mFFy7NQZESIB+9aVbTQ5ETc1huTYExDL/z9yPBA0fPelZJN6yzmB5erI8OAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182061; x= 1661268461; bh=Ds5W69sMUjID7EvzJTjybVIdCA4RPqmY+2ZVAuvq13Y=; b=D 6K94sSxn8Q3AHXmhBXc2twEj3B6CRSePQR8KS7phGZ9qXhg7tT+JdIcuoSqTkIqO JTCBwilzHrP5Ee4rzLEt5QUVT0xuOQlYyeZ8weqF87oOVKFOxvnuK38tKcMNa0tC DPsJeF5wCD570fib71/VIjLIHgPz4/q2wI88lhMSnOGqa8O4Zy+wMruGyY6UIMaj u08PpTw7oRJim5dpK0XS8UcB40L4lQH2i58l6B6RcHKmJAEvaDqPyBAvMDawdpQY 0YJqriVTl7SJyl3SOL4RKwM3xJ0PlEEIXD+MasYY3ATmpndmD1Pa1LYaeDksUDQp 9RlKUvGaws+jeneazm7qg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpeegnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Jan Beulich , Andrew Cooper , George Dunlap , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v5 7/9] drivers/char: add RX support to the XHCI driver Date: Mon, 22 Aug 2022 17:27:07 +0200 Message-Id: <7520ee704004e5464ab13bacb38baea4e986992e.1661181584.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 Add another work ring buffer for received data, and point IN TRB at it. Ensure there is always at least one pending IN TRB, so the controller has a way to send incoming data to the driver. Note that both "success" and "short packet" completion codes are okay - in fact it will be "short packet" most of the time, as the TRB length is about maximum size, not required size. Signed-off-by: Marek Marczykowski-Górecki Acked-by: Jan Beulich --- Changes in v4: - adjust return types - add some const New patch in v3 --- docs/misc/xen-command-line.pandoc | 4 +- xen/drivers/char/xhci-dbc.c | 129 +++++++++++++++++++++++++++++++- 2 files changed, 131 insertions(+), 2 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index e31300ea3408..a5883ef49a88 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -736,8 +736,8 @@ over the PCI busses sequentially) or by PCI device (must be on segment 0). Specify the USB controller to use, either by instance number (when going over the PCI busses sequentially) or by PCI device (must be on segment 0). -Output only console. XHCI driver will wait indefinitely for the debug host to -connect - make sure the cable is connected. +XHCI driver will wait indefinitely for the debug host to connect - make sure +the cable is connected. ### debug_stack_lines > `= ` diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c index 32e9efeb0f77..afcd3312e2aa 100644 --- a/xen/drivers/char/xhci-dbc.c +++ b/xen/drivers/char/xhci-dbc.c @@ -111,6 +111,7 @@ enum { enum { XHCI_TRB_CC_SUCCESS = 1, XHCI_TRB_CC_TRB_ERR = 5, + XHCI_TRB_CC_SHORT_PACKET = 13, }; /* DbC endpoint types */ @@ -239,6 +240,7 @@ struct dbc { struct xhci_trb_ring dbc_oring; struct xhci_trb_ring dbc_iring; struct dbc_work_ring dbc_owork; + struct dbc_work_ring dbc_iwork; struct xhci_string_descriptor *dbc_str; pci_sbdf_t sbdf; @@ -443,6 +445,16 @@ static void xhci_trb_norm_set_ioc(struct xhci_trb *trb) trb->ctrl |= 0x20; } +static uint64_t xhci_trb_norm_buf(const struct xhci_trb *trb) +{ + return trb->params; +} + +static uint32_t xhci_trb_norm_len(const struct xhci_trb *trb) +{ + return trb->status & 0x1FFFF; +} + /** * Fields for Transfer Event TRBs (see section 6.4.2.1). Note that event * TRBs are read-only from software @@ -452,6 +464,17 @@ static uint64_t xhci_trb_tfre_ptr(const struct xhci_trb *trb) return trb->params; } +static uint32_t xhci_trb_tfre_cc(const struct xhci_trb *trb) +{ + return trb->status >> 24; +} + +/* Amount of data _not_ transferred */ +static uint32_t xhci_trb_tfre_len(const struct xhci_trb *trb) +{ + return trb->status & 0x1FFFF; +} + /* Fields for link TRBs (section 6.4.4.1) */ static void xhci_trb_link_set_rsp(struct xhci_trb *trb, uint64_t rsp) { @@ -493,6 +516,14 @@ static bool xhci_trb_ring_full(const struct xhci_trb_ring *ring) return ((ring->enq + 1) & (DBC_TRB_RING_CAP - 1)) == ring->deq; } +static unsigned int xhci_trb_ring_size(const struct xhci_trb_ring *ring) +{ + if ( ring->enq >= ring->deq ) + return ring->enq - ring->deq; + + return DBC_TRB_RING_CAP - ring->deq + ring->enq; +} + static bool dbc_work_ring_full(const struct dbc_work_ring *ring) { return ((ring->enq + 1) & (DBC_WORK_RING_CAP - 1)) == ring->deq; @@ -506,6 +537,14 @@ static unsigned int dbc_work_ring_size(const struct dbc_work_ring *ring) return DBC_WORK_RING_CAP - ring->deq + ring->enq; } +static unsigned int dbc_work_ring_space_to_end(const struct dbc_work_ring *ring) +{ + if ( ring->enq >= ring->deq ) + return DBC_WORK_RING_CAP - ring->enq; + + return ring->deq - ring->enq; +} + static void dbc_push_trb(struct dbc *dbc, struct xhci_trb_ring *ring, uint64_t dma, uint64_t len) { @@ -566,6 +605,31 @@ static unsigned int dbc_push_work(struct dbc *dbc, struct dbc_work_ring *ring, return i; } +static void dbc_rx_trb(struct dbc *dbc, struct xhci_trb *trb, + uint64_t not_transferred) +{ + struct dbc_work_ring *ring = &dbc->dbc_iwork; + unsigned int rx_len; + unsigned int end, start = ring->enq; + + if ( xhci_trb_type(trb) != XHCI_TRB_NORM ) + /* Can be Link TRB for example. */ + return; + + ASSERT(xhci_trb_norm_buf(trb) == ring->dma + ring->enq); + ASSERT(xhci_trb_norm_len(trb) >= not_transferred); + rx_len = xhci_trb_norm_len(trb) - not_transferred; + + /* It can hit the ring end, but should not wrap around. */ + ASSERT(ring->enq + rx_len <= DBC_WORK_RING_CAP); + ring->enq = (ring->enq + rx_len) & (DBC_WORK_RING_CAP - 1); + + end = ring->enq; + + if ( end > start ) + cache_flush(&ring->buf[start], end - start); +} + /* * Note that if IN transfer support is added, then this * will need to be changed; it assumes an OUT transfer ring only @@ -575,6 +639,7 @@ static void dbc_pop_events(struct dbc *dbc) struct dbc_reg *reg = dbc->dbc_reg; struct xhci_trb_ring *er = &dbc->dbc_ering; struct xhci_trb_ring *tr = &dbc->dbc_oring; + struct xhci_trb_ring *ir = &dbc->dbc_iring; struct xhci_trb *event = &er->trb[er->deq]; uint64_t erdp = readq(®->erdp); uint32_t portsc; @@ -600,6 +665,14 @@ static void dbc_pop_events(struct dbc *dbc) trb_idx = (event_ptr - tr->dma) >> XHCI_TRB_SHIFT; tr->deq = (trb_idx + 1) & (DBC_TRB_RING_CAP - 1); } + else if ( event_ptr - ir->dma < DBC_TRB_RING_BYTES ) + { + trb_idx = (event_ptr - ir->dma) >> XHCI_TRB_SHIFT; + if ( xhci_trb_tfre_cc(event) == XHCI_TRB_CC_SUCCESS || + xhci_trb_tfre_cc(event) == XHCI_TRB_CC_SHORT_PACKET ) + dbc_rx_trb(dbc, &ir->trb[trb_idx], xhci_trb_tfre_len(event)); + ir->deq = (trb_idx + 1) & (DBC_TRB_RING_CAP - 1); + } else dbc_alert("event: TRB 0x%lx not found in any ring\n", event_ptr); @@ -870,6 +943,7 @@ static bool __init dbc_open(struct dbc *dbc) return false; dbc_init_work_ring(dbc, &dbc->dbc_owork); + dbc_init_work_ring(dbc, &dbc->dbc_iwork); dbc_enable_dbc(dbc); dbc->open = true; @@ -946,6 +1020,33 @@ static void dbc_flush(struct dbc *dbc, struct xhci_trb_ring *trb, } /** + * Ensure DbC has a pending transfer TRB to receive data into. + * + * @param dbc the dbc to flush + * @param trb the ring for the TRBs to transfer + * @param wrk the work ring to receive data into + */ +static void dbc_enqueue_in(struct dbc *dbc, struct xhci_trb_ring *trb, + struct dbc_work_ring *wrk) +{ + struct dbc_reg *reg = dbc->dbc_reg; + uint32_t db = (readl(®->db) & 0xFFFF00FF) | (trb->db << 8); + + /* Check if there is already queued TRB */ + if ( xhci_trb_ring_size(trb) >= 1 ) + return; + + if ( dbc_work_ring_full(wrk) ) + return; + + dbc_push_trb(dbc, trb, wrk->dma + wrk->enq, + dbc_work_ring_space_to_end(wrk)); + + wmb(); + writel(db, ®->db); +} + +/** * Queue a single character to the DbC. A transfer TRB will be created * if the character is a newline and the DbC will be notified that data is * available for writing to the debug host. @@ -968,6 +1069,19 @@ static int64_t dbc_putc(struct dbc *dbc, char c) return 1; } +static int dbc_getc(struct dbc *dbc, char *c) +{ + struct dbc_work_ring *wrk = &dbc->dbc_iwork; + + if ( dbc_work_ring_size(wrk) == 0 ) + return 0; + + *c = wrk->buf[wrk->deq]; + wrk->deq = (wrk->deq + 1) & (DBC_WORK_RING_CAP - 1); + + return 1; +} + struct dbc_uart { struct dbc dbc; struct timer timer; @@ -986,10 +1100,16 @@ static void cf_check dbc_uart_poll(void *data) if ( spin_trylock_irqsave(&port->tx_lock, flags) ) { if ( dbc_ensure_running(dbc) ) + { dbc_flush(dbc, &dbc->dbc_oring, &dbc->dbc_owork); + dbc_enqueue_in(dbc, &dbc->dbc_iring, &dbc->dbc_iwork); + } spin_unlock_irqrestore(&port->tx_lock, flags); } + while ( dbc_work_ring_size(&dbc->dbc_iwork) ) + serial_rx_interrupt(port, guest_cpu_user_regs()); + serial_tx_interrupt(port, guest_cpu_user_regs()); set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL)); } @@ -1028,6 +1148,12 @@ static void cf_check dbc_uart_putc(struct serial_port *port, char c) dbc_putc(&uart->dbc, c); } +static int cf_check dbc_uart_getc(struct serial_port *port, char *c) +{ + struct dbc_uart *uart = port->uart; + return dbc_getc(&uart->dbc, c); +} + static void cf_check dbc_uart_flush(struct serial_port *port) { s_time_t goal; @@ -1047,6 +1173,7 @@ static struct uart_driver dbc_uart_driver = { .init_postirq = dbc_uart_init_postirq, .tx_ready = dbc_uart_tx_ready, .putc = dbc_uart_putc, + .getc = dbc_uart_getc, .flush = dbc_uart_flush, }; @@ -1056,6 +1183,7 @@ struct dbc_dma_bufs { struct xhci_trb out_trb[DBC_TRB_RING_CAP]; struct xhci_trb in_trb[DBC_TRB_RING_CAP]; uint8_t out_wrk_buf[DBC_WORK_RING_CAP]; + uint8_t in_wrk_buf[DBC_WORK_RING_CAP]; struct xhci_erst_segment erst __aligned(16); struct xhci_dbc_ctx ctx __aligned(16); struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT]; @@ -1108,6 +1236,7 @@ void __init xhci_dbc_uart_init(void) dbc->dbc_oring.trb = dbc_dma_bufs.out_trb; dbc->dbc_iring.trb = dbc_dma_bufs.in_trb; dbc->dbc_owork.buf = dbc_dma_bufs.out_wrk_buf; + dbc->dbc_iwork.buf = dbc_dma_bufs.in_wrk_buf; dbc->dbc_str = dbc_dma_bufs.str_buf; if ( dbc_open(dbc) ) From patchwork Mon Aug 22 15:27:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950806 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63EE9C28D13 for ; Mon, 22 Aug 2022 15:27:58 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391479.629395 (Exim 4.92) (envelope-from ) id 1oQ9LC-0005r6-IT; Mon, 22 Aug 2022 15:27:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391479.629395; Mon, 22 Aug 2022 15:27:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9LC-0005qa-9Z; Mon, 22 Aug 2022 15:27:50 +0000 Received: by outflank-mailman (input) for mailman id 391479; Mon, 22 Aug 2022 15:27:48 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9LA-0003iu-Cv for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:48 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f920e2d7-222e-11ed-9250-1f966e50362f; Mon, 22 Aug 2022 17:27:47 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 06E403200971; Mon, 22 Aug 2022 11:27:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 22 Aug 2022 11:27:45 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:43 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f920e2d7-222e-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182064; x=1661268464; bh=JeIh207uzZ pb1aNaI9jycqnoSZkqV0i0vjOTN7pf/5Q=; b=LtyZhkiSLkkhUCK7hJdj4Gkhiu xvs7MC4xxv4k5DiHtU3T1Q7/Vrow0sOWR3nEKU1mhGl1E/xW0w7UGCu20IL4xexo tOijEaN2fXONZz6zy/jBUkqpv9AOPvdUSomfaCkDcIzHFIgPKKLqMQlStqsrOkPd gcSuIiSbj0JQDfhpQ3hQbYG5mm0DmznXrMelVChrG8twpwiaSVWCYEAKI2Eputbl eIKdAF28Z6jevAzJ0Bqv0sn5QI2CvNEZ8NP7W2Pa9CmVKiUDHBu/ZWpr7DW4SwZH A1owg5aiNvoVN97jnkV8d30pmk1TE18mAj4qkdCNTN6gG4pcBh3hVs7KIAzw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182064; x= 1661268464; bh=JeIh207uzZpb1aNaI9jycqnoSZkqV0i0vjOTN7pf/5Q=; b=P z5E4lH9m5nj+7llt36Lvn9fKjA6UT0s6XdcEwm10U24kB1pobDSV7D24j8oiY+Mw t+UP3vM4qISaFtrL9eP1wj1v0i+Rn2vVlESgOkupHF7sXp5jn6MyT5pddDT3u2mi llHUEtoGhuqA4+2j+EQz0+mwms7scdCJXLnY91VrnCJnyXQvGXzEMQgfMEAL3+oM WMQc3noLtpJTYHRmCHz932+DRoMRm2LRk0kPD9fh2GgiwezDvBzGr02nEm4b8Z69 ty2x0x9hSTwhz0rCB0ViZ1jNOeGM8cf0fA8TTsDn57MiN/pO5o+i8HVBvs7bahqq 6WtonL3Jf9DKxqMJ2bw7g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgkeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v5 8/9] drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC Date: Mon, 22 Aug 2022 17:27:08 +0200 Message-Id: <1e2e4177eecc1da762d519926fa206971a9997a0.1661181584.git-series.marmarek@invisiblethingslab.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 That's possible, because the capability was designed specifically to allow separate driver handle it, in parallel to unmodified xhci driver (separate set of registers, pretending the port is "disconnected" for the main xhci driver etc). It works with Linux dom0, although requires an awful hack - re-enabling bus mastering behind dom0's backs. Linux driver does similar thing - see drivers/usb/early/xhci-dbc.c:xdbc_handle_events(). When controller sharing is enabled in kconfig (option marked as experimental), dom0 is allowed to use the controller even if Xen uses it for debug console. Additionally, option `dbgp=xhci,share=` is available to either prevent even dom0 from using it (`no` value), or allow any domain using it (`any` value). In any case, to avoid Linux messing with the DbC, mark this MMIO area as read-only. This might cause issues for Linux's driver (if it tries to write something on the same page - like anoter xcap), but makes Xen's use safe. In practice, as of Linux 5.18, it seems to work without issues. Signed-off-by: Marek Marczykowski-Górecki Reviewed-by: Jan Beulich --- Changes in v5: - drop CONFIG_XHCI_SHARE - make XHCI_SHARE_HWDOM = 0 - use parse_boolean - add comment about mmio_ro_ranges - fix doc Changes in v4: - minor fix for cmdline parsing - make sharing opt-in build time, with option marked as EXPERIMENTAL - change cmdline syntax to share=|hwdom - make share=hwdom default (if enabled build-time) Changes in v3: - adjust for xhci-dbc rename - adjust for dbc_ensure_running() split - wrap long lines - add runtime option for sharing USB controller --- docs/misc/xen-command-line.pandoc | 14 ++- xen/drivers/char/xhci-dbc.c | 129 +++++++++++++++++++++++++++++-- 2 files changed, 134 insertions(+), 9 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index a5883ef49a88..5aa184a4a06c 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -731,13 +731,25 @@ Specify the USB controller to use, either by instance number (when going over the PCI busses sequentially) or by PCI device (must be on segment 0). ### dbc -> `= xhci[ | @pci:. ]` +> `= xhci[ | @pci:. ][,share=|hwdom]` Specify the USB controller to use, either by instance number (when going over the PCI busses sequentially) or by PCI device (must be on segment 0). XHCI driver will wait indefinitely for the debug host to connect - make sure the cable is connected. +The `share` option for xhci controls who else can use the controller: +* `no`: use the controller exclusively for console, even hardware domain + (dom0) cannot use it +* `hwdom`: hardware domain may use the controller too, ports not used for debug + console will be available for normal devices; this is the default +* `yes`: the controller can be assigned to any domain; it is not safe to assign + the controller to untrusted domain + +Choosing `share=hwdom` (the default) or `share=yes` allows a domain to reset the +controller, which may cause small portion of the console output to be lost. + +The `share=yes` configuration is not security supported. ### debug_stack_lines > `= ` diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c index afcd3312e2aa..e838e285fb75 100644 --- a/xen/drivers/char/xhci-dbc.c +++ b/xen/drivers/char/xhci-dbc.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -232,6 +233,12 @@ struct dbc_work_ring { uint64_t dma; }; +enum xhci_share { + XHCI_SHARE_HWDOM = 0, + XHCI_SHARE_NONE, + XHCI_SHARE_ANY +}; + struct dbc { struct dbc_reg __iomem *dbc_reg; struct xhci_dbc_ctx *dbc_ctx; @@ -249,6 +256,7 @@ struct dbc { void __iomem *xhc_mmio; bool open; + enum xhci_share share; unsigned int xhc_num; /* look for n-th xhc */ }; @@ -951,13 +959,56 @@ static bool __init dbc_open(struct dbc *dbc) } /* - * Ensure DbC is still running, handle events, and possibly re-enable if cable - * was re-plugged. Returns true if DbC is operational. + * Ensure DbC is still running, handle events, and possibly + * re-enable/re-configure if cable was re-plugged or controller was reset. + * Returns true if DbC is operational. */ static bool dbc_ensure_running(struct dbc *dbc) { struct dbc_reg *reg = dbc->dbc_reg; uint32_t ctrl; + uint16_t cmd; + + if ( dbc->share != XHCI_SHARE_NONE ) + { + /* + * Re-enable memory decoding and later bus mastering, if dom0 (or + * other) disabled it in the meantime. + */ + cmd = pci_conf_read16(dbc->sbdf, PCI_COMMAND); + if ( !(cmd & PCI_COMMAND_MEMORY) ) + { + cmd |= PCI_COMMAND_MEMORY; + pci_conf_write16(dbc->sbdf, PCI_COMMAND, cmd); + } + + /* + * FIXME: Make Linux coordinate XHCI reset, so the DbC driver can + * prepare for it properly, instead of only detecting it after the + * fact. See EHCI driver for similar handling. + */ + if ( dbc->open && !(readl(®->ctrl) & (1U << DBC_CTRL_DCE)) ) + { + if ( !dbc_init_dbc(dbc) ) + return false; + + dbc_init_work_ring(dbc, &dbc->dbc_owork); + dbc_enable_dbc(dbc); + } + else + { + /* + * dbc_init_dbc() takes care about it, so check only if it wasn't + * called. + */ + cmd = pci_conf_read16(dbc->sbdf, PCI_COMMAND); + if ( !(cmd & PCI_COMMAND_MASTER) ) + { + cmd |= PCI_COMMAND_MASTER; + pci_conf_write16(dbc->sbdf, PCI_COMMAND, cmd); + } + } + } dbc_pop_events(dbc); @@ -1128,10 +1179,38 @@ static void __init cf_check dbc_uart_init_postirq(struct serial_port *port) init_timer(&uart->timer, dbc_uart_poll, port, 0); set_timer(&uart->timer, NOW() + MILLISECS(1)); - if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) ) - printk(XENLOG_WARNING - "Failed to mark read-only %pp used for XHCI console\n", - &uart->dbc.sbdf); + switch ( uart->dbc.share ) + { + case XHCI_SHARE_NONE: + if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) ) + printk(XENLOG_WARNING + "Failed to mark read-only %pp used for XHCI console\n", + &uart->dbc.sbdf); + break; + case XHCI_SHARE_HWDOM: + if ( pci_hide_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) ) + printk(XENLOG_WARNING + "Failed to hide %pp used for XHCI console\n", + &uart->dbc.sbdf); + break; + case XHCI_SHARE_ANY: + /* Do not hide. */ + break; + } +#ifdef CONFIG_X86 + /* + * This marks the whole page as R/O, which may include other registers + * unrelated to DbC. Xen needs only DbC area protected, but it seems + * Linux's XHCI driver (as of 5.18) works without writting to the whole + * page, so keep it simple. + */ + if ( rangeset_add_range(mmio_ro_ranges, + PFN_DOWN(uart->dbc.xhc_mmio_phys + uart->dbc.xhc_dbc_offset), + PFN_UP(uart->dbc.xhc_mmio_phys + uart->dbc.xhc_dbc_offset + + sizeof(*uart->dbc.dbc_reg)) - 1) ) + printk(XENLOG_INFO + "Error while adding MMIO range of device to mmio_ro_ranges\n"); +#endif } static int cf_check dbc_uart_tx_ready(struct serial_port *port) @@ -1203,13 +1282,15 @@ void __init xhci_dbc_uart_init(void) { struct dbc_uart *uart = &dbc_uart; struct dbc *dbc = &uart->dbc; - const char *e; + const char *e, *opt; + int val; if ( strncmp(opt_dbc, "xhci", 4) ) return; memset(dbc, 0, sizeof(*dbc)); + e = &opt_dbc[4]; if ( isdigit(opt_dbc[4]) ) { dbc->xhc_num = simple_strtoul(opt_dbc + 4, &e, 10); @@ -1219,7 +1300,7 @@ void __init xhci_dbc_uart_init(void) unsigned int bus, slot, func; e = parse_pci(opt_dbc + 8, NULL, &bus, &slot, &func); - if ( !e || *e ) + if ( !e || (*e && *e != ',') ) { printk(XENLOG_ERR "Invalid dbc= PCI device spec: '%s'\n", @@ -1229,6 +1310,38 @@ void __init xhci_dbc_uart_init(void) dbc->sbdf = PCI_SBDF(0, bus, slot, func); } + opt = e; + + /* other options */ + while ( *opt == ',' ) + { + opt++; + e = strchr(opt, ','); + if ( !e ) + e = strchr(opt, '\0'); + + if ( (val = parse_boolean("share", opt, e)) != -1 ) + { + if ( val == -2 && !cmdline_strcmp(opt + 6, "hwdom") ) + dbc->share = XHCI_SHARE_HWDOM; + else if ( val == 0 ) + dbc->share = XHCI_SHARE_NONE; + else if ( val == 1 ) + dbc->share = XHCI_SHARE_ANY; + else + break; + } + else + break; + + opt = e; + } + + if ( *opt ) + { + printk(XENLOG_ERR "Invalid dbc= parameters: '%s'\n", opt); + return; + } dbc->dbc_ctx = &dbc_dma_bufs.ctx; dbc->dbc_erst = &dbc_dma_bufs.erst; From patchwork Mon Aug 22 15:27:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= X-Patchwork-Id: 12950807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1B33C32772 for ; Mon, 22 Aug 2022 15:28:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.391482.629406 (Exim 4.92) (envelope-from ) id 1oQ9LE-0006Ih-UR; Mon, 22 Aug 2022 15:27:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 391482.629406; Mon, 22 Aug 2022 15:27:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9LE-0006Hf-N9; Mon, 22 Aug 2022 15:27:52 +0000 Received: by outflank-mailman (input) for mailman id 391482; Mon, 22 Aug 2022 15:27:51 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1oQ9LD-0003iu-3F for xen-devel@lists.xenproject.org; Mon, 22 Aug 2022 15:27:51 +0000 Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id fae6ba82-222e-11ed-9250-1f966e50362f; Mon, 22 Aug 2022 17:27:50 +0200 (CEST) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 02AEE3200A3C; Mon, 22 Aug 2022 11:27:47 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Mon, 22 Aug 2022 11:27:48 -0400 Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 22 Aug 2022 11:27:46 -0400 (EDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fae6ba82-222e-11ed-9250-1f966e50362f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= invisiblethingslab.com; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1661182067; x=1661268467; bh=xj7bP3qj5V gPvcpXH0+9UDlbrSUvdrbGTDOQmCOLAFU=; b=lVdYxd7OMDMpoWEzPlSB6orzII svSLcmI7LAHqXmG9EElY31MfPQmwiSgCSddFJ2EAePlAEf63+hLRCLUfiBx01xoM gilIEG89lbJpdqxFcKmHAt5X8S5rm73waHCHzlWGZzjH7dBKNob2xG+bKjv8ZpBB Rn2XkQTmdDF/yKHeGFSmqyB74pCiGz+g/5cmWBJ1+pWT6Oi1lc6EWGYNMPaL+r32 ZSlUtWbhtrnnYpxN6CBo0zPnpGhAzS4F55tk1S0/4wA8pDV/vzrNMhC1UDyZlv+1 w2AOGPh/GA8nAk+mI4w7eWaF+zaY6tzhwUsBNLGDPSfFHosFjbE0sEOvt76A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1661182067; x= 1661268467; bh=xj7bP3qj5VgPvcpXH0+9UDlbrSUvdrbGTDOQmCOLAFU=; b=R b45L44hfa+9QOZ/zMt3kAlxTGY+9Fmf7KmjUeX/qfLLhwR8wQKdtMz+ZwDaL31Zs 4jCm1/pz5huCnFkMhlx1W5p0YS7V+94yxkQY45qy+m7WDbxc6rW1oxStOFwUE6T2 y4k1dINxltvQ9xDyAu5RDc+866q68ORu6jIg2LmSdGx4YxZIemfrPrtoMQLD5Li1 nV2c+PcI565KHmvoud0/6iofLqXbn8ECgm7neGXKp0qigGo1tKUzsac0EbQ2bUIi uqygazSI4gL7EumMtSlOV6KRMK4Jz63LDnLWHor08j5tEz6Vyk7IkG2tpxoUE02A ix/utwOvxMc4vG+f/wFnw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeijedgledtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpeforghr vghkucforghrtgiihihkohifshhkihdqifpkrhgvtghkihcuoehmrghrmhgrrhgvkhesih hnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhmqeenucggtffrrghtthgvrhhnpefg ueduhefgvdefheehudejheefudevueeghfekhfehleegveduteeuiedugffgffenucevlh hushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehmrghrmhgrrhgv khesihhnvhhishhisghlvghthhhinhhgshhlrggsrdgtohhm X-ME-Proxy: Feedback-ID: i1568416f:Fastmail From: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= To: xen-devel@lists.xenproject.org Cc: =?utf-8?q?Marek_Marczykowski-G=C3=B3recki?= , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v5 9/9] drivers/char: fix handling cable re-plug in XHCI console driver Date: Mon, 22 Aug 2022 17:27:09 +0200 Message-Id: X-Mailer: git-send-email 2.35.3 In-Reply-To: References: MIME-Version: 1.0 When cable is unplugged, dbc_ensure_running() correctly detects this situation (DBC_CTRL_DCR flag is clear), and prevent sending data immediately to the device. It gets only queued in work ring buffers. When cable is plugged in again, subsequent dbc_flush() will send the buffered data. But there is a corner case, where no subsequent data was buffered in the work buffer, but a TRB was still pending. Ring the doorbell to let the controller re-send them. For console output it is rare corner case (TRB is pending for a very short time), but for console input it is very normal case (there is always one pending TRB for input). Extract doorbell ringing into separate function to avoid duplication. Signed-off-by: Marek Marczykowski-Górecki --- xen/drivers/char/xhci-dbc.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c index e838e285fb75..0ff340dac103 100644 --- a/xen/drivers/char/xhci-dbc.c +++ b/xen/drivers/char/xhci-dbc.c @@ -553,6 +553,15 @@ static unsigned int dbc_work_ring_space_to_end(const struct dbc_work_ring *ring) return ring->deq - ring->enq; } +static void dbc_ring_doorbell(struct dbc *dbc, int doorbell) +{ + uint32_t __iomem *db_reg = &dbc->dbc_reg->db; + uint32_t db = (readl(db_reg) & ~DBC_DOORBELL_TARGET_MASK) | + (doorbell << DBC_DOORBELL_TARGET_SHIFT); + + writel(db, db_reg); +} + static void dbc_push_trb(struct dbc *dbc, struct xhci_trb_ring *ring, uint64_t dma, uint64_t len) { @@ -1023,6 +1032,8 @@ static bool dbc_ensure_running(struct dbc *dbc) writel(ctrl | (1U << DBC_CTRL_DRC), ®->ctrl); writel(readl(®->portsc) | (1U << DBC_PSC_PED), ®->portsc); wmb(); + dbc_ring_doorbell(dbc, dbc->dbc_iring.db); + dbc_ring_doorbell(dbc, dbc->dbc_oring.db); } return true; @@ -1040,10 +1051,6 @@ static bool dbc_ensure_running(struct dbc *dbc) static void dbc_flush(struct dbc *dbc, struct xhci_trb_ring *trb, struct dbc_work_ring *wrk) { - struct dbc_reg *reg = dbc->dbc_reg; - uint32_t db = (readl(®->db) & ~DBC_DOORBELL_TARGET_MASK) | - (trb->db << DBC_DOORBELL_TARGET_SHIFT); - if ( xhci_trb_ring_full(trb) ) return; @@ -1066,8 +1073,7 @@ static void dbc_flush(struct dbc *dbc, struct xhci_trb_ring *trb, } } - wmb(); - writel(db, ®->db); + dbc_ring_doorbell(dbc, trb->db); } /**