From patchwork Mon Aug 1 21:12:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AF37BC19F2C for ; Mon, 1 Aug 2022 21:15:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vWnG1o3p9pwbhHhzUtJH4GCxi8EMx9WfShxB5eTtRhU=; b=2hbpmCud9kf5EF G9rX44SQEz+kJpCvNGhIzJrxn4Ogtb0TREqWBTgDzQ0GHHQ9mEc/3Yv2L1xgdmh1uxRegSBD7cE7l gy7a8dk+nLJET2gsAPPooLLGsoMVoCr5NxGU4SeyC2O+kdPHCslMpfUigJvF65IIDPPaeFwCUG0jk KPRdm8C3FlMGoQ9Fpmpm3YP8ehr79qWDhfchj0OpQkTWCq7Q0yEaU/5YGDC4Ve/Y++04Ghj3oXCfZ sVOgcDOZaiTYILJ9e8/KjtGVGOQljwhiAXjkFDZwOM2fiQEPrToCfnMKTzyHWwDvgSEZQgYRoT8H9 EioXgEqEfffpEraDXdtg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjx-00AhMC-Lf; Mon, 01 Aug 2022 21:14:17 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjh-00Ah74-Jp for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388441; x=1690924441; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9xD/zs9qtNWomxU7JBnnn4p050BQ9Xldp5Az/0aV2IA=; b=l0uc7Ftr+ajYQZncHAOO1O9eygqCftRM99g50XwPTqhN+oPUW6FYAHV2 qyYNwDvu9t6gyAXQ9BgaTOU8ds7j81WyueZ1ObXvN4OUE/Y80ZRlCgS94 x6Tlmh5XF23omyWxjASFuRG8OuPkuHwg/j2qzRec68r9DmNb/hAlW57lt k=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:56 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:56 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:55 -0700 From: Elliot Berman To: Bjorn Andersson , , Jonathan Corbet CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Will Deacon , Catalin Marinas , , Subject: [PATCH v2 01/11] docs: gunyah: Introduce Gunyah Hypervisor Date: Mon, 1 Aug 2022 14:12:30 -0700 Message-ID: <20220801211240.597859-2-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141401_733228_390B08F8 X-CRM114-Status: GOOD ( 26.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Signed-off-by: Elliot Berman Reported-by: kernel test robot --- Documentation/virt/gunyah/index.rst | 92 +++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 52 ++++++++++++ Documentation/virt/index.rst | 1 + MAINTAINERS | 7 ++ 4 files changed, 152 insertions(+) create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst new file mode 100644 index 000000000000..e7bb2b14543e --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,92 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Gunyah Hypervisor +================= + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in +a higher CPU privilege level. It does not depend on any lower-privileged operating system +for its core functionality. This increases its security and can support a much smaller +trusted computing base than a Type-2 hypervisor. + +Gunyah is an open source hypervisor. The source repo is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + A scheduler for virtual CPUs (VCPUs) on physical CPUs and enables time-sharing + of the CPUs. +- Memory Management: + APIs handling memory, abstracted as objects, limiting direct use of physical + addresses. Memory ownership and usage tracking of all memory under its control. + Memory partitioning between VMs is a fundamental security feature. +- Interrupt Virtualization: + Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled + in the hypervisor and routed to the assigned VM. +- Inter-VM Communication: + There are several different mechanisms provided for communicating between VMs. +- Virtual platform: + Architectural devices such as interrupt controllers and CPU timers are directly provided + by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI. +- Device Virtualization: + Para-virtualization of devices is supported using inter-VM communication. + +Architectures supported +======================= +AArch64 with a GIC + +Resources and Capabilities +========================== + +Some services or resources provided by the Gunyah hypervisor are described by capability IDs. +For instance, inter-VM communication is performed with doorbells and message queues. The specific +instance of a doorbell is described by a capability ID. These devices are described in Linux as a +struct gunyah_device. + +High level management of these resources is performed by the resource manager VM. RM informs a +guest VM about resources it can access through either the device tree or via guest-initiated RPC. + +Resource Manager +================ + +The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor. +It provides policy enforcement aspects of the virtualization system. The resource manager can +be treated as an extension of the Hypervisor but is separated to its own partition to ensure +that the hypervisor layer itself remains small and secure and to maintain a separation of policy +and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines. + +Communication with the resource manager from each guest VM happens with message-queue.rst. Details +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c + +:: + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \==========/ | | + | \========================/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- Generate device-tree overlay +- VM creation and deletion +- VM device-tree management +- VM access control policy +- Interrupt routing configuration + +When booting a virtual machine which uses a devicetree, resource manager overlays a +/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM, +how to communicate with resource manager, and basic description and capabilities of +this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description +of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst new file mode 100644 index 000000000000..afd405f3a5e1 --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,52 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +============== +Message queue is a simple low-capacity IPC channel between two VMs. It is +intended for sending small control and configuration messages. Each message +queue object is unidirectional, so a full-duplex IPC channel requires a pair of +objects. + +Messages can be up to 240 bytes in length. Longer messages require a further +protocol on top of the message queue messages themselves. For instance, communication +with the resource manager adds a header field for sending longer messages via multiple +message fragments. + +The diagram below shows how message queue works. A typical configuration involves +2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message +queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall + with the message to inform the hypervisor to add the message to + message queue 1's queue. +2. Gunyah raises the corresponding interrupt for VM_B when any of these happens: + a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case. + b. Explicility with gh_msgq_push command from VM_A. + c. Message queue has reached a threshold depth. +3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer. + +For VM_B to send a message to VM_A, the process is identical, except that hypercalls +reference message queue 2's capability ID. + +:: + + +---------------+ +-----------------+ +---------------+ + | VM_A | |Gunyah hypervisor| | VM_B | + | | | | | | + | | | | | | + | | Tx | | | | + | |-------->| | Rx vIRQ | | + |gh_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gh_msgq_recv() | + | |<------- | | | | + | | | | | | + | Message Queue | | | | Message Queue | + | driver | | | | driver | + | | | | | | + | | | | | | + | | | | Tx | | + | | Rx vIRQ | |<--------| | + |gh_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gh_msgq_send() | + | | | |-------->| | + | | | | | | + | | | | | | + +---------------+ +-----------------+ +---------------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index 492f0920b988..dd4e8ef284eb 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -14,6 +14,7 @@ Linux Virtualization Support ne_overview acrn/index coco/sev-guest + gunyah/index .. only:: html and subproject diff --git a/MAINTAINERS b/MAINTAINERS index 04ec80ee7352..18fb034526e1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8736,6 +8736,13 @@ L: linux-efi@vger.kernel.org S: Maintained F: block/partitions/efi.* +GUNYAH HYPERVISOR DRIVER +M: Elliot Berman +M: Murali Nalajala +L: linux-arm-msm@vger.kernel.org +S: Maintained +F: Documentation/virt/gunyah/ + HABANALABS PCI DRIVER M: Oded Gabbay S: Supported From patchwork Mon Aug 1 21:12:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6DB1C3F6B0 for ; Mon, 1 Aug 2022 21:15:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OeQ42FdwsnLdC7MpiIcGYNa4mh+lLb34pgLKwTXEnNg=; b=sUVMfEc60N7dI/ b3GzrgVCl88E/pKzDb4kjMBb4/hQnCfEZbaPDnDmJMj+EHjnoRkDTtg/9RwtRSes0E37SED+N72Y7 rhU70/P8XFJvK6ue/jIlXVFw4ECxH+U8FLzLDvjccx0vTjsfn1KJWNEkcNP4Q2s723diUPkX89nBL 5i1xDPcZjfoaNRBBoduC3e+95xqVcRAn6+ure02QmUd1AZaQ7l2scDfpVUGN8wWqR+OT+Bktvo1Os JGuVVsgdwDnRGzoVZDxy+2XbnZqq406xWaI2EJAsQpNDU35/GgF7FdylvxVP99r4keD3ailIL6lwq MyDA1LM5HenXB4M2C9vQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIck6-00AhSL-71; Mon, 01 Aug 2022 21:14:26 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcji-00Ah8U-CL for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388442; x=1690924442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vIfmWBafylqnlRoEuinALPR3jmyroFOxK4aJ7M4ecys=; b=XDEg3iAGJZ0nVlwu2CptK3ainxwtohxQEidRSyG9TfXCsX8UnF0KvmaU gNFbPMnbOEqYAEOA8puQIq32QCkZeYXAeHB/ecn8LcCgyxM/WTPJgGTvh wC0L+R1Arc3tMEv1eWfU8UPHLqYKMzDXwNikeXt9Ih+wfwjHpYVogaaQ+ s=; Received: from unknown (HELO ironmsg02-sd.qualcomm.com) ([10.53.140.142]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:57 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg02-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:56 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:56 -0700 From: Elliot Berman To: Bjorn Andersson , Rob Herring , Krzysztof Kozlowski , CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Jonathan Corbet , Will Deacon , Catalin Marinas , , Subject: [PATCH v2 02/11] dt-bindings: Add binding for gunyah hypervisor Date: Mon, 1 Aug 2022 14:12:31 -0700 Message-ID: <20220801211240.597859-3-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141402_496957_69A23160 X-CRM114-Status: GOOD ( 17.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When Linux is booted as a guest under the Gunyah hypervisor, Gunyah applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Signed-off-by: Elliot Berman --- .../bindings/firmware/gunyah-hypervisor.yaml | 84 +++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 85 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..e50d932e768c --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,84 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Murali Nalajala + - Elliot Berman + +description: |+ + On systems which support devicetree, Gunyah generates and overlays a deviceetree overlay which + describes the basic configuration of the hypervisor. Virtual machines use this information for + initial discovery that they are running as a Gunyah guest VM. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + oneOf: + - items: + - const: gunyah-hypervisor-1.0 + - const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + oneOf: + - items: + - const: gunyah-resource-manager-1-0 + - const: gunyah-resource-manager + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + additionalProperties: false + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor-1.0", "gunyah-hypervisor"; + + gunyah-resource-mgr@1 { + compatible = "gunyah-resource-manager-1-0", "gunyah-resource-manager"; + interrupts = , /* TX full IRQ */ + ; /* RX empty IRQ */ + reg = <0x00000000 0x00000000>, <0x00000000 0x00000001>; + /* TX, RX cap ids */ + }; + }; diff --git a/MAINTAINERS b/MAINTAINERS index 18fb034526e1..0cd12ea6c11c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8741,6 +8741,7 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Maintained +F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ HABANALABS PCI DRIVER From patchwork Mon Aug 1 21:12:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69EB2C00144 for ; Mon, 1 Aug 2022 21:15:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=POT126uA2r1RvAJCXKu1eQBgbgP4awHONDjsXZpp92Q=; b=uwEGKx1BV2bo1n 2ctlshSMLBWIkLW5JHyNg6WKF6UY9OIIq/IbAWbjWOQUa11ebm5MQBQ8Ym8Nm8QvE/wNPuJZXJiR2 FPzTiUzaexpR3U/Iez2xFLqUDKQ2Duxi6CpCN9TI8ermXFexJNXRXV6JR+kpL4ScKZdIE4iKcMdZI +FMkwZqORWTSiG6RPnK2O2cBsO4hwoB2s22y8+N2LSNXqXBETrHa1rw0i2AHnlnTflwlh65WdhB9b DbUPP8KXF9bJnhfxOgJLm4L503crw/P1b8pofrdHKjus8acqyoBGu96Jqie7HXcnJ0UmZbuKwEBDT wHDQCrTq8aqsUnsoRP9w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIckE-00AhXF-SQ; Mon, 01 Aug 2022 21:14:35 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjj-00Ah6P-Uz for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388443; x=1690924443; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2wwXDsSDy8neRX8+Ed9SYEvVYy0LIc3XuXU/Jlw0R6s=; b=gXy5rb1XWJGnSXXDVNVc8DzNfZo+lpNjZrpP5X/Wyz5wym3LOFc/DIa1 wbMmmtlZTnCzpqxYwL8gg3KiQdOKuCP82ffeYaXWHf1dxaE/T10b2px4h Q8WMZdyOIX3EvZxq+zBTRM421e9JfVuI3f7ftEmXvOkzjwkaSpioAR7GR c=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:57 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:57 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:56 -0700 From: Elliot Berman To: Bjorn Andersson , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Rob Herring , Krzysztof Kozlowski , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 03/11] arm64: gunyah: Add Gunyah hypercalls ABI Date: Mon, 1 Aug 2022 14:12:32 -0700 Message-ID: <20220801211240.597859-4-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141404_084854_A783015D X-CRM114-Status: GOOD ( 17.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add initial support to perform Gunyah hypercalls. The arm64 ABI for Gunyah hypercalls generally follows the SMC Calling Convention. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + arch/arm64/include/asm/gunyah.h | 134 ++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 arch/arm64/include/asm/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index 0cd12ea6c11c..02f97ac90cdf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8743,6 +8743,7 @@ L: linux-arm-msm@vger.kernel.org S: Maintained F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ +F: arch/arm64/include/asm/gunyah.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h new file mode 100644 index 000000000000..4820e9389f40 --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,134 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __ASM_GUNYAH_H +#define __ASM_GUNYAH_H + +#include +#include + +#define GH_CALL_TYPE_PLATFORM_CALL 0 +#define GH_CALL_TYPE_HYPERCALL 2 +#define GH_CALL_TYPE_SERVICE 3 +#define GH_CALL_TYPE_SHIFT 14 +#define GH_CALL_FUNCTION_NUM_MASK 0x3fff + +#define GH_SERVICE(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + (GH_CALL_TYPE_SERVICE << GH_CALL_TYPE_SHIFT) \ + | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) + +#define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + (GH_CALL_TYPE_HYPERCALL << GH_CALL_TYPE_SHIFT) \ + | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) + +#define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x + +#define __gh_count_args(...) \ + ___gh_count_args(_, ## __VA_ARGS__, 8, 7, 6, 5, 4, 3, 2, 1, 0) + +#define __gh_skip_0(...) __VA_ARGS__ +#define __gh_skip_1(a, ...) __VA_ARGS__ +#define __gh_skip_2(a, b, ...) __VA_ARGS__ +#define __gh_skip_3(a, b, c, ...) __VA_ARGS__ +#define __gh_skip_4(a, b, c, d, ...) __VA_ARGS__ +#define __gh_skip_5(a, b, c, d, e, ...) __VA_ARGS__ +#define __gh_skip_6(a, b, c, d, e, f, ...) __VA_ARGS__ +#define __gh_skip_7(a, b, c, d, e, f, g, ...) __VA_ARGS__ +#define __gh_skip_8(a, b, c, d, e, f, g, h, ...) __VA_ARGS__ +#define __gh_to_res(nargs, ...) __gh_skip_ ## nargs (__VA_ARGS__) + +#define __gh_declare_arg_0(...) + +#define __gh_declare_arg_1(arg1, ...) \ + .a1 = (arg1) + +#define __gh_declare_arg_2(arg1, arg2, ...) \ + __gh_declare_arg_1(arg1), \ + .a2 = (arg2) + +#define __gh_declare_arg_3(arg1, arg2, arg3, ...) \ + __gh_declare_arg_2(arg1, arg2), \ + .a3 = (arg3) + +#define __gh_declare_arg_4(arg1, arg2, arg3, arg4, ...) \ + __gh_declare_arg_3(arg1, arg2, arg3), \ + .a4 = (arg4) + +#define __gh_declare_arg_5(arg1, arg2, arg3, arg4, arg5, ...) \ + __gh_declare_arg_4(arg1, arg2, arg3, arg4), \ + .a5 = (arg5) + +#define __gh_declare_arg_6(arg1, arg2, arg3, arg4, arg5, arg6, ...) \ + __gh_declare_arg_5(arg1, arg2, arg3, arg4, arg5), \ + .a6 = (arg6) + +#define __gh_declare_arg_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7, ...) \ + __gh_declare_arg_6(arg1, arg2, arg3, arg4, arg5, arg6), \ + .a7 = (arg7) + +#define __gh_declare_arg_8(arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, ...) \ + __gh_declare_arg_7(arg1, arg2, arg3, arg4, arg5, arg6, arg7), \ + .a8 = (arg8) + +#define ___gh_declare_args(nargs) __gh_declare_arg_ ## nargs +#define __gh_declare_args(nargs) ___gh_declare_args(nargs) +#define _gh_declare_args(nargs, ...) __gh_declare_args(nargs)(__VA_ARGS__) + +#define __gh_assign_res_0(...) + +#define __gh_assign_res_1(r1) \ + r1 = __res.a0 + +#define __gh_assign_res_2(r1, r2) \ + __gh_assign_res_1(r1); \ + r2 = __res.a1 + +#define __gh_assign_res_3(r1, r2, r3) \ + __gh_assign_res_2(r1, r2); \ + r3 = __res.a2 + +#define __gh_assign_res_4(r1, r2, r3, r4) \ + __gh_assign_res_3(r1, r2, r3); \ + r4 = __res.a3 + +#define __gh_assign_res_5(r1, r2, r3, r4, r5) \ + __gh_assign_res_4(r1, r2, r3, r4); \ + r5 = __res.a4 + +#define __gh_assign_res_6(r1, r2, r3, r4, r5, r6) \ + __gh_assign_res_5(r1, r2, r3, r4, r5); \ + r6 = __res.a5 + +#define __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7) \ + __gh_assign_res_6(r1, r2, r3, r4, r5, r6); \ + r7 = __res.a6 + +#define __gh_assign_res_8(r1, r2, r3, r4, r5, r6, r7, r8) \ + __gh_assign_res_7(r1, r2, r3, r4, r5, r6, r7); \ + r8 = __res.a7 + +#define ___gh_assign_res(nargs) __gh_assign_res_ ## nargs +#define __gh_assign_res(nargs) ___gh_assign_res(nargs) +#define _gh_assign_res(...) __gh_assign_res(__gh_count_args(__VA_ARGS__))(__VA_ARGS__) + +/** + * arch_gh_hypercall() - Performs an AArch64-specific call into hypervisor using Gunyah ABI + * @hcall_num: Hypercall function ID to invoke + * @nargs: Number of input arguments + * @...: First nargs are the input arguments. Remaining arguments are output variables. + */ +#define arch_gh_hypercall(hcall_num, nargs, ...) \ + do { \ + struct arm_smccc_1_2_regs __res; \ + struct arm_smccc_1_2_regs __args = { \ + .a0 = hcall_num, \ + _gh_declare_args(nargs, __VA_ARGS__) \ + }; \ + arm_smccc_1_2_hvc(&__args, &__res); \ + _gh_assign_res(__gh_to_res(nargs, __VA_ARGS__)); \ + } while (0) + +#endif From patchwork Mon Aug 1 21:12:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12EF3C00144 for ; Mon, 1 Aug 2022 21:15:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5avHQD3KIp2kN9zRZ73rjhzbIEkBcWFBTf6MAp/up4w=; b=4HwywpdPyWREWl C9F0Scd+2G0OAvhjEd7rOGOYDTDybLqZED2qV8timQq5v7okRQHfg+jPKu1uTkH68ggZHrCFNRDjP ncl7dPFHtHg6WBEag3DcO8YymkZDrDiJBUwEslOR3A+rNxOlT6TFubMVOTrFv3k+XeI4InuhebGc5 bxBzqXBox8EKhrJgYhSNkcJIo92M+ccR7OgQk0P+LGgnGQkXhkdnmUn8UiVMHTaSTTWxuC6gO4jpK eLi2IMy98Ocn/OqIxhqmVaUWpOIE0LPyfQfc1r4fmolruOBo0GiyudRjtLgB7AYUHqF7N1c+N1HxB TlHsX20osKxm4pgTLIbg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIckN-00AheI-Us; Mon, 01 Aug 2022 21:14:44 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjk-00Ah74-2W for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388444; x=1690924444; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8Zu1w/0V0h7dug7aLTMZQuEKcNOe7CoCHzakPAKWRP0=; b=H++RttK6HwnwXqYk7Vh3sRUJN6qMHJ/Z+DLUUViTmwQZZc9f/dZ39vl5 tUqr3/IZFLtQSTGN3HqDEvVGkS/CMf3py1Mv3zj6AMAzcVUIgysSuaZnR KKZNGKQQettEuPXnfHnOTAJRzALJN+3Bn4pL3OPCxgGQhjDPhmFruFwTp w=; Received: from unknown (HELO ironmsg05-sd.qualcomm.com) ([10.53.140.145]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:58 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg05-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:58 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:57 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 04/11] gunyah: Common types and error codes for Gunyah hypercalls Date: Mon, 1 Aug 2022 14:12:33 -0700 Message-ID: <20220801211240.597859-5-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141404_213525_9A4BB58D X-CRM114-Status: GOOD ( 15.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Signed-off-by: Elliot Berman --- MAINTAINERS | 1 + include/linux/gunyah.h | 75 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/MAINTAINERS b/MAINTAINERS index 02f97ac90cdf..2e4f1d9ed47b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8744,6 +8744,7 @@ S: Maintained F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah.h +F: include/linux/gunyah.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..69931a0f5736 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,75 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_H +#define _GUNYAH_H + +#include +#include +#include + +typedef u64 gh_capid_t; + +/* Common Gunyah macros */ +#define GH_CAPID_INVAL U64_MAX + +#define GH_ERROR_OK 0 +#define GH_ERROR_UNIMPLEMENTED -1 +#define GH_ERROR_RETRY -2 + +#define GH_ERROR_ARG_INVAL 1 +#define GH_ERROR_ARG_SIZE 2 +#define GH_ERROR_ARG_ALIGN 3 + +#define GH_ERROR_NOMEM 10 + +#define GH_ERROR_ADDR_OVFL 20 +#define GH_ERROR_ADDR_UNFL 21 +#define GH_ERROR_ADDR_INVAL 22 + +#define GH_ERROR_DENIED 30 +#define GH_ERROR_BUSY 31 +#define GH_ERROR_IDLE 32 + +#define GH_ERROR_IRQ_BOUND 40 +#define GH_ERROR_IRQ_UNBOUND 41 + +#define GH_ERROR_CSPACE_CAP_NULL 50 +#define GH_ERROR_CSPACE_CAP_REVOKED 51 +#define GH_ERROR_CSPACE_WRONG_OBJ_TYPE 52 +#define GH_ERROR_CSPACE_INSUF_RIGHTS 53 +#define GH_ERROR_CSPACE_FULL 54 + +#define GH_ERROR_MSGQUEUE_EMPTY 60 +#define GH_ERROR_MSGQUEUE_FULL 61 + +static inline int gh_remap_error(int gh_error) +{ + switch (gh_error) { + case GH_ERROR_OK: + return 0; + case GH_ERROR_NOMEM: + return -ENOMEM; + case GH_ERROR_DENIED: + case GH_ERROR_CSPACE_CAP_NULL: + case GH_ERROR_CSPACE_CAP_REVOKED: + case GH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GH_ERROR_CSPACE_INSUF_RIGHTS: + case GH_ERROR_CSPACE_FULL: + return -EACCES; + case GH_ERROR_BUSY: + case GH_ERROR_IDLE: + return -EBUSY; + case GH_ERROR_IRQ_BOUND: + case GH_ERROR_IRQ_UNBOUND: + case GH_ERROR_MSGQUEUE_FULL: + case GH_ERROR_MSGQUEUE_EMPTY: + return -EPERM; + default: + return -EINVAL; + } +} + +#endif From patchwork Mon Aug 1 21:12:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA4F8C25B06 for ; Mon, 1 Aug 2022 21:15:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IinGH/sFUvVQ5vuhxoMnccGi6jLNoDK1lIIckUU7rH8=; b=by5meNilF4xI+S 3OlmNJnQGZqwoHPg7O9PlM3P+YR/RjqpJKDWoTTlQT1gTnyd/lhQ41BLYCoraz6NbQV1r09yMgWAz LXrZNsKSvtB46sjG8LrbgQt4aKvR6j6MD8whYLNa2YYbsksbcr3y0z5Uw+A/9BWHysQ7N8h9HEinb EHJEdwjxIxoypL7Dk0VTa+dMrH5bmw759dvra52RGh+F2+mzfLZt4Nz9vChqKH5sB7iCEF1HyhQ2X hxQNNxpHJi/EuWMdg279kPaDJKQAcO4XDJFclsugz7i2AqR0KpJSEB4qaNry9LjczpBDIUNanTvhh Uceq3JZzbvTQeFpf3Ewg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIckZ-00AhlG-3q; Mon, 01 Aug 2022 21:14:55 +0000 Received: from alexa-out-sd-02.qualcomm.com ([199.106.114.39]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcji-00Ah7A-EJ for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388442; x=1690924442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J9519SgLqABIUbX23vSTKPiAzykXv5PmWxk67sQPv0w=; b=CRWpLBUT6AY+XLnA6V/HD+k/8i0a3qJ/KKIHwbCGY8R4LXLAWw0Img9b EVdewIAN/0UwxneQtYu2sNT+qoYXPDhpmY3nMdmT6/Kt+JgqvJbbxWaAo TkCplKJVHfgBpMZZUkHKpBJ4tkP5Dp0Yiz3lqIMVET4dvzzT6CguYVrMo U=; Received: from unknown (HELO ironmsg01-sd.qualcomm.com) ([10.53.140.141]) by alexa-out-sd-02.qualcomm.com with ESMTP; 01 Aug 2022 14:13:58 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg01-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:58 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:57 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 05/11] virt: gunyah: Add sysfs nodes Date: Mon, 1 Aug 2022 14:12:34 -0700 Message-ID: <20220801211240.597859-6-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141402_570976_E0934F79 X-CRM114-Status: GOOD ( 23.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add /sys/hypervisor support when detecting that Linux is running in a Gunyah environment. Export the version of Gunyah which is reported via the hyp_identify hypercall. Signed-off-by: Elliot Berman --- .../ABI/testing/sysfs-hypervisor-gunyah | 37 +++++ MAINTAINERS | 2 + arch/arm64/include/asm/gunyah.h | 4 + drivers/virt/Kconfig | 1 + drivers/virt/Makefile | 1 + drivers/virt/gunyah/Kconfig | 13 ++ drivers/virt/gunyah/Makefile | 4 + drivers/virt/gunyah/sysfs.c | 139 ++++++++++++++++++ 8 files changed, 201 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-hypervisor-gunyah create mode 100644 drivers/virt/gunyah/Kconfig create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/sysfs.c diff --git a/Documentation/ABI/testing/sysfs-hypervisor-gunyah b/Documentation/ABI/testing/sysfs-hypervisor-gunyah new file mode 100644 index 000000000000..41bce1965c91 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-hypervisor-gunyah @@ -0,0 +1,37 @@ +What: /sys/hypervisor/type +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + Type of hypervisor: + "gunyah": Gunyah hypervisor + +What: /sys/hypervisor/features +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + Space separated list of features supported by Linux and Gunyah: + "cspace": Gunyah devices + "doorbell": Sending/receiving virtual interrupts via Gunyah doorbells + "message-queue": Sending/receiving messages via Gunyah message queues + "vic": Interrupt lending + "vpm": Virtual platform management + "vcpu": Virtual CPU management + "memextent": Memory lending/management + "trace": Gunyah hypervisor tracing + + +What: /sys/hypervisor/version/api +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + The Gunyah API version. + +What: /sys/hypervisor/version/variant +Date: August 2022 +KernelVersion: 5.20 +Contact: linux-arm-msm@vger.kernel.org +Description: If running under Gunyah: + The Gunyah variant (build) version. \ No newline at end of file diff --git a/MAINTAINERS b/MAINTAINERS index 2e4f1d9ed47b..e63c51ee1a2a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8741,9 +8741,11 @@ M: Elliot Berman M: Murali Nalajala L: linux-arm-msm@vger.kernel.org S: Maintained +F: Documentation/ABI/testing/sysfs-hypervisor-gunyah F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah.h +F: drivers/virt/gunyah/ F: include/linux/gunyah.h HABANALABS PCI DRIVER diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h index 4820e9389f40..3aee35009910 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -19,11 +19,15 @@ (GH_CALL_TYPE_SERVICE << GH_CALL_TYPE_SHIFT) \ | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) +#define GH_HYPERCALL_CALL_UID GH_SERVICE(0x3f01) + #define GH_HYPERCALL(fn) ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ ARM_SMCCC_OWNER_VENDOR_HYP, \ (GH_CALL_TYPE_HYPERCALL << GH_CALL_TYPE_SHIFT) \ | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) +#define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) + #define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x #define __gh_count_args(...) \ diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index 87ef258cec64..259dc2be6cad 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -52,4 +52,5 @@ source "drivers/virt/coco/efi_secret/Kconfig" source "drivers/virt/coco/sev-guest/Kconfig" +source "drivers/virt/gunyah/Kconfig" endif diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index 093674e05c40..ec4fdfa8eef1 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -6,6 +6,7 @@ obj-$(CONFIG_FSL_HV_MANAGER) += fsl_hypervisor.o obj-$(CONFIG_VMGENID) += vmgenid.o obj-y += vboxguest/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..1e493017885e --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + select SYS_HYPERVISOR + help + The Gunyah drivers are the helper interfaces that runs in a guest VM + such as basic inter-VM IPC and signaling mechanisms and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..0aa086f9149f --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only + +gunyah-y += sysfs.o +obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c new file mode 100644 index 000000000000..253433a939cf --- /dev/null +++ b/drivers/virt/gunyah/sysfs.c @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gunyah: " fmt + +#include +#include +#include +#include +#include +#include + +#define QC_HYP_UID0 0x19bd54bd +#define QC_HYP_UID1 0x0b37571b +#define QC_HYP_UID2 0x946f609b +#define QC_HYP_UID3 0x54539de6 + +#define GUNYAH_UID0 0x673d5f14 +#define GUNYAH_UID1 0x9265ce36 +#define GUNYAH_UID2 0xa4535fdb +#define GUNYAH_UID3 0xc1d58fcd + +#define gh_uid_matches(prefix, uid) \ + ((uid)[0] == prefix ## _UID0 && (uid)[1] == prefix ## _UID1 && \ + (uid)[2] == prefix ## _UID2 && (uid)[3] == prefix ## _UID3) + +#define GH_API_INFO_API_VERSION(x) (((x) >> 0) & 0x3fff) +#define GH_API_INFO_BIG_ENDIAN(x) (((x) >> 14) & 1) +#define GH_API_INFO_IS_64BIT(x) (((x) >> 15) & 1) +#define GH_API_INFO_VARIANT(x) (((x) >> 56) & 0xff) + +#define GH_IDENTIFY_PARTITION_CSPACE(flags) (((flags)[0] >> 0) & 1) +#define GH_IDENTIFY_DOORBELL(flags) (((flags)[0] >> 1) & 1) +#define GH_IDENTIFY_MSGQUEUE(flags) (((flags)[0] >> 2) & 1) +#define GH_IDENTIFY_VIC(flags) (((flags)[0] >> 3) & 1) +#define GH_IDENTIFY_VPM(flags) (((flags)[0] >> 4) & 1) +#define GH_IDENTIFY_VCPU(flags) (((flags)[0] >> 5) & 1) +#define GH_IDENTIFY_MEMEXTENT(flags) (((flags)[0] >> 6) & 1) +#define GH_IDENTIFY_TRACE_CTRL(flags) (((flags)[0] >> 7) & 1) + +struct gh_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static struct gh_hypercall_hyp_identify_resp gunyah_api; + +static ssize_t type_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "gunyah\n"); +} +static struct kobj_attribute type_attr = __ATTR_RO(type); + +static ssize_t api_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_API_VERSION(gunyah_api.api_info)); +} +static struct kobj_attribute api_attr = __ATTR_RO(api); + +static ssize_t variant_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "%d\n", (int)GH_API_INFO_VARIANT(gunyah_api.api_info)); +} +static struct kobj_attribute variant_attr = __ATTR_RO(variant); + +static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) +{ + return sysfs_emit(buffer, "\n"); +} +static struct kobj_attribute features_attr = __ATTR_RO(features); + +static struct attribute *version_attrs[] = { + &api_attr.attr, + &variant_attr.attr, + NULL +}; + +static const struct attribute_group version_group = { + .name = "version", + .attrs = version_attrs, +}; + +static int __init gh_sysfs_register(void) +{ + int ret; + + ret = sysfs_create_file(hypervisor_kobj, &type_attr.attr); + if (ret) + return ret; + + ret = sysfs_create_group(hypervisor_kobj, &version_group); + if (ret) + return ret; + + return sysfs_create_file(hypervisor_kobj, &features_attr.attr); +} + +static void gh_sysfs_unregister(void) +{ + sysfs_remove_file(hypervisor_kobj, &type_attr.attr); + sysfs_remove_group(hypervisor_kobj, &version_group); +} + +static int __init gunyah_init(void) +{ + unsigned long uid[4]; + + arch_gh_hypercall(GH_HYPERCALL_CALL_UID, 0, uid[0], uid[1], uid[2], uid[3]); + + if (!(gh_uid_matches(GUNYAH, uid) || gh_uid_matches(QC_HYP, uid))) + return 0; + + arch_gh_hypercall(GH_HYPERCALL_HYP_IDENTIFY, 0, gunyah_api.api_info, + gunyah_api.flags[0], gunyah_api.flags[1], gunyah_api.flags[2]); + + if (GH_API_INFO_API_VERSION(gunyah_api.api_info) != 1) { + pr_warn("Unrecognized gunyah version: %llu. Currently supported: 1\n", + GH_API_INFO_API_VERSION(gunyah_api.api_info)); + return 0; + } + + pr_notice("Running under Gunyah hypervisor v%lld/%llx\n", + GH_API_INFO_API_VERSION(gunyah_api.api_info), + GH_API_INFO_VARIANT(gunyah_api.api_info)); + + return gh_sysfs_register(); +} +module_init(gunyah_init); + +static void __exit gunyah_exit(void) +{ + gh_sysfs_unregister(); +} +module_exit(gunyah_exit); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Driver"); From patchwork Mon Aug 1 21:12:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E2DFC00144 for ; Mon, 1 Aug 2022 21:16:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rcinRtaqKDCxBKdd+NhL6cEF9u4BP2zsLmfsbR4p41A=; b=Vv4esfSCSE9IoB 4sZIVVKeaYKMEONw5XSyVJ13tpgvZkuHax66tdCYmouBveJJM/WvVj0lmJ3VDAQCOWjiernCTf+gA R5IE6kyj3V+wE+45nrlekdW6KH/Fq01eYzMnwY+nJop9Pf8WzwM7VsGbY8nkfyNI44laTy1+ublxi 0vfQF6CXq/M0oneaGfPaL9paaJEZpAwP2TxXvaJ4bHc5y+AzkiK4O33yKhznHVxngoC9EKABuf0d8 p/C7Z27khBbGr9cSnQI8kF3trrYwkYPkU+4YPaPQvf63bUJNIrNb6aAZEZr2GQQBwP/ul5+0X3orU RDGbYtgUJEtTFK5A2Nxw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIckk-00AhuL-BG; Mon, 01 Aug 2022 21:15:06 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjl-00Ah8U-J0 for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388445; x=1690924445; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wckTDe8qexURvwCtE2aZc4qm1X8M+6zM/y6Z0jnbyIo=; b=hFSo/Kru2rfCCR2lZE1syh0ZYe81zOBb1MXLJyU6jptZ11PXPDnnPNIG +foX9yW+h0kOL0WkDHBaoneXYsUNgvLbK7S/Fykf0jDWv7CnAYk6hLvaK Dt7NKOPIHdv6xHaMer4Bmx6U3+EcJky4GTmCzUJ9Qs/mDDwng5qMaoWt+ Q=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:13:59 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:59 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:58 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 06/11] virt: gunyah: Add capabilities bus and devices Date: Mon, 1 Aug 2022 14:12:35 -0700 Message-ID: <20220801211240.597859-7-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141405_687495_7DBBBB46 X-CRM114-Status: GOOD ( 25.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Some resources provided by the Gunyah hypervisor are described as objects. The objects are identified with a capability ID. For instance, Inter-VM communication is performed with doorbells and message queues. Each doorbell and message queue endpoint can be described consisely as a Linux device. These resources are discovered either on the devicetree or reported by the Resource Manager. Devices on the Gunyah bus are matched with drivers according to the type ID reported by resource manager. Most resources will be discovered directly from the resource manager, so matching directly on type ID seems like sensible design. Each resource may also optionally have an interrupt associated with it and a known partner VM (e.g. which VM is the receiver of a message queue). Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/device.c | 108 +++++++++++++++++++++++++++ drivers/virt/gunyah/gunyah_private.h | 12 +++ drivers/virt/gunyah/sysfs.c | 25 ++++++- include/linux/gunyah.h | 45 +++++++++++ 5 files changed, 189 insertions(+), 3 deletions(-) create mode 100644 drivers/virt/gunyah/device.c create mode 100644 drivers/virt/gunyah/gunyah_private.h diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 0aa086f9149f..3869fb7371df 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -gunyah-y += sysfs.o +gunyah-y += sysfs.o device.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/device.c b/drivers/virt/gunyah/device.c new file mode 100644 index 000000000000..93595f9a65b9 --- /dev/null +++ b/drivers/virt/gunyah/device.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "ghdev: " fmt + +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +static int gunyah_match(struct device *dev, struct device_driver *drv) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(drv); + + return ghdev->type == ghdrv->type; +} + +static int gunyah_probe(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver); + + return ghdrv->probe ? ghdrv->probe(ghdev) : 0; +} + +static void gunyah_remove(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + struct gunyah_driver *ghdrv = to_gunyah_driver(dev->driver); + + if (ghdrv->remove) + ghdrv->remove(ghdev); +} + +static struct bus_type gunyah_bus = { + .name = "gunyah", + .match = gunyah_match, + .probe = gunyah_probe, + .remove = gunyah_remove, +}; + +int gunyah_register_driver(struct gunyah_driver *ghdrv) +{ + ghdrv->driver.bus = &gunyah_bus; + return driver_register(&ghdrv->driver); +} + +void gunyah_unregister_driver(struct gunyah_driver *ghdrv) +{ + driver_unregister(&ghdrv->driver); +} + +static void gunyah_device_release(struct device *dev) +{ + struct gunyah_device *ghdev = to_gunyah_device(dev); + + kfree(ghdev); +} + +struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type) +{ + struct gunyah_device *ghdev; + + ghdev = kzalloc(sizeof(*ghdev), GFP_KERNEL); + if (!ghdev) + return NULL; + + ghdev->capid = capid; + ghdev->type = type; + ghdev->irq = IRQ_NOTCONNECTED; + ghdev->dev.parent = parent; + ghdev->dev.release = gunyah_device_release; + ghdev->dev.bus = &gunyah_bus; + device_initialize(&ghdev->dev); + return ghdev; +} + +int gunyah_device_add(struct gunyah_device *ghdev) +{ + int ret; + + ret = dev_set_name(&ghdev->dev, "%u.%08llx", ghdev->type, ghdev->capid); + if (ret) + return ret; + + return device_add(&ghdev->dev); +} + +void gunyah_device_remove(struct gunyah_device *ghdev) +{ + device_unregister(&ghdev->dev); +} + +int __init gunyah_bus_init(void) +{ + return bus_register(&gunyah_bus); +} + +void gunyah_bus_exit(void) +{ + bus_unregister(&gunyah_bus); +} diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h new file mode 100644 index 000000000000..5f3832608020 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_private.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_PRIVATE_H +#define _GUNYAH_PRIVATE_H + +int __init gunyah_bus_init(void); +void gunyah_bus_exit(void); + +#endif diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 253433a939cf..220560cb3b1c 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -12,6 +12,8 @@ #include #include +#include "gunyah_private.h" + #define QC_HYP_UID0 0x19bd54bd #define QC_HYP_UID1 0x0b37571b #define QC_HYP_UID2 0x946f609b @@ -67,7 +69,13 @@ static struct kobj_attribute variant_attr = __ATTR_RO(variant); static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, char *buffer) { - return sysfs_emit(buffer, "\n"); + int len = 0; + + if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags)) + len += sysfs_emit_at(buffer, len, "cspace "); + + len += sysfs_emit_at(buffer, len, "\n"); + return len; } static struct kobj_attribute features_attr = __ATTR_RO(features); @@ -105,6 +113,7 @@ static void gh_sysfs_unregister(void) static int __init gunyah_init(void) { + int ret; unsigned long uid[4]; arch_gh_hypercall(GH_HYPERCALL_CALL_UID, 0, uid[0], uid[1], uid[2], uid[3]); @@ -125,12 +134,24 @@ static int __init gunyah_init(void) GH_API_INFO_API_VERSION(gunyah_api.api_info), GH_API_INFO_VARIANT(gunyah_api.api_info)); - return gh_sysfs_register(); + ret = gh_sysfs_register(); + if (ret) + return ret; + + ret = gunyah_bus_init(); + if (ret) + goto err_sysfs; + + return ret; +err_sysfs: + gh_sysfs_unregister(); + return ret; } module_init(gunyah_init); static void __exit gunyah_exit(void) { + gunyah_bus_exit(); gh_sysfs_unregister(); } module_exit(gunyah_exit); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 69931a0f5736..ce35f4491773 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,6 +6,7 @@ #ifndef _GUNYAH_H #define _GUNYAH_H +#include #include #include #include @@ -72,4 +73,48 @@ static inline int gh_remap_error(int gh_error) } } +/* Follows resource manager's resource types for VM_GET_HYP_RESOURCES */ +#define GUNYAH_DEVICE_TYPE_BELL_TX 0 +#define GUNYAH_DEVICE_TYPE_BELL_RX 1 +#define GUNYAH_DEVICE_TYPE_MSGQ_TX 2 +#define GUNYAH_DEVICE_TYPE_MSGQ_RX 3 +#define GUNYAH_DEVICE_TYPE_VCPU 4 + +struct gunyah_device { + u8 type; + gh_capid_t capid; + int irq; + + struct device dev; +}; + +#define to_gunyah_device(dev) container_of(dev, struct gunyah_device, dev) + +static inline void *ghdev_get_drvdata(const struct gunyah_device *ghdev) +{ + return dev_get_drvdata(&ghdev->dev); +} + +static inline void ghdev_set_drvdata(struct gunyah_device *ghdev, void *data) +{ + dev_set_drvdata(&ghdev->dev, data); +} + +struct gunyah_device *gunyah_device_alloc(struct device *parent, gh_capid_t capid, u8 type); + +int gunyah_device_add(struct gunyah_device *ghdev); +void gunyah_device_remove(struct gunyah_device *ghdev); + +struct gunyah_driver { + struct device_driver driver; + u8 type; + int (*probe)(struct gunyah_device *ghdev); + int (*remove)(struct gunyah_device *ghdev); +}; + +#define to_gunyah_driver(drv) container_of(drv, struct gunyah_driver, driver) + +int gunyah_register_driver(struct gunyah_driver *ghdrv); +void gunyah_unregister_driver(struct gunyah_driver *ghdrv); + #endif From patchwork Mon Aug 1 21:12:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ED34EC00144 for ; Mon, 1 Aug 2022 21:16:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jEIF0sAWoJBG8QNFCaP6pNy6hp930ZYQAwuaptMBdBU=; b=yBfpAktyuZmeoA CwEVMXfXa9nC0WhqkvFUGpxHz/YZY9jku0X6omlVVrAE8FCq5eonU1QG887vcmfMQdWLgrHNsPYvs 1J4JsOrN+omzGeQlZOX0xOdLMGE6IlBRRjJTvQ1fL6kjYWDT/1saavRxkACG0Jo61vylDrXQMt5Ks CCP0hJJbhu9r0LXY8kfMw18MpmGyIrQSb0iPDFmZ9rg0gaA9ORo/0DhWNGLI5nciJDJszjedsXWn2 LI17f/jhVFZOavWXprebuj30FmhsGoTAWxRMW++4wDJY0LSdvOAShci3BwHgrWRwQi3sCt2zp+FFy k6/qVBf3x9CV5LAC9hZw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIclG-00AiKX-K2; Mon, 01 Aug 2022 21:15:38 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjl-00Ah74-Q9 for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388445; x=1690924445; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SWnaPKeSYzf43pvBC3iC3Et/H+kvep4Pqvwdek+EBCk=; b=L1xcAIzWoS5K4kHczC7IZjW1w4n8x9KrzyhvJPAOFKsp2dytFTkcEgZX w8V0qgX8pgntNNYMlJ2vY6s2QD6mrsk1rq167ka+XzHxGoQeQZt9omG/i qZ7qz3UVxKeOcS68W5WsDn0Oxx8pWMuCgE8Nhh/yUIRfdphw4SIez9tpH I=; Received: from unknown (HELO ironmsg-SD-alpha.qualcomm.com) ([10.53.140.30]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:00 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg-SD-alpha.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:13:59 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:58 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 07/11] gunyah: msgq: Add Gunyah message queues Date: Mon, 1 Aug 2022 14:12:36 -0700 Message-ID: <20220801211240.597859-8-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141405_929848_52477E52 X-CRM114-Status: GOOD ( 31.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Gunyah message queues are unidirectional pipelines to communicate between 2 virtual machines, but are typically paired to allow bidirectional communication. The intended use case is for small control messages between 2 VMs, as they support a maximum of 240 bytes. Message queues can be discovered either by resource manager or on the devicetree. To support discovery on the devicetree, client drivers can use gh_msgq_platform_host_attach to allocate the tx and rx message queues according to Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml. Signed-off-by: Elliot Berman --- arch/arm64/include/asm/gunyah.h | 4 + drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/gunyah_private.h | 3 + drivers/virt/gunyah/msgq.c | 223 +++++++++++++++++++++++++++ drivers/virt/gunyah/sysfs.c | 9 ++ include/linux/gunyah.h | 13 ++ 6 files changed, 253 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/msgq.c diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h index 3aee35009910..ba7398bd851b 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -27,6 +27,10 @@ | ((fn) & GH_CALL_FUNCTION_NUM_MASK)) #define GH_HYPERCALL_HYP_IDENTIFY GH_HYPERCALL(0x0000) +#define GH_HYPERCALL_MSGQ_SEND GH_HYPERCALL(0x001B) +#define GH_HYPERCALL_MSGQ_RECV GH_HYPERCALL(0x001C) + +#define GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH BIT(0) #define ___gh_count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 3869fb7371df..94dc8e738911 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only -gunyah-y += sysfs.o device.o +gunyah-y += sysfs.o device.o msgq.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h index 5f3832608020..2ade32bd9bdf 100644 --- a/drivers/virt/gunyah/gunyah_private.h +++ b/drivers/virt/gunyah/gunyah_private.h @@ -9,4 +9,7 @@ int __init gunyah_bus_init(void); void gunyah_bus_exit(void); +int __init gh_msgq_init(void); +void gh_msgq_exit(void); + #endif diff --git a/drivers/virt/gunyah/msgq.c b/drivers/virt/gunyah/msgq.c new file mode 100644 index 000000000000..afc2572d3e7d --- /dev/null +++ b/drivers/virt/gunyah/msgq.c @@ -0,0 +1,223 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" + +struct gh_msgq { + bool ready; + wait_queue_head_t wq; + spinlock_t lock; +}; + +static irqreturn_t gh_msgq_irq_handler(int irq, void *dev) +{ + struct gh_msgq *msgq = dev; + + spin_lock(&msgq->lock); + msgq->ready = true; + spin_unlock(&msgq->lock); + wake_up_interruptible_all(&msgq->wq); + + return IRQ_HANDLED; +} + +static int __gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, u64 tx_flags) +{ + unsigned long flags, gh_error; + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + bool ready; + + spin_lock_irqsave(&msgq->lock, flags); + arch_gh_hypercall(GH_HYPERCALL_MSGQ_SEND, 5, + ghdev->capid, size, (uintptr_t)buff, tx_flags, 0, + gh_error, ready); + switch (gh_error) { + case GH_ERROR_OK: + ret = 0; + msgq->ready = ready; + break; + case GH_ERROR_MSGQUEUE_FULL: + ret = -EAGAIN; + msgq->ready = false; + break; + default: + ret = gh_remap_error(gh_error); + break; + } + spin_unlock_irqrestore(&msgq->lock, flags); + + return ret; +} + +/** + * gh_msgq_send() - Send a message to the client running on a different VM + * @client: The client descriptor that was obtained via gh_msgq_register() + * @buff: Pointer to the buffer where the received data must be placed + * @buff_size: The size of the buffer space available + * @flags: Optional flags to pass to receive the data. For the list of flags, + * see linux/gunyah/gh_msgq.h + * + * Returns: The number of bytes copied to buff. <0 if there was an error. + * + * Note: this function may sleep and should not be called from interrupt context + */ +ssize_t gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags) +{ + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + u64 tx_flags = 0; + + if (flags & GH_MSGQ_TX_PUSH) + tx_flags |= GH_HYPERCALL_MSGQ_SEND_FLAGS_PUSH; + + do { + ret = __gh_msgq_send(ghdev, buff, size, tx_flags); + + if (ret == -EAGAIN) { + if (flags & GH_MSGQ_NONBLOCK) + goto out; + if (wait_event_interruptible(msgq->wq, msgq->ready)) + ret = -ERESTARTSYS; + } + } while (ret == -EAGAIN); + +out: + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_send); + +static ssize_t __gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size) +{ + unsigned long flags, gh_error; + size_t recv_size; + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + bool ready; + + spin_lock_irqsave(&msgq->lock, flags); + + arch_gh_hypercall(GH_HYPERCALL_MSGQ_RECV, 4, + ghdev->capid, (uintptr_t)buff, size, 0, + gh_error, recv_size, ready); + switch (gh_error) { + case GH_ERROR_OK: + ret = recv_size; + msgq->ready = ready; + break; + case GH_ERROR_MSGQUEUE_EMPTY: + ret = -EAGAIN; + msgq->ready = false; + break; + default: + ret = gh_remap_error(gh_error); + break; + } + spin_unlock_irqrestore(&msgq->lock, flags); + + return ret; +} + +/** + * gh_msgq_recv() - Receive a message from the client running on a different VM + * @client: The client descriptor that was obtained via gh_msgq_register() + * @buff: Pointer to the buffer where the received data must be placed + * @buff_size: The size of the buffer space available + * @flags: Optional flags to pass to receive the data. For the list of flags, + * see linux/gunyah/gh_msgq.h + * + * Returns: The number of bytes copied to buff. <0 if there was an error. + * + * Note: this function may sleep and should not be called from interrupt context + */ +ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags) +{ + struct gh_msgq *msgq = ghdev_get_drvdata(ghdev); + ssize_t ret; + + do { + ret = __gh_msgq_recv(ghdev, buff, size); + + if (ret == -EAGAIN) { + if (flags & GH_MSGQ_NONBLOCK) + goto out; + if (wait_event_interruptible(msgq->wq, msgq->ready)) + ret = -ERESTARTSYS; + } + } while (ret == -EAGAIN); + +out: + return ret; +} +EXPORT_SYMBOL_GPL(gh_msgq_recv); + +static int gh_msgq_probe(struct gunyah_device *ghdev) +{ + struct gh_msgq *msgq; + + msgq = devm_kzalloc(&ghdev->dev, sizeof(*msgq), GFP_KERNEL); + if (!msgq) + return -ENOMEM; + ghdev_set_drvdata(ghdev, msgq); + + msgq->ready = true; /* Assume we can use the message queue right away */ + init_waitqueue_head(&msgq->wq); + spin_lock_init(&msgq->lock); + + return devm_request_irq(&ghdev->dev, ghdev->irq, gh_msgq_irq_handler, 0, + dev_name(&ghdev->dev), msgq); +} + +static struct gunyah_driver gh_msgq_tx_driver = { + .driver = { + .name = "gh_msgq_tx", + .owner = THIS_MODULE, + }, + .type = GUNYAH_DEVICE_TYPE_MSGQ_TX, + .probe = gh_msgq_probe, +}; + +static struct gunyah_driver gh_msgq_rx_driver = { + .driver = { + .name = "gh_msgq_rx", + .owner = THIS_MODULE, + }, + .type = GUNYAH_DEVICE_TYPE_MSGQ_RX, + .probe = gh_msgq_probe, +}; + +int __init gh_msgq_init(void) +{ + int ret; + + ret = gunyah_register_driver(&gh_msgq_tx_driver); + if (ret) + return ret; + + ret = gunyah_register_driver(&gh_msgq_rx_driver); + if (ret) + goto err_rx; + + return ret; +err_rx: + gunyah_unregister_driver(&gh_msgq_tx_driver); + return ret; +} + +void gh_msgq_exit(void) +{ + gunyah_unregister_driver(&gh_msgq_rx_driver); + gunyah_unregister_driver(&gh_msgq_tx_driver); +} diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 220560cb3b1c..7589689e5e92 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -73,6 +73,8 @@ static ssize_t features_show(struct kobject *kobj, struct kobj_attribute *attr, if (GH_IDENTIFY_PARTITION_CSPACE(gunyah_api.flags)) len += sysfs_emit_at(buffer, len, "cspace "); + if (GH_IDENTIFY_MSGQUEUE(gunyah_api.flags)) + len += sysfs_emit_at(buffer, len, "message-queue "); len += sysfs_emit_at(buffer, len, "\n"); return len; @@ -142,7 +144,13 @@ static int __init gunyah_init(void) if (ret) goto err_sysfs; + ret = gh_msgq_init(); + if (ret) + goto err_bus; + return ret; +err_bus: + gunyah_bus_exit(); err_sysfs: gh_sysfs_unregister(); return ret; @@ -151,6 +159,7 @@ module_init(gunyah_init); static void __exit gunyah_exit(void) { + gh_msgq_exit(); gunyah_bus_exit(); gh_sysfs_unregister(); } diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index ce35f4491773..099224f9d6d1 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,6 +6,7 @@ #ifndef _GUNYAH_H #define _GUNYAH_H +#include #include #include #include @@ -117,4 +118,16 @@ struct gunyah_driver { int gunyah_register_driver(struct gunyah_driver *ghdrv); void gunyah_unregister_driver(struct gunyah_driver *ghdrv); +#define GH_MSGQ_MAX_MSG_SIZE 1024 + +/* Possible flags to pass for Tx or Rx */ +#define GH_MSGQ_TX_PUSH BIT(0) +#define GH_MSGQ_NONBLOCK BIT(32) + +ssize_t gh_msgq_send(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags); +ssize_t gh_msgq_recv(struct gunyah_device *ghdev, void *buff, size_t size, + const unsigned long flags); + + #endif From patchwork Mon Aug 1 21:12:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2202C19F29 for ; Mon, 1 Aug 2022 21:17:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cOm7hYH8XSZc8+s/9ZuW3bGUPhhl75ms9RIw+0qS0wg=; b=OYUcRVekqHK7sA VR2dyWqREbq/qfWupvri+j7IbANuZzdH1MC4CG2CM81F+GPzDmlULzFLNa0pufBVmA1xrdxWa62Aj svztq2mVxh8T96McqUkm8VpRKdbXQPvZjF9BehM6xtrmyMtZ1ilYY2akGNJPz1dMCL2jBWCfwMQeQ R/5U0hQSNBT2kpnGYGekJjbQcIrhIPyrHA0BheyfBFD0qzpsjs00xukmOHfPzBTZ8tYoVWRV2AhE0 lgZl6ftSDUBlhdLEyyRjBrEf0uEafEZOPjAS5LJn2CJQ1w1TxfScF22qheMa860mcXGAlSMgyWGlT eQnQegUzQBDwXj0aNtsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIclf-00Aih4-DQ; Mon, 01 Aug 2022 21:16:03 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjl-00Ah6P-Rs for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388445; x=1690924445; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5/aSZM4f3z/9XJTcxshtcyVWDSuHSE81mwr8fNk8a3s=; b=lUTnlI+dZbJPfdJkCzIKA9emmK3z06t0Hu2jPWFkrN3xnrBIa6BxnRew jI6UDxrWLscPucAM9QGlXSmsFCQzECHCfymwxPIbRMlynt6WcI+XLh5aM 7KP1VpKfExxOQyZKaI67bh7JZTWhkAHI8j72UBXf2daRM+Gheli8QD1SV k=; Received: from unknown (HELO ironmsg04-sd.qualcomm.com) ([10.53.140.144]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:00 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg04-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:00 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:13:59 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 08/11] gunyah: rsc_mgr: Add resource manager RPC core Date: Mon, 1 Aug 2022 14:12:37 -0700 Message-ID: <20220801211240.597859-9-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141405_998843_3FD20D8A X-CRM114-Status: GOOD ( 32.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Signed-off-by: Elliot Berman --- MAINTAINERS | 2 +- drivers/virt/gunyah/Kconfig | 1 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_private.h | 3 + drivers/virt/gunyah/rsc_mgr.c | 623 +++++++++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr.h | 34 ++ drivers/virt/gunyah/sysfs.c | 7 + include/linux/gunyah_rsc_mgr.h | 29 ++ 8 files changed, 699 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 drivers/virt/gunyah/rsc_mgr.h create mode 100644 include/linux/gunyah_rsc_mgr.h diff --git a/MAINTAINERS b/MAINTAINERS index e63c51ee1a2a..dcd961d6623c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8746,7 +8746,7 @@ F: Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml F: Documentation/virt/gunyah/ F: arch/arm64/include/asm/gunyah.h F: drivers/virt/gunyah/ -F: include/linux/gunyah.h +F: include/linux/gunyah*.h HABANALABS PCI DRIVER M: Oded Gabbay diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 1e493017885e..0adb6efd4848 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -4,6 +4,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 select SYS_HYPERVISOR + select AUXILIARY_BUS help The Gunyah drivers are the helper interfaces that runs in a guest VM such as basic inter-VM IPC and signaling mechanisms and higher level diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 94dc8e738911..86655bca8944 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only gunyah-y += sysfs.o device.o msgq.o +gunyah-y += rsc_mgr.o obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file diff --git a/drivers/virt/gunyah/gunyah_private.h b/drivers/virt/gunyah/gunyah_private.h index 2ade32bd9bdf..6483ffa8c15d 100644 --- a/drivers/virt/gunyah/gunyah_private.h +++ b/drivers/virt/gunyah/gunyah_private.h @@ -12,4 +12,7 @@ void gunyah_bus_exit(void); int __init gh_msgq_init(void); void gh_msgq_exit(void); +int __init gh_rsc_mgr_init(void); +void gh_rsc_mgr_exit(void); + #endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c new file mode 100644 index 000000000000..b8268ee02fab --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -0,0 +1,623 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "gunyah_private.h" +#include "rsc_mgr.h" + +/* Resource Manager Header */ +struct gh_rm_rpc_hdr { + u8 version:4, + hdr_words:4; + u8 type:2, + fragments:6; + u16 seq; + u32 msg_id; +} __packed; + +/* Standard reply header */ +struct gh_rm_rpc_reply_hdr { + struct gh_rm_rpc_hdr rpc_hdr; + u32 err_code; +} __packed; + +/* RPC Header versions */ +#define GH_RM_RPC_HDR_VERSION_ONE 0x1 + +/* RPC Header words */ +#define GH_RM_RPC_HDR_WORDS 0x2 + +/* RPC Message types */ +#define GH_RM_RPC_TYPE_CONT 0x0 +#define GH_RM_RPC_TYPE_REQ 0x1 +#define GH_RM_RPC_TYPE_RPLY 0x2 +#define GH_RM_RPC_TYPE_NOTIF 0x3 + +#define GH_RM_MAX_NUM_FRAGMENTS 62 + +#define GH_RM_MAX_MSG_SIZE (GH_MSGQ_MAX_MSG_SIZE - sizeof(struct gh_rm_rpc_hdr)) + +/** + * struct gh_rm_connection - Represents a complete message from resource manager + * @payload: Combined payload of all the fragments (i.e. msg headers stripped off). + * @size: Size of the payload. + * @ret: Linux return code, set in case there was an error processing the connection. + * @msg_id: Message ID from the header. + * @type: GH_RM_RPC_TYPE_RPLY or GH_RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received for this connection. + * @fragments_recieved: fragments received so far. + * @rm_error: For request/reply sequences with standard replies. + * @seq: Sequence ID for the main message. + */ +struct gh_rm_connection { + void *payload; + size_t size; + int ret; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /* only for req/reply sequence */ + u32 rm_error; + u16 seq; + struct completion seq_done; +}; + +struct gh_rm_notif_complete { + struct gh_rm_connection *conn; + struct work_struct work; +}; + +struct gh_rsc_mgr { + struct task_struct *recv_task; + struct gunyah_device *msgq_tx, *msgq_rx; + + struct idr call_idr; + struct mutex call_idr_lock; + + struct mutex send_lock; +}; + +static struct gh_rsc_mgr *__rsc_mgr; +SRCU_NOTIFIER_HEAD_STATIC(gh_rm_notifier); + +static struct gh_rm_connection *gh_rm_alloc_connection(u32 msg_id, u8 type) +{ + struct gh_rm_connection *connection; + + connection = kzalloc(sizeof(*connection), GFP_KERNEL); + if (!connection) + return NULL; + + connection->type = type; + connection->msg_id = msg_id; + + return connection; +} + +/** + * gh_rm_init_connection_payload() - Fills the first message for a connection. + */ +static int gh_rm_init_connection_payload(struct gh_rm_connection *connection, void *msg, + size_t hdr_size, size_t payload_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t max_buf_size; + + connection->num_fragments = hdr->fragments; + connection->fragments_received = 0; + connection->type = hdr->type; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !connection->num_fragments) + return 0; + + /* + * maximum payload size is GH_MSGQ_MAX_MSG_SIZE - hdr_size + * and can received (hdr->fragments + 1) of those + */ + max_buf_size = (GH_MSGQ_MAX_MSG_SIZE - hdr_size) * (hdr->fragments + 1); + + connection->payload = kzalloc(max_buf_size, GFP_KERNEL); + if (!connection->payload) + return -ENOMEM; + + memcpy(connection->payload, msg + hdr_size, payload_size); + connection->size = payload_size; + return 0; +} + +static void gh_rm_notif_work(struct work_struct *work) +{ + struct gh_rm_notif_complete *notif = container_of(work, struct gh_rm_notif_complete, work); + struct gh_rm_connection *connection = notif->conn; + u32 notif_id = connection->msg_id; + struct gh_rm_notification notification = { + .buff = connection->payload, + .size = connection->size, + }; + + srcu_notifier_call_chain(&gh_rm_notifier, notif_id, ¬ification); + + kfree(connection->payload); + kfree(connection); + kfree(notif); +} + +static struct gh_rm_connection *gh_rm_process_notif(struct gh_rsc_mgr *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + connection = gh_rm_alloc_connection(hdr->msg_id, hdr->type); + if (!connection) { + pr_err("Failed to alloc connection for notification, dropping.\n"); + return NULL; + } + + if (gh_rm_init_connection_payload(connection, msg, sizeof(*hdr), msg_size - sizeof(*hdr))) { + pr_err("Failed to alloc connection buffer for notification, dropping.\n"); + kfree(connection); + return NULL; + } + + return connection; +} + +static struct gh_rm_connection *gh_rm_process_rply(struct gh_rsc_mgr *rsc_mgr, + void *msg, size_t msg_size) +{ + struct gh_rm_rpc_reply_hdr *reply_hdr = msg; + struct gh_rm_rpc_hdr *hdr = msg; + struct gh_rm_connection *connection; + + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) + return ERR_PTR(-ERESTARTSYS); + + connection = idr_find(&rsc_mgr->call_idr, hdr->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + if (!connection) { + pr_err("Failed to find connection for sequence %u\n", hdr->seq); + return NULL; + } + if (connection->msg_id != hdr->msg_id) { + pr_err("Reply for sequence %u expected msg_id: %x but got %x\n", hdr->seq, + connection->msg_id, hdr->msg_id); + /* + * Don't complete connection and error the client, maybe resource manager will + * send us the expected reply sequence soon. + */ + return NULL; + } + + if (gh_rm_init_connection_payload(connection, msg, sizeof(*reply_hdr), + msg_size - sizeof(*reply_hdr))) { + pr_err("Failed to alloc connection buffer for sequence %d\n", hdr->seq); + /* Send connection complete and error the client. */ + connection->ret = -ENOMEM; + complete(&connection->seq_done); + return NULL; + } + + connection->rm_error = reply_hdr->err_code; + return connection; +} + +static void gh_rm_process_cont(struct gh_rm_connection *connection, void *msg, size_t msg_size) +{ + struct gh_rm_rpc_hdr *hdr = msg; + size_t payload_size = msg_size - sizeof(*hdr); + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply or notif message. + * For sake of sanity, check if it's still intact. + */ + if (connection->msg_id != hdr->msg_id) + pr_warn("Appending mismatched continuation with id %d to connection with id %d\n", + hdr->msg_id, connection->msg_id); + if (connection->num_fragments != hdr->fragments) + pr_warn("Number of fragments mismatch for seq: %d\n", hdr->seq); + + memcpy(connection->payload + connection->size, msg + sizeof(*hdr), payload_size); + connection->size += payload_size; + connection->fragments_received++; +} + +static bool gh_rm_complete_connection(struct gh_rm_connection *connection) +{ + struct gh_rm_notif_complete *notif_work; + + if (!connection) + return false; + + if (connection->fragments_received != connection->num_fragments) + return false; + + switch (connection->type) { + case GH_RM_RPC_TYPE_RPLY: + complete(&connection->seq_done); + break; + case GH_RM_RPC_TYPE_NOTIF: + notif_work = kzalloc(sizeof(*notif_work), GFP_KERNEL); + if (notif_work == NULL) + break; + + notif_work->conn = connection; + INIT_WORK(¬if_work->work, gh_rm_notif_work); + + schedule_work(¬if_work->work); + break; + default: + pr_err("Invalid message type (%d) received\n", connection->type); + break; + } + + return true; +} + +static void gh_rm_abort_connection(struct gh_rm_connection *connection) +{ + switch (connection->type) { + case GH_RM_RPC_TYPE_RPLY: + connection->ret = -EIO; + complete(&connection->seq_done); + break; + case GH_RM_RPC_TYPE_NOTIF: + fallthrough; + default: + kfree(connection->payload); + kfree(connection); + } +} + +static int gh_rm_recv_task_fn(void *data) +{ + struct gh_rsc_mgr *rsc_mgr = data; + struct gh_rm_connection *connection = NULL; + struct gh_rm_rpc_hdr *hdr = NULL; + ssize_t msg_size; + void *msg; + + msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + while (!kthread_should_stop()) { + /* Block until a new message is received */ + msg_size = gh_msgq_recv(rsc_mgr->msgq_rx, msg, GH_MSGQ_MAX_MSG_SIZE, 0); + if (msg_size < 0) { + pr_err("Failed to receive the message: %ld\n", msg_size); + continue; + } else if (msg_size <= sizeof(struct gh_rm_rpc_hdr)) { + pr_err("Invalid message size received: %ld is too small\n", msg_size); + continue; + } + + hdr = msg; + switch (hdr->type) { + case GH_RM_RPC_TYPE_NOTIF: + if (connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + pr_warn("Received start of new notification without finishing existing message series.\n"); + gh_rm_abort_connection(connection); + } + connection = gh_rm_process_notif(rsc_mgr, msg, msg_size); + break; + case GH_RM_RPC_TYPE_RPLY: + if (connection) { + /* Not possible per protocol. Do something better than BUG_ON */ + pr_warn("Received start of new reply without finishing existing message series.\n"); + gh_rm_abort_connection(connection); + } + connection = gh_rm_process_rply(rsc_mgr, msg, msg_size); + break; + case GH_RM_RPC_TYPE_CONT: + if (!connection) { + pr_warn("Received a continuation message without receiving initial message\n"); + break; + } + gh_rm_process_cont(connection, msg, msg_size); + break; + default: + pr_err("Invalid message type (%d) received\n", hdr->type); + continue; + } + + if (gh_rm_complete_connection(connection)) + connection = NULL; + } + + return 0; +} + +static int gh_rm_send_request(struct gh_rsc_mgr *rsc_mgr, u32 message_id, + const void *req_buff, size_t req_buff_size, + struct gh_rm_connection *connection) +{ + size_t buff_size_remaining = req_buff_size; + const void *req_buff_curr = req_buff; + struct gh_rm_rpc_hdr *hdr; + unsigned long tx_flags; + u32 num_fragments = 0; + size_t payload_size; + void *msg; + int i, ret = 0; + + if (req_buff_size > GH_RM_MAX_MSG_SIZE) + num_fragments = req_buff_size / GH_RM_MAX_MSG_SIZE; + + if (WARN(num_fragments > GH_RM_MAX_NUM_FRAGMENTS, + "Limit exceeded for the number of fragments: %u\n", num_fragments)) + return -E2BIG; + + /* + * The above calculation also includes the count for the 'request' packet. + * Exclude it as the header needs to fill the num. of fragments to follow. + */ + if (num_fragments) + num_fragments--; + + if (mutex_lock_interruptible(&rsc_mgr->send_lock)) + return -ERESTARTSYS; + + msg = kzalloc(GH_MSGQ_MAX_MSG_SIZE, GFP_KERNEL); + if (!msg) { + mutex_unlock(&rsc_mgr->send_lock); + return -ENOMEM; + } + + /* Consider also the 'request' packet for the loop count */ + for (i = 0; i <= num_fragments; i++) { + if (buff_size_remaining > GH_RM_MAX_MSG_SIZE) { + payload_size = GH_RM_MAX_MSG_SIZE; + buff_size_remaining -= payload_size; + } else { + payload_size = buff_size_remaining; + } + + memset(msg, 0, GH_MSGQ_MAX_MSG_SIZE); + + /* Fill header */ + hdr = msg; + hdr->version = GH_RM_RPC_HDR_VERSION_ONE; + hdr->hdr_words = GH_RM_RPC_HDR_WORDS; + hdr->type = i == 0 ? GH_RM_RPC_TYPE_REQ : GH_RM_RPC_TYPE_CONT; + hdr->fragments = num_fragments; + hdr->seq = connection->seq; + hdr->msg_id = message_id; + + /* Copy payload */ + memcpy(msg + sizeof(*hdr), req_buff_curr, payload_size); + req_buff_curr += payload_size; + + /* Force the last fragment to be sent immediately to the receiver */ + tx_flags = (i == num_fragments) ? GH_MSGQ_TX_PUSH : 0; + + ret = gh_msgq_send(rsc_mgr->msgq_tx, msg, sizeof(*hdr) + payload_size, tx_flags); + + if (ret < 0) + break; + } + + mutex_unlock(&rsc_mgr->send_lock); + return ret < 0 ? ret : 0; +} + +/** + * gh_rm_call: Achieve request-response type communication with RPC + * @message_id: The RM RPC message-id + * @req_buff: Request buffer that contains the payload + * @req_buff_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buff_size: Size of the response buffer + * @reply_err_code: Returns Gunyah standard error code for the response + * + * Make a request to the RM-VM and wait for reply back. For a successful + * response, the function returns the payload. The size of the payload is set in resp_buff_size. + * The resp_buf should be freed by the caller. + * + * Context: Process context. Will sleep waiting for reply. + * Return: >0 is standard reply error from RM. <0 on internal error. + */ +int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size) +{ + struct gh_rm_connection *connection; + int ret; + struct gh_rsc_mgr *rsc_mgr = __rsc_mgr; + + /* messaged_id 0 is reserved */ + if (!message_id) + return -EINVAL; + + if (!rsc_mgr) + return -EPROBE_DEFER; + + connection = gh_rm_alloc_connection(message_id, GH_RM_RPC_TYPE_RPLY); + if (!connection) + return -ENOMEM; + + init_completion(&connection->seq_done); + + /* Allocate a new seq number for this connection */ + if (mutex_lock_interruptible(&rsc_mgr->call_idr_lock)) { + kfree(connection); + return -ERESTARTSYS; + } + connection->seq = idr_alloc_cyclic(&rsc_mgr->call_idr, connection, 0, U16_MAX, GFP_KERNEL); + mutex_unlock(&rsc_mgr->call_idr_lock); + + /* Send the request to the Resource Manager */ + ret = gh_rm_send_request(rsc_mgr, message_id, req_buff, req_buff_size, connection); + if (ret < 0) + goto out; + + /* Wait for response */ + wait_for_completion(&connection->seq_done); + + if (connection->ret) { + ret = connection->ret; + kfree(connection->payload); + goto out; + } + + if (connection->rm_error) { + ret = connection->rm_error; + kfree(connection->payload); + goto out; + } + + *resp_buf = connection->payload; + *resp_buff_size = connection->size; + +out: + mutex_lock(&rsc_mgr->call_idr_lock); + idr_remove(&rsc_mgr->call_idr, connection->seq); + mutex_unlock(&rsc_mgr->call_idr_lock); + + kfree(connection); + return ret; +} + +int gh_rm_register_notifier(struct notifier_block *nb) +{ + return srcu_notifier_chain_register(&gh_rm_notifier, nb); +} +EXPORT_SYMBOL_GPL(gh_rm_register_notifier); + +int gh_rm_unregister_notifier(struct notifier_block *nb) +{ + return srcu_notifier_chain_unregister(&gh_rm_notifier, nb); +} +EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier); + +static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev, + u8 gh_type, int idx) +{ + int irq, ret; + u64 capid; + struct device_node *node = pdev->dev.of_node; + struct gunyah_device *ghdev; + + irq = platform_get_irq(pdev, idx); + if (irq < 0) { + dev_err(&pdev->dev, "Failed to get irq%d: %d\n", idx, irq); + return ERR_PTR(irq); + } + + ret = of_property_read_u64_index(node, "reg", idx, &capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get capid%d: %d\n", idx, ret); + return ERR_PTR(ret); + } + + ghdev = gunyah_device_alloc(&pdev->dev, capid, gh_type); + ghdev->irq = irq; + ret = gunyah_device_add(ghdev); + if (ret) { + kfree(ghdev); + return ERR_PTR(ret); + } + + return ghdev; +} + +static int gh_rm_drv_probe(struct platform_device *pdev) +{ + struct gh_rsc_mgr *rsc_mgr; + int ret; + + rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); + if (!rsc_mgr) + return -ENOMEM; + platform_set_drvdata(pdev, rsc_mgr); + + mutex_init(&rsc_mgr->call_idr_lock); + idr_init(&rsc_mgr->call_idr); + mutex_init(&rsc_mgr->send_lock); + + rsc_mgr->msgq_tx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, 0); + if (IS_ERR(rsc_mgr->msgq_tx)) + return PTR_ERR(rsc_mgr->msgq_tx); + rsc_mgr->msgq_rx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_RX, 1); + if (IS_ERR(rsc_mgr->msgq_rx)) { + ret = PTR_ERR(rsc_mgr->msgq_rx); + goto err_msgq_tx; + } + + rsc_mgr->recv_task = kthread_run(gh_rm_recv_task_fn, rsc_mgr, "gh_rm_recv_task"); + if (IS_ERR_OR_NULL(rsc_mgr->recv_task)) { + ret = PTR_ERR(rsc_mgr->recv_task); + goto err_msgq; + } + + __rsc_mgr = rsc_mgr; + + return 0; + +err_msgq: + gunyah_device_remove(rsc_mgr->msgq_rx); +err_msgq_tx: + gunyah_device_remove(rsc_mgr->msgq_tx); + return ret; +} + +static int gh_rm_drv_remove(struct platform_device *pdev) +{ + struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + + gunyah_device_remove(rsc_mgr->msgq_tx); + gunyah_device_remove(rsc_mgr->msgq_rx); + + return 0; +} + +static const struct of_device_id gh_rm_of_match[] = { + { .compatible = "gunyah-resource-manager" }, + { } +}; +MODULE_DEVICE_TABLE(of, gh_rm_of_match); + +static struct platform_driver gh_rsc_mgr_driver = { + .probe = gh_rm_drv_probe, + .remove = gh_rm_drv_remove, + .driver = { + .name = "gh_rsc_mgr", + .of_match_table = gh_rm_of_match, + }, +}; + +int __init gh_rsc_mgr_init(void) +{ + return platform_driver_register(&gh_rsc_mgr_driver); +} + +void gh_rsc_mgr_exit(void) +{ + platform_driver_unregister(&gh_rsc_mgr_driver); +} diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h new file mode 100644 index 000000000000..e4f2499267bf --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __GH_RSC_MGR_PRIV_H +#define __GH_RSC_MGR_PRIV_H + +#include + +/* RM Error codes */ +#define GH_RM_ERROR_OK 0x0 +#define GH_RM_ERROR_UNIMPLEMENTED 0xFFFFFFFF +#define GH_RM_ERROR_NOMEM 0x1 +#define GH_RM_ERROR_NORESOURCE 0x2 +#define GH_RM_ERROR_DENIED 0x3 +#define GH_RM_ERROR_INVALID 0x4 +#define GH_RM_ERROR_BUSY 0x5 +#define GH_RM_ERROR_ARGUMENT_INVALID 0x6 +#define GH_RM_ERROR_HANDLE_INVALID 0x7 +#define GH_RM_ERROR_VALIDATE_FAILED 0x8 +#define GH_RM_ERROR_MAP_FAILED 0x9 +#define GH_RM_ERROR_MEM_INVALID 0xA +#define GH_RM_ERROR_MEM_INUSE 0xB +#define GH_RM_ERROR_MEM_RELEASED 0xC +#define GH_RM_ERROR_VMID_INVALID 0xD +#define GH_RM_ERROR_LOOKUP_FAILED 0xE +#define GH_RM_ERROR_IRQ_INVALID 0xF +#define GH_RM_ERROR_IRQ_INUSE 0x10 +#define GH_RM_ERROR_IRQ_RELEASED 0x11 + +int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, + void **resp_buf, size_t *resp_buff_size); + +#endif diff --git a/drivers/virt/gunyah/sysfs.c b/drivers/virt/gunyah/sysfs.c index 7589689e5e92..7c0efc80f85e 100644 --- a/drivers/virt/gunyah/sysfs.c +++ b/drivers/virt/gunyah/sysfs.c @@ -148,7 +148,13 @@ static int __init gunyah_init(void) if (ret) goto err_bus; + ret = gh_rsc_mgr_init(); + if (ret) + goto err_msgq; + return ret; +err_msgq: + gh_msgq_exit(); err_bus: gunyah_bus_exit(); err_sysfs: @@ -159,6 +165,7 @@ module_init(gunyah_init); static void __exit gunyah_exit(void) { + gh_rsc_mgr_exit(); gh_msgq_exit(); gunyah_bus_exit(); gh_sysfs_unregister(); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..015bd851e1a3 --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _GUNYAH_RSC_MGR_H +#define _GUNYAH_RSC_MGR_H + +#include +#include +#include + +typedef u16 gh_vmid_t; +typedef u32 gh_virq_handle_t; + +#define GH_VMID_INVAL U16_MAX + +/* Gunyah recognizes VMID0 as an alias to the current VM's ID */ +#define GH_VMID_SELF 0 + +struct gh_rm_notification { + const void *buff; + const size_t size; +}; + +int gh_rm_register_notifier(struct notifier_block *nb); +int gh_rm_unregister_notifier(struct notifier_block *nb); + +#endif From patchwork Mon Aug 1 21:12:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E562FC00144 for ; Mon, 1 Aug 2022 21:17:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oK42YdDjGiPqrqrF6gTURbRAdD7EV3wk9diyHJGJnWg=; b=0bAcumk6wGYDRi OdGbn2ccDLGX7f7QBvk2uv5wQ339yciWLf+/djUL5sV+9W6D5Zk5G4yhfgGQcCYInbMfTBjUYglVY +hmvditsEckh43qd0O/F1ZKXG/kchDrpL0yYHCxjg7j6kQKFXYLzYmAClCkuBeeu9UH0lRc7bscjA 8zQcIeEQpQAUnqPWLh5dlzU4JV9mYJXIrTJ7zZWTAKAz6tnHVZ7E8HMtf6NTjcZK9/xWqxD0RAwfp XXB8g4JHV+bVw0UnBiDqfVdOjFGHo6ZudWFQbTfB+RorKqff0Yok7Ky6I9afGVUSwRQ8V4EX42vLv E3KYH9AQZ+knolcQx0UA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcm0-00Aj1k-2I; Mon, 01 Aug 2022 21:16:24 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjo-00Ah8U-1e for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388448; x=1690924448; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yzAyPrU/qCIlo6Cf2CdARPBaTkg8+aYwmjRonAsM36o=; b=wP89LbdzksjxUmERYdT0ahyxtpyxPLLzK2KnrGRLShYesvLi4I0LAyIX cb5xDF9Be7Dx3xOjzLcD421iGbfirpr5QHGEihSYtrnY34n5SEooQ4qTF k/6g6bUbiory31186Lx3kIP0gEbi4cDZzbK8QGqM52yLUbet/TsoAbGbA 4=; Received: from unknown (HELO ironmsg05-sd.qualcomm.com) ([10.53.140.145]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:01 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg05-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:01 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:14:00 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 09/11] gunyah: rsc_mgr: Add auxiliary devices for console Date: Mon, 1 Aug 2022 14:12:38 -0700 Message-ID: <20220801211240.597859-10-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141408_160911_4813589D X-CRM114-Status: GOOD ( 17.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Gunyah resource manager exposes a concrete functionalities which complicate a single resource manager driver. Use auxiliary bus to help split high level functions for the resource manager and keep the primary resource manager driver focused on the RPC with RM itself. Delegate Resource Manager's console functionality to the auxiliary bus. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/rsc_mgr.c | 61 ++++++++++++++++++++++++++++++++++- 1 file changed, 60 insertions(+), 1 deletion(-) diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index b8268ee02fab..44b22cef7d44 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -91,6 +91,11 @@ struct gh_rm_notif_complete { struct work_struct work; }; +struct gh_rsc_mgr_adev { + struct auxiliary_device adev; + struct list_head list; +}; + struct gh_rsc_mgr { struct task_struct *recv_task; struct gunyah_device *msgq_tx, *msgq_rx; @@ -99,6 +104,13 @@ struct gh_rsc_mgr { struct mutex call_idr_lock; struct mutex send_lock; + + struct list_head adevs; +}; + +/* List of auxiliary devices which resource manager creates */ +static const char * const adev_names[] = { + "console", }; static struct gh_rsc_mgr *__rsc_mgr; @@ -516,6 +528,14 @@ int gh_rm_unregister_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(gh_rm_unregister_notifier); +static void gh_rm_adev_release(struct device *dev) +{ + struct gh_rsc_mgr_adev *rm_adev = container_of(dev, struct gh_rsc_mgr_adev, adev.dev); + + list_del(&rm_adev->list); + kfree(rm_adev); +} + static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_device *pdev, u8 gh_type, int idx) { @@ -550,7 +570,9 @@ static struct gunyah_device *gh_msgq_platform_probe_direction(struct platform_de static int gh_rm_drv_probe(struct platform_device *pdev) { struct gh_rsc_mgr *rsc_mgr; - int ret; + struct gh_rsc_mgr_adev *rm_adev; + struct list_head *l, *n; + int ret, i; rsc_mgr = devm_kzalloc(&pdev->dev, sizeof(*rsc_mgr), GFP_KERNEL); if (!rsc_mgr) @@ -560,6 +582,7 @@ static int gh_rm_drv_probe(struct platform_device *pdev) mutex_init(&rsc_mgr->call_idr_lock); idr_init(&rsc_mgr->call_idr); mutex_init(&rsc_mgr->send_lock); + INIT_LIST_HEAD(&rsc_mgr->adevs); rsc_mgr->msgq_tx = gh_msgq_platform_probe_direction(pdev, GUNYAH_DEVICE_TYPE_MSGQ_TX, 0); if (IS_ERR(rsc_mgr->msgq_tx)) @@ -576,10 +599,38 @@ static int gh_rm_drv_probe(struct platform_device *pdev) goto err_msgq; } + for (i = 0; i < ARRAY_SIZE(adev_names); i++) { + rm_adev = kzalloc(sizeof(*rm_adev), GFP_KERNEL); + + rm_adev->adev.dev.parent = &pdev->dev; + rm_adev->adev.dev.release = gh_rm_adev_release; + rm_adev->adev.name = adev_names[i]; + ret = auxiliary_device_init(&rm_adev->adev); + if (ret) { + kfree(rm_adev); + goto err_adevs; + } + + list_add(&rm_adev->list, &rsc_mgr->adevs); + + ret = auxiliary_device_add(&rm_adev->adev); + if (ret) { + auxiliary_device_uninit(&rm_adev->adev); + goto err_adevs; + } + } + __rsc_mgr = rsc_mgr; return 0; +err_adevs: + list_for_each_safe(l, n, &rsc_mgr->adevs) { + rm_adev = container_of(l, struct gh_rsc_mgr_adev, list); + auxiliary_device_delete(&rm_adev->adev); + auxiliary_device_uninit(&rm_adev->adev); + } + err_msgq: gunyah_device_remove(rsc_mgr->msgq_rx); err_msgq_tx: @@ -590,6 +641,14 @@ static int gh_rm_drv_probe(struct platform_device *pdev) static int gh_rm_drv_remove(struct platform_device *pdev) { struct gh_rsc_mgr *rsc_mgr = platform_get_drvdata(pdev); + struct gh_rsc_mgr_adev *rm_adev; + struct list_head *l, *n; + + list_for_each_safe(l, n, &rsc_mgr->adevs) { + rm_adev = container_of(l, struct gh_rsc_mgr_adev, list); + auxiliary_device_delete(&rm_adev->adev); + auxiliary_device_uninit(&rm_adev->adev); + } gunyah_device_remove(rsc_mgr->msgq_tx); gunyah_device_remove(rsc_mgr->msgq_rx); From patchwork Mon Aug 1 21:12:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D28C9C00144 for ; Mon, 1 Aug 2022 21:17:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ExA51CER8tOk8EIcxIKDSbc4edXQJYgGYANu6UqePEo=; b=KQJU5UXepRx9XV bFGIRFWapJxyGlLraz49giTxsxV9HQIaxsgXGJ1/UvU16AMWCJgZtbrkLqn1ya2rtXFwnB/jBtiY5 Z5cCjPHVt8oIXsJSCJwzshjjAmZ0iCh5qNue6W7RU7UFNdOm0Gcyg0sJ8sWt1vQgRmvYhtOQXWjZy DpneOrnsPIt1HmRLR1w6rR0vNDDuuiKTj7S1eRizXWph1cM35m0ObwhUmY2aVs/TCfCwRAbqoqIwA wRCSDw6O3ork5sN8NwDNZQCOZrxNK/KngrcbZ8TOXuPD+icNdCTxhXKk4HAI4cxqnpfgdgA3dI79S yzkVNzIwiLu2rQAbFQPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcmI-00AjHf-Ea; Mon, 01 Aug 2022 21:16:42 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjo-00Ah74-EW for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388448; x=1690924448; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z7EpwCPH7kpklNw4k3PRnXjN7KwyE+imFag8eqKdl+0=; b=n+DDOLQ8x+agMxKw+6DCykyBDgnJcE1VUOfiK23Le0bEMycP0DL3VluD oAQocIcroZ5mq8pL8PgtXoAMztpO1pjUhGM9wxJk9GI+iSv/ms4Pl0yDz fFBunEt1cs2d6P6g0vwh04npWVk5xiEU8XYtKWd61iK6LuqmNycYeci9M s=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:01 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:01 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:14:00 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 10/11] gunyah: rsc_mgr: Add RPC for console services Date: Mon, 1 Aug 2022 14:12:39 -0700 Message-ID: <20220801211240.597859-11-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141408_582476_FE3223B5 X-CRM114-Status: GOOD ( 22.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Gunyah resource manager defines a simple API for virtual machine log sharing with the console service. A VM's own log can be opened by using GH_VMID_SELF. Another VM's log can be accessed via its VMID. Once opened, characters can be written to the log with a write command. Characters are received with resource manager notifications (using ID GH_RM_NOTIF_VM_CONSOLE_CHARS). These high level rpc calls are kept in drivers/virt/gunyah/rsc_mgr_rpc.c. Future RPC calls, e.g. to launch a VM will also be maintained in this file. Signed-off-by: Elliot Berman --- drivers/virt/gunyah/Makefile | 4 +- drivers/virt/gunyah/rsc_mgr.h | 22 +++++ drivers/virt/gunyah/rsc_mgr_rpc.c | 151 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 16 ++++ 4 files changed, 191 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 86655bca8944..b3f15c052297 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only gunyah-y += sysfs.o device.o msgq.o -gunyah-y += rsc_mgr.o -obj-$(CONFIG_GUNYAH) += gunyah.o \ No newline at end of file +gunyah-y += rsc_mgr.o rsc_mgr_rpc.o +obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index e4f2499267bf..c82a2e97ad49 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -28,6 +28,28 @@ #define GH_RM_ERROR_IRQ_INUSE 0x10 #define GH_RM_ERROR_IRQ_RELEASED 0x11 +/* Message IDs: VM Management */ +#define GH_RM_RPC_VM_GET_VMID 0x56000024 + +/* Message IDs: VM Services */ +#define GH_RM_RPC_VM_CONSOLE_OPEN_ID 0x56000081 +#define GH_RM_RPC_VM_CONSOLE_CLOSE_ID 0x56000082 +#define GH_RM_RPC_VM_CONSOLE_WRITE_ID 0x56000083 +#define GH_RM_RPC_VM_CONSOLE_FLUSH_ID 0x56000084 + +/* Call: CONSOLE_OPEN, CONSOLE_CLOSE, CONSOLE_FLUSH */ +struct gh_vm_console_common_req { + gh_vmid_t vmid; + u16 reserved0; +} __packed; + +/* Call: CONSOLE_WRITE */ +struct gh_vm_console_write_req { + gh_vmid_t vmid; + u16 num_bytes; + u8 data[0]; +} __packed; + int gh_rm_call(u32 message_id, void *req_buff, size_t req_buff_size, void **resp_buf, size_t *resp_buff_size); diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c new file mode 100644 index 000000000000..be9317968537 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr: " fmt + +#include +#include +#include +#include + +#include "rsc_mgr.h" + +/** + * gh_rm_get_vmid: Retrieve VMID of this virtual machine + * @vmid: Filled with the VMID of this VM + */ +int gh_rm_get_vmid(gh_vmid_t *vmid) +{ + void *resp; + size_t resp_size; + int ret; + int payload = 0; + + ret = gh_rm_call(GH_RM_RPC_VM_GET_VMID, &payload, sizeof(payload), &resp, &resp_size); + if (ret) + return ret; + + if (resp_size != sizeof(*vmid)) + return -EIO; + *vmid = *(gh_vmid_t *)resp; + kfree(resp); + + return ret; +} + +/** + * gh_rm_console_open: Open a console with a VM + * @vmid: VMID of the other vmid whose console to open. If VMID is GH_VMID_SELF, the + * console associated with this VM is opened. + */ +int gh_rm_console_open(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_OPEN_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_OPEN: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_open); + +/** + * gh_rm_console_close: Close a console with a VM + * @vmid: The vmid of the vm whose console to close. + */ +int gh_rm_console_close(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_CLOSE_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_CLOSE: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_close); + +/** + * gh_rm_console_write: Write to a VM's console + * @vmid: The vmid of the vm whose console to write to. + * @buf: Buffer to write to the VM's console + * @size: Size of the buffer + */ +int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size) +{ + void *resp; + struct gh_vm_console_write_req *req_payload; + size_t resp_size; + int ret = 0; + size_t req_payload_size = sizeof(*req_payload) + size; + + if (size < 1 || size > (U32_MAX - sizeof(*req_payload))) + return -EINVAL; + + req_payload = kzalloc(req_payload_size, GFP_KERNEL); + + if (!req_payload) + return -ENOMEM; + + req_payload->vmid = vmid; + req_payload->num_bytes = size; + memcpy(req_payload->data, buf, size); + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_WRITE_ID, + req_payload, req_payload_size, + &resp, &resp_size); + kfree(req_payload); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_WRITE: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_write); + +/** + * gh_rm_console_flush: Flush a console with a VM + * @vmid: The vmid of the vm whose console to flush + */ +int gh_rm_console_flush(gh_vmid_t vmid) +{ + void *resp; + struct gh_vm_console_common_req req_payload = {0}; + size_t resp_size; + int ret; + + req_payload.vmid = vmid; + + ret = gh_rm_call(GH_RM_RPC_VM_CONSOLE_FLUSH_ID, + &req_payload, sizeof(req_payload), + &resp, &resp_size); + kfree(resp); + + if (!ret && resp_size) + pr_warn("Received unexpected payload for CONSOLE_FLUSH: %lu\n", resp_size); + + return ret; +} +EXPORT_SYMBOL_GPL(gh_rm_console_flush); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 015bd851e1a3..4211ac9a219c 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -26,4 +26,20 @@ struct gh_rm_notification { int gh_rm_register_notifier(struct notifier_block *nb); int gh_rm_unregister_notifier(struct notifier_block *nb); +/* Notification type Message IDs */ +#define GH_RM_NOTIF_VM_CONSOLE_CHARS 0x56100080 + +struct gh_rm_notif_vm_console_chars { + gh_vmid_t vmid; + u16 num_bytes; + u8 bytes[0]; +} __packed; + +/* RPC Calls */ +int gh_rm_get_vmid(gh_vmid_t *vmid); +int gh_rm_console_open(gh_vmid_t vmid); +int gh_rm_console_close(gh_vmid_t vmid); +int gh_rm_console_write(gh_vmid_t vmid, const char *buf, size_t size); +int gh_rm_console_flush(gh_vmid_t vmid); + #endif From patchwork Mon Aug 1 21:12:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 12934080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 38F43C00144 for ; Mon, 1 Aug 2022 21:18:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5/Ieev+lzj4g8vlfkdI7/soxPzlIO9XYOCFeRUtW4vg=; b=ysnNaRzyfGoZ4E oX3KL9Fhf6Giqab83cvtFzEFFpgaOKDcoA+k0eIbTeKO1lQPTCkmY+CBPmjHz8cG/o/msotlwI0CO rs5Z3syGHX2997RHgpWr+yXbHgQS9dI/qdz6zHcJ5FPacuXaNBxJoqaSeXkDP7ivnxB8NrzutrmQ9 wwbdrSV7P4uelsf8QsLK2nokkq7S1A1EGeAwx2VPwuC2DU/xHg22rgZ9JI0wK1JHX/2/jZ1Z6is6w umo1WHotS7w5xbDyTIPSAtd8EOXD2dddV2oPDV6/5vzxXZqv5W3fw1fO3/nrp9gMysEYm0IjpT+ik oFWBS+GFol9zDOIAPNiw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcn4-00AjuF-Td; Mon, 01 Aug 2022 21:17:31 +0000 Received: from alexa-out-sd-01.qualcomm.com ([199.106.114.38]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oIcjp-00Ah6P-DW for linux-arm-kernel@lists.infradead.org; Mon, 01 Aug 2022 21:14:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; i=@quicinc.com; q=dns/txt; s=qcdkim; t=1659388449; x=1690924449; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=apoLqHIFfY6O5tPy1Y3FUD9tFxQUIrkjkeX6NZD7iW0=; b=No8ix4aJPSJWMjfJBrPmIyQT1X5NjVPHwqsHOozukrH9LpV8mEvVuJmG T4BDDCVFlZr4WOFXhhxrJHANXadPF7MI5MjY6vNr3gddvDf2m/B12zf/G 88ZRQ6hIOVhjq4l23E16QnyuhwBWoG22vv3Bb/NfjjpfDFinp00ylU3JA 4=; Received: from unknown (HELO ironmsg03-sd.qualcomm.com) ([10.53.140.143]) by alexa-out-sd-01.qualcomm.com with ESMTP; 01 Aug 2022 14:14:02 -0700 X-QCInternal: smtphost Received: from nasanex01b.na.qualcomm.com ([10.46.141.250]) by ironmsg03-sd.qualcomm.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Aug 2022 14:14:02 -0700 Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 1 Aug 2022 14:14:01 -0700 From: Elliot Berman To: Bjorn Andersson CC: Elliot Berman , Murali Nalajala , Trilok Soni , "Srivatsa Vaddagiri" , Carl van Schaik , Andy Gross , , Lorenzo Pieralisi , Sudeep Holla , "Marc Zyngier" , Rob Herring , "Krzysztof Kozlowski" , Jonathan Corbet , Will Deacon , Catalin Marinas , , , Subject: [PATCH v2 11/11] gunyah: Add tty console driver for RM Console Serivces Date: Mon, 1 Aug 2022 14:12:40 -0700 Message-ID: <20220801211240.597859-12-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220801211240.597859-1-quic_eberman@quicinc.com> References: <20220801211240.597859-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220801_141409_530528_E21E8598 X-CRM114-Status: GOOD ( 31.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Gunyah provides a console for each VM using the VM console resource manager APIs. This driver allows console data from other VMs to be accessed via a TTY device and exports a console device to dump Linux's own logs to our console. Signed-off-by: Elliot Berman --- Documentation/virt/gunyah/index.rst | 7 + drivers/virt/gunyah/Kconfig | 10 + drivers/virt/gunyah/Makefile | 3 + drivers/virt/gunyah/rsc_mgr_console.c | 405 ++++++++++++++++++++++++++ 4 files changed, 425 insertions(+) create mode 100644 drivers/virt/gunyah/rsc_mgr_console.c diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst index e7bb2b14543e..95ba9b71ab30 100644 --- a/Documentation/virt/gunyah/index.rst +++ b/Documentation/virt/gunyah/index.rst @@ -90,3 +90,10 @@ When booting a virtual machine which uses a devicetree, resource manager overlay how to communicate with resource manager, and basic description and capabilities of this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description of this node. + +Resource Manager Consoles +------------------------- +RM provides infrastructure for virtual machines to share an interactive console. This can be used to +interact with a VM which may not have access to a serial port. Linux will register a printk console: +ttyGH0. That console and other VM's consoles can be accessed via ttyGHX. +/sys/class/tty/ttyGHX/vmid will print the VM which is associated with that TTY. diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 0adb6efd4848..ba7bd058e959 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -12,3 +12,13 @@ config GUNYAH Say Y here to enable the drivers needed to interact in a Gunyah virtual environment. + +if GUNYAH +config GUNYAH_RESOURCE_MANAGER_CONSOLE + tristate "Gunyah Resource Manager Consoles" + depends on TTY + help + This enables support for console output using Gunyah's Resource Manager RPC. + This is normally used when a secondary VM which does not have exclusive access + to a real serial device and virtio-console is unavailable. +endif diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b3f15c052297..001cf1630c03 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -3,3 +3,6 @@ gunyah-y += sysfs.o device.o msgq.o gunyah-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o + +gunyah_console-y += rsc_mgr_console.o +obj-$(CONFIG_GUNYAH_RESOURCE_MANAGER_CONSOLE) += gunyah_console.o diff --git a/drivers/virt/gunyah/rsc_mgr_console.c b/drivers/virt/gunyah/rsc_mgr_console.c new file mode 100644 index 000000000000..eda25100314f --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_console.c @@ -0,0 +1,405 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#define pr_fmt(fmt) "gh_rsc_mgr_console: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * The Linux TTY code does not support dynamic addition of tty derived devices so we need to know + * how many tty devices we might need when space is allocated for the tty device. Since VMs might be + * added/removed dynamically, we need to make sure we have enough allocated. + */ +#define RSC_MGR_TTY_ADAPTERS 16 + +/* # of payload bytes that can fit in a 1-fragment CONSOLE_WRITE message */ +#define RM_CONS_WRITE_MSG_SIZE ((1 * (GH_MSGQ_MAX_MSG_SIZE - 8)) - 4) + +struct rm_cons_port { + struct tty_port port; + gh_vmid_t vmid; + bool open; + unsigned int index; + + DECLARE_KFIFO(put_fifo, char, 1024); + spinlock_t fifo_lock; + struct work_struct put_work; + + struct rm_cons_data *cons_data; +}; + +struct rm_cons_data { + struct tty_driver *tty_driver; + struct device *dev; + + spinlock_t ports_lock; + struct rm_cons_port *ports[RSC_MGR_TTY_ADAPTERS]; + + struct notifier_block rsc_mgr_notif; + struct console console; +}; + +static void put_work_fn(struct work_struct *ws) +{ + char buf[RM_CONS_WRITE_MSG_SIZE]; + int count, ret; + struct rm_cons_port *port = container_of(ws, struct rm_cons_port, put_work); + + while (!kfifo_is_empty(&port->put_fifo)) { + count = kfifo_out_spinlocked(&port->put_fifo, buf, sizeof(buf), &port->fifo_lock); + if (count <= 0) + continue; + + ret = gh_rm_console_write(port->vmid, buf, count); + if (ret) { + pr_warn_once("failed to send characters: %d\n", ret); + break; + } + } +} + +static int rsc_mgr_console_notif(struct notifier_block *nb, unsigned long cmd, void *data) +{ + int count, i; + struct rm_cons_port *rm_port; + struct tty_port *tty_port = NULL; + struct rm_cons_data *cons_data = container_of(nb, struct rm_cons_data, rsc_mgr_notif); + const struct gh_rm_notification *notif = data; + struct gh_rm_notif_vm_console_chars const * const msg = notif->buff; + + if (cmd != GH_RM_NOTIF_VM_CONSOLE_CHARS || + notif->size < sizeof(*msg)) + return NOTIFY_DONE; + + spin_lock(&cons_data->ports_lock); + for (i = 0; i < RSC_MGR_TTY_ADAPTERS; i++) { + if (!cons_data->ports[i]) + continue; + if (cons_data->ports[i]->vmid == msg->vmid) { + rm_port = cons_data->ports[i]; + break; + } + } + if (rm_port) + tty_port = tty_port_get(&rm_port->port); + spin_unlock(&cons_data->ports_lock); + + if (!rm_port) + pr_warn("Received unexpected console characters for VMID %u\n", msg->vmid); + if (!tty_port) + return NOTIFY_DONE; + + count = tty_buffer_request_room(tty_port, msg->num_bytes); + tty_insert_flip_string(tty_port, msg->bytes, count); + tty_flip_buffer_push(tty_port); + + tty_port_put(tty_port); + return NOTIFY_OK; +} + +static ssize_t vmid_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(dev); + + if (rm_port->vmid == GH_VMID_SELF) + return sysfs_emit(buf, "self\n"); + + return sysfs_emit(buf, "%u\n", rm_port->vmid); +} + +static DEVICE_ATTR_RO(vmid); + +static struct attribute *rsc_mgr_tty_dev_attrs[] = { + &dev_attr_vmid.attr, + NULL +}; + +static const struct attribute_group rsc_mgr_tty_dev_attr_group = { + .attrs = rsc_mgr_tty_dev_attrs, +}; + +static const struct attribute_group *rsc_mgr_tty_dev_attr_groups[] = { + &rsc_mgr_tty_dev_attr_group, + NULL +}; + +static int rsc_mgr_tty_open(struct tty_struct *tty, struct file *filp) +{ + int ret; + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + if (!rm_port->open) { + ret = gh_rm_console_open(rm_port->vmid); + if (ret) { + pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = true; + } + + return tty_port_open(&rm_port->port, tty, filp); +} + +static void rsc_mgr_tty_close(struct tty_struct *tty, struct file *filp) +{ + int ret; + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + if (rm_port->open) { + if (rm_port->vmid != GH_VMID_SELF) { + ret = gh_rm_console_close(rm_port->vmid); + if (ret) + pr_warn("Failed to close RM console for vmid %d: %d\n", + rm_port->vmid, ret); + } + rm_port->open = false; + + tty_port_close(&rm_port->port, tty, filp); + } + +} + +static int rsc_mgr_tty_write(struct tty_struct *tty, const unsigned char *buf, int count) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + int ret; + + ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock); + if (ret > 0) + schedule_work(&rm_port->put_work); + + return ret; +} + +static unsigned int rsc_mgr_mgr_tty_write_room(struct tty_struct *tty) +{ + struct rm_cons_port *rm_port = dev_get_drvdata(tty->dev); + + return kfifo_avail(&rm_port->put_fifo); +} + +static void rsc_mgr_console_write(struct console *co, const char *buf, unsigned count) +{ + struct rm_cons_port *rm_port = co->data; + int ret; + + ret = kfifo_in_spinlocked(&rm_port->put_fifo, buf, count, &rm_port->fifo_lock); + if (ret > 0) + schedule_work(&rm_port->put_work); +} + +static struct tty_driver *rsc_mgr_console_device(struct console *co, int *index) +{ + struct rm_cons_port *rm_port = co->data; + + *index = rm_port->index; + return rm_port->port.tty->driver; +} + +static int rsc_mgr_console_setup(struct console *co, char *unused) +{ + int ret; + struct rm_cons_port *rm_port = co->data; + + if (!rm_port->open) { + ret = gh_rm_console_open(rm_port->vmid); + if (ret) { + pr_err("Failed to open RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = true; + } + + return 0; +} + +static int rsc_mgr_console_exit(struct console *co) +{ + int ret; + struct rm_cons_port *rm_port = co->data; + + if (rm_port->open) { + ret = gh_rm_console_close(rm_port->vmid); + if (ret) { + pr_err("Failed to close RM console for vmid %x: %d\n", rm_port->vmid, ret); + return ret; + } + rm_port->open = false; + } + + return 0; +} + +static const struct tty_operations rsc_mgr_tty_ops = { + .open = rsc_mgr_tty_open, + .close = rsc_mgr_tty_close, + .write = rsc_mgr_tty_write, + .write_room = rsc_mgr_mgr_tty_write_room, +}; + +static void rsc_mgr_port_destruct(struct tty_port *port) +{ + struct rm_cons_port *rm_port = container_of(port, struct rm_cons_port, port); + struct rm_cons_data *cons_data = rm_port->cons_data; + + spin_lock(&cons_data->ports_lock); + WARN_ON(cons_data->ports[rm_port->index] != rm_port); + cons_data->ports[rm_port->index] = NULL; + spin_unlock(&cons_data->ports_lock); + kfree(rm_port); +} + +static const struct tty_port_operations rsc_mgr_port_ops = { + .destruct = rsc_mgr_port_destruct, +}; + +static struct rm_cons_port *rsc_mgr_port_create(struct rm_cons_data *cons_data, gh_vmid_t vmid) +{ + struct rm_cons_port *rm_port; + struct device *ttydev; + unsigned int index; + int ret; + + rm_port = kzalloc(sizeof(*rm_port), GFP_KERNEL); + rm_port->vmid = vmid; + INIT_KFIFO(rm_port->put_fifo); + spin_lock_init(&rm_port->fifo_lock); + INIT_WORK(&rm_port->put_work, put_work_fn); + tty_port_init(&rm_port->port); + rm_port->port.ops = &rsc_mgr_port_ops; + + spin_lock(&cons_data->ports_lock); + for (index = 0; index < RSC_MGR_TTY_ADAPTERS; index++) { + if (!cons_data->ports[index]) { + cons_data->ports[index] = rm_port; + rm_port->index = index; + break; + } + } + spin_unlock(&cons_data->ports_lock); + if (index >= RSC_MGR_TTY_ADAPTERS) { + ret = -ENOSPC; + goto err_put_port; + } + + ttydev = tty_port_register_device_attr(&rm_port->port, cons_data->tty_driver, index, + cons_data->dev, rm_port, rsc_mgr_tty_dev_attr_groups); + if (IS_ERR(ttydev)) { + ret = PTR_ERR(ttydev); + goto err_put_port; + } + + return rm_port; +err_put_port: + tty_port_put(&rm_port->port); + return ERR_PTR(ret); +} + +static int rsc_mgr_console_probe(struct auxiliary_device *auxdev, + const struct auxiliary_device_id *aux_dev_id) +{ + struct rm_cons_data *cons_data; + struct rm_cons_port *rm_port; + int ret; + gh_vmid_t vmid; + + cons_data = devm_kzalloc(&auxdev->dev, sizeof(*cons_data), GFP_KERNEL); + if (!cons_data) + return -ENOMEM; + dev_set_drvdata(&auxdev->dev, cons_data); + cons_data->dev = &auxdev->dev; + + cons_data->tty_driver = tty_alloc_driver(RSC_MGR_TTY_ADAPTERS, + TTY_DRIVER_REAL_RAW | TTY_DRIVER_DYNAMIC_DEV); + if (IS_ERR(cons_data->tty_driver)) + return PTR_ERR(cons_data->tty_driver); + + cons_data->tty_driver->driver_name = "gh"; + cons_data->tty_driver->name = "ttyGH"; + cons_data->tty_driver->type = TTY_DRIVER_TYPE_SYSTEM; + cons_data->tty_driver->init_termios = tty_std_termios; + tty_set_operations(cons_data->tty_driver, &rsc_mgr_tty_ops); + + ret = tty_register_driver(cons_data->tty_driver); + if (ret) { + dev_err(&auxdev->dev, "Could not register tty driver: %d\n", ret); + goto err_put_tty; + } + + spin_lock_init(&cons_data->ports_lock); + + cons_data->rsc_mgr_notif.notifier_call = rsc_mgr_console_notif; + ret = gh_rm_register_notifier(&cons_data->rsc_mgr_notif); + if (ret) { + dev_err(&auxdev->dev, "Could not register for resource manager notifications: %d\n", + ret); + goto err_put_tty; + } + + rm_port = rsc_mgr_port_create(cons_data, GH_VMID_SELF); + if (IS_ERR(rm_port)) { + ret = PTR_ERR(rm_port); + dev_err(&auxdev->dev, "Could not create own console: %d\n", ret); + goto err_unreg_notif; + } + + strncpy(cons_data->console.name, "ttyGH", sizeof(cons_data->console.name)); + cons_data->console.write = rsc_mgr_console_write; + cons_data->console.device = rsc_mgr_console_device; + cons_data->console.setup = rsc_mgr_console_setup; + cons_data->console.exit = rsc_mgr_console_exit; + cons_data->console.index = rm_port->index; + cons_data->console.data = rm_port; + register_console(&cons_data->console); + + ret = gh_rm_get_vmid(&vmid); + if (!ret) + rsc_mgr_port_create(cons_data, vmid); + else + pr_warn("Failed to get this VM's VMID: %d. Not creating loop-back console\n", ret); + + return 0; +err_unreg_notif: + gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif); +err_put_tty: + tty_driver_kref_put(cons_data->tty_driver); + return ret; +} + +static void rsc_mgr_console_remove(struct auxiliary_device *auxdev) +{ + struct rm_cons_data *cons_data = dev_get_drvdata(&auxdev->dev); + + unregister_console(&cons_data->console); + gh_rm_unregister_notifier(&cons_data->rsc_mgr_notif); + tty_driver_kref_put(cons_data->tty_driver); +} + +static struct auxiliary_device_id rsc_mgr_console_ids[] = { + { .name = "gunyah.console" }, + {} +}; +MODULE_DEVICE_TABLE(auxiliary, rsc_mgr_console_ids); + +static struct auxiliary_driver rsc_mgr_console_drv = { + .probe = rsc_mgr_console_probe, + .remove = rsc_mgr_console_remove, + .id_table = rsc_mgr_console_ids, +}; +module_auxiliary_driver(rsc_mgr_console_drv); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Console");