From patchwork Tue Jan 9 19:37:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 13515277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D75AC4707B for ; Tue, 9 Jan 2024 19:38:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF40D8D0001; Tue, 9 Jan 2024 14:38:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DA2A68D0006; Tue, 9 Jan 2024 14:38:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF55C8D0001; Tue, 9 Jan 2024 14:38:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ACDBD8D0006 for ; Tue, 9 Jan 2024 14:38:07 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8215680A11 for ; Tue, 9 Jan 2024 19:38:07 +0000 (UTC) X-FDA: 81660783414.01.9CC0732 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf24.hostedemail.com (Postfix) with ESMTP id 670D7180006 for ; Tue, 9 Jan 2024 19:38:05 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=Sjt98MuC; spf=pass (imf24.hostedemail.com: domain of quic_eberman@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_eberman@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704829085; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zaO4CN5sb0ZBvCi/8rrxssP7owlddgLH9ngiDzhb0F8=; b=bT7qGJIbUN9pXZYGnzRPXbEJGZAgoTlTPZdIiont2UdBEJrHeb6SY50Z6UCycYZaIUoz8q H+ovrExrmY4lPf+uNCw3ujNO08rp9LmC6stiqfNKF4cxqNuomNPkALufEldWrFnnttGzhK wO61Oot6dvzrF2QnUu3e0Q8+PyOPAoc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704829085; a=rsa-sha256; cv=none; b=mjaa84ViG7SIDSWtiCy8oYxTnGhfxKlQtpC82BUmRaKzwO3n2YQE1/RsY5jmnW1yIltjvi Cu4V16K78ZIIqpEr+iD7OCji6mZ7GWET3T98z0smtt2pyjjXunygp3ilOQwkW0vjQC8+pX gtB5NMPOqoSamhDoIjEaprE5C7GuT98= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=Sjt98MuC; spf=pass (imf24.hostedemail.com: domain of quic_eberman@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_eberman@quicinc.com; dmarc=pass (policy=none) header.from=quicinc.com Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 409D9R98015182; Tue, 9 Jan 2024 19:37:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:date:subject:mime-version:content-type :content-transfer-encoding:message-id:references:in-reply-to:to :cc; s=qcppdkim1; bh=zaO4CN5sb0ZBvCi/8rrxssP7owlddgLH9ngiDzhb0F8 =; b=Sjt98MuCOMs/tT++NREfn+nkZgrQ/4wdoIOHtvC0hwulbxMqGo48ahHNIsj iyf5nTWs/L0jnK6BDZgDgWBAczooHX4qrt5ptIK9R77YM4pGuCTqLKNAgrPCAjtZ Gk1qOzHyOWTMulZ4Own1ROhD4hy/YSqSDNd9nGrfvQGjD/TZhbzx1IYgxjgplAOc HS6vw6j6FoAYmJDvTFyxLSShn+si+FBFAGkJCDwOSECuTd9BxAHCMiAXHvt0x5hJ YlOAU5FnEbZU6it6NWNiKotEwmTVGNIe8BMYQL7DkFX8RYh3C4PMuVF/VLnxn8Nq 13SCwoBwRUX8TjdEBMmdcVAOmgQ== Received: from nasanppmta03.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3vh3g699p4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 09 Jan 2024 19:37:57 +0000 (GMT) Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA03.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 409JbuHp011429 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 9 Jan 2024 19:37:56 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Tue, 9 Jan 2024 11:37:55 -0800 From: Elliot Berman Date: Tue, 9 Jan 2024 11:37:49 -0800 Subject: [PATCH v16 11/34] virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource MIME-Version: 1.0 Message-ID: <20240109-gunyah-v16-11-634904bf4ce9@quicinc.com> References: <20240109-gunyah-v16-0-634904bf4ce9@quicinc.com> In-Reply-To: <20240109-gunyah-v16-0-634904bf4ce9@quicinc.com> To: Alex Elder , Srinivas Kandagatla , Murali Nalajal , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Philip Derrin , Prakruthi Deepak Heragu , Jonathan Corbet , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Catalin Marinas , Will Deacon , Konrad Dybcio , Bjorn Andersson , Dmitry Baryshkov , "Fuad Tabba" , Sean Christopherson , "Andrew Morton" CC: , , , , , , Elliot Berman X-Mailer: b4 0.13-dev X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: uIJ4ZLH2FZ4hRQD_l7p3KGjz_2w7OoFa X-Proofpoint-ORIG-GUID: uIJ4ZLH2FZ4hRQD_l7p3KGjz_2w7OoFa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 bulkscore=0 spamscore=0 malwarescore=0 phishscore=0 clxscore=1015 impostorscore=0 suspectscore=0 lowpriorityscore=0 mlxlogscore=867 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2401090158 X-Rspamd-Queue-Id: 670D7180006 X-Rspam-User: X-Stat-Signature: mxmny7s3ub8wgiampzz8ztr5ywoq7qb4 X-Rspamd-Server: rspam03 X-HE-Tag: 1704829085-953384 X-HE-Meta: U2FsdGVkX193ZoALZr3yD5gQSbyCaYqGZSj7nB0Zh8Y42GBQW63dkZa67veqrye5kfCtoopvpyto0u3ctApdDqZHc5z40JPHuIh2fxgLmx6YquDnXr4zmG+B9Bz7ozP8r8A20CDYL9dSXCMdhIX+pD4Mc8dz0LdCh03HARAu8yCG15YneLA11D4U2vJhiJ3amXubQDqmVkRrzq45pHhDV+BxvHdI/lqIkqHrjJBWuoevBh4WAHLuj3wlJmypxLI1GXkwpakWi3btYsGt/w+jZgpOjCpqUZ4IY73BlYHjPZjAu9lOTl0QjLPJ2dIaWu1602sC9+oQeQtMAerrT1gYIWQXrN7ng/po8MtkP1GoD/oJ4clr3bB5poZnoch91jUwekpzBoJV+YyAP8c2Pb9Dvzcz3hF/S2B6TKlPkRQwigoAjzAyXSzqpSHN1mHECPAJJg54Ll+9H27dsivqRA+mUA0/XKaTdyJnK8hEQfMe0pne52wbYT5M9x7lqiNV6YZoBuuE1Tr1itK2FLHSt+2sZYnufzTypeKhHIrFw5TRwGeAu28yE8QXdETrex0b4ANSuNeMW7sz/VNndtmIDE3x7WxFgvgcibtOJsn3lPOwX7VVvcBox3jLZCQy1hkASSOTryf2Ue6nQICIG3jn+hazKZ7/a2oOlBL9kLbHPixRtfoYQVOISlQxCVH9LRTQdvP1j4MttaiIwdeFRbHO9VUXf1bFzJIsTEGuD4eCYjvMMiJQbE4Zj8H8HmveUfy32oHRlYiOFJwonp4JCrxRZBkjRurRDb0y3XBbkv5OkUaoGQ0zstkY4uhB1eydkW9loLYYiuKi+VaTLPFQgqBsMe0pirwlNMSq8iyyvbkBSjUIOLXXxSinF/Cb/IX55/PMkyjp5JqK3D4UeFWev/FNDKKDG03PSsG81FRKGlUMoIp93Dfrg/QgGyUsCEvCmH1CfH4Q6qJfOmL+y4h5Mhuahrh Su0WkX2I WDllnD/4WnINXiL1cv3aQqJ3HYFZwMD8BkUaKpCkGLxDOQdT54AVow6cjOfKVGrNnAGJwhW6CSv6h482+dko2zx64t0lPongeQDLF6EFb7cXi6kGXLkfA91fva/TkQaO0DceYoA8dN5ZueaoFwlEiwklWLIY+ASXqtAH8zYObsNLgrmL4bok8a7WiNCSW8PirhQYfLXCkGeASCVwGQ+hvLOq4tj5S2kmAi8gQW3VzSRUEO3HR8KcKB7GTIJEd6TxfEoLPybEG/1CeS1YdrE6A+GqXIuZ+9tDFoN4JsQK9lLu0CmuhTFhUqBUVBDXFUuQ0sX+V7dxWKPfgaBcTkcYqZ+t179zvR7xOwvvGPjVN6SH8ZjLBifFR66Vw1T+d0hMV7Eho3qMQdyOu2h3iELZUvRrGpa5BKz5urmHZ7Zz4kTMcwRB4bO/GFExa5tDQKfyzHFv4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When booting a Gunyah virtual machine, the host VM may gain capabilities to interact with resources for the guest virtual machine. Examples of such resources are vCPUs or message queues. To use those resources, we need to translate the RM response into a gunyah_resource structure which are useful to Linux drivers. Presently, Linux drivers need only to know the type of resource, the capability ID, and an interrupt. On ARM64 systems, the interrupt reported by Gunyah is the GIC interrupt ID number and always a SPI or extended SPI. Signed-off-by: Elliot Berman --- arch/arm64/include/asm/gunyah.h | 36 +++++++++ drivers/virt/gunyah/rsc_mgr.c | 175 +++++++++++++++++++++++++++++++++++++++- drivers/virt/gunyah/rsc_mgr.h | 5 ++ include/linux/gunyah.h | 3 + 4 files changed, 218 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h new file mode 100644 index 000000000000..0cd3debe22b6 --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef _ASM_GUNYAH_H +#define _ASM_GUNYAH_H + +#include +#include + +static inline int arch_gunyah_fill_irq_fwspec_params(u32 virq, + struct irq_fwspec *fwspec) +{ + /* Assume that Gunyah gave us an SPI or ESPI; defensively check it */ + if (WARN(virq < 32, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq <= 1019) { + fwspec->param_count = 3; + fwspec->param[0] = 0; /* GIC_SPI */ + fwspec->param[1] = virq - 32; /* virq 32 -> SPI 0 */ + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + } else if (WARN(virq < 4096, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq < 5120) { + fwspec->param_count = 3; + fwspec->param[0] = 2; /* GIC_ESPI */ + fwspec->param[1] = virq - 4096; /* virq 4096 -> ESPI 0 */ + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + } else { + WARN(1, "Unexpected virq: %d\n", virq); + return -EINVAL; + } + return 0; +} + +#endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 45f9514cfe0e..efc0970ae4cc 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -9,9 +9,12 @@ #include #include #include +#include #include #include +#include + #include "rsc_mgr.h" #include "vm_mgr.h" @@ -121,6 +124,7 @@ struct gunyah_rm_message { * @send_ready: completed when we know Tx message queue can take more messages * @nh: notifier chain for clients interested in RM notification messages * @miscdev: /dev/gunyah + * @irq_domain: Domain to translate Gunyah hwirqs to Linux irqs */ struct gunyah_rm { struct device *dev; @@ -138,6 +142,7 @@ struct gunyah_rm { struct blocking_notifier_head nh; struct miscdevice miscdev; + struct irq_domain *irq_domain; }; /** @@ -178,6 +183,143 @@ static inline int gunyah_rm_error_remap(enum gunyah_rm_error rm_error) } } +struct gunyah_irq_chip_data { + u32 gunyah_virq; +}; + +static struct irq_chip gunyah_rm_irq_chip = { + /* clang-format off */ + .name = "Gunyah", + .irq_enable = irq_chip_enable_parent, + .irq_disable = irq_chip_disable_parent, + .irq_ack = irq_chip_ack_parent, + .irq_mask = irq_chip_mask_parent, + .irq_mask_ack = irq_chip_mask_ack_parent, + .irq_unmask = irq_chip_unmask_parent, + .irq_eoi = irq_chip_eoi_parent, + .irq_set_affinity = irq_chip_set_affinity_parent, + .irq_set_type = irq_chip_set_type_parent, + .irq_set_wake = irq_chip_set_wake_parent, + .irq_set_vcpu_affinity = irq_chip_set_vcpu_affinity_parent, + .irq_retrigger = irq_chip_retrigger_hierarchy, + .irq_get_irqchip_state = irq_chip_get_parent_state, + .irq_set_irqchip_state = irq_chip_set_parent_state, + .flags = IRQCHIP_SET_TYPE_MASKED | + IRQCHIP_SKIP_SET_WAKE | + IRQCHIP_MASK_ON_SUSPEND, + /* clang-format on */ +}; + +static int gunyah_rm_irq_domain_alloc(struct irq_domain *d, unsigned int virq, + unsigned int nr_irqs, void *arg) +{ + struct gunyah_irq_chip_data *chip_data, *spec = arg; + struct irq_fwspec parent_fwspec = {}; + struct gunyah_rm *rm = d->host_data; + u32 gunyah_virq = spec->gunyah_virq; + int ret; + + if (nr_irqs != 1) + return -EINVAL; + + chip_data = kzalloc(sizeof(*chip_data), GFP_KERNEL); + if (!chip_data) + return -ENOMEM; + + chip_data->gunyah_virq = gunyah_virq; + + ret = irq_domain_set_hwirq_and_chip(d, virq, chip_data->gunyah_virq, + &gunyah_rm_irq_chip, chip_data); + if (ret) + goto err_free_irq_data; + + parent_fwspec.fwnode = d->parent->fwnode; + ret = arch_gunyah_fill_irq_fwspec_params(chip_data->gunyah_virq, + &parent_fwspec); + if (ret) { + dev_err(rm->dev, "virq translation failed %u: %d\n", + chip_data->gunyah_virq, ret); + goto err_free_irq_data; + } + + ret = irq_domain_alloc_irqs_parent(d, virq, nr_irqs, &parent_fwspec); + if (ret) + goto err_free_irq_data; + + return ret; +err_free_irq_data: + kfree(chip_data); + return ret; +} + +static void gunyah_rm_irq_domain_free_single(struct irq_domain *d, + unsigned int virq) +{ + struct irq_data *irq_data; + + irq_data = irq_domain_get_irq_data(d, virq); + if (!irq_data) + return; + + kfree(irq_data->chip_data); + irq_data->chip_data = NULL; +} + +static void gunyah_rm_irq_domain_free(struct irq_domain *d, unsigned int virq, + unsigned int nr_irqs) +{ + unsigned int i; + + for (i = 0; i < nr_irqs; i++) + gunyah_rm_irq_domain_free_single(d, virq); +} + +static const struct irq_domain_ops gunyah_rm_irq_domain_ops = { + .alloc = gunyah_rm_irq_domain_alloc, + .free = gunyah_rm_irq_domain_free, +}; + +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource) +{ + struct gunyah_resource *ghrsc; + int ret; + + ghrsc = kzalloc(sizeof(*ghrsc), GFP_KERNEL); + if (!ghrsc) + return NULL; + + ghrsc->type = hyp_resource->type; + ghrsc->capid = le64_to_cpu(hyp_resource->cap_id); + ghrsc->irq = IRQ_NOTCONNECTED; + ghrsc->rm_label = le32_to_cpu(hyp_resource->resource_label); + if (hyp_resource->virq) { + struct gunyah_irq_chip_data irq_data = { + .gunyah_virq = le32_to_cpu(hyp_resource->virq), + }; + + ret = irq_domain_alloc_irqs(rm->irq_domain, 1, NUMA_NO_NODE, + &irq_data); + if (ret < 0) { + dev_err(rm->dev, + "Failed to allocate interrupt for resource %d label: %d: %d\n", + ghrsc->type, ghrsc->rm_label, ret); + kfree(ghrsc); + return NULL; + } + ghrsc->irq = ret; + } + + return ghrsc; +} + +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc) +{ + irq_dispose_mapping(ghrsc->irq); + kfree(ghrsc); +} + static int gunyah_rm_init_message_payload(struct gunyah_rm_message *message, const void *msg, size_t hdr_size, size_t msg_size) @@ -712,6 +854,8 @@ static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, static int gunyah_rm_probe(struct platform_device *pdev) { + struct irq_domain *parent_irq_domain; + struct device_node *parent_irq_node; struct gunyah_rm *rm; int ret; @@ -737,15 +881,43 @@ static int gunyah_rm_probe(struct platform_device *pdev) if (ret) return ret; + parent_irq_node = of_irq_find_parent(pdev->dev.of_node); + if (!parent_irq_node) { + dev_err(&pdev->dev, + "Failed to find interrupt parent of resource manager\n"); + return -ENODEV; + } + + parent_irq_domain = irq_find_host(parent_irq_node); + if (!parent_irq_domain) { + dev_err(&pdev->dev, + "Failed to find interrupt parent domain of resource manager\n"); + return -ENODEV; + } + + rm->irq_domain = irq_domain_add_hierarchy(parent_irq_domain, 0, 0, + pdev->dev.of_node, + &gunyah_rm_irq_domain_ops, + NULL); + if (!rm->irq_domain) { + dev_err(&pdev->dev, "Failed to add irq domain\n"); + return -ENODEV; + } + rm->irq_domain->host_data = rm; + + rm->miscdev.parent = &pdev->dev; rm->miscdev.name = "gunyah"; rm->miscdev.minor = MISC_DYNAMIC_MINOR; rm->miscdev.fops = &gunyah_dev_fops; ret = misc_register(&rm->miscdev); if (ret) - return ret; + goto err_irq_domain; return 0; +err_irq_domain: + irq_domain_remove(rm->irq_domain); + return ret; } static void gunyah_rm_remove(struct platform_device *pdev) @@ -753,6 +925,7 @@ static void gunyah_rm_remove(struct platform_device *pdev) struct gunyah_rm *rm = platform_get_drvdata(pdev); misc_deregister(&rm->miscdev); + irq_domain_remove(rm->irq_domain); } static const struct of_device_id gunyah_rm_of_match[] = { diff --git a/drivers/virt/gunyah/rsc_mgr.h b/drivers/virt/gunyah/rsc_mgr.h index 205b9ea735e5..52711de77bb7 100644 --- a/drivers/virt/gunyah/rsc_mgr.h +++ b/drivers/virt/gunyah/rsc_mgr.h @@ -99,6 +99,11 @@ struct gunyah_rm_hyp_resources { int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, struct gunyah_rm_hyp_resources **resources); +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource); +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); + int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index acd70f982425..ede8abb1b276 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -29,6 +29,9 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; unsigned int irq; + + struct list_head list; + u32 rm_label; }; /******************************************************************************/