From patchwork Wed Feb 1 12:53:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 13124408 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C6DEC636D3 for ; Wed, 1 Feb 2023 14:26:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4ItyPRCBUskcgkG6/IvScSzIwSTG/8ooEvXAgEHx8nw=; b=eIfJ3OuYs6Nbfl CbnkSLEyFpkrtBKnENgjHVuYV2L8Lmb6TRFL5FgIFzuLuJMhOt2r/uy1O8fTldJ9F0hTE98XZdfZs ZU8lP0TIqAd+tBst4OQB/aqNOOEr7L6PPGLD/Ng8+j9tRGnylue+mMpHj3+jufgDP2q20zAwOYKY5 IVa97zSVn3JFU0A4U9QxCIg9nzoKtbHcie9tIxWv2WJf0YclpToKiNoMVKbsExVdET/u75Ocqvuhi bsy8xhKVr6dHkwye/PWzXP5jkH75PYZBD0cxsMDLDw8u2WVHMZoXbnL4u2cHqebjQe7V3SikFinsp OAU2UNWc5M8MJ7/qqsgA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pNE3S-00CKsR-0k; Wed, 01 Feb 2023 14:25:42 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pNDhY-00CBiH-JU for linux-arm-kernel@bombadil.infradead.org; Wed, 01 Feb 2023 14:03:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9Dh/SuicpAUNOw64qrqjagLAfYYRP7GDUICNSAPWkEY=; b=W7rXHMENSCMebFZKwhasZIccvC Kgtd3qClTP3Ad0Dr3DV2QYXdNYg/3BEmsSR+E7TF8yVw+ALOwcr6HwyfHyiEmsxtYt5WmQKhjvUrN 9sKInDLWUEc/lz4Lx7mryq2xFfEdKXAlAZHoF651FRJ7qrrBwWJ51E586zVtkw3Ujw7bRiE5/6Xp6 WG5fOobE7JmUbXwSv89Yr56DcRRwHNJBm+2QihZdqx4o5X6HNrNUKCKYGPRuKI6MX92tHinx7wO4Q nyzs3XB/J4DOcXyivwqEk+YDdeyXxqYzHBnLmZx0RGLCh2seYsXJnkflXOHL/Gyx43PNP+Iz/87NK 2BN3DazA==; Received: from mail-wr1-x432.google.com ([2a00:1450:4864:20::432]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pNChk-004m2X-27 for linux-arm-kernel@lists.infradead.org; Wed, 01 Feb 2023 12:59:17 +0000 Received: by mail-wr1-x432.google.com with SMTP id t7so8870520wrp.5 for ; Wed, 01 Feb 2023 04:59:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9Dh/SuicpAUNOw64qrqjagLAfYYRP7GDUICNSAPWkEY=; b=sXyiSrRhQ03FjrQeAHXjhFoy/QwVyYKEscKoUYb3OZRIaH21HsrZEtSD5AL7zuiIFV vDmfHFdxA8m9jWRbzQBklMRmgEyQ8+yrurctzwyqsImPtLAcqi2b+RXEPT6gAqI7QsND +2RffjhJqi6hLIOeiiX9cDf5HGfIXxluoUmwfM5XmIHifSZEe86EmJPHIsdVqVX6OOzg e1uI55R5fQEInMR7Q4YM0Y6z0F2RxFH8mM5Slfd91Sypx3NOF0Hf+r9f59c975D6zMRc mL3qqvwrhHy14hSMyfJGg5gGthQWrjaqR0dYTk08kP4fcqUHPg0Bbxxmk0MaT4GaZEDO x2jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9Dh/SuicpAUNOw64qrqjagLAfYYRP7GDUICNSAPWkEY=; b=aKaYOfOqzokoKp4Q0AEiaTSlXRVGH4sL0YixKPiK3ZWN9UErlqpdupWY6156rrgKjR azq3Q3AyDFuQzaNoQ4HgV0dFTBG7IuYl4MqaOR6ONDAmCSUzxyQp1nVMZQOIm7VqX5Dz 7W/D6MC3Fq1w78KBJ4j2Ykb2DesxRUH79G9APvkK2qOsLytiISIwppmw8HtXMBBk+sXP ApZoWHZHfOgTssKLUB2OX0+vzuFPVEiThnmRnEPDrC+riiRuNhkV7FcZvQaeFiJBcZyV SjopWWVdY6AYT/gPK6qakkI9gqDWuxFJ3A57v6XtbfLwg3LxSJr0iPdaLEnYkOJru3nx ztLw== X-Gm-Message-State: AO0yUKUm/bivI89wC4N79x3NooqOD+xG5l42IQra4G35RQn6znGWG4E5 BE2M9jBmxxO3cY+dSOahTSZWIg== X-Google-Smtp-Source: AK7set+68/Lq7+7dw7L5pKpw0HIza+anyCOmSfiVtB5lzEVXRlZZF1Y8GjmD8XI2Ho4LfqXQo9439g== X-Received: by 2002:a5d:5092:0:b0:2bf:c753:573f with SMTP id a18-20020a5d5092000000b002bfc753573fmr1910182wrt.24.1675256387073; Wed, 01 Feb 2023 04:59:47 -0800 (PST) Received: from localhost.localdomain (054592b0.skybroadband.com. [5.69.146.176]) by smtp.gmail.com with ESMTPSA id m15-20020a056000024f00b002bfae16ee2fsm17972811wrz.111.2023.02.01.04.59.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 04:59:46 -0800 (PST) From: Jean-Philippe Brucker To: maz@kernel.org, catalin.marinas@arm.com, will@kernel.org, joro@8bytes.org Cc: robin.murphy@arm.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, smostafa@google.com, dbrazdil@google.com, ryan.roberts@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, iommu@lists.linux.dev, Jean-Philippe Brucker Subject: [RFC PATCH 35/45] iommu/arm-smmu-v3-kvm: Pass a list of SMMU devices to the hypervisor Date: Wed, 1 Feb 2023 12:53:19 +0000 Message-Id: <20230201125328.2186498-36-jean-philippe@linaro.org> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230201125328.2186498-1-jean-philippe@linaro.org> References: <20230201125328.2186498-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230201_125913_157253_032F4949 X-CRM114-Status: GOOD ( 24.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Build a list of devices and donate the page to the hypervisor. At this point the host is trusted and this would be a good opportunity to provide more information about the system. For example, which devices are owned by the host (perhaps via the VMID and SW bits in the stream table, although we populate the stream table lazily at the moment.) Signed-off-by: Jean-Philippe Brucker --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c | 123 +++++++++++++++++- 1 file changed, 120 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c index 4092da8050ef..1e0daf9ea4ac 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-kvm.c @@ -4,15 +4,78 @@ * * Copyright (C) 2022 Linaro Ltd. */ +#include #include #include #include "arm-smmu-v3.h" +struct host_arm_smmu_device { + struct arm_smmu_device smmu; + pkvm_handle_t id; +}; + +#define smmu_to_host(_smmu) \ + container_of(_smmu, struct host_arm_smmu_device, smmu); + +static size_t kvm_arm_smmu_cur; +static size_t kvm_arm_smmu_count; +static struct hyp_arm_smmu_v3_device *kvm_arm_smmu_array; + static int kvm_arm_smmu_probe(struct platform_device *pdev) { - return -ENOSYS; + int ret; + bool bypass; + size_t size; + phys_addr_t ioaddr; + struct resource *res; + struct arm_smmu_device *smmu; + struct device *dev = &pdev->dev; + struct host_arm_smmu_device *host_smmu; + struct hyp_arm_smmu_v3_device *hyp_smmu; + + if (kvm_arm_smmu_cur >= kvm_arm_smmu_count) + return -ENOSPC; + + hyp_smmu = &kvm_arm_smmu_array[kvm_arm_smmu_cur]; + + host_smmu = devm_kzalloc(dev, sizeof(*host_smmu), GFP_KERNEL); + if (!host_smmu) + return -ENOMEM; + + smmu = &host_smmu->smmu; + smmu->dev = dev; + + ret = arm_smmu_fw_probe(pdev, smmu, &bypass); + if (ret || bypass) + return ret ?: -EINVAL; + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + size = resource_size(res); + if (size < SZ_128K) { + dev_err(dev, "unsupported MMIO region size (%pr)\n", res); + return -EINVAL; + } + ioaddr = res->start; + host_smmu->id = kvm_arm_smmu_cur; + + smmu->base = devm_ioremap_resource(dev, res); + if (IS_ERR(smmu->base)) + return PTR_ERR(smmu->base); + + ret = arm_smmu_device_hw_probe(smmu); + if (ret) + return ret; + + platform_set_drvdata(pdev, host_smmu); + + /* Hypervisor parameters */ + hyp_smmu->mmio_addr = ioaddr; + hyp_smmu->mmio_size = size; + kvm_arm_smmu_cur++; + + return 0; } static int kvm_arm_smmu_remove(struct platform_device *pdev) @@ -33,6 +96,36 @@ static struct platform_driver kvm_arm_smmu_driver = { .remove = kvm_arm_smmu_remove, }; +static int kvm_arm_smmu_array_alloc(void) +{ + int smmu_order; + struct device_node *np; + + kvm_arm_smmu_count = 0; + for_each_compatible_node(np, NULL, "arm,smmu-v3") + kvm_arm_smmu_count++; + + if (!kvm_arm_smmu_count) + return 0; + + /* Allocate the parameter list shared with the hypervisor */ + smmu_order = get_order(kvm_arm_smmu_count * sizeof(*kvm_arm_smmu_array)); + kvm_arm_smmu_array = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, + smmu_order); + if (!kvm_arm_smmu_array) + return -ENOMEM; + + return 0; +} + +static void kvm_arm_smmu_array_free(void) +{ + int order; + + order = get_order(kvm_arm_smmu_count * sizeof(*kvm_arm_smmu_array)); + free_pages((unsigned long)kvm_arm_smmu_array, order); +} + /** * kvm_arm_smmu_v3_init() - Reserve the SMMUv3 for KVM * @count: on success, number of SMMUs successfully initialized @@ -44,12 +137,36 @@ int kvm_arm_smmu_v3_init(unsigned int *count) { int ret; + /* + * Check whether any device owned by the host is behind an SMMU. + */ + ret = kvm_arm_smmu_array_alloc(); + *count = kvm_arm_smmu_count; + if (ret || !kvm_arm_smmu_count) + return ret; + ret = platform_driver_probe(&kvm_arm_smmu_driver, kvm_arm_smmu_probe); if (ret) - return ret; + goto err_free; - *count = 0; + if (kvm_arm_smmu_cur != kvm_arm_smmu_count) { + /* A device exists but failed to probe */ + ret = -EUNATCH; + goto err_free; + } + + /* + * These variables are stored in the nVHE image, and won't be accessible + * after KVM initialization. Ownership of kvm_arm_smmu_array will be + * transferred to the hypervisor as well. + */ + kvm_hyp_arm_smmu_v3_smmus = kern_hyp_va(kvm_arm_smmu_array); + kvm_hyp_arm_smmu_v3_count = kvm_arm_smmu_count; return 0; + +err_free: + kvm_arm_smmu_array_free(); + return ret; } void kvm_arm_smmu_v3_remove(void)