From patchwork Wed May 8 18:57:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6FA2C25B5F for ; Wed, 8 May 2024 18:58:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=b/fNP2YVNlHLeRtKozAij7HN6ck0wqSZ3ehLmk5QhTs=; b=4SM5jHbCypfT8E 49mF/7utNyxNgbNH0uSWR9DW7V9IXS7fsjCmstVYvfUc9fyLV16Jy1KAerJPYEM9Ffm/TyHkChdZz Nuzbo5xUIPPbnbQ7mcQSWwA/Z8lCJbaNv1txShf8+CKJT5vy08wLMBxENCNUjbSGwSfGo8yqab7Yz nHW9buZF1jetTYTrwmcvxxUwGGZFbOaUwQQfpZhDPuhhltw6lSz1Svc/LbnHN5anoiG6we9sIGr+U UwZE+43YpcEzXUaFAiGoj3i6RE6XBU0QVdSOgW5uncXtpcPRwu2GKgbsnV5pA35Ms0ay1UJ0Qiqsl gOtE6VTJhCCO1elWo3dQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU9-0000000GcG8-3niq; Wed, 08 May 2024 18:57:49 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mTy-0000000Gc7U-1kus for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WoOQ0Gf0c+FzBoh1uOkLFql0ew+CKN1i1tPCy2+j5O37viVvz1vYywXgXjZH7+ajA69F4u747AHw+NdExRX3GgbyCpmBgd99VX2VqNPMtnumCNYn1YzrAsatKHWdZcUUXb4Wve0CdyASg4BlM/dssWjKOP8k6RjuebGc0wJ4dhFm/zV77+oG6I4oIV2Is3hC3egAKF83Xrk3mCouUa2fOoi//rp48ZdoZLHkI+O8kydGC1QIO91pje+3pL5lEAwgJVNpigHPgYu2q58SqZWMHiEO0tPBep6ATiwjQYox2lUiRdyv4AD4U75fSggFEUYBaSXA711mY5FbkeLWgv01+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zA4LpcSGupkkN6Fq3FTszpjZ3K+o8PFd02Wzq1eQV8o=; b=NfQtKIkHOqZ4s6zlgSp0wzcc2hCcWnxtPwV42IXUcUh0OKb+/lZ1r2Cz0nlq8qH3xwyrLIjFGRizKyjUmuwvQpsKVvUH8EdhWZkRb2ulpjOsE1NR9pQYTHQ/QJQGKkIgyJ+I3C7znU4RpDcB5PWb08ja8q9jJYX1LeFW9avtvSshBb0mxYTkv6wqZm5mHIEbGqSOwbiQTi/heVW3yN2QO6W6Vc2z81iwmCr94HyV8PskXpoFRMvLuwOIRXz+PmOVxYIOq3Ho7yyjjG4Zp0GqdJ3+mXqxA26+W8XK31qHb+/2xQK/EWpJTzWL0eWcEItJPOzFBO7W2H2dpNh/ibQr8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zA4LpcSGupkkN6Fq3FTszpjZ3K+o8PFd02Wzq1eQV8o=; b=c7K50nfUN12xrG/E6q7x/PGHiUv3uJosZMVLRxTZNdbMqllqmdwrhnU0wAUXxe8tTFMgjDGPI9fUoYCy56+opAQSpWCOTTRAvTqi7sr12tGinZN4xvLgjI6iFcwCoFiLmvbejn0eoCM0oRjreukBH82ojmT1ZaS68FOl5IYuCoTrm6McSNH+WmFe6VwnMpXzIiOP2FwKGzM0zljAD36SjCbOkiSDKQPTPCbNKLb2pn+8CwwxaaPtqzqnEYe8rCufqP+yZdMyQQH4dsjqm0/SlOWBW30aJ10d7hcmWY4NcPKCqy4J23865rrxafxfUwpU0YxnfFkvfz1KxOfrjjT5Zw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:24 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:24 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 01/14] iommu/arm-smmu-v3: Convert to domain_alloc_sva() Date: Wed, 8 May 2024 15:57:09 -0300 Message-ID: <1-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P221CA0022.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::16) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 7b3f0850-b106-4bf7-2237-08dc6f90b28d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: pF3r7IQ4dohks5tT5YKmgjHm4DLejXSJlq+3MmqATRf0q0bB4GALePOHs2+J23xRRzllKBJojqyTC5ON0k7ekgV8VkeLSf1K7IS0DIDTO+VX6nr+hbfPyBufyErRNs80OTKFNC4/XrHlIaZZHT7+JUIaFU10Am9ONonxYDm/Jt+80vFWAovz4pWyRcXuwNoinUvDTO3IklV5e2S1tW25pu7E77duHeJ6IW1MZBfDFU8hlCLg9aTeKose7Z5dvd8TNT8aj6kwGd+vq8n4wvwZ5Aq5VG1FHjjU4ACddJRsQ0mS3BK3Q9O00kCDzWQaakanro2f688KYr42ZjhmhRLOIxH8/nsStEuda49hrEjWZh+mS+HDrwvnTnfDAH4bJoiD8pz0TDpynjBuqudE1M0IwoeWZ/Xr30mfht/pglx/BRlCRJMQYlq1Igu1sKeAq07M1iWoIukyboOVeuPdoHPx5ivkf/421xa6fdcRPNus8Iq2855FZsulYXnHTksDXWGC/mFZaq56pBtVrBYUIdpefVe2RoSSCdmpSTLjagXbprkLFIzK9MrZ7U9va5lBq/JFZqyB6ffAvTy0PODv9oinOFVEMBm7irVWVZfYiQMVAUW5C9gjzRAZGMvyUCDmMSwLE5Rla29LDkHPEZrUKXvpKlFyR+QDuP3aP859GXZzIT1ngWqd2NTd4DJsgbWNzAI/60QhMkzoJZPZwlYH9dTfQM9qQgZ1VVCljG4GF2NUOavNRZbIBnaA0Sfk8WJQBBybw4plRKh9cOwcqB22TGwc7gLdM3Cg8l+thbXgPicAP8YG6o4zq5TydpYtZN72JPRXgiYW1GkJih1+vsjFQ62YRgdy2UM/RdimBMtVmJqLnJujFsBOXCZGBi/ML/7GeN/OnywzpxQD/yDWKx193ttV9jHeEFri0BGbyFGvOKO3J4ThOX4UhpztrB3Nnruja/J08q4a2iqL92ZuaAOcjizYmzMxAr9hLtjwcD8bbvutOenjqsZCyM79FZEoI2+KeEjQoFNPpI++rfD8wRKWuZy1hfpzvJJB5rBbzkktloNTZWtyQNmgReY6DnJMSsYZ0niYFjKn+AStfZNb7Br8NY6ccgfgnJxq6VTaoQaVXgwxUabbT1Kwd0PWIF2o4B55JGTjr2YZeTkckXM+5/9wuFNhpfnIlx+1kzjjFDQb1pXVN+2oTwsXMsdDr64nnyiSRk00Ser6OVQV+kWG/Ju99sBocSfRLZdzrfDa1Mp+rcJGqAaHaNKhkmAFJMNmp38V9gLvcrjl6TiO/5uy/XcK2HAyJw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AqEoZUUqR+1N61bL046esm6RsqXRo5W+xXuePUCpsWV2aUDs5bnc5UaomwMwMFNw1Q9Uhssv4Nb1B0qddqI1jqJDhfl43rpxrAG+Yprusghio3UAUGmZ6lsPPSpmRA07g6MurVQw420uUQ8ufxpHmxJlql0qE9sMoEsOCL+Zd4KbGcwkdhKntnzSSdqV0lPc1/ittbH6LJyoSiMDkAcGwaf0EEJUUxjTzv34c1nSxSN8uQFEu957cO04HExNjQACGOyzN24KhRcDqyTJur3CCo+qkPW+u9FWJ8opqlbBIrpBUkwLNaT52pv/B+fvBkjiuhBYRQFL1aYWH2wRnLX2FYvGPxeB4BiZDhDxTcg0BTl1QxcAakQDPnnBhnZVeQZtxfqfOwgLG8JWgQLnafYDaRML2UNjvbzi31BCI2Szw4l7ZggaPFw3vg4RjNij4SnZzR7R1ySbbUKU9MHC2Z7w++9sMkHgyFLCu7Daht97S/G2njLCXEOR3gmg/C49xRJEu0Nw8g1xtqeLqbtVDex8i/1An4BDrJqAceMWf+GkTGedWKgUKUr/O89JvHFMG088GSflJP1CHwFe+ubyyey0wzF4DsRMQb1F44tIKELGl5v56IevB7UftMDr0XaR7O9QWbY5Uer2kk2vycOU0caOiDtZ0KTu2DKtOameXVkndxTeI11WXVZlj36N0X4mxoVpw6GtlRAwpzBkX0m+4S6843WiW0c7WZmoi4ft7UytYWVSSw5pr7MfknmT0/imj6vyGPPWfLg7AqTE38YiBbJ6YSNBZ74W3ZqNKXM7EpPYkRBpnpvIRo0I5WGfUitQcFIlqqTnVlhdM+b9ae58cuk35kTat3x281QyTELU6b2lray0crnzVkU8OsxCfvrsrmd/DfnybInTIly8vvQoJM3uRP7yrBpvnZ105wgJaoLnFwGjk+5yTORkQrTLREu/gs73F8ljkKXHfVuKPgI9OFkLGWipw4xgc4NslQQ/sFLYRCvURgwx/GgwrseytaqHQdBV4MTO6wHT3GI8zzW9IVzuE/kpoSVbhuLDqf1wb891iKIW0nDraksxVENGigabjr/p5Ia7sStNRq4NbIeUBJapcguxZcomjK+wOv+aG4B/i4a63B1Rax+Czy82b8pkiRWMiwHe/d1DnzPG0MrSijufGjXd335EfTa1s9fvjCxulXFbrM/FJFTFb29LheMoe4LKZIcTRsx0Ocgs4AUPdvIhTe+2fTNTE1TEuqBu0ziNH4qbttDAaT5tN0VzRkt4Kq63wWxfjWPFqtJZ9YCwAcUVtIEnm4DlZaeOWXMqG53NoDld1jts9/gKBaGrt2yTuRHmGBoX8MMgkcsQ+QTGidd83z2u1Ef6SDJgTgC+76W1de8IA/5fo9JXMAQoRf6EtvxYNgZBWyBD3EChjFRYeA/dqURRbcDg5DevUrm/Sgneo4lbmWaISTuK6vyespH7TWvEhQB06gsKyF4wwg8xZPPy+2NIFrrxX2+SwIQMX0LrP453cj0Um4kiyKxOL+rbZIecmL3LjQTf1oLNIIOmeEW/to7xqQKq5lst66XMrbYTBZSVx3Z9up2yEmcjMmGrdxsD X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7b3f0850-b106-4bf7-2237-08dc6f90b28d X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:24.3283 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NddKqfuEeunMHL07fynZFbndX6uLiGdMciXq7RE5T2ySZsoHzHOe7qhEFcsMqrAN X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115738_903325_07FC6209 X-CRM114-Status: GOOD ( 12.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This allows the driver the receive the mm and a always a device during allocation. Later patches need this to properly setup the notifier when the domain is first allocated. Remove ops->domain_alloc() as SVA was the only remaining purpose. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Reviewed-by: Michael Shavit Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 +--------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +++++- 3 files changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index e490ffb3801545..28f8bf4327f69a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -656,13 +656,15 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { .free = arm_smmu_sva_domain_free }; -struct iommu_domain *arm_smmu_sva_domain_alloc(void) +struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev, + struct mm_struct *mm) { struct iommu_domain *domain; domain = kzalloc(sizeof(*domain), GFP_KERNEL); if (!domain) - return NULL; + return ERR_PTR(-ENOMEM); + domain->type = IOMMU_DOMAIN_SVA; domain->ops = &arm_smmu_sva_domain_ops; return domain; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index ab415e107054c1..bd79422f7b6f50 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2237,14 +2237,6 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap) } } -static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) -{ - - if (type == IOMMU_DOMAIN_SVA) - return arm_smmu_sva_domain_alloc(); - return ERR_PTR(-EOPNOTSUPP); -} - static struct iommu_domain *arm_smmu_domain_alloc_paging(struct device *dev) { struct arm_smmu_domain *smmu_domain; @@ -3097,8 +3089,8 @@ static struct iommu_ops arm_smmu_ops = { .identity_domain = &arm_smmu_identity_domain, .blocked_domain = &arm_smmu_blocked_domain, .capable = arm_smmu_capable, - .domain_alloc = arm_smmu_domain_alloc, .domain_alloc_paging = arm_smmu_domain_alloc_paging, + .domain_alloc_sva = arm_smmu_sva_domain_alloc, .probe_device = arm_smmu_probe_device, .release_device = arm_smmu_release_device, .device_group = arm_smmu_device_group, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 1242a086c9f948..4c6c843669aaf9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -802,7 +802,8 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master); int arm_smmu_master_disable_sva(struct arm_smmu_master *master); bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); void arm_smmu_sva_notifier_synchronize(void); -struct iommu_domain *arm_smmu_sva_domain_alloc(void); +struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev, + struct mm_struct *mm); void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id); #else /* CONFIG_ARM_SMMU_V3_SVA */ @@ -848,5 +849,8 @@ static inline void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, ioasid_t id) { } + +#define arm_smmu_sva_domain_alloc NULL + #endif /* CONFIG_ARM_SMMU_V3_SVA */ #endif /* _ARM_SMMU_V3_H */ From patchwork Wed May 8 18:57:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 87F6FC04FFE for ; Wed, 8 May 2024 18:57:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/N1JoJ8fp5a6BINhrNjBfeZvw9EzGRwy7eXsqMKW36U=; b=Ssa7IZAdj5J+a6 mK1RZoN92dU4phygTmoIxIGGOOQ0XLhm7d8ljnqXds8Oin9TjwZ3u3SueFxzI9ZA8fijveIyMI9VY GCYSj+sLMPQlqH1KH/A1l7Qiz9ApQYHp1Hmr958md+j56ufftSdHTV6oQ+QBkkdQYDmR5EVjR6bTH vzahnbgaxronOpCxrs+hbdk06yOZasdUjMRphZb97zohOuUP/p/WQQZa7/u2S16xYZWPkHaswx/xj AyiF5O5RhIsyUCyaEprB7qR3Tf2yO6OXeTH0Whc2u7HMIrGeeMXwXT2nRELFLkRraXa84MQFaLvrP ABWD6N0KtbbFhn/EFVhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU3-0000000GcBg-1TlH; Wed, 08 May 2024 18:57:43 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mTx-0000000Gc5t-2UVt for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LAFxbN3LknkqSvZsIcLr2xRoFgSws92qEWKvNb7FQ9FhPavzgy17bQcGO6B070NQFSTFDM3AMX8/f1ngWGVon0RCCi4Q1aoG9msaOyPCRq0iBqm4ARAvJeK/g7YQPilk+PtlC51xDuImL2PRMvtlh8mrklwUVN/PfVVA6uT62WANKjw9d3o2DCXGpuUz/2klrNo8s8A/t/A8PTS+uPsAhsoChSEPHyOoawatsEenz1CY5HjZAcytIEmrumF5saKT0mRpMIHVC1jLdnwFE9AK58tWXVJhl9okNhtUQ1dWrR5kxYZBt3HT4/i3wcOlGal7sfDVGe46Tfz3UK1JLK4/Qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=u25uE5oWSzghTR4y+8jL3OuatCgTBPJ50yUGkygAQMM=; b=CWceAabMip/sPcjR9eoyfJgrC2Hn90gb2wK7ZzhYg6NSXVFayr2W4xSmd6qC1Zd1pjL1aAdhbyifHjsRbcyY5tQmuHOwQZw9hdzKq4mG9yU62vHPyqR3ODTMWAuzskMSJ9qkWsvNnupKmk9ARW3h9KNLVBgPQNczNPdvVabhO6YXxyBdkaOaclUAsYKsl64cLqB9Vzl052g/7PxFj3NBu8bgXHzRggnZtUHadvv7R3ut7f4OaX0TKCU/25+jhlMhdZJ2AjgwzAvx5HkHoJQrZ4zDMs4PUypC3NNOZGavTWbt5Y9QZQ8h3Fg+tba2UrBkQAbGX8MM11lT3KXc1gXP8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u25uE5oWSzghTR4y+8jL3OuatCgTBPJ50yUGkygAQMM=; b=ZtrcspbBmPs7yujv8qcKgKPvisHjWu/hyJ+C1sQAsjK82hYDRORsr9nKcZoonNf3KV4rybtcBhsC06LDUpzvWOlEC6HzD4nQf6VdHYlBoIZLbLLPAsT7BJ+pqzI3TsyWpnArolDhGZNlTeqpU3Y3g9Jb5HpE2J5JYutY1Hyeez+RCuqYqeJoSG0u1epISlaQhiweq5VaRC/McMHK2nuhy2QikAVBHggkKeJULNs0kKMDd2JIu0lGKQHZKpxKa2i5KtF0tmhjo2qRFuoD7VrORmh/AjK12zfs5N/7K8Mnw0cofFe8Xi6KP0BTrQnsr1tpv10jF13EV5fC7TvKfW2eaA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:25 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:25 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 02/14] iommu/arm-smmu-v3: Start building a generic PASID layer Date: Wed, 8 May 2024 15:57:10 -0300 Message-ID: <2-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P222CA0018.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:2c7::23) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 9ee1773a-74d8-40c6-9de1-08dc6f90b2af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: twcAolKT3uYOzi/O1nY0i3i6+YFxjpwKWT+49xhAYXOLAGsRlys2+Qh1FlWrrh4WB/EyEl8Ne5SfZBX6h6znFtdvBzg6iIdhmZsY/kXyh+p6VFspFm5+ch4fAy7Dydph+KudLsXxMERFmps1hJAss9WThIlFVpUidFP9yIYo06vwHo/vHet4u3WEDlWlPSkc6pnqggHAR8MpGFO74a/D9L0+Nli14+ekmhqSeLRAd42mj9beFAX1wOqd+fUweKfgJngKfdDivwY+Xve362pVAVGxdKPMis9s2wD1tsLue5AvJ2fXnXdaqcVG6LiFJEBGtQjSnKEcOIP4MapPBtnBoA614P52UQsbGeeQErLGxqgoWzKqqVFVBegVz4yQ8Hi6HIjaR4Ekbebzc7fXkL1fc38PhfTTlYxqW5FnYfwNWqPNgIPd01M/Y1iH8j2JVE30SZCqp9MI2A9YU7JkAh6cUSHsoK44U/iG0giNWcldJZLoWUY0gdKRdxLoMVBz25cb3Fq+kE/ybpcqRC+uhdT6HoWzR30ieKMgYwKBPucT/SUxZqswTcNz71ZgZA1/mzUAAfHn44n3OLJ7WeS6WeJOSrKy6HF15zHyNDmRdIwdZle3mKO89kyk9E/XyCo96hQd/ZSABMM75fL81qK2s9EmpPUXA6kBFbqPKJbf69YlWh2VG4NNh4Fg7JPqyzdGNtDbyfiH/zA7jLW6604RvaIbKavaVVAvHyIEwLQOXw0jqP6oo7Ite3j5QIZ/5XVt/O6SWL+cwn4r0XQBFpKtwhGTuvosqa9XoD7PIJJ35NJ6ARPfxCe/2pq27E52WsD0BtBUv/ATAmujqx372n32Tqjw1CgqyFyej8TA/TD99/li2N2REZWLk8xCbMjRB4JYRwFG8llqgIgNzQ5p9g8PJRu8bpga/kTqsgf2jL1J7FNuoC5usZUoMa22QDWMh3sXjt5GYWeQd4Ny1gsnmkRVgNstgOYz9+CgmtZXpp4pzHybakS9+t06zk66l1H2ZUdtba0WWSxr/5PcVCjHIum3LfQHXrTrsMSJEPPVKhbfig8UdYUrjWTV0Czs47YD881QHXTeWSLsU6fdLiyWYgR5suhKfPkw4feMw/LMHp8aY7LQ/xlzEa06LAkb+9/8Tggf6tZziqSc21LBSAWvaIS30SXxxG1fkZ0gqLcfDMCucIcq9uRkLAeAEivTKclmji/TPixy8WWIrRFA773Y4PZGp3oTxTAw0aV5Cg4fPCpq/LXZfiXsXCqCiHXvVG3/WLgYKr+DpwojWwBDc61BaM2heN/i4Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hiInYl4YdxxKNvbTDbVFntwlX1qIfFGFv0YbjfVvDjeLwHRvJW/gyrtbhWwniv5sOQUj9iNvNFviu7jgdWSn85tARKvn63Y1STCEAM2rG5PQUD60jhGH5qNDGPFSyrhdPiR2dpSf9U7Y6b0/E6cqybV3VoulWXh4opRLuqUu5TBtTIWcFj6fBzNyxwxyQh8h/RH1Gf8h30+iCZpls+C4AYRmXtU5Hp5smMiLPOQmqKjpZvjBixU7IyX8SdwGRc1ewZksgvDl8NcHjHFAXEpujZNFuW39V0LsoXeybfM2SRBU74vT7EUaPC9jAunHWECIvRz4olprZWCkyqYSBCDcSsEKnLYxL8IwG+r3h9/DPasrU6IYp73srBti4LhFg0Xbwco/34X6jQjZV3JYRixzya3Dyw5XwNn/CO5JAlIkJ1z+lnrZkseVDAfwF40x4zH8enDn221WRpPd5obnUjf6JOTp94LcOcQ22D/QYPWKPz0ORQBiN7+JzL77iZ82ptOrbZfp6mZ3Pvc/kV8bi6cPV6k++Kk9S7a49IpR7Q0MFBT5frDd2Dwmaxnn8pOUheAF7/6AYVUzXwhnogX27oUx61BkpAsLRewdsDpMHss8XKbMf5iOvhb2YnMXis/bU47ichmHs8AQZX8FBHRKFJDuN62omDRAAbhWOciwJG04ugk1C339ATYAwklwZvrl6icogBvKACtXeL2hzEz57fxoiWhmm+d/eQF8kN/ZJJbsGIpLUUkkkTpDWFrLeEffGFXPwR7BRah94J4aIobBWg2SnhvOO93BZYq9mNUjk6lOUtC7O0ENqCVdUDshOajdHTNY0lSpAJkDrHwq5+KxbH4fcRa4gcuNHjP3ZFH1KCbieqK43xwwCh6GxAdZS2ED/pLAGanF1U/EntMLOLM3u+Opyqeldz6YLOteSEOFh/tYmJliq3EOkhbhPVtHFbm191IItNReUApoLzdV8Y3GTt4t40LbOF0YP4KtMkyY+n8flYM8f1EhP7ejVDVJZoYY42LP4ISHKS4XD6fBchPoxY1OxTWghwBh9QLI3JwXW4WYzRFX7cy4wAjmH6Dd9qtQNs0Ujbr6a8jTOXHSJkZ2kwbwTcyEal+iO+2iYI1VSfQmPiJV/o8aEKDC5P18Aw4j1bQZOgZ3gTrVFEU4lD8PyUkZ5ADZPfLZfNwyDYcORfcfL6Qy5Z2FHc94PL98a+a2x6gubUo6NYtCDOjJOPJ/6SVqVopAteV4WxiYtelFhw595mnSl9epHgOABvJVWFiulXqPZlCPAvrZ7AOH7BtW8bkFVJR3F3139FuarGUC74nYLGPA15MFLYG73h3k1lcpp3MKuAVYPH6ZrYGo8cTDUWIC18FYKkQf4K9+knM0XoGUdZK8Zdq5+EjKxvPtlAOvg6NmUcGD2dz2VW54fHsXVQCiTZYl+GjvoviE3DiODj299YPtzJpkzPNjhw+UhhQmULvwje/zwFkRu4GE+apW7f21jcanofLka4+4Ua4Z6ZUwtQlFirnOKUQYjXXlkpbcPucqenBrITT7ByeX2QhDDEJ7RNuL189sf2gRKuKHthn5wNquahLeqt8HvLUd2MLs7Rym X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9ee1773a-74d8-40c6-9de1-08dc6f90b2af X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:24.5061 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: iJdUpOmN6cxoR4CXobuzPIqjI3dI9Fysmwt+YzwCBJs7Q/xyj4LWUZByZOB8t/HI X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115737_706882_01EB410F X-CRM114-Status: GOOD ( 19.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add arm_smmu_set_pasid()/arm_smmu_remove_pasid() which are to be used by callers that already constructed the arm_smmu_cd they wish to program. These functions will encapsulate the shared logic to setup a CD entry that will be shared by SVA and S1 domain cases. Prior fixes had already moved most of this logic up into __arm_smmu_sva_bind(), move it to it's final home. Following patches will relieve some of the remaining SVA restrictions: - The RID domain is a S1 domain and has already setup the STE to point to the CD table - The programmed PASID is the mm_get_enqcmd_pasid() - Nothing changes while SVA is running (sva_enable) SVA invalidation will still iterate over the S1 domain's master list, later patches will resolve that. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 57 ++++++++++--------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 32 ++++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 9 ++- 3 files changed, 67 insertions(+), 31 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 28f8bf4327f69a..71ca87c2c5c3b6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -417,29 +417,27 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } -static int __arm_smmu_sva_bind(struct device *dev, ioasid_t pasid, - struct mm_struct *mm) +static struct arm_smmu_bond *__arm_smmu_sva_bind(struct device *dev, + struct mm_struct *mm) { int ret; - struct arm_smmu_cd target; - struct arm_smmu_cd *cdptr; struct arm_smmu_bond *bond; struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct iommu_domain *domain = iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain; if (!(domain->type & __IOMMU_DOMAIN_PAGING)) - return -ENODEV; + return ERR_PTR(-ENODEV); smmu_domain = to_smmu_domain(domain); if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) - return -ENODEV; + return ERR_PTR(-ENODEV); if (!master || !master->sva_enabled) - return -ENODEV; + return ERR_PTR(-ENODEV); bond = kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return -ENOMEM; + return ERR_PTR(-ENOMEM); bond->mm = mm; @@ -449,22 +447,12 @@ static int __arm_smmu_sva_bind(struct device *dev, ioasid_t pasid, goto err_free_bond; } - cdptr = arm_smmu_alloc_cd_ptr(master, mm_get_enqcmd_pasid(mm)); - if (!cdptr) { - ret = -ENOMEM; - goto err_put_notifier; - } - arm_smmu_make_sva_cd(&target, master, mm, bond->smmu_mn->cd->asid); - arm_smmu_write_cd_entry(master, pasid, cdptr, &target); - list_add(&bond->list, &master->bonds); - return 0; + return bond; -err_put_notifier: - arm_smmu_mmu_notifier_put(bond->smmu_mn); err_free_bond: kfree(bond); - return ret; + return ERR_PTR(ret); } bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -611,10 +599,9 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct arm_smmu_bond *bond = NULL, *t; struct arm_smmu_master *master = dev_iommu_priv_get(dev); + arm_smmu_remove_pasid(master, to_smmu_domain(domain), id); + mutex_lock(&sva_lock); - - arm_smmu_clear_cd(master, id); - list_for_each_entry(t, &master->bonds, list) { if (t->mm == mm) { bond = t; @@ -633,17 +620,33 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - int ret = 0; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct mm_struct *mm = domain->mm; + struct arm_smmu_bond *bond; + struct arm_smmu_cd target; + int ret; if (mm_get_enqcmd_pasid(mm) != id) return -EINVAL; mutex_lock(&sva_lock); - ret = __arm_smmu_sva_bind(dev, id, mm); - mutex_unlock(&sva_lock); + bond = __arm_smmu_sva_bind(dev, mm); + if (IS_ERR(bond)) { + mutex_unlock(&sva_lock); + return PTR_ERR(bond); + } - return ret; + arm_smmu_make_sva_cd(&target, master, mm, bond->smmu_mn->cd->asid); + ret = arm_smmu_set_pasid(master, NULL, id, &target); + if (ret) { + list_del(&bond->list); + arm_smmu_mmu_notifier_put(bond->smmu_mn); + kfree(bond); + mutex_unlock(&sva_lock); + return ret; + } + mutex_unlock(&sva_lock); + return 0; } static void arm_smmu_sva_domain_free(struct iommu_domain *domain) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index bd79422f7b6f50..3f19436fe86a37 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1211,8 +1211,8 @@ struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, return &l1_desc->l2ptr[ssid % CTXDESC_L2_ENTRIES]; } -struct arm_smmu_cd *arm_smmu_alloc_cd_ptr(struct arm_smmu_master *master, - u32 ssid) +static struct arm_smmu_cd *arm_smmu_alloc_cd_ptr(struct arm_smmu_master *master, + u32 ssid) { struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; struct arm_smmu_device *smmu = master->smmu; @@ -2412,6 +2412,10 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master, int i, j; struct arm_smmu_device *smmu = master->smmu; + master->cd_table.in_ste = + FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(target->data[0])) == + STRTAB_STE_0_CFG_S1_TRANS; + for (i = 0; i < master->num_streams; ++i) { u32 sid = master->streams[i].id; struct arm_smmu_ste *step = @@ -2632,6 +2636,30 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; } +int arm_smmu_set_pasid(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, ioasid_t pasid, + const struct arm_smmu_cd *cd) +{ + struct arm_smmu_cd *cdptr; + + /* The core code validates pasid */ + + if (!master->cd_table.in_ste) + return -ENODEV; + + cdptr = arm_smmu_alloc_cd_ptr(master, pasid); + if (!cdptr) + return -ENOMEM; + arm_smmu_write_cd_entry(master, pasid, cdptr, cd); + return 0; +} + +void arm_smmu_remove_pasid(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, ioasid_t pasid) +{ + arm_smmu_clear_cd(master, pasid); +} + static int arm_smmu_attach_dev_ste(struct device *dev, struct arm_smmu_ste *ste) { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 4c6c843669aaf9..a1b400e3e41878 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -602,6 +602,7 @@ struct arm_smmu_ctx_desc_cfg { dma_addr_t cdtab_dma; struct arm_smmu_l1_ctx_desc *l1_desc; unsigned int num_l1_ents; + u8 in_ste; u8 s1fmt; /* log2 of the maximum number of CDs supported by this table */ u8 s1cdmax; @@ -777,8 +778,6 @@ extern struct mutex arm_smmu_asid_lock; void arm_smmu_clear_cd(struct arm_smmu_master *master, ioasid_t ssid); struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid); -struct arm_smmu_cd *arm_smmu_alloc_cd_ptr(struct arm_smmu_master *master, - u32 ssid); void arm_smmu_make_s1_cd(struct arm_smmu_cd *target, struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain); @@ -786,6 +785,12 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, struct arm_smmu_cd *cdptr, const struct arm_smmu_cd *target); +int arm_smmu_set_pasid(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, ioasid_t pasid, + const struct arm_smmu_cd *cd); +void arm_smmu_remove_pasid(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain, ioasid_t pasid); + void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, From patchwork Wed May 8 18:57:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A3CFBC04FFE for ; Wed, 8 May 2024 20:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sLMSdOi75raJLVAvM4uu+4z1Z4+BSYOke2rBiB4+5uY=; b=liPP9/IJuNOKfQ 13fAPwsuSQqEetZZK3awDRyXOJKm5qU78MWPpsAx37syi0vq/OZ63PjiJ0pcqX+oU5HDMSHPJxW76 s6JmVZDbvimj5oV/yZpGcBrqclN0dHQiA4wmfiQHYrlKFZyXocxTXVpQmXwT2qG7LoKOVmEACqLaT ZELKFeX65tw6Ywoy5FDvkn7i5YSu7Xm0hvTA+ymYHTVun4GPYPQqP/YjhPcG3z65LtUSCLr1UO42Q kQRvilbBKZUGncLwqSKlylYy90ShJenmXlvl/TZ4uhEuhjSewjt59XKPxHkEbiQDttwkdBSm58a87 DGjHMOBa1GttnYz1FmIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4nTf-0000000Gp1L-24hy; Wed, 08 May 2024 20:01:23 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUJ-0000000Gc7U-3AXv for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:58:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d3yXwJ1+7HkoBotx4gFjdkrDOmXICBqQ3LSWgGXjVkrvqYXcKuxbB/uYd5kr9PRvrqdhF0+SQXBdraoX+1z1v0qSyz7loGDzT4Dwn128q6CC8NHHf/DpQ3ZwrH0RixoOFWOwiVfvURJ8qAyVhwoA/clgsKLxKJAWyW7n88zzmJeNB1Rz5FjD3oF8Ooa387UOLHFmr6oEeAkPmo4b7yWpn5rrfkwVOgJG8kjYNGZjaOPelxNeAk1fu8waRgWx/7e6w0ymL+AdyViPz7RdTnQQrNjfRqh75q/IVDo8opP1Ldz5CI+GhUx8KgUZE5jV99GKWkLVsXuSa/brlNyLQNWllA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rHrDdFn03CQuhDxPeEhiYssYd1GLkbrfxQW7fV56mbw=; b=RtPNvZ6Rm8EZyH/R6apwuYY2trrifF2OqVFEPIxG3cmVO6H03UYyzVqqg7o8Ak/bq4py66UEYsSOwt1cxSX58ndIFDlJ8W18Xv8+a7rounF8xNeA5AUBjQyR3AQv1jAVRcL4YpgmjMtcdENjEQ0UzgNyadJb4YR29NhWL4aor7ygnQk6Sd8XERwHCzHrDlYOBEp2DeNgqvjWWRfjSynM1py50+RIOVFWB37XWk/CM39fXvGgjX7/KtEnVkqih/XqR+N5sM+gr0yg0SN52Oc107V/z9lUvMfASR0gTNxDnFvCx8r5oZvMG7ragz2IMH/Bp07FfTp1QJeMvg6XY+a94Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rHrDdFn03CQuhDxPeEhiYssYd1GLkbrfxQW7fV56mbw=; b=lj2ZiNwi3w+d9u5jrrbKNMAufuiWmW3m8AJVQ6lUDUlZ/4husuv8BQ6GDWQu5rECcmAFyR0G4Mp2n2SVV1y55JJW5YQoCRU10l4qIVPAly/nVXHxs9KgTOgpwN/n8i30dH1wYtyFHCPGEQZiTk9t1RGxLyD0YpcLV9mi5X9ud38TUoiAjAg2SRI4oDRFpfqiDHPXMjSWKJ4OwnTxUIFpx1q1/+IfkU91odgs6p694lOvomc+OAstMAS1CsRMuYL7QXEcTqMY/WJnGuShXfKuq3aQA2GyBguQAsxYklg0cPRKniIdMy3pPLnm1gnMWEyrJpNyzxi4RxqDrGoGeNXUWw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:28 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:28 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 03/14] iommu/arm-smmu-v3: Make smmu_domain->devices into an allocated list Date: Wed, 8 May 2024 15:57:11 -0300 Message-ID: <3-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P222CA0019.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:2c7::24) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a8b830b-0119-483b-0b3e-08dc6f90b4ac X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: UtzU34EGnFMsrT7sciCUnzpPdQSoY7na+OCmGat+Zb3IrSt3pCO7nrjzRW/rzTRu2KWMe8oh2/38cADVOUfwZKSDZVDwFfaqTQJKlX3fp5OStGJ9hqdkWNkyHrbYU+C4gh9Ogq5w/3wYuj33l2zNVvoVombhY1MpKWa2ViSOG7+nIvnIBS5QrdBqZWKHurAqtu2xhMIcgNSilXjw/pibe8cbSsdefRHFr/MhbQYyzYt67m871pTbSuJmHyquczSyIBacUcFC4tCVr04gqvVBu1sFIoX7KlZljhrqNwDBscPa5Z3zgqhoffVg5m+tLQzyo4eXBiAFTAZNfsbLIz9xqtR7lHL0OaEeH+/tm1u8gmccaCTfKdlM/GrDhq6qS58w7APw89gqcMYNEyHt4hJtod6K7MJHoYc2xmSqgkShA/rj/LdBJdxTSbQYRYBlfAeKmC9ukufZT0UuD/jQg5Rif5vDdcLZdH+sCQ69MtqbvuYxowX1O1LmFWKVcIaxTgxXxYmQJ7GxK45mXNjk9B143xZ0i5AOcTQpUOLk4iyfHXREgQpKdogITxEAzm91kKd2d8GEuFtjLLjTUHwLetlbK2oLXURnSH4xJk6aRvMXESGvNv2KvqU9mQQzwmVNo1662HRAEgZAzJtRF09/Hot4+Is6R/L3vDSjPtGg+5k+T6IexbNlwvBm8AES7i53rfHY+4uFUFMj7fat34L3UMkVu4WKtlsI+up1DbFcqhq6dZZ8UkIhfzdcuCuIi36ZaGSzozZo0OcGHnR3UUGSbKurNwxI4LDPgelYX7gpSgBvo6/vsqvEbCZS95pYcXEV6Vw6EoxGiptUAcBIUWk7flo3a1yY2vs53CBuvctrTuvFK6/PgTBvsYq9ViianmtPUhgDLWrM/6BJaHmByP4FxzX4MdytLJ10cDTMQMa5s/D2ABEGZxZ6be08sOm75XpT9o5nTY9YDZnzXnk7/C9hr1BJFSWxB7xfH2l6SyVxgfaEVB5LHCAvXVQdxMHynlq0gsY2J6UIGnD4EmfWxaHkQsEnfO2EgS1s5Odd4NUaSIoe+jlK4uphO46i4pItSJBctQRbsZ/cXiWsocwhUY8Y7iwUgMVQXt+1eIlH3R6+YK5C0wf5tKnxRO4/IEdoEc6sfTe7tLTlpaJy9APZXy2pgB89rzQOcMGhf+xrqjjj+mFutxHrL4JGkZrd8EehV64JXymBwPdouH7UwJ+CuUDPy9s8xbGmikq4srSieFrLp1pmh9tGvgax0TXcmVka3Pv7HdRmHDsp/TnGSoIyWkrUPrSEyA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: yLk8VkRKbWos/R3PU8gnzSs1x7jhRGM23WavjnvtVj7onrMU/BDQed9RPKu3YeMZs6VZSYnVoWt2Z1Frk3H159zAz0GLqzPTfnxXqIw0DEcCiHOpuJRn+oqqWvSuhK61lATMXJiKyy9NZ+5nv5ad2MnIpHO12toHaO6LCpHCWtybU59OQOAhu+ckrIYUIL1P1+JGiJajdaXk/z634t7KncVSd3JIXE82zYhIBRpBhG0AoDIDJ9IgF0f1M/ACU34Lf9y1+Ot1Wfah9hZiYDKTPcF5cKNyF+iW924lA1IrMOWL1OtUtOgMUKHeeQ89BcTZj73x9BIkTWQ3TMXhKEIZ/PZnGdt5osiD73mkRGCKreSflg/JNK+SUX+UB2TaJzUN53gZ9g0DrHXwHBXTKG8GAg6P89u8+y9LPwRwTzvyrwV2pE/Y9N2QBYgAS4IvE5DZTwVVepPqyV/TUXsdd56nNkR+iyX7ofefVV3qLo+0qHVJjq89/El050MZfRujsrqMOT4Sv25udmBwwJdsCTZiskfwGvefBYhI8qzauternNeGUQ+2/xfAa+fsDrT+cfosx6ojgoNF6x684vEbBcGlipC2CSKxdLa0QO7BSb4Xn4GpY2jeoMdAXNkzPQNLnhyU8afWZKifsCXiZMopG9F+r8VtPPx1B+VQ+evZ9Zd6Fp+4z1hypw9qhmN/mBWmBqF0keGXQZQixm2xRy7yy6vBtfvVPOCcUdDHZL2MDEqTuATbEMkxZZEKqFiM+MCIyLJLo3ZV7WGGorfRAsDr269tdsUTPxQc2CjvHi1fU3pQqf2Qh4VoOcSMdZ5YnB90sUGN5jurrA5K+aeDAkEflmfFXk5d1zseVxfuqhZhBgldCJh+29UaP02yiGOZeBmzkEAmQ2MU8AilmJTO9BPCSh7nxfk1zbxkOTtV2lUgwFyE9HQ7gnJMtk4MDgmi381QFkFjzKpjRDSfc4kRRPe5xSf2mir73Iyd1Wn82sNoWiiqL36AQo+xSElS9kwalVJuriOndKDPpuW3ZgkF542jblWRn0AOxA/50md2JsQtvQRc3pZOOk7gMYv+IcP9cbhRj+71wvZS/QLMqroTvpsibaZDmoAR1fT59ZrQSPECWwKfx9wManawsrUrdkH1FM5mD37Fr/mIwhxS+e/yMo1F6OmfRMo4wKcudZI+YOkFGMSxrSVMqZT6WkFqHq1A2PB/psRIsFqs+9liTtHMLHUiVte0ZWKlkltPK6CyFbkv1nFIDP6YbxhqW+Xg2Nc6a6mAqUy1lO3ceQGDT8pMORlaSfjOy3da4kDkIV5PPGhsLYB908G3LY/TWEeI4+crgXDtx4HQWjdFol6euwKP/yClOdzNhC+kNAVqJjFnkT0SayTh8D3/Dwu7sBUPtsjeOXKRnTzh8hxhJFV/KC6Eorb9yLVF84P6DYQquehKyP2rJNeqdn6qE0Dem7akBqLIXGQTCCDn8FxTp/rn9kfpuBoVzcBs/nWqXhW+WGuFN/Nx4Yw4XVDuPjXK0KyOAIxUT5sGdalJugcR4a85xOnHLHoMZwEPFYPXJ37QZvfi6fCI1PcFjnz8gEqNo2vjmoxzDO7mCmw/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a8b830b-0119-483b-0b3e-08dc6f90b4ac X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:27.8613 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cL/LDchzLo52h4Rp32bmnnC7H+OZSwbV66+xxXQcPv3hhLemq3sf3a7FlZL8VRxK X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115759_984645_B3B54FD4 X-CRM114-Status: GOOD ( 16.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The next patch will need to store the same master twice (with different SSIDs), so allocate memory for each list element. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Reviewed-by: Michael Shavit Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 11 ++++-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 39 ++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 7 +++- 3 files changed, 47 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 71ca87c2c5c3b6..cb3a0e4143c84a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -38,12 +38,13 @@ static DEFINE_MUTEX(sva_lock); static void arm_smmu_update_s1_domain_cd_entry(struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_master *master; + struct arm_smmu_master_domain *master_domain; struct arm_smmu_cd target_cd; unsigned long flags; spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + list_for_each_entry(master_domain, &smmu_domain->devices, devices_elm) { + struct arm_smmu_master *master = master_domain->master; struct arm_smmu_cd *cdptr; /* S1 domains only support RID attachment right now */ @@ -301,7 +302,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain = smmu_mn->domain; - struct arm_smmu_master *master; + struct arm_smmu_master_domain *master_domain; unsigned long flags; mutex_lock(&sva_lock); @@ -315,7 +316,9 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) * but disable translation. */ spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + list_for_each_entry(master_domain, &smmu_domain->devices, + devices_elm) { + struct arm_smmu_master *master = master_domain->master; struct arm_smmu_cd target; struct arm_smmu_cd *cdptr; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3f19436fe86a37..532fe17f28bfe5 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2015,10 +2015,10 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size) { + struct arm_smmu_master_domain *master_domain; int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -2046,7 +2046,10 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, cmds.num = 0; spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + list_for_each_entry(master_domain, &smmu_domain->devices, + devices_elm) { + struct arm_smmu_master *master = master_domain->master; + if (!master->ats_enabled) continue; @@ -2534,9 +2537,26 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) pci_disable_pasid(pdev); } +static struct arm_smmu_master_domain * +arm_smmu_find_master_domain(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) +{ + struct arm_smmu_master_domain *master_domain; + + lockdep_assert_held(&smmu_domain->devices_lock); + + list_for_each_entry(master_domain, &smmu_domain->devices, + devices_elm) { + if (master_domain->master == master) + return master_domain; + } + return NULL; +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { struct iommu_domain *domain = iommu_get_domain_for_dev(master->dev); + struct arm_smmu_master_domain *master_domain; struct arm_smmu_domain *smmu_domain; unsigned long flags; @@ -2547,7 +2567,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) arm_smmu_disable_ats(master, smmu_domain); spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del_init(&master->domain_head); + master_domain = arm_smmu_find_master_domain(smmu_domain, master); + if (master_domain) { + list_del(&master_domain->devices_elm); + kfree(master_domain); + } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); master->ats_enabled = false; @@ -2561,6 +2585,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_master_domain *master_domain; struct arm_smmu_master *master; struct arm_smmu_cd *cdptr; @@ -2597,6 +2622,11 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -ENOMEM; } + master_domain = kzalloc(sizeof(*master_domain), GFP_KERNEL); + if (!master_domain) + return -ENOMEM; + master_domain->master = master; + /* * Prevent arm_smmu_share_asid() from trying to change the ASID * of either the old or new domain while we are working on it. @@ -2610,7 +2640,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->ats_enabled = arm_smmu_ats_supported(master); spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); + list_add(&master_domain->devices_elm, &smmu_domain->devices); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); switch (smmu_domain->stage) { @@ -2925,7 +2955,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; INIT_LIST_HEAD(&master->bonds); - INIT_LIST_HEAD(&master->domain_head); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index a1b400e3e41878..125b52ea5ad71c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -697,7 +697,6 @@ struct arm_smmu_stream { struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct list_head domain_head; struct arm_smmu_stream *streams; /* Locked by the iommu core using the group mutex */ struct arm_smmu_ctx_desc_cfg cd_table; @@ -731,6 +730,7 @@ struct arm_smmu_domain { struct iommu_domain domain; + /* List of struct arm_smmu_master_domain */ struct list_head devices; spinlock_t devices_lock; @@ -767,6 +767,11 @@ void arm_smmu_make_sva_cd(struct arm_smmu_cd *target, u16 asid); #endif +struct arm_smmu_master_domain { + struct list_head devices_elm; + struct arm_smmu_master *master; +}; + static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) { return container_of(dom, struct arm_smmu_domain, domain); From patchwork Wed May 8 18:57:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60223C04FFE for ; Wed, 8 May 2024 18:58:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=md8DqpIc6VNt1wkR+veB1Zt+8Vrg4H0+WaIcYwLIk2g=; b=MFon15Q1v+hmxP /z7FQar+QO19kNfhGtWbtQl3K9ya8btth/h4YNPdm9nJOz6bC6EvopXh7DZ0HCKRDKt3OlEQGOwyk 6HCBziwxZZg0gHEcKqxx58yXeQLWPqd1cUrrFSpF4bl5lNNaURKPIcdyuf3LdXNKZnwxlBfoMugdq Gp3bN6gIYD02hrUKfbrXhmhHpC7HT+i0CAYRO2W5frvcKvgrJSRZYbnW8WGsU7XEDDHBCFwbQSbkC KNnZBr1FCCXpgaru63rgcHTx/StdbIEKfoczKrwlVAPW7Ij2tyjWiXhTBldVLD0khWwlS03Qkb1Sa eN5SMN+Tz86lZ8de7mMw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUf-0000000Gcbu-2bIl; Wed, 08 May 2024 18:58:21 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU8-0000000Gc5t-165L for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T+NkMlFQB9OAyS6gnFo4Rmggi9IK+dVquUc3uZsNTWBkXV8PTT272aWI6XnYjpxJGZpsFb71IgYrExC5XQi5hqpgnGcfYlm5elkvyrXKJiP+9Nt/riUCfgWPnbyYpYf9WR1Q4JWdF2RteyCzyowDRZ0CkIIfxZq4NfERXqV7alJ+RwATr5v5itpH1hAK2LMSgYU9MMN45og8XbAB6X2AFGNMeKJVnFOabmNJ/w61y7t3O3O+VHIsJ94iJMjGB6ukKOoUAPDDoooExfNDN3iewMhl5ZURMpftuFV3nzbbF9XN9/wJS/6oe6hXXMRzZ25tAxBZcqwqXo896G2x2AsvIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C7v3lutwpuP7PIuMzMt9fFRmd+GNq7ZG2T8w7WcpurA=; b=fviiOC8ErGJLgqw4SfUMmEzszmK/XpZyZ5X+Pw728KZ48DFlkEjKtSrVFSraUuEfelJHoLN2XJ9dbS7ZQEebdp7XCtNBLtVBtAO7XJ16/bFWCsPeocGB8HhDC1hAv1xxUEFioIipVz/0RT0Rc5Gr0eCOAHTMATJMIvnmZRI3dA3MUeMjT5SI6GoVYCFkQGvaxt2CPJXU09yREsKmAHcC77Bw7MmJj+F92evnGgCYKH/oKvhuXaFiSoKFJjIpD+QehS8TO8ymzR3nbAf6aA1NGGH+Ad506Qp3nW7DF805m6X0NVzAy6r6FBv00g1zIE99rm4Apdc+8L+nXFg7rMWnDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C7v3lutwpuP7PIuMzMt9fFRmd+GNq7ZG2T8w7WcpurA=; b=g3rd5H06gPnvmgDMwYQwuLXASUInMCypqyGJ3Givh1CIybfpydtIt2Dc/DWxi0ppJXLiNaxSIM4gk4zlnhKDHe2pGeeABRg4NFcD1BX3kqSn2rukJnZ5LZWL1KVf9IoSFcdnw67TWKRhCLUtrxId8Bkm//delQOnq74a5Fm1aY/ROlIRuO5lz4xcPS5IcZJQ49sgPb7I3GUP6bo7VB+tIeYRFZ7hxCMs+vk1AdM7NwqbLh3lNvyfddofhqQ0gORHnRMhIvduMSp61Ylj6tWvLRCKnpXSTIUe4GcsX2Q6PaV5qqOclKcSNTsm/9j8MZ86hCWrmImPUWgYp5T+aQ3p/g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:26 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:26 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 04/14] iommu/arm-smmu-v3: Make changing domains be hitless for ATS Date: Wed, 8 May 2024 15:57:12 -0300 Message-ID: <4-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0089.namprd13.prod.outlook.com (2603:10b6:208:2b8::34) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 51730357-9a0a-4f1f-0839-08dc6f90b32d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: Jh6/0GuedKShb3RTq3vWpUN8fPx0tpPZbKSps/esRuMrM7GxWcWpgA5q5iss3ePI1NtVrTHLBpgfEU4W0s1IoLTILk4DKmWUFEfWmGHD//fFKPAYUCVkioqls/nLSSQH0N8HT0+vc/kJx4iQ2moA9sMwnfz/SXR6g167ySo4qVJkNMHoulOuXZjThkFThRBD0lE+4Sfdqz9wkRdZ03O50stNHDCxxpG7RiZhuVHvu8kl+emCgygVgf/d3S8e2Q6tADrBdGsttEizY1YsfllRIEDaECz+9RT7/KWeGRZe0HKwF+P7mfXpR4/yj6mUohZlLW9Zky87CQM5ItalUlHJlZcMk+lBIprgN+YLpw9YCF/HpeUM9XNaqBtpy3fSTQgCNfwGSgQNxm8MFQlrObKEB9GMuOKWvHrA8K8uToBZx/f7VE1Qj8mzzaCQIYQbc7LKeYgycGozHwT0JLWQ33j8/Pj5/rusbU2qCR5aCGjMG0HCwRWY9pS63k3Nhz9pNZANCqJEaJrxcNiDD90MvO2YYTJudEcXeH5r65slKrRE9+8jGBljAgWeuIN0aQuuGcT96WHUI9297ntDuYtJLlR7V3nB8jXaDQpo2W+lJyzY1BnqBzOWUab7LDtbtdQ1J8VWqLr/QP/nX58qK/fgiKRGW0SJfj3s1SpsEAeRcmkc9xYrWVNwgBdOE57icH5+TCHxmhg19JfJ9loCbOOwxPG4ObmPjPLIc89oVq8qQX4FPPUwZvCPsPN0SZDC2mn9Q8v4o0pC3RPXxJHtSuWcNIGa2Vk2lb4Tg0M1Lgp/TnUh1sWM/CsQ4LTt7NXH46uvFNxA9QVN9pYRgTMxXBxtHAxwB0LpqPawRyf6qSIYnXfRalRfCOvsTmn4aK1XFfA88tmrPq+B7spgN23NXY38YpDEU6hrlWoyV9MOhx2NdpzaguFo+fUwQw8w3tjvz0dyP5svRTX7WeV68enOE7zlcrG//l3MZk3srJy4zY4Q66Fj82wAxecP/YrmKdH8+OPBxKMZbYqQXAwXAqQHJa1wYvT9i99JIj6faTa6UW0m4eGHC8xilcoQIIeL7CDpJECDB5qhYHwYPXbQ/JGzy/7ndwoJGH/MFIiyEcO+vh1wcHAlVnDqBI8KcEGlNMuw5AzQy4oFkBEPnsM35rJiDI+UZV7NVoyaSzV60BtMB7EMYtcx+U4oaMX9ldHKvbCiSYoGmGIwTz4loHwJOm3PqjFC7fPpVHCigxkhSe9MQU3HH3xwclW6KTP3x1PMHzoSdrzYlw8vOnyYxrD8Y7hMbpkpBBznaQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CYaWQo2TgLeCyro/D4uua6gkqQInEuaDGaQmI88VEFg9auKutUssBQMbE8NOy/fsK/KepEUreVNSnZq+SreqkEHQZ2aIJPZHsErFDfadQqsj0XBNrbK5JluzoraWsz4+jxLi4rGswg7ECuD3qTtiT/QzbDZyyGn9J9z9MIxDV5cxxoV2Q3bHmHE94qCRZnqyG/9py1rVwT1mo8oIb0DpLn0GkNpTXMfEseTmtSAjpLATPC11AeVLH46kGy/uXWWRQNF2NpkFB0ipfMYIQJlFVrJUXm0N9Kcb+9uyS58xzkJ5R9odnjj0kNaWC07XZm43ZQDkV5fmfB++tLD2EY+/lPn65wZda+chaS9PhsJiEUipyOFCG6zzpgrlcH0A4S087yHJ9XrNpXFTT8bodR8DzcpmU0KxCputr7CbdS7sjAtMv+c14pmP/Gz9NquHUbQZZgBQZLQbSQXD+Es+tGvkI/LLo6ZLMiBEu6R91hj7IDxARJD98ZCfBz3dCkp+cr9i5eeyCd7aZjOpBw79lNXZ1yeWpZnw17sIvqGNNxlxk/xqxcK2mXXgquBXAtIRyTS/7pFVPa+btB5je5fKa4HzvIyl1sMMY4fAJduRKk37pn+FmpCj6rwReCrVGh4UjuAz0dbPAp/B+01VUjmVReoAO0dUIUkTUjkoyIDF1wfVJFvMUBZkNuj0f4v/mhhos8hNvI6Ji9JQV3XQlviBO5obEbCrTlJkCCkxpIye5TIKbgE9YzdLMCsMpz09d4uXbAad93h6W2GIsxiod5qT62kVOReKZ5JVL8as0q9m9EW3ZznGoJLoJvMMchn5bDQzeCk62lPaCkZIE8JiA8+byrrjpOwxhCHnJqd7GUdGyWtfgljcvnOZobWmf23Mc80UpDzZAfYQAI7p28JPHdX1BFPJMJg2wmJJcPrzoTe3gSLg4pqhS9GrBOAaXzbuAyGTDWZ9TyBUuw3qYsiDQA5gqJQTuLZMqS8L06D1sK6bB9bSQ04F9SEo+vIzAE9bncUHgFolcfQnCCFVElP5iRrRRR0wQ2K1nLiQOBuRierQJBKAF/5xoXY2f/+mKLYfMUQ8pmdfV3WCWBqvRYj/30B/P3G4R4msHjhwfjPtID2K8GzajJ6QO6xSHo4xB2UhO/fIVtRzJlkuroCG58X5/H4N9nByMfuJ9Ogv6NssPrlaXXGDJxCZ/x4qeyuQn5O66gdcnvKDg2rZNEtuFeJBe6WLx76BaHfR1WIzj0YGs381IAYl+lHy7gBMvHGqynj91KFP0mjgwJAOBpPCTpGbchpPPtnp0eNMI+VeMXJzDDc6EJ+JAaCaFJv9N5fmFKPCcuQDnYfn21x0weTo8l3EnVTgW1VvRgC/2YQAQZWLdYMNLTmjFfaccZcWnz86PpYYLFIDD3mtHpTdKbS7TZqLuFm37a+Dc5AVvP7IWea1l17Y3qKS64NhEl21mMovH1iHXV/7ZcFgra0lkK7PFNg7sgO91/aw35PwpWCnJLrMja8nR9Jzr4OV5MwyrG0dpJM7G+xrJLMcFSSLB/QDN67IQfI9oxO3jWRYHJ7zEeVAF2AzQ5R/kB+nFfKtV0y48QbPI9tbqIfg X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 51730357-9a0a-4f1f-0839-08dc6f90b32d X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:25.6144 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WXK50U06CVW1uToGixz63FrZfOe7ENyt71pA9RAgX0gjTOAMcu+KUjrdw5Hgj5Im X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115748_423479_B1FA395D X-CRM114-Status: GOOD ( 25.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The core code allows the domain to be changed on the fly without a forced stop in BLOCKED/IDENTITY. In this flow the driver should just continually maintain the ATS with no change while the STE is updated. ATS relies on a linked list smmu_domain->devices to keep track of which masters have the domain programmed, but this list is also used by arm_smmu_share_asid(), unrelated to ats. Create two new functions to encapsulate this combined logic: arm_smmu_attach_prepare() arm_smmu_attach_commit() The two functions can sequence both enabling ATS and disabling across the STE store. Have every update of the STE use this sequence. Installing a S1/S2 domain always enables the ATS if the PCIe device supports it. The enable flow is now ordered differently to allow it to be hitless: 1) Add the master to the new smmu_domain->devices list 2) Program the STE 3) Enable ATS at PCIe 4) Remove the master from the old smmu_domain This flow ensures that invalidations to either domain will generate an ATC invalidation to the device while the STE is being switched. Thus we don't need to turn off the ATS anymore for correctness. The disable flow is the reverse: 1) Disable ATS at PCIe 2) Program the STE 3) Invalidate the ATC 4) Remove the master from the old smmu_domain Move the nr_ats_masters adjustments to be close to the list manipulations. It is a count of the number of ATS enabled masters currently in the list. This is stricly before and after the STE/CD are revised, and done under the list's spin_lock. This is part of the bigger picture to allow changing the RID domain while a PASID is in use. If a SVA PASID is relying on ATS to function then changing the RID domain cannot just temporarily toggle ATS off without also wrecking the SVA PASID. The new infrastructure here is organized so that the PASID attach/detach flows will make use of it as well in following patches. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c | 5 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 201 +++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 149 insertions(+), 63 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c index 315e487fd990eb..a460b71f585789 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c @@ -164,7 +164,7 @@ static void arm_smmu_test_make_cdtable_ste(struct arm_smmu_ste *ste, .smmu = &smmu, }; - arm_smmu_make_cdtable_ste(ste, &master); + arm_smmu_make_cdtable_ste(ste, &master, true); } static void arm_smmu_v3_write_ste_test_bypass_to_abort(struct kunit *test) @@ -231,7 +231,6 @@ static void arm_smmu_test_make_s2_ste(struct arm_smmu_ste *ste, { struct arm_smmu_master master = { .smmu = &smmu, - .ats_enabled = ats_enabled, }; struct io_pgtable io_pgtable = {}; struct arm_smmu_domain smmu_domain = { @@ -247,7 +246,7 @@ static void arm_smmu_test_make_s2_ste(struct arm_smmu_ste *ste, io_pgtable.cfg.arm_lpae_s2_cfg.vtcr.sl = 3; io_pgtable.cfg.arm_lpae_s2_cfg.vtcr.tsz = 4; - arm_smmu_make_s2_domain_ste(ste, &master, &smmu_domain); + arm_smmu_make_s2_domain_ste(ste, &master, &smmu_domain, ats_enabled); } static void arm_smmu_v3_write_ste_test_s2_to_abort(struct kunit *test) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 532fe17f28bfe5..01e970f6ee4363 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1538,7 +1538,7 @@ EXPORT_SYMBOL_IF_KUNIT(arm_smmu_make_bypass_ste); VISIBLE_IF_KUNIT void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, - struct arm_smmu_master *master) + struct arm_smmu_master *master, bool ats_enabled) { struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; struct arm_smmu_device *smmu = master->smmu; @@ -1561,7 +1561,7 @@ void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, STRTAB_STE_1_S1STALLD : 0) | FIELD_PREP(STRTAB_STE_1_EATS, - master->ats_enabled ? STRTAB_STE_1_EATS_TRANS : 0)); + ats_enabled ? STRTAB_STE_1_EATS_TRANS : 0)); if (smmu->features & ARM_SMMU_FEAT_E2H) { /* @@ -1591,7 +1591,8 @@ EXPORT_SYMBOL_IF_KUNIT(arm_smmu_make_cdtable_ste); VISIBLE_IF_KUNIT void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target, struct arm_smmu_master *master, - struct arm_smmu_domain *smmu_domain) + struct arm_smmu_domain *smmu_domain, + bool ats_enabled) { struct arm_smmu_s2_cfg *s2_cfg = &smmu_domain->s2_cfg; const struct io_pgtable_cfg *pgtbl_cfg = @@ -1608,7 +1609,7 @@ void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target, target->data[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_EATS, - master->ats_enabled ? STRTAB_STE_1_EATS_TRANS : 0)); + ats_enabled ? STRTAB_STE_1_EATS_TRANS : 0)); if (smmu->features & ARM_SMMU_FEAT_ATTR_TYPES_OVR) target->data[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_SHCFG, @@ -2450,22 +2451,16 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } -static void arm_smmu_enable_ats(struct arm_smmu_master *master, - struct arm_smmu_domain *smmu_domain) +static void arm_smmu_enable_ats(struct arm_smmu_master *master) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu = master->smmu; - /* Don't enable ATS at the endpoint if it's not enabled in the STE */ - if (!master->ats_enabled) - return; - /* Smallest Translation Unit: log2 of the smallest supported granule */ stu = __ffs(smmu->pgsize_bitmap); pdev = to_pci_dev(master->dev); - atomic_inc(&smmu_domain->nr_ats_masters); /* * ATC invalidation of PASID 0 causes the entire ATC to be flushed. */ @@ -2474,22 +2469,6 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master, dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } -static void arm_smmu_disable_ats(struct arm_smmu_master *master, - struct arm_smmu_domain *smmu_domain) -{ - if (!master->ats_enabled) - return; - - pci_disable_ats(to_pci_dev(master->dev)); - /* - * Ensure ATS is disabled at the endpoint before we issue the - * ATC invalidation via the SMMU. - */ - wmb(); - arm_smmu_atc_inv_master(master); - atomic_dec(&smmu_domain->nr_ats_masters); -} - static int arm_smmu_enable_pasid(struct arm_smmu_master *master) { int ret; @@ -2553,39 +2532,147 @@ arm_smmu_find_master_domain(struct arm_smmu_domain *smmu_domain, return NULL; } -static void arm_smmu_detach_dev(struct arm_smmu_master *master) +/* + * If the domain uses the smmu_domain->devices list return the arm_smmu_domain + * structure, otherwise NULL. These domains track attached devices so they can + * issue invalidations. + */ +static struct arm_smmu_domain * +to_smmu_domain_devices(struct iommu_domain *domain) { - struct iommu_domain *domain = iommu_get_domain_for_dev(master->dev); + /* The domain can be NULL only when processing the first attach */ + if (!domain) + return NULL; + if (domain->type & __IOMMU_DOMAIN_PAGING) + return to_smmu_domain(domain); + return NULL; +} + +static void arm_smmu_remove_master_domain(struct arm_smmu_master *master, + struct iommu_domain *domain) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain_devices(domain); struct arm_smmu_master_domain *master_domain; - struct arm_smmu_domain *smmu_domain; unsigned long flags; - if (!domain || !(domain->type & __IOMMU_DOMAIN_PAGING)) + if (!smmu_domain) return; - smmu_domain = to_smmu_domain(domain); - arm_smmu_disable_ats(master, smmu_domain); - spin_lock_irqsave(&smmu_domain->devices_lock, flags); master_domain = arm_smmu_find_master_domain(smmu_domain, master); if (master_domain) { list_del(&master_domain->devices_elm); kfree(master_domain); + if (master->ats_enabled) + atomic_dec(&smmu_domain->nr_ats_masters); } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +} - master->ats_enabled = false; +struct attach_state { + bool want_ats; + bool disable_ats; + struct iommu_domain *old_domain; +}; + +/* + * Prepare to attach a domain to a master. If disable_ats is not set this will + * turn on ATS if supported. smmu_domain can be NULL if the domain being + * attached does not have a page table and does not require invalidation + * tracking. + */ +static int arm_smmu_attach_prepare(struct arm_smmu_master *master, + struct iommu_domain *domain, + struct attach_state *state) +{ + struct arm_smmu_domain *smmu_domain = + to_smmu_domain_devices(domain); + struct arm_smmu_master_domain *master_domain; + unsigned long flags; + + /* + * arm_smmu_share_asid() must not see two domains pointing to the same + * arm_smmu_master_domain contents otherwise it could randomly write one + * or the other to the CD. + */ + lockdep_assert_held(&arm_smmu_asid_lock); + + state->want_ats = !state->disable_ats && arm_smmu_ats_supported(master); + + if (smmu_domain) { + master_domain = kzalloc(sizeof(*master_domain), GFP_KERNEL); + if (!master_domain) + return -ENOMEM; + master_domain->master = master; + + /* + * During prepare we want the current smmu_domain and new + * smmu_domain to be in the devices list before we change any + * HW. This ensures that both domains will send ATS + * invalidations to the master until we are done. + * + * It is tempting to make this list only track masters that are + * using ATS, but arm_smmu_share_asid() also uses this to change + * the ASID of a domain, unrelated to ATS. + * + * Notice if we are re-attaching the same domain then the list + * will have two identical entries and commit will remove only + * one of them. + */ + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + if (state->want_ats) + atomic_inc(&smmu_domain->nr_ats_masters); + list_add(&master_domain->devices_elm, &smmu_domain->devices); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + } + + if (!state->want_ats && master->ats_enabled) { + pci_disable_ats(to_pci_dev(master->dev)); + /* + * This is probably overkill, but the config write for disabling + * ATS should complete before the STE is configured to generate + * UR to avoid AER noise. + */ + wmb(); + } + return 0; +} + +/* + * Commit is done after the STE/CD are configured with the EATS setting. It + * completes synchronizing the PCI device's ATC and finishes manipulating the + * smmu_domain->devices list. + */ +static void arm_smmu_attach_commit(struct arm_smmu_master *master, + struct attach_state *state) +{ + lockdep_assert_held(&arm_smmu_asid_lock); + + if (state->want_ats && !master->ats_enabled) { + arm_smmu_enable_ats(master); + } else if (master->ats_enabled) { + /* + * The translation has changed, flush the ATC. At this point the + * SMMU is translating for the new domain and both the old&new + * domain will issue invalidations. + */ + arm_smmu_atc_inv_master(master); + } + master->ats_enabled = state->want_ats; + + arm_smmu_remove_master_domain(master, state->old_domain); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; - unsigned long flags; struct arm_smmu_ste target; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_master_domain *master_domain; + struct attach_state state = { + .old_domain = iommu_get_domain_for_dev(dev), + }; struct arm_smmu_master *master; struct arm_smmu_cd *cdptr; @@ -2622,11 +2709,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -ENOMEM; } - master_domain = kzalloc(sizeof(*master_domain), GFP_KERNEL); - if (!master_domain) - return -ENOMEM; - master_domain->master = master; - /* * Prevent arm_smmu_share_asid() from trying to change the ASID * of either the old or new domain while we are working on it. @@ -2635,13 +2717,11 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) */ mutex_lock(&arm_smmu_asid_lock); - arm_smmu_detach_dev(master); - - master->ats_enabled = arm_smmu_ats_supported(master); - - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master_domain->devices_elm, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + ret = arm_smmu_attach_prepare(master, domain, &state); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + return ret; + } switch (smmu_domain->stage) { case ARM_SMMU_DOMAIN_S1: { @@ -2650,18 +2730,19 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_make_s1_cd(&target_cd, master, smmu_domain); arm_smmu_write_cd_entry(master, IOMMU_NO_PASID, cdptr, &target_cd); - arm_smmu_make_cdtable_ste(&target, master); + arm_smmu_make_cdtable_ste(&target, master, state.want_ats); arm_smmu_install_ste_for_dev(master, &target); break; } case ARM_SMMU_DOMAIN_S2: - arm_smmu_make_s2_domain_ste(&target, master, smmu_domain); + arm_smmu_make_s2_domain_ste(&target, master, smmu_domain, + state.want_ats); arm_smmu_install_ste_for_dev(master, &target); arm_smmu_clear_cd(master, IOMMU_NO_PASID); break; } - arm_smmu_enable_ats(master, smmu_domain); + arm_smmu_attach_commit(master, &state); mutex_unlock(&arm_smmu_asid_lock); return 0; } @@ -2690,10 +2771,13 @@ void arm_smmu_remove_pasid(struct arm_smmu_master *master, arm_smmu_clear_cd(master, pasid); } -static int arm_smmu_attach_dev_ste(struct device *dev, - struct arm_smmu_ste *ste) +static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, + struct device *dev, struct arm_smmu_ste *ste) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct attach_state state = { + .old_domain = iommu_get_domain_for_dev(dev), + }; if (arm_smmu_master_sva_enabled(master)) return -EBUSY; @@ -2711,9 +2795,10 @@ static int arm_smmu_attach_dev_ste(struct device *dev, * the stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry). */ - arm_smmu_detach_dev(master); - + state.disable_ats = true; + arm_smmu_attach_prepare(master, domain, &state); arm_smmu_install_ste_for_dev(master, ste); + arm_smmu_attach_commit(master, &state); mutex_unlock(&arm_smmu_asid_lock); /* @@ -2732,7 +2817,7 @@ static int arm_smmu_attach_dev_identity(struct iommu_domain *domain, struct arm_smmu_master *master = dev_iommu_priv_get(dev); arm_smmu_make_bypass_ste(master->smmu, &ste); - return arm_smmu_attach_dev_ste(dev, &ste); + return arm_smmu_attach_dev_ste(domain, dev, &ste); } static const struct iommu_domain_ops arm_smmu_identity_ops = { @@ -2750,7 +2835,7 @@ static int arm_smmu_attach_dev_blocked(struct iommu_domain *domain, struct arm_smmu_ste ste; arm_smmu_make_abort_ste(&ste); - return arm_smmu_attach_dev_ste(dev, &ste); + return arm_smmu_attach_dev_ste(domain, dev, &ste); } static const struct iommu_domain_ops arm_smmu_blocked_ops = { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 125b52ea5ad71c..ce01d3fc768ad6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -758,10 +758,12 @@ void arm_smmu_make_abort_ste(struct arm_smmu_ste *target); void arm_smmu_make_bypass_ste(struct arm_smmu_device *smmu, struct arm_smmu_ste *target); void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, - struct arm_smmu_master *master); + struct arm_smmu_master *master, + bool ats_enabled); void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target, struct arm_smmu_master *master, - struct arm_smmu_domain *smmu_domain); + struct arm_smmu_domain *smmu_domain, + bool ats_enabled); void arm_smmu_make_sva_cd(struct arm_smmu_cd *target, struct arm_smmu_master *master, struct mm_struct *mm, u16 asid); From patchwork Wed May 8 18:57:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9F59C04FFE for ; Wed, 8 May 2024 18:58:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cia5qKBna4G4Uljrn6DYiymyzefFaqirgnHEHQCP8iA=; b=BCd7HPljyu92zl +kDMSB2lZR7pi/wlQrzNu+TodtCOcwFzuxnT5ZvFertprF3G+8SKHoh/WoUYNo3ov54QWiZNRt6dM zfczp6ugsGoexOkR14ux9Cpy3I5oVhwV64FnLwJczimsVltzr4QKF7MkYXB9L46ewD5FPY3YHOsvB KNUo40i1BYnrEvbj02RksPoH+U22UP5kRUVb9uRPWwex5rsb3i0HRwtdTZK/J7Z25hlFuiI+NeizQ 3JVrpFu/xBjq0DeI5Zrt2j2/BKtMog5z0eK5QBZD6/48nEa09TlyJaesV/lKJ8JUV2Hn6ZxubNMm9 JvfvbZQDBISdoEDFwNqQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUW-0000000GcVs-3oaw; Wed, 08 May 2024 18:58:12 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU6-0000000Gc8t-0lqp for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CZ0Xe0Kz2cgxKEmyXlF9SCNM7k2hkMghGi0lKR3+634d7oxR3EfXHJNiZV9VZQ73CuBDf+OCFwbFTaSnD1hWKg56fFJggqVhJ3eoDBhlDfW6xmCV8Bd3SPsP3kw0kYU/R+6GXe8DzJLMiXQjDGbuqKEZ6jkyZ163lYuN4JkO8zhVOnsHLB2DBF/VBMC/TRw8ydt5b/ekqdXnWlmfF8hN0y7EYqHxmpQRwAqZwCI3DoJkkCsSLzLpPzRfHHrUk+cxEJKo1nbhEM9xEIIiSHl84NCSP+74+9b/bxWmd/QcWSMnLfsJTCYMYGuQiDZii4fx112Ssb/k+bcJytb8j1AhWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O4YOnaEiRj6hBwAdgo+aFd8H2MqvXXS+W9MsIEdNjqQ=; b=IrJlDlJmKDm0oxBHaK1rP0o1rSwA0mIMepwplc4XtjGKR1U5+sCT+sHUoXRQwDXaEt1UtQJmhbn9GFyYwe+6SLuG93tLnmZpYVi9hyZatSuCruu9l6SM5XnhTFc4VHGDtbQkY58VwVOgtIyNJOnu6AdGkvUS9NATSNjq+u7ahrFLBIjASns953sMxr/2nzHgIitdQ0BmK4qqbJQd4yQe/p8emEvb6AEyJ+Pe0VlQlzm6lrM+GJTtuXM/35YbusLBaTtkow3PHIVOJGX/fd0EJLmr0XQysKau0xxg/CxY8HCxcSn7sDRl0kEsiLqBLqKffrqGrLjJdN7twEGUua/+tw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O4YOnaEiRj6hBwAdgo+aFd8H2MqvXXS+W9MsIEdNjqQ=; b=pv5yALMMioFId1XxgZvQuN8dx5pm75PTEf3Uy3PyY6D/GnPV1vExPPkFZRHZkzNo2AtiCOBuyOb+d549Kum5ob8/4hg1x27W3Z3Di8ILnrvIkVKvpuDMet0tOglKiMMveFPXtIpJjLIsXEBTxIvqfmEbdGq6k3aBohfLO7ur/74R10sQbqz22rxa2NFcn9+R8qA4Y4/SfyTUNHICc80w6S9iZUnzov/3h8TTKwVRsN6CIBZFNm+8c8JMpqhcGQ72ewlc/MfZRA31i9dwCrBGs1gS+ad1L24s3LfKXsI9dXzQpuZ2PiUcQ045lSs7AfIInaJSM3aRv3z6P6XrU2XSxA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:27 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:27 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 05/14] iommu/arm-smmu-v3: Add ssid to struct arm_smmu_master_domain Date: Wed, 8 May 2024 15:57:13 -0300 Message-ID: <5-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P222CA0020.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:2c7::25) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 4b727ee0-f716-4485-1694-08dc6f90b37d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: jS60I++WJloUYcN3RYPrjFSRiTS7hrjqswWTzd7B7GLjoBDeXpgUVFnQxWrS0I1axOyTYAMwjQJlC1Ebqum6FTZlqfw93e1xd0dNUBdPHSLJazozCndglga7vkGPeOMzGW3BjcqLusNI9BjU0bl1ayYx8sSN+4xFRj4488aGxn/KSyQzQJmmbN+/xMdkdq1CchapY7nhEvuk1MSE22kW7X3tLWlt577W+4dCUe2RAdzh6IXTElOaVm8No9UoYFk4wcsRvBPdByYU4eAvCc3d5v5N9+2XjGX/GKSNaJVUnV8UK2ir7MqA6gS2Y+OZZYX+CaDVsJMrEkxtYzHU/gIhrU3p/ePIjiAjeumYoB4y3PwPJhynnl0MCjiP5CzxZYmeCXvT2RytjGsyamhkkAfaCYYme0fKjxfRSvGWyPI/+mMqzlgBohs9FmuuZ3m0ocvrzMWHSs8yLECQKNuTNUvG4jvkkxAoRZFlI9/Qv1eFZX0RZOHprd/8O+XUantrs5jvOauf7PJ7pqJaH/pTZ7Jrp0abOPIEbQ84y2oEO8Kqu3EW6Fq4DsCPPBVITgzUFbZ029FW5A+Fh98Wo+aDSHcfxMVgtt6NsMiJdynUTfc4sItkCq08NI3ApVBjZusaoFt5ypvx1Cv3ojzrOPW56lvfa95VnsfhpGndKyTgMW84QJP5nIPdHRIXAhPKEjVo+RpQa84lvdNlKkrje2aOfbnZPT9CUnOMsXhnyNQvYB5SuMBqt12tgMgSyz+nUyZmIHreNO+af3hmRWmuRbvzXCsWdWcjR6ILm+V52CqPWMQI3EZ4Z7eq4e2jAnU2mmZTANtMwGrVzSSeLHIujQH7pWtTNxqRZlSa3rFX7m/Ujascn6lJozvVS5LZlkI4tM8MDYiMPUI2a9iMoQWEyKBogrT71CqRYcE+eHN5OS5BBxU4a79gp2k2fHYb5G+cSOCFoTIFTgFLUWqvMhZY/mVdjDXBghuxCm7f/FidfMiqlZR22KyUjXF1Z9EoPABFBACU6WXxzwYg04E7ipxfTiRQjYtk04/6HrIGh67PVsg0rnEoKKoaUdK8W1XBcbBCKcj5EXgAqE4rhr32wMzB8q6ZWQ7bLGWiuX7JW8jNqozxplAn/oJTo7D/xotNHZigB0ssXYXhR4OYMxgxBd+UbJVLvd2yY/DyFPXM/z7tRu98JzXlbySeb3vTLOyMMvmbO+ysqr9U5gToiGJFUDsWxYJP0CvVYDJtsUe1pqWEEI1XUMoEJ8y0/KFtO6NpEKVe5m6yR/IqUX5SXFUv6Uw3VVnG4So+CQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7NBstXE2rWY9XfxOpMLc5Gokhp7azghRMZySfpvwmTBoDc7vH5Opcs8j6OsQYZbwXWDJkZAmo48DempSP8GpFkL6eV7w22aynLOmafrZ0GK4iFdF9g/01p6URTNibshcPx2m+dPjGHeH14ixVth4fYEkymNUWIm1Th7vtRyX+O13CF/w8ojtlnC8PqvEUmuQzDo7CmJGYI3th8c0tPK1cqiZk2IWLE+j6hplaUbLOSeuOW+kYqz9cgajxpUGqCJjIq+MaNbbthr35bqt3SjR5PKH7ONhU7TIS4EBUMTO4epmfiwo5762heM2L23dOvxSrwnLEiOrfkOxVspXf9EO4PFXzc5e57gGFkcZPeJI54eK12Ndzs17OPuhsdsrd3lr6P3MFx39ZyPgGVCHHu+8NcKJwTcso1AyCeKq45iG8sexWFxB/qVTYvzt2kx5re4dvTzQrq3ccuks+1XADffCfA5FKLAV/cg+H/KF0mz+S/z83RbcVz4isKnf67fhIfyJSOktGCf601neHgUsPQvm+uh3eGRdPkeWP7JUfe26oaZDGp38SetuUumqgJRG62dkc6UJufB0P+VrnULoPo80OuyL5Pva2A/wtBEPogvNlshxPqccsABd3RrB2RB88HYqSELDqb45RiPUCw//G5m7tP66tQsCE5AXaSCbSNm/nHcLafixiYR2397FYB4TK73zodETmhwTeUPiVq7g3NCMSIpuC7NjivqdLKmjqqTnbvAbOCsbeT/kW+l1t8XXGiBFxEm//LkcPdpl3jVzp8fEUeDHRMvOuc4+WYoe1mGYf9CAxaE5V4mod/HbiuV3C4sqVY19JF+26yoFUK2TYp1ecfDrt5J93LFmIewFea+4PTAmwJqQO9XugXjBG3DNE/ipAi63ibFQuxXj47FZqB5og5v/odMv11+41fiZFGwWLDtj4/B3TIv/9tqAujz6TqE5pKAdmmXKlutumrqOwOsfqKZQPkISoGqDEH9uOlelWiBtL1oF+zcF3Y4NgGCWVY+C09Z7osgFsUQr2l6HGOxPM6Gf6aJ7yxirw1tkSUSPaKQXNiFTWS2MQOSC+dcTZTXJxSYNeB0XNtTe6p+b5x0JFIAzKe/zfOCIIAuBIesX3Fltuf4wgRuhoYiSG+aNtaK3971/+c9a2kT47OCquMSsVHCq5NS0wVAlT0RlGKEW29iFVVK1UPLd3rROEWelmzcxFBKzDIyV85DmkYOKm6S49YOAfinX5XaLU8TkCOcWzI8OTGvqjH0kXlfmRIfKgcT6YnvigYJpZazwvfW4cz9lneX+lKOXgZTiI8a4Czg09cQ75fhMSOrzbNfTMLKCkoY2hBtqlIMysKup1Bj71kFUB8Q/6nEx0oaUikuntHILk7M+H1GBLODPMGg8TTjkjQ8FuNaxYZebPqTf/KPv3M+oKStrFwV4TR3z6gsalsaZvasDVq8SQIXgzZBqIiDScOIRPSkQvNcIStxZ5LyLiSv6sh1HkjkW1uxqovzu3KTjjrNJNUgQUmhOc79Z1xLS4tRQRTtIL01J3XzjSJeCk+5pJCjt7PR13LS5ifG0z9Hea/zelVWcg2Iz8ZXpC/vACZra X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b727ee0-f716-4485-1694-08dc6f90b37d X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:25.8719 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jg4AatnrH6c6ISUFoenZhWtq8uR84fS+7BzpwhIU2rOiO86l/xPgUhi/WFoUUrl3 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115746_435818_87B7BCDB X-CRM114-Status: GOOD ( 15.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare to allow a S1 domain to be attached to a PASID as well. Keep track of the SSID the domain is using on each master in the arm_smmu_master_domain. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Reviewed-by: Michael Shavit Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 15 ++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 42 +++++++++++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 ++- 3 files changed, 43 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index cb3a0e4143c84a..d31caceb584984 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -47,13 +47,12 @@ arm_smmu_update_s1_domain_cd_entry(struct arm_smmu_domain *smmu_domain) struct arm_smmu_master *master = master_domain->master; struct arm_smmu_cd *cdptr; - /* S1 domains only support RID attachment right now */ - cdptr = arm_smmu_get_cd_ptr(master, IOMMU_NO_PASID); + cdptr = arm_smmu_get_cd_ptr(master, master_domain->ssid); if (WARN_ON(!cdptr)) continue; arm_smmu_make_s1_cd(&target_cd, master, smmu_domain); - arm_smmu_write_cd_entry(master, IOMMU_NO_PASID, cdptr, + arm_smmu_write_cd_entry(master, master_domain->ssid, cdptr, &target_cd); } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); @@ -294,8 +293,8 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn, smmu_domain); } - arm_smmu_atc_inv_domain(smmu_domain, mm_get_enqcmd_pasid(mm), start, - size); + arm_smmu_atc_inv_domain_sva(smmu_domain, mm_get_enqcmd_pasid(mm), start, + size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) @@ -332,7 +331,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm_get_enqcmd_pasid(mm), 0, 0); + arm_smmu_atc_inv_domain_sva(smmu_domain, mm_get_enqcmd_pasid(mm), 0, 0); smmu_mn->cleared = true; mutex_unlock(&sva_lock); @@ -411,8 +410,8 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm_get_enqcmd_pasid(mm), 0, - 0); + arm_smmu_atc_inv_domain_sva(smmu_domain, + mm_get_enqcmd_pasid(mm), 0, 0); } /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 01e970f6ee4363..3a7ef73994b57b 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2013,8 +2013,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +static int __arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + ioasid_t ssid, unsigned long iova, size_t size) { struct arm_smmu_master_domain *master_domain; int i; @@ -2042,8 +2042,6 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); - cmds.num = 0; spin_lock_irqsave(&smmu_domain->devices_lock, flags); @@ -2054,6 +2052,16 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, if (!master->ats_enabled) continue; + /* + * Non-zero ssid means SVA is co-opting the S1 domain to issue + * invalidations for SVA PASIDs. + */ + if (ssid != IOMMU_NO_PASID) + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + else + arm_smmu_atc_inv_to_cmd(master_domain->ssid, iova, size, + &cmd); + for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); @@ -2064,6 +2072,19 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } +static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return __arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, + size); +} + +int arm_smmu_atc_inv_domain_sva(struct arm_smmu_domain *smmu_domain, + ioasid_t ssid, unsigned long iova, size_t size) +{ + return __arm_smmu_atc_inv_domain(smmu_domain, ssid, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -2085,7 +2106,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -2183,7 +2204,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2518,7 +2539,8 @@ static void arm_smmu_disable_pasid(struct arm_smmu_master *master) static struct arm_smmu_master_domain * arm_smmu_find_master_domain(struct arm_smmu_domain *smmu_domain, - struct arm_smmu_master *master) + struct arm_smmu_master *master, + ioasid_t ssid) { struct arm_smmu_master_domain *master_domain; @@ -2526,7 +2548,8 @@ arm_smmu_find_master_domain(struct arm_smmu_domain *smmu_domain, list_for_each_entry(master_domain, &smmu_domain->devices, devices_elm) { - if (master_domain->master == master) + if (master_domain->master == master && + master_domain->ssid == ssid) return master_domain; } return NULL; @@ -2559,7 +2582,8 @@ static void arm_smmu_remove_master_domain(struct arm_smmu_master *master, return; spin_lock_irqsave(&smmu_domain->devices_lock, flags); - master_domain = arm_smmu_find_master_domain(smmu_domain, master); + master_domain = arm_smmu_find_master_domain(smmu_domain, master, + IOMMU_NO_PASID); if (master_domain) { list_del(&master_domain->devices_elm); kfree(master_domain); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index ce01d3fc768ad6..23d8a5b7912069 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -772,6 +772,7 @@ void arm_smmu_make_sva_cd(struct arm_smmu_cd *target, struct arm_smmu_master_domain { struct list_head devices_elm; struct arm_smmu_master *master; + ioasid_t ssid; }; static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) @@ -803,8 +804,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_sva(struct arm_smmu_domain *smmu_domain, + ioasid_t ssid, unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); From patchwork Wed May 8 18:57:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A1F9C04FFE for ; Wed, 8 May 2024 18:58:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bA5a3ROtG7PjL/8Nn93w/7XgRg2x6rbeBedaLJjIdpo=; b=PyVkS8KxAs2QsH 2mmwt2ZPEqSKV5taCaduptpSQKt2WCOJEAZvPLKAC+3KYN0Wnh77XVli2s8bLzdllC0uqz6G26c0U THT/jj3n/NxaaTXUbCZeRGrRUScUBxR3fxt1HLhv4GgcOxGqcFHFynm0wfcpfmzcMgRkseBTF1/wC vWJXtSf7lCpcHSP3uF6ywbvqdk2d3CvlPvvLpaE6OjVYi5u39f26dPtO/PstI+VzJQa6bGrec+o4R fcVo8BFPsx+lF/u+N3JJ/FKvogULCa4wjkJjhA6gvQNPy2IID2VcKYs78SC17H8vjKNw/XVL/6vAn eDgBsECa/Z7OcB6G7bGg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUp-0000000Gcj8-1SSr; Wed, 08 May 2024 18:58:31 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUE-0000000Gc5t-1Jeu for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HCWlreo8FStuQmulBElM2FLAe+UHmQSYCU6Zo7TmxyF4oLkOL0Ek5AxCqjvWU6OceN50k1pmSpGs03LagcZUKLsIL0SdfY0LIMD0AfDj8psXsqDfMyW7U8/mggRsoR1BFQmpBjH2SV2yfDdJvJkUd6mPrZWOwm1ZXrcc/g+SmGFL7bAAPq9NMrNek7V5aJJSdg6q/Wm0KnBhOS3pN60GLRVhvG43hZHyEcaAIf+3b80Cxzc+OHqpF5dpg9CZGipntF5EaIs4+E+EK1HJ19Elg/Kus7JbLNes1MvXlHJA1MbjUVbzLC/jEOvHAdGMYdBOb24ihb/geqvhOPjlV1Iy0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IUodXpyO7aK3vOyswz2aXpOEtIdXRX+5P/d2jL/C30k=; b=n79GpPPDC18KbHgqHUOoM+t+ywVHzqw86luw9hFFllAqLrbMNf8qLLv1IiqG7T4I5gKuC7qDf7Ul86KVHDqMuAdmOUHs1D+gKnx9c7FLa/aE9U9Z//u7OUtCwXWKoIeJqFTuTELvJRAJgUt3uvVLtWDJKdsp+HQzJ7vsDZuM1IVFs2lKAYQo5fDVR0PhejRGXZc8IUyyjzbbaR1sOyqn1lVOIx1qUB8mQJ8X4vlf7qSKX8glGvqyRcJSXrZUisfsmjDc1WkKMVo8XJrkO/fZMi9Io3/q6PnA3pfSQVqEkHdsbqyLVzQR/4Y3WOC5wkbnsL+EFliC5V6ppCfiCcl6Ww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IUodXpyO7aK3vOyswz2aXpOEtIdXRX+5P/d2jL/C30k=; b=e0BkymJLeE0a5DmGrHPYzmtMpy/IeHnetyHT7vqMLerkR8kFuAf4EA37eZDzKkaE7ObNh7XyZXMv9KcH3pXUM5sWv6geghgtp9tTq4+ZtVx1WOUMYVz8phsbt986tfH5dhdtatZHocOuml/+hIHtJSNMRK41hfqHhYPoUG6ORFYwO7o47YR9O+4RfYBrq9U0I6/lSTjJVVfZHRiY8noKOZM82s1Ktr4FHtmPuAt0boJNv0Gi6RATzYSLbTD488G/xGw1E0MKcbae5iwEkYfRBGFFvUgadvaHHinIwB+F9Qmqqt/nZml5PNlkbm+p3GzitoKmXz/eIJS/38Dcb+v6ZA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:28 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:28 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 06/14] iommu/arm-smmu-v3: Do not use master->sva_enable to restrict attaches Date: Wed, 8 May 2024 15:57:14 -0300 Message-ID: <6-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0079.namprd13.prod.outlook.com (2603:10b6:208:2b8::24) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 73381cfb-5aae-45f2-97d1-08dc6f90b3f0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: XNQYvjHvhjL0DdAumGVUyb+pLky+zj2toFbNrALuinS6yv1yDcM/xrZEvJCgY5mdnvr/JkSSKmDnLKIa3GXhU738jUT14oDfMXeMnzpOPb/YCeXLdhQaFkxObv57+7IPJ0EPba91ZUo7cxI71rMj+GEwXkP+0guzIvXVy25PJJgbdeOALUoMpb4itaIg8oYz3LJrvimbVxZ3xtC3wYb8WYlXtlFrcUqMolQi0b+gIg9C7z+/r0wSr5ZA4gQyAwvLoTh80cm2Vu6Vf8DPtVwvS9HCpIspmHPf1FkQHlAkI/gmVh+uRq7hVGpFTBKacTazbbrWJMCb4LD+EvyTJV/TTQrzCU7XlF/khO+qescBZ0kONnD9T1vUsZMRlyNqvv8wkDb8efIs6WT3jiH28oOxC9pIHLkqjPFugPeDp6uW741ZnTWK/X2/mv71i/0xWxKNtHCOZyHY3Bxd6k5HbSTfWCz5p0z+tqaup9rCOwfKUl3ORZxo/kzDurNG72ta3v/rU/C7zebS9mWEoGCfFpjzL/GSn4Dbe0Tc1JhgOCyJkLwbw+3YA9zYxwXN7mgEutNTkrejpxY2ECek8yWOS7JYx8Se1kCAeUT6cAg5s66BBVIVYcnDF/m1zvbm0NPBORNFlHpCsIzAenhNz3/xiBOB41kAjMXRlfl3tZqEpPQjdnkIBpFgdEk8bUnQReOcdjfDDktHch4WhDLuCEbUM6oA/91CcoisyjNcrsf6qZjCeEWMXgMn7tHZrQbfgeEtWtd0tuJZoMKcAf8GhrKZFeDsMlYIAs30fadcAcRqPe9/nBy9mHXxEIpGlaBTFhBvYYE0HE/yB/JmcLCEyyK+pR0lyWi2SLvk62uI/vqZuVMTG8qdyR4gXz1olk0CTjGNI1qdLhAzsEvalPUE4ytsRdIf01sJd/7d01P1+MNsBe/XYsnFL0CRFELvR62Gs7cybsxFVoSr8UNAdYoDca/zf5ZPl3tWJAbCUKb8V/R8mTW/v6L6fmswQGzvyjMjcwOq7ki86R6+7DuvHAXuxF/MYcNq+8m3IJ4JLuswWgvwd/E1YNz6Q/XuyaAS9b6LzlZgLYg92UPyQBmPfujQLmzwsLf9dCcjf2ocvDYMKXjirsBgNuxI/q8689aHqJ+6yWeX5Ur8aMjIw/NvXiARHGpnrXu7rNu8guF5a63chUR5Z487vSqS292IYRSjt0JQwa5Hpsii1QivqzFAodZjN5n/o66/rk1oAIDSajqbQbQ7rW8bcwCRo7YoEZNu2cbTa+nahXpR+AR++RW809QV5xJ0tX/zFQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: czmip6XJdZ397HnE6HVQvSHChFgF6F+LChVxHOAesSZ0gqyPjuEBiakCO8iVJ+P4sG8Kdq5rkLDLUH2o1i31D092GIcfaxQ8iNztwjJXLofIknHg3LiHSj4wHba8aKgi/dSTLVvmry7tldgFY3toDwLpd6Fyhw44Ui51zeXFujV1i8Pqpq69zKozMIiWpW3ptfU6KShjnIrGDNdW1mElpudXerSY1r68vbCrS/UWpQhwJP5qyTnnOKliuYrVWVxTaR0011tgDrSelixWYvsIr4oGw0OLTn69BTqOeS26eCIOX96TTh+fjQ4Ez2J5kOeQo1bLfCfbWxcLxAvAv3NFxLEH0SVDnO190rxw9aHbZe0FOfWTVCz0bgRpGbH2rbebFsKMH++u+6TAvFBxE6fkVd5KuZHBHkkrBZOAY0dJFuiY8oX5UpGRF3/q8tHO4nn0XezHdlxOyZN5kDLG6d6gBBA6faU0qmAo4dOgXTwAYAQ16HGYVtcAPizno804WC+nNC3eD32a7IObdjLPv3eQniJ/R++d/IpTxl4v582BRh7hZ0WVS23VBXeOehpejvelxtT9aze6YksVB+/C/u71VHtttp42m7FAynWeUaGdqqDjzHf7eh43N9XBo3lXPgmIw/YbR5+/ocPQ/RROwN7KUXlptKoSSLyINF1SckaJ/sng8MunMMtXl3SOzUwHGce7PVQv00ts0GRxVl6LI1yPvG7+QO9PhMKU3qvioiRp6kzGq8f8z3oPUz+yd8NIr1i0J9nEyMFKfL11sg33kQ4GLa3LfFthZyPbAQTAgb1VdCHb0cNHgOE+Pf7uWkxmahgaCRzuCVShCPBd400EHygnhgXs3HKN1PXiDST/YvdkzGNcpRWRaVLN/XmIf/YCTNh1wnCaPd4eDs/jhWRVORIZS8GWtBGJJXlT0kAAoP/m1wwEMTjQjUC/hUgZBJ3+ITebEKPHRNq/pa1otYDVyCscwKibGPOsFysMpUJUoDCWIUVUUimgGiyJ+ZCF17yJgxthum5N+EW9rKo/BEkjRnUOQUfgXXihXSIaq9B3EUXIwdNHx+uuL+nZfIj4B7kvlqpV6uIU7KZBb6G3qVlfcWvQJ014HFny93Mv7ktAZpCccZ1Rwaq4ePNf8rOwdz0yJNyG2Va9SIyYrdrhK6ePW0PWIvEzMki9wkz6LHRzKdtCT/I0y17b7VCILFBd51kuwnx4+6rTa2hfOnE1qVLz6bhFfkYTnAPbD+I+jQYNLxxRA0+kEqILUu+td6wZ3qPNEWnnQ4wQv///YK3S+OtF/1378P6Dyhx9oQMvIcKmfT9dOV0lkDeQBij2kGZK3Td1ZUlNE5rQ3l3xqcP7ijRTppyv8Siucu5Vg4xSvLJm7TkMncLqT085mjthNZSaejcod2tnJTNZ2RuSk8a5z4xyKZSnvicJDVMe5H66UJFwjocnDCpeHuXD5o0eliu4bPEJBFh8HcJeQTflB2fvQp+0lQBeBjL8voBYAoEW6e/cXuEeuNaoptU1yLsO0d3QbbjZHNzZ3tQpo5PWjxytTwhpvqsyh01QfI8ZNAY2RgeLRJ1K/mcwYqHKDolZTwoZ7KkHexzJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 73381cfb-5aae-45f2-97d1-08dc6f90b3f0 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:26.6219 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Sc4ctNaRps8esyauIF3svT+FzqcZR0XoePpvQGZO6ntQjaq9PpmuLhjaW3H39vI1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115754_704402_4F279AE8 X-CRM114-Status: GOOD ( 18.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We no longer need a master->sva_enable to control what attaches are allowed. Instead we can tell if the attach is legal based on the current configuration of the master. Keep track of the number of valid CD entries for SSID's in the cd_table and if the cd_table has been installed in the STE directly so we know what the configuration is. The attach logic is then made into: - SVA bind, check if the CD is installed - RID attach of S2, block if SSIDs are used - RID attach of IDENTITY/BLOCKING, block if SSIDs are used Signed-off-by: Jason Gunthorpe --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 24 +++++++++---------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 7 ++++++ 3 files changed, 20 insertions(+), 13 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d31caceb584984..7627cb53da55b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -628,7 +628,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct arm_smmu_cd target; int ret; - if (mm_get_enqcmd_pasid(mm) != id) + if (mm_get_enqcmd_pasid(mm) != id || !master->cd_table.in_ste) return -EINVAL; mutex_lock(&sva_lock); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3a7ef73994b57b..5f418b96679c88 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1289,6 +1289,8 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, struct arm_smmu_cd *cdptr, const struct arm_smmu_cd *target) { + bool target_valid = target->data[0] & cpu_to_le64(CTXDESC_CD_0_V); + bool cur_valid = cdptr->data[0] & cpu_to_le64(CTXDESC_CD_0_V); struct arm_smmu_cd_writer cd_writer = { .writer = { .ops = &arm_smmu_cd_writer_ops, @@ -1297,6 +1299,13 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, .ssid = ssid, }; + if (ssid != IOMMU_NO_PASID && cur_valid != target_valid) { + if (cur_valid) + master->cd_table.used_ssids--; + else + master->cd_table.used_ssids++; + } + arm_smmu_write_entry(&cd_writer.writer, cdptr->data, target->data); } @@ -2706,16 +2715,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; - /* - * Checking that SVA is disabled ensures that this device isn't bound to - * any mm, and can be safely detached from its old domain. Bonds cannot - * be removed concurrently since we're holding the group mutex. - */ - if (arm_smmu_master_sva_enabled(master)) { - dev_err(dev, "cannot attach - SVA enabled\n"); - return -EBUSY; - } - mutex_lock(&smmu_domain->init_mutex); if (!smmu_domain->smmu) { @@ -2731,7 +2730,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) cdptr = arm_smmu_alloc_cd_ptr(master, IOMMU_NO_PASID); if (!cdptr) return -ENOMEM; - } + } else if (arm_smmu_ssids_in_use(&master->cd_table)) + return -EBUSY; /* * Prevent arm_smmu_share_asid() from trying to change the ASID @@ -2803,7 +2803,7 @@ static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, .old_domain = iommu_get_domain_for_dev(dev), }; - if (arm_smmu_master_sva_enabled(master)) + if (arm_smmu_ssids_in_use(&master->cd_table)) return -EBUSY; /* diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 23d8a5b7912069..e850b7913e67a1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -602,12 +602,19 @@ struct arm_smmu_ctx_desc_cfg { dma_addr_t cdtab_dma; struct arm_smmu_l1_ctx_desc *l1_desc; unsigned int num_l1_ents; + unsigned int used_ssids; u8 in_ste; u8 s1fmt; /* log2 of the maximum number of CDs supported by this table */ u8 s1cdmax; }; +/* True if the cd table has SSIDS > 0 in use. */ +static inline bool arm_smmu_ssids_in_use(struct arm_smmu_ctx_desc_cfg *cd_table) +{ + return cd_table->used_ssids; +} + struct arm_smmu_s2_cfg { u16 vmid; }; From patchwork Wed May 8 18:57:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1558BC25B5F for ; Wed, 8 May 2024 18:58:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LdkS249cneuf8SMGg05VXigrmrREqALrrUS6bYRzDB0=; b=qIR+l6Tm/aaaoM mf0eaZ9lUWk+L79Ze7ID3t9UTVueoE8TZPJfS153dI9TfjTTMoDCAJnWeyKsMsprEUQ13gMyc07Za 7vG2HVHBQ0nTua17ZemXiQFvPQXu1bqOUwDGbkzt9JXAlrCrT7M5WeB7e8FbeaxdStwnSybphIjTB TN+EmXoVZ6pSbg35QfOOS7Cas2CC5RY5hkUYyq0YD5xBNGMn6NbZB5e7sbAmJumeFfa9SSYLS92yp DurCFzwYvHHdqOfSntoCCmZAPwrRBk/V8JErUJifkBumeWQX7AvqVcpFhXgvM/XlXTt0yqKAuVTHD yaxPyY22vS9z97EJd2IA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUl-0000000Gcgh-3eNK; Wed, 08 May 2024 18:58:27 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUD-0000000Gc7U-2qLt for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CREU+s1XUG5jcIvwCpKV882FIUoL47xrb7AK/fFpf+/1KhrW+gfpP7CIVYcf1Iv6UYDFiJ2HULUv3fxN0DZaMWO5YDHVrBlyMfMVQonJa54fSRs5xDMtlYrMvLo2IJapnn86Qvrjl8RRIc1BSkfbgol55zV7rbPDPQNzQ201x6xnUG4ewibm2jXL9JjICJgiyKNz19RsaOG7oaZen6ijWPA/6m9l1QCK/ybqvrIYCS1gj5dajTsp1YUo+7t6UrojHhV/Oyrm1CG4VaXQOqzyTmei58wjGlR/80/5zzilgsHOcpxkbizoREKpmWqIqpmIvFZo5d7MF10DW5Fyein1eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OfKEWo7ACqbvcuUbrtoSc5CScqzXZmbs/gxNzZH9glg=; b=QtDi1aDREOGOCM0ATeF/RjHJ469j8+JrVqCLjLKDz+jmnirMzKIA4AV2bl9mGvai0HglVRraR8ZsvWjqTwj2IqgJXxXS5frhdi5a9jF1dJzZqILx5Mmt+0RjOGsPLnwSgJd43tZ4/YUO2e1FkX6HtlCt3z2llMPvvHcHODQIbJHhqd2ts8t9h4TjGa2f2nNPwjHc/njlgUxu7kH48GZXr+utZidm16VO8YwLZ+TrvIwKK9J+OvzibUkH+DPC8tM0c2/G/lsn0JpjXwaKKmMtvBWYAimCwFMFSQKx0nMyxSirjFQD3pJajNEdO+wBicaxfwJ9zzQqUZ2065VNbYDqiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OfKEWo7ACqbvcuUbrtoSc5CScqzXZmbs/gxNzZH9glg=; b=Nr7G/M6mF8PpGlqDpHDsJzE8YwZb4OCJXyf+e+NrMtOM1JWb2CDA13MycsYB1p9f9ir0X4RQIeAUihzT324TggTGz7VyZ2m4KBJT4iV4PnsDpsQ9ay1NpYQUTQBQGfTyrP6Ip8WzEaT1bt/ZjXH1BenVOK/KbisTwtZz7ILw0nn25OdjnD17cIQtXKxMCJEV7OWIYGs6L2V6sUQ+0V/ZjBTvAe9GcrmDwlq5JblE8ME15plW9uQAV+w+f4XvvkDhS7QpV5jHvy6OxaZoZE7R6g11suHKzzQwB3EG54rS/G3m++Jl42J7S3WEt5s45s5g59cP3klsoJrrYZjLxH67MQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:27 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:27 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 07/14] iommu/arm-smmu-v3: Thread SSID through the arm_smmu_attach_*() interface Date: Wed, 8 May 2024 15:57:15 -0300 Message-ID: <7-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P221CA0020.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::7) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: d4550059-1b2c-4859-1e11-08dc6f90b3b4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: OyMPOY5kpFpYP961iX1EKH1RyYGVQ3P6X1MJW9QWU7XvYMOArgOgP4/Q5erSCgYd2/HODFJRgn176RpXffLB1rVhvfFh6n72XtRooG4T3LpmpTmW19WF3khbFqN1BBlKwfDvESFiA0VL5Xoax1mlzcco7NkXswijWKIRcgesXm/RV5OfF4281MaEuA7wevH7Ydd5b3FSvbD88f9GtTlJAHn8nwC0eq8KLt+PjvkrDJ2vSOFubGxqqapKiHPKouFw1MzIRXJDGbC6jaobmhYlfWbr0qa2KW+rAeZqo2ea4weuDPasMPo0j+9rkqcMCujaWzzOjNwsOWHgE+c8kjNiuYoW3KLfVGW58whtNfi8dHOkXiEMi6oesx6fEzPwhCTKLUDy9T1d2tTd5TnRxhIBfdwR8vGXAijpxVK2G16545evhMt8BvRi5xcYdW19j9yLF+khXkPfgF2C5rBlx/03k98oxvzAxCmjIJ3K+czhMaR/jsz5qQpbhMsLHo5uhPoL1oDI3zEjLkWieb5BvuvY3c4ScB106HTdNAqDb+FjBGZ/PUSCFpK3nq2NQojQGgDyXb9wEHltNxvz1T3+8iEO2/uAEKWT0bm6mS/I0N+Sy+Kt92i0kDZzKTr1F8M+QkdCjNPTXzf4FQrd8THIexLoEQAHS2WTE37f+LPXC705DCf/r35tUqCC+IE4anN5wyRDjFDW5E3+OycbnoohqcIiUykANBuZ8o+feq9q8oVLYilbVSbxFA1C7iXL75doKZLFEGAZImPyDsPzAeV8ms/gInbAN+kw5r4uJLMe1B6XwpHmfkUOIed7XYJG5tSsRXjsxVBvozTNJIVG7O8KitS3YhT7blx6HqJKqRoQ/SfBvBr4Uqf33OR/O5ZL5uS7+nlXHEhPwU3gv+F8VFyVhnQfA8prND+TsC0bSjWNYASbeULk5w1Gk7X7umNvWwcnPck5Ahl1+Inqq6s+Kfojw+5+yyHj7vCAOb09ROzokluHNBQRqImLFIynKtjH0abaqjJ6TU+FEQrANOMqYeUoWO/Zq1HNfdVAXYIB535yfl/nXsTRrflmYcc3Qc1HHPVWdAp2Ky9dn9dRcTjJL0/7hgRD+wOPshTurlsMwRmkKFaG56QgZVYLrGJofq5ahOdfCdrTp9bVbJrBi22h8JYu4v51eTUYzmx/zzd3/+i0huZmHY8nNOMzaNgUsJLZwL2T1SGC5mgYk6OjimFklmCMVwGqCM0I/scgYMdU4n0Z7HQtpLnAiLfpaCwO5X2rAQq7rGqY2TZe9umgRreFPu0XxJhcWg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ozsoz0RqBoZYZxZolD2iGStkIzQTSp7w1W9dHRzz3dHgZhs41qrmkAbNQsA0rdaWdK4rVkbYzK2NbIad8ALLRXvhy57sqG5vPTdAq8j2Mdy9gccoWCUCxriVBIv90dXKAA6VsZnP+1o/R7yxNWuXq7s8LZhgkwo7/v0d38W3U0uVrGf/gv4gRjo9GoiAQC9c4Zu0hHWjBreozDuuuypr8DByfErJb6OsKLfHdVD9gsczkHtUHB3cLJiGBjTUKWujGSoyN9Fd57W+86Br/7oRGSD+uSArbilQuZjv5L/wMov2MafNBb7fdeUXuziR+sYRPaQlVA0ZNkhYoq7Chu2zVzBtuo+te/LA37r6//SMO5h76iWHG1Dkzd1rcWM3vlWmNgX3KLTVJcBtztv+Pwi5UZaJcEe6AUnsClXSXMFWZYTaW/uJVOkGixnPJQ6+MtCzCr9fwZ//Qsyg5T3NqxKf6xuN2JHX0w7vFSns1lu/zD5hTkJU0VdEbjN05qq0M8duNzihGwQvg8RKmE7H7R6K589bc9oyGfoPuoDWbmZI7JuyECaHtEXJYLS+tEB01NCkEb9hzFgna4GS6ckrQYPaKd4YFEUBp09M34X1B90U9noqLHVPWVIOefYBB+rzmvdblBK8apuAWtXWVWxnpIUhBv3uVqXHK73jOtkAQwHLRcvD6d8KK1Xc9f8fCsWuLPY9OFlg3CbFqqrVVAYAYqXsj3MI5H28ClJoQZ6ORPv8jvu33dlQggDpy9NX4F0cbMuegM704E8T0BP7pTFvEtz7mnog50Wmocy5te5XnFF67uciWqyXBIUNQ2u/nZBfxpLGCYP+NbFlORWzPk0IfMh+8jtZlYioUlHYYYImDA0WFS5kZxx9C0wWDhrDuZe+wKG4bEdhqsQ+lSqTXRv/SE/PD3NyI4czOskOumN1kdSH0GGDnMBkSqgv+/GvPBI5VWc/8+qYb2CaAeSYoGf5O+DdvkKXTdpukpWuLAXeIPlU0v4zdLxFUQGY5O1Xe/chB+3oJkRXSxflLSAY/Odvio1nbCvCVc0IJ/CnflCstCpwJQd6jjEE6m2DlOCEJBCItA83H6GH/87j9WbZHchBFEYIjUpc22ffs/0RROPGvipbml7suUHEgON3DHQQ9+R4kujkV0z8jkpGzfrfPFLQ81ZTj3B4iGdXMQz7sIQtHRIMBfVzEJN9xgKJMGLBErYSIRd0grxqdNz6OK9JMVUv/LPkna1wft0CBMBEdCsBibGP0D8Gk8THLHbpr7eyO7UoOX0WF7EZuv3vJQsIEtHhVki14P6xfFwqjY5agZrVrnIc/IaqSM6ybWyov7Vb7bM9bdDP2URGoa3k4cTRT+yWoZkLVNllmezw6au8n7ubX/XML5qK3sk3qcWeEN9iMt3OTGfdKcf/+widV57TrW3ign1YFkfxUMEczsbkP32q+rUrV4B7fgOPFtoDPLcBwL7xTB+ovpjjLbXISx/SKq/ECXJIXTpjgGye4QcqNSd5cZ/g/T2vGn/h/wzGkFbDET4ruZO3+D2BsKYbieOBA8SDuiTMO6vC9S21nnCE5D6iqZSlHtWUX2vkqNoVil4FqO7UghdO X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4550059-1b2c-4859-1e11-08dc6f90b3b4 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:26.1711 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uAT8MzD0KkMzG0uituD3Ju02xKZM7YEkYfnYIwjqgaSerhluQbJd5M3ffQt+FsW9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115753_922984_30CE30E6 X-CRM114-Status: GOOD ( 14.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow creating and managing arm_smmu_mater_domain's with a non-zero SSID through the arm_smmu_attach_*() family of functions. This triggers ATC invalidation for the correct SSID in PASID cases and tracks the per-attachment SSID in the struct arm_smmu_master_domain. Generalize arm_smmu_attach_remove() to be able to remove SSID's as well by ensuring the ATC for the PASID is flushed properly. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 24 ++++++++++++++------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 5f418b96679c88..b2d8b7a1df343a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2005,13 +2005,14 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, cmd->atc.size = log2_span; } -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, + ioasid_t ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; - arm_smmu_atc_inv_to_cmd(IOMMU_NO_PASID, 0, 0, &cmd); + arm_smmu_atc_inv_to_cmd(ssid, 0, 0, &cmd); cmds.num = 0; for (i = 0; i < master->num_streams; i++) { @@ -2494,7 +2495,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master) /* * ATC invalidation of PASID 0 causes the entire ATC to be flushed. */ - arm_smmu_atc_inv_master(master); + arm_smmu_atc_inv_master(master, IOMMU_NO_PASID); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } @@ -2581,7 +2582,8 @@ to_smmu_domain_devices(struct iommu_domain *domain) } static void arm_smmu_remove_master_domain(struct arm_smmu_master *master, - struct iommu_domain *domain) + struct iommu_domain *domain, + ioasid_t ssid) { struct arm_smmu_domain *smmu_domain = to_smmu_domain_devices(domain); struct arm_smmu_master_domain *master_domain; @@ -2591,8 +2593,7 @@ static void arm_smmu_remove_master_domain(struct arm_smmu_master *master, return; spin_lock_irqsave(&smmu_domain->devices_lock, flags); - master_domain = arm_smmu_find_master_domain(smmu_domain, master, - IOMMU_NO_PASID); + master_domain = arm_smmu_find_master_domain(smmu_domain, master, ssid); if (master_domain) { list_del(&master_domain->devices_elm); kfree(master_domain); @@ -2606,6 +2607,7 @@ struct attach_state { bool want_ats; bool disable_ats; struct iommu_domain *old_domain; + ioasid_t ssid; }; /* @@ -2637,6 +2639,7 @@ static int arm_smmu_attach_prepare(struct arm_smmu_master *master, if (!master_domain) return -ENOMEM; master_domain->master = master; + master_domain->ssid = state->ssid; /* * During prepare we want the current smmu_domain and new @@ -2689,11 +2692,14 @@ static void arm_smmu_attach_commit(struct arm_smmu_master *master, * SMMU is translating for the new domain and both the old&new * domain will issue invalidations. */ - arm_smmu_atc_inv_master(master); + if (state->want_ats) + arm_smmu_atc_inv_master(master, state->ssid); + else + arm_smmu_atc_inv_master(master, IOMMU_NO_PASID); } master->ats_enabled = state->want_ats; - arm_smmu_remove_master_domain(master, state->old_domain); + arm_smmu_remove_master_domain(master, state->old_domain, state->ssid); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -2705,6 +2711,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct attach_state state = { .old_domain = iommu_get_domain_for_dev(dev), + .ssid = IOMMU_NO_PASID, }; struct arm_smmu_master *master; struct arm_smmu_cd *cdptr; @@ -2801,6 +2808,7 @@ static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct attach_state state = { .old_domain = iommu_get_domain_for_dev(dev), + .ssid = IOMMU_NO_PASID, }; if (arm_smmu_ssids_in_use(&master->cd_table)) From patchwork Wed May 8 18:57:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 727C3C04FFE for ; Wed, 8 May 2024 18:58:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UpfWoa56HYWWpMOEZPfJpBoZA7gxArtUYjq7xifBT8k=; b=nFrG48nyAyONnB Ju50y5q+IhvnlAIGmT/iSEHubIE3BGAK0yseMZ1Y3OZ6tgFa5ZeSXMI+exELaQd7X0v/ZWSZbx4fl QVs8CXM/3OFl88hU7biw1eODPjUv4uBtgrmsrpNemwdrnfOAUlbvZ3MnnMVuqLEPc1A/vVutlLuZF x1uM2hfwaKnbs2Knx0ezRkO0YYeyF4UWVXXOC3xw36yOWrVP6c3N0oLBDdMctdfkZ+DrrJLxKmoIP /M8P9xNpXTsW40DejRjMLLz4wG+JenmqMXgCndkFcGoqA7Ea6j5tbqjsNzU9yEPi8xpLKFqmBh4Cx VTq+XkQPHWFPKnwSLP1A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUh-0000000Gcdl-4BMl; Wed, 08 May 2024 18:58:24 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUB-0000000Gc8t-0zHH for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ab82/+a8d8q66b2Re/8DdVfsz0g4bBR0B3jtCmS1Vv3CALQwmiAdKEv3iBqrhtrlfYt3gB6omjgModPx0INZ/R7mILqpC7Nh4jt7dat56WaWh8ljmoLB860HKQxFQ0FWZmkZ8/U9hxrnK3gFHdTx0fDoPQS7PMXiykPzT4gN/C0KXNxnep7ElWxjl3XrF1SpsOBEMwnwTJinQaXlK9/h5AQE5izNJdyCVDuz+2AFx7HsMVztOCtoNmlX+/OuPq5u+SRP8wJltZSH1mJIQSg8M+/5Wj1h6mLGh3vYYC8FNOPX+6kWnSaJ5aag9kKdTjJ1G+PoY6hWb2NGGRV1EGgU6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=F5FTUpjpMSI/6olZMRtQP+S/FJoJlN0+xp6bzMXdtVY=; b=aA7oix9/0n2pupmXEcysk8t+xb2wGwoqJ021+UqDNZDstuhmPM75FllRo9JgGOzvJiyjkLfDR2DouISEMTTZlcTjhSPNeL9lVfZzosgHnGTM99sWbZGo4112++AZsnSQUICjviro+gtQXD/tMOJzoItvPWgbFko/FlVqdde9QV+nbcuMfDjxlvo8/jI+gE7z3X+JSVBoRBmSe81ZJqjN4wC8EKWqee99l04dXx7Rxf4SYjvEkpNmlH8Jz5FLZAFGzuW4ajxvr+Z4en2RM5hKaKOV9dkvHDf4+VVIleRMc0PSbLL7nypd8KaYdGORD6BsmhgVgXlPgMC2ClyEB6iCQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F5FTUpjpMSI/6olZMRtQP+S/FJoJlN0+xp6bzMXdtVY=; b=DIUhOHHCZdz95xneCW4ItWMrFJocjVrAtFt04FxhxwEroydYyuOtEjQStBYregXUYBZMJjJqJ4nlXWGafnyqDZ19gLTCT8o6sVfqUqK1aFCOheOeJdhWJkNekaFbYsmhUkHmHz6a+08j3heqd7ZvQH8RuT1QYfWqjCTmxS7tZWLMgUvJkrdvir3O8C5oeAlApdQNRLHivdnphy0ehfEOZ/ONf1TLxhMwFR+5dYWG7eNXyB3EMskVVhw5/ZROI/i19Su5g67StlsiqhaONXZL45qsEoBUz7UOIAPGZt3NiK3yG/lBkUIj1h988x5Mqtov+TubiY36ZEUyAgIDutwsow== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:28 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:27 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 08/14] iommu/arm-smmu-v3: Make SVA allocate a normal arm_smmu_domain Date: Wed, 8 May 2024 15:57:16 -0300 Message-ID: <8-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0074.namprd13.prod.outlook.com (2603:10b6:208:2b8::19) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 634611c7-01d0-4abd-6c1d-08dc6f90b3f5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: Uz5wvK85S+6l9/rmHoTHagH3ps7faIS9Ifsc2HCPJzeXZ7WhCIHX6qkcCyLkWOuGgxSWBUNibQHpzlEVo4OPQGGWAuesXnoPRd355GB6fxPs5dj3gx6qvWLD7dt8DZInFTo9UIX7rCTaiHgPJggwAbGFgC6otoId7M+r8udYLf5mNry3jQLM+XwwvSthAuMhDKFi2Nfl3p6cOLYhoyTcxQDGoumzBuaJsJVhw6r5AjF1O+bc8AiWWRVjFLSim77LfjlGmMI+f7ixRVjigQERAoZ2IgAiKcXFEkCeWJeofytYxPOWGLHpBXPG9J8KZwyxCf5yY1L0+5g1/GhlKhQBOdynRNWTdNz8rJAN0arxl9mr+odP2OXUd6jALP8Ubnt/xanKJl3BTV4fKPPniurXLw6bVolQ42JCdkLrXNjHRp040Ez7XJzHGejW7HnvOAaQD4DTg8BvxdxXQh5ltkGr5n7A3BW4ryZDlW0bAvoRcIyaCOOAUKY1lQIob6ID0cmSByXX5nvW0x/Skn4v4LCCoVZtAEUCFyF1DOdrCTy+lne2nNq05baL0AgTBCsu30Y74ZDxmpDGuD0zGSVPJ0LKOymcjtRpN2mG2VTovp38EtX2ZnI+wa0i3PDJ6P4HLu0MoAamYwzxhw5JiJKNioQ0waE8LRRv6birqsFH6B4MkjgmuzbNUa33Gt/+jmxX5XYRG8C9jHs8+XRmXnzhD9NQFQJzj19eYQAQto/Pgn1UCxLh2HpclyNnguT+EjAw1tc5qV9t9ybMU9BpW1gt7++E67c/6ct7xl+yXKUZS6rvQooaIMx75RLdZlgXb85h5V0MWrp3y1TWiKlvKlRU+2f8i5R62+S/5FCpSUDyTOyxwhR/r58A2czM+XzNtrTMPkUK6sNDUZ1O07n8VkDbXp6YB4riCO453GsrVLpJJPPpPCJ0Q7G7isI+x6r/cEl/WKxFaq5+uhpOdldbsjEBOCABf45E63bMreuDAwVJduZVq27gP9u75gmqUcA7FhoIOnvXhI5MZnYN8isGBdeWhwOXoSh/o5Ufcy6n9ZTyPoiIEG4bEq+4o8nCWop6s0ecM2hpTvaRQqycOecBibsiMdsEK/cjE5nzcIzBLmx+f0GPRMQDZtrPrXWR6z0yYNS9tgrb1P26oojwxCu+m09uhxfGNNE/+0/nLbDrcTBmEGMFp1m6TUXbgupERmsy6H/9D+DCLSrmzKry90vUKSbNx5nJBfooDh4iCFVj0wn9lZIXZBuR84KYsk3BVNQDAQgFXjpPU1yCfiAfiakdEekPDoM63Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CGVhfALsfECYrwLYImK+yrYRJG1zJHbd2zWDuVfjLdckBm7jeDAvXzWZb3AxyIdvUZf3FRxLWWExraf/16pHiAbsKbkgd08N3kod+Gcg6Pw2tta1gbK7cq8UahAItox9logOUab/QZEL2103QIQ6r4w1jTuDnRRa4huSb+Xt2lPZzT67qyw/NLPGaddNIihKAL7pbVpbXYtCDYzP0Q5GdWCqtPFAVWEPffLrfc7VkJzxCf5fUxqMU1jMsbhgsNNeP3ZmyuG70/Yhyru7JYCE7Ag1wZQkrnBDUN4Uf08M4YLram6Y3SpYrOxf4X17v4Z3MtorVeecFO/metyhybdTzqvM2dV3lmLMA4KidXRQHAo5w/ollLBmORTR7FJfF2WJSFHyTk3zHIsI/Rsu3Sx38QLmc14rAOZ9MSkxcX8+MKt/i5Vues8bJmG7EbPkO2wHPtKWKdRBH/e0tCwks6I+UowAYKz9bBe41cjYAWfbSgEAdLrxst0+DLEsESYLI7NFsi67A0B+TpFBZMka58egysK+0innwXQTSlagQYcB8bh3wP6p9AqmPD7Q3Sm4UIRjTGo/kp5sUaC7IEuhTLmd3AD5z8h4K4lapBZsCoC+qwflwMrDcZKQP1Xa7OtGewXDpADydpqeB10D8AXKy7CDE0StpB0bIcyHF2arJPGYwh2tDz/678ht0WAXKJebJXdHa71iYy8dl4SarjJgK45vdOQxaeg5JJgSoeICRSeWyZAhptz9wd/4/gsxXAanTjcN6Ac4K7PcVWYSeu3btle7S167YKb/MDf3kR3pdhwONL2TPlN/snXBL4xcdsbKFRnvX+FkVpqYO4wRcxFW6ApSu7gsZrvPABFWMJ7zGVIV85wSTcg3nIH4oAniJ5Jcth3UIbJTzOMHznXokLo7EIo/KFOH8cpOGtLsOnVpzHavvSV9jDO96hCUlF5iEEQpKIXkiCBFW7qeyauYCgwmeicqzu3qSfRmqu1x1rb6jBICkaHMcUWiVwYJcLKGN9Wg+/ZhM4QwjiqYDCiwLPyLiVRUh9b5CWboF9p+yz1jU8pV8XD2fn1UeiPIedfwRn7vMMOTgADfRDfzwG5a+vKjOo+vEz7oB4mXIcbmgrGxSDiBOXvj8I3K6QnzoUMaasD4Fg+yJ8zKzWgj+nKQ13RJknXwxJP1c/klPdogeQmoVSCmoCAe8Rt6jFavg0MmaRMsRjFUiRUH2Y1uJ2KCB3AF7hA2bdFHntRC2+hsHi5HJFF9GD5zYBxpLbLwi+kxXooRFS7WEiw8fu7Hy1V+JEJpJZIH50qdfFnK3XECrHoen8M75j1k/+j3nY+fMoYbWsswA+yFT+Ao1i2wI9QrUCOpffMyY3X7T85ymhL627nWZd4V06HJ/qWMEOeXYbcvA0f7BSPNKiSapN0ypYFMj3mfVFIJ0YBqYz8u5OE3BDbJGQIq8KfAAKTpaHk72zDQyDxFZAyoLL75Ks3+5Obbqu5PbPOitg7lQQTpLYWhStjzsrsgoUbjOdRwTwhhK+2f4ZCLI/w+TEvt77Vsa0QSzwtkoLfH6qULvZrT1+axoAXAFXsLN5bIgsKoBsKR/aOBMwVgW6bL X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 634611c7-01d0-4abd-6c1d-08dc6f90b3f5 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:26.6120 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jPY7RUBypwjseqjR1u2BfmPCfu7fGcTbW6/UEDHeJyf8lCae9mnrWDv/CIhNiuMf X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115751_482002_7455C62C X-CRM114-Status: GOOD ( 14.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently the SVA domain is a naked struct iommu_domain, allocate a struct arm_smmu_domain instead. This is necessary to be able to use the struct arm_master_domain mechanism. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Reviewed-by: Michael Shavit Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 21 +++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 31 +++++++++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 3 files changed, 35 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 7627cb53da55b9..39ba6766513457 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -639,7 +639,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, } arm_smmu_make_sva_cd(&target, master, mm, bond->smmu_mn->cd->asid); - ret = arm_smmu_set_pasid(master, NULL, id, &target); + ret = arm_smmu_set_pasid(master, to_smmu_domain(domain), id, &target); if (ret) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); @@ -653,7 +653,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, static void arm_smmu_sva_domain_free(struct iommu_domain *domain) { - kfree(domain); + kfree(to_smmu_domain(domain)); } static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { @@ -664,13 +664,16 @@ static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev, struct mm_struct *mm) { - struct iommu_domain *domain; + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_domain *smmu_domain; - domain = kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) - return ERR_PTR(-ENOMEM); - domain->type = IOMMU_DOMAIN_SVA; - domain->ops = &arm_smmu_sva_domain_ops; + smmu_domain = arm_smmu_domain_alloc(); + if (IS_ERR(smmu_domain)) + return ERR_CAST(smmu_domain); + smmu_domain->domain.type = IOMMU_DOMAIN_SVA; + smmu_domain->domain.ops = &arm_smmu_sva_domain_ops; + smmu_domain->smmu = smmu; - return domain; + return &smmu_domain->domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b2d8b7a1df343a..3011eb10763bee 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2272,6 +2272,22 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap) } } +struct arm_smmu_domain *arm_smmu_domain_alloc(void) +{ + struct arm_smmu_domain *smmu_domain; + + smmu_domain = kzalloc(sizeof(*smmu_domain), GFP_KERNEL); + if (!smmu_domain) + return ERR_PTR(-ENOMEM); + + mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->devices); + spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); + + return smmu_domain; +} + static struct iommu_domain *arm_smmu_domain_alloc_paging(struct device *dev) { struct arm_smmu_domain *smmu_domain; @@ -2281,14 +2297,9 @@ static struct iommu_domain *arm_smmu_domain_alloc_paging(struct device *dev) * We can't really do anything meaningful until we've added a * master. */ - smmu_domain = kzalloc(sizeof(*smmu_domain), GFP_KERNEL); - if (!smmu_domain) - return ERR_PTR(-ENOMEM); - - mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); - INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); + smmu_domain = arm_smmu_domain_alloc(); + if (IS_ERR(smmu_domain)) + return ERR_CAST(smmu_domain); if (dev) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); @@ -2303,7 +2314,7 @@ static struct iommu_domain *arm_smmu_domain_alloc_paging(struct device *dev) return &smmu_domain->domain; } -static void arm_smmu_domain_free(struct iommu_domain *domain) +static void arm_smmu_domain_free_paging(struct iommu_domain *domain) { struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; @@ -3285,7 +3296,7 @@ static struct iommu_ops arm_smmu_ops = { .iotlb_sync = arm_smmu_iotlb_sync, .iova_to_phys = arm_smmu_iova_to_phys, .enable_nesting = arm_smmu_enable_nesting, - .free = arm_smmu_domain_free, + .free = arm_smmu_domain_free_paging, } }; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index e850b7913e67a1..a036610e50da14 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -790,6 +790,8 @@ static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; +struct arm_smmu_domain *arm_smmu_domain_alloc(void); + void arm_smmu_clear_cd(struct arm_smmu_master *master, ioasid_t ssid); struct arm_smmu_cd *arm_smmu_get_cd_ptr(struct arm_smmu_master *master, u32 ssid); From patchwork Wed May 8 18:57:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C97E4C04FFE for ; Wed, 8 May 2024 18:57:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aBTL+A06eZRZ4/VOMDwlL7/b3YrlE67KnbCDWiPPRSY=; b=1C0qGrOO8cAXGC tYT9LiWKDA5k1d4pBbV176VHhPxW+GI24E7Y5XJiFy2i4oHb0bIBFgvqVoUTUkSq7/v24I4E3eR5b uZ8YsHSqosrZEEj23o8jLrz2v5OLgt/rgpfUWMSMZQqrUDrcCiFvPxqYAt7GYa4QYPSIv/XkBQVOl 1QcRfLiYvxoy9K00+lPHu2v6VVRH973t/mYQj41HDSUWsvzvua8/MmRnFZw/ZWzhQh3b6P3LgWLSX sShQIuQX+JdH3ZUNS1a2vjd2vWBeBhodOhwaajdZ7br4CoAilpk9s6GB8IAzlQInoKSaOszmNWjX9 u8ti7y0/Jj3UGQyt9DAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU0-0000000Gc8z-1aaf; Wed, 08 May 2024 18:57:40 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mTt-0000000Gc5t-2T05 for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bUgQDJAWDCsj1o9lBUUADb1Fwglf2kBZ5qBb4GLJHpWD9OsVbkxmPJE/bBVAIXC+jSfWUCQYnVUW3/mG1xvjdvJkqte9wc+jNVQ4X4h8zh02+ni2JrKGh2myG99AMUzGNo2Rm5zNhnOlmjF+GAl/7v6HW3jJEJNC7+GPFR2/4uOFC170MUApcqlQSEaZQ3F1cHLqklniEEb0Nowgf7Ruabe/1gG8qrqrc2knctgNrclpoDX4BTfVNCiNA0eJVAD4WYwJqiDLYW2fIzk/AW3X68cdRngWaAmq5OiuET+dEMBsZax3IFqV2Ccz655sxSDcPPxV2VyGRwjDx8xPoBKQHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2eFo+6WoFpIg+epk1sHwLf15bM46gwJfBKn6cttP35g=; b=kNDkSeW0r4ilPYJtWvpYcdRreRNINYh1MzybneEN1ZHcaBdYXI53Uf2MyDefmguiF3zuSFLR3Lm4dR8sXv8Iy9g9nR8m4yYLOjnZ9FxuKLdrqW59/MIh9Z2ToVr15B46DuEB3xvaEc3VsSAYOFVnmk0kvs/c9Qn9N2ynJepdW2OgYLpjoOa4umwDPSbFs7BBpHOkGYQM3a8CE8DsaW07aby4ZKwl++tOS8oLBRes8j2A6uSwZR8QdMBT6hh4Yi+38rO9bZPdtpc5w8S0DF594lz03TaXJwuYn622yNUQ684AIiWUNyPMXhSGQxYqrRb7Z+XtQ0f27BoF+UwNnR4r6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2eFo+6WoFpIg+epk1sHwLf15bM46gwJfBKn6cttP35g=; b=WBTKpaDTSPpj5/YA3xWmiGnB1tVeIJn1rt1RLEyJVNneSDnEds4+S9Qf6J3/Lne96nJowLUE7LazwWmYB8s/vDMyq8wuaIOhPkA/3ADMOK9drl3T3st2YPlwgNoXEn+O953e4VTOHI89frATUugRs0xPeeM4aTF+hphJLaYhU8ETlERS4ujNsJFbp5oiyQTEh07vmb7q8AxUUjwYwe3tPBKKoDamx3KefovoqhJr7awMD8KYFKdkpha7k1Een5NRfDIK67m5g1kufvgaRG6V4eUXPgyT9dvDL2xBVwwrnpCwbp+sDWkjMtRA3sthG3HeoFK7KI/VKHCWwfzzVyLnuQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:24 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:24 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 09/14] iommu/arm-smmu-v3: Keep track of arm_smmu_master_domain for SVA Date: Wed, 8 May 2024 15:57:17 -0300 Message-ID: <9-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P222CA0026.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:2c7::31) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 4bde1df9-a641-4c1b-2bdf-08dc6f90b28f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: 0bj2ja3dtW7ljf9I9qpeRbQ4uFyTdsNCjL+OunrtmnLkRwJURR0LxH6RToS/M3wKtrtVX5fQzYZI4gmFdwwBOKK7Te6Lf8b+nTF6Sz2vPCSsiu/yOzpLq2c2DNxsfye8ZSdBqEQNfZRdZU3RQNE5XiMk4CJ80UzGOVAsKpfjkXYaRFoy7tyN3g83OBpPXJpBh/4IUx6mcjrJ+7CJ6EEki1MUlG4t6FhZl1dSk8qsQv14eUfgoo/TTBA97oe/h5YhHWzHBuHm2TpK6EHb7j2xxsFxCG5Eo8WhDAkU1v9+1x2MaTkjLwlBHr4B00Z6UghRmIKJ4h9dlHzNJD7torwXttBQwffwNq1A/g3i+FV0ufdO73FHkg5nvMWp5gAv3FAd9ZMnEpBfXCLBnxx3nlet0xTnyjMmIs71A5PqZ0LByr4p9cF8J9eDF33gV5HmAfiUtB3CB53e4Gs2d59fcp6Xl8c/sCVaLO0FCQGf9LH8aONQ0cLGnvSrNyzTikAVXZn1gylvqhsheYo/CaLzXTgIhL1VobE6KLmuf/z3JlLm9EIDaHM4Py0WcFWJ73O1RZYo4uw8DC130EvzNFTinWWbTQW0c85UcdRgOT0isDTzmc/6RfaOsPZBy2wrNY84tn/fczfae7aWVCHj/jz6WiC/2GptTvwHriCTEmbX7PLL+4z2aLyBMOaZ2/NDx+ogfWN9/jDZi2FfEWSSJj+HugHRAWRzIPrkGSTihtr6FC6pH/TPzReZpy5vierx6IaYijHWFLkjgLgNNE74xh7iqWCwe9pc+U0N5l8fyBXWK9u1CIQcYklTGHEhFHSWJltRDbiRRhghhfiH4VruF6ot+zNgP90rvvGSqWtIikEGwQgsG81uRw0Tvz8RDIDd8C4EOaKvJ1E6/cBkWVdz1dLJ09PllXTTm2B0ZwWl4/9rp1BlvAoCrQG57WOwR1CncJzGsv8Y/ph4T47V3S6mi3IoTBEBc20rG8G+Jnqj7WBZ1AfIlhPGatnm2F0ZF4vLHx8jTXiwik97gO7VFbo55gAqYq3m9H/FMWznFjxtmbKYOFpdrBi9e5ISgAlO4UqlWuwULWhkfPegNlVmxuGyhdS/l9z+PluSgGAjn9ledb7UuhMBmmx19a1S7yf1vth9pE7RE6ZTn2mKn1vMbDojoifmrvAJK1F30f3Mz7rSj7Xl67PusYsk80BhYDFhQ2qwd7clHFPP89W1jJevVIBChvJjDR2aoirSkFSv8T39bNthzY2ktpuwJoae2Ny/bAaHp3aLugUCvDrpBhiSHHbRetdcGPK3Og== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: jxBhpp9qT3oIEnJrVcq3sq3j1AxCvf0DZ02Rdf+vZAWjMIBUFQokxE+P2hTceuB0261jbXS6UJ3EemOUK9Xxmir4wdViJoWaC839KlnUcqnBKAbAnLrmUhwXPKjsqBGaoQhZ8g2eHohKJRMmACrc73MRpPVJ24N6N1kKNlnjqDq3do8WBDTINoyG0zhqNkHXaTQ50ll3sJtSDeG/ewMIIFU/CfDmOUbdcfbA2zg51kjDji3PD8Mk0OGUbW7+gZ+fNXSNjY8RUA8yX7LRHvutEmpzGXTbpnRaVjOZybA6/dRyslqljyQCMa7ySttTyOX/g/rpbvPM2Rv/ErwEKqm/pXhU9kyfruCtjt2ukj8wB6I6PybX8fa1YQJ52OFP9aQ436M+Dgs02i3YXUeqYdZ94pmbZo9EC09BwUdl7N+rQJL82Q0E/RE58of6vpgT8V3gDUONZpi2hgXo/uiYxhRkzH6evtCX1kPuEAUe3TeCnlzn9iSgHvD1+LXSO0KigFqT3jgD2mCDSKb53SctFDwlNBNnjza6cdJm/kXmRaFtHMlR8th/cnp6gUaUkTAFlb7hFGehSc7EEMNguEkKtz1NGcFD1MBB1H1lJcsQbp+GF9byvdiGrXpEpDNefP3PJt8a7NXatWoJtMiS0lt2eimHbj4HjKvNt3K+UN4vkHA5QHBVJCdb2rV/2pDfJOqWQ2laJocycRv15fUAbEDaSTMwY70xC2N7xeMA0wib/oOKLIse0+epysWguj18wmLhSgxM28kbHxuY4cAR1tEzZriM39DrYmM6RjwtpI8Ug5IFzHKYrCd0Vh3ef5fjgQzFtF63DhwDiNjQQScWfRIQP+xAAl8aS5VXQug/uW5krr+ydKOamabkC+te5c8yoJaaw7CxK+684XIp5vdvNnzMZ2sVyTFdLdo/HEOohIuj6xetSDrbahMOV6KlE5clah3YyuN2TxqFGxxiiYDk2sGtYoOcznLEyHI31mFBCGsWOFHIaJL3vE7WbftP2oH2PZo6rz8wCKYk2O2eFfHsDLE7M76Gxjtt5C9RJpWWkhq1tYJddInzEVy10KAYmXRFvFXp6SbPxLn8MxM9ISkh5+vj529zzg38R878nEJUSoFbRfGZON7ubVqQ74LiNXz/UqBpnMPLS8r6dSH93uD2YXGYYIszwW5iQcZuDdo5jVLMD2pIGkmzGG1wjDmlmlNWyrqpcZANTGorMYUEVE6rw0w1Dsngtd3Int2J+ym8YoPnc0GGRjGtGsWSuZ2+opgp1lNUPziXdFpjwtF/uXJ9Z25U1oleGlakbgfd2NsW8tOFAroYT6FRu7fO+cuJQpyKezex0DdQMRt4euYA10Fu6USouqn0sSOkp40ERH18pnNU9Daw9aXlN157NAI5RYqUNFzqX0EaB2ip5tnS/E/6jjmB+uTK4P1Z/khYTQ2ZaT+Dg5Ko1GkoFUFb45UWcSDvJG5BCbmOQ7aKFPpHvkFaR5pjFxwNlw7Dj2T8QfkZhKnLCttocOVJdcNS3iuv5sSSfrVpTFj+8eSUP0MpmMNBX/z5YGVKTFpW94V7WVljysJTpcy++J1W3uyhkYt8XkrBPHu23550 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4bde1df9-a641-4c1b-2bdf-08dc6f90b28f X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:24.2940 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5mwK+hUp8izQpSiY0FuiOJPSWJzQWi0jj+sR7W3/MZRj3VV4cknbS+DQrqsLIEuS X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115734_107126_C72C67AE X-CRM114-Status: GOOD ( 14.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Fill in the smmu_domain->devices list in the new struct arm_smmu_domain that SVA allocates. Keep track of every SSID and master that is using the domain reusing the logic for the RID attach. This is the first step to making the SVA invalidation follow the same design as S1/S2 invalidation. At present nothing will read this list. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 29 +++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3011eb10763bee..020e4efdf8fc06 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2587,7 +2587,8 @@ to_smmu_domain_devices(struct iommu_domain *domain) /* The domain can be NULL only when processing the first attach */ if (!domain) return NULL; - if (domain->type & __IOMMU_DOMAIN_PAGING) + if ((domain->type & __IOMMU_DOMAIN_PAGING) || + domain->type == IOMMU_DOMAIN_SVA) return to_smmu_domain(domain); return NULL; } @@ -2793,7 +2794,15 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid, const struct arm_smmu_cd *cd) { + struct attach_state state = { + /* + * For now the core code prevents calling this when a domain is + * already attached, no need to set old_domain. + */ + .ssid = pasid, + }; struct arm_smmu_cd *cdptr; + int ret; /* The core code validates pasid */ @@ -2803,14 +2812,30 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, cdptr = arm_smmu_alloc_cd_ptr(master, pasid); if (!cdptr) return -ENOMEM; + + mutex_lock(&arm_smmu_asid_lock); + ret = arm_smmu_attach_prepare(master, &smmu_domain->domain, &state); + if (ret) + goto out_unlock; + arm_smmu_write_cd_entry(master, pasid, cdptr, cd); - return 0; + + arm_smmu_attach_commit(master, &state); + +out_unlock: + mutex_unlock(&arm_smmu_asid_lock); + return ret; } void arm_smmu_remove_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid) { + mutex_lock(&arm_smmu_asid_lock); arm_smmu_clear_cd(master, pasid); + if (master->ats_enabled) + arm_smmu_atc_inv_master(master, pasid); + arm_smmu_remove_master_domain(master, &smmu_domain->domain, pasid); + mutex_unlock(&arm_smmu_asid_lock); } static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, From patchwork Wed May 8 18:57:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C8791C25B5F for ; Wed, 8 May 2024 18:58:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9GdYM8zRGxpYKAEpAGT4IU1+Tj9GYp2x1sQBSEbtU50=; b=LeEOnYw91hHx58 3Uw6PslNskECUdTKTE5mcvtrCMfrzaawTRH2WkEVpUcYRc/9RHGnziZRYMH9ANCsSSQqaOgO/9au9 k41H98+rh5AyCdEE8EJFAMQNNi11Hp2Gk7K3F5es/2hO3qe+IJTNuAnHwo+OOuzp/fmVP93e7eqzm h8rdoHzI5YnW8XMlnWdFRpFFhYSNtnerIvqN5nMgipKCaZZP02rrOTYbIkrGgWjz0RQVNarOvWpGk Jo760SzCgkcPv/aD4g4knJgxYfSM6otIiD3HG6/ZQa8THb4Sta1p46eRqa7xDiswvz5/fJIee8B/a OrBwkBUNUidxWvvZoo8Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUc-0000000Gca2-01f4; Wed, 08 May 2024 18:58:18 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU6-0000000Gc7U-0Txa for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IrVhWNuAZ0MI0XoCvqCqAYXqcbkMnswbD5WT8BAxLiJZcnPQE0q0qkDXlhJTgzgEFk2yJV79nAKHXMqBlDZqIHZtGTwsvXyEC+r49vwzQZ8EzVca117DwI5htIjroi8XwUWYygx0JsjS0SMxXt4/HLDpdOhYI6x8YiV5Nem3JsVbBn2Ai/ixELg1CjtCKMYJbXF+Xm/VlYSa7jegLh9ZMJ9wY+8Qf+NPWdqf5TjkhmP8/ol2VbdCyfTDfs77AZlt244O6WRhRBaypLbY17E5uBHSmTe3B4nz0JM1MtR5yys50fzFZfJejom+uBlEJEmE0f/OcEu1tX5HaNcA9awlFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+ACxAnTnuYV0Dl/daixsQJNVd0lWjEHqwttskA5sfkE=; b=UakHfmv+PEHtsgMFRxrlE04qUPldWXmEh92Iv3u0KxiOn80MdvL5tuRMUgmB4RXKyFkbTazX+AkPZDYGZPDpib1oTU5Zx9p4QHnoDxdm06fadMgLktQp+4szInq8VkVFyJRrdB9cl9sdoqkOR8FUkWd3u7P+3nW/hODymitlbLvrmjQI7rtCnbBlFTY4T3KwLh+go50cI3BVjJjCDonIT9yW+B04c9Xsqcqqy6ex3Kl3F7ORIcn5vLf6KP3bkaZLuXviVIF/AH34HhmJLdhyF2iPpyDwplSzCXH+BudNfvopVcZZqsst5nNLJLSxJfEYS6tgo+z25phJ+z3/x+nhNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+ACxAnTnuYV0Dl/daixsQJNVd0lWjEHqwttskA5sfkE=; b=EdguU+sRZ+cPT4/W3KJ5nEAi7C1sgIzhICmMfBRVUbpRDEt0wJHC6tTpdy1NWmLV++0JdJN6Nm61mOz3OhN4CctiDoi/642YtzlkFjbpQOEGrJXGHrF1M4uGrAQnBKDASr6eiAqW81BsMn6MCWs+3pztUsR87GMRPejj/MkzFZx2cKKxJ1fPJypNp+P//XD4ttUehjgGOAw7QZg54BEM2UgKApeOdDBQk/DtKhacVcx2eaghSgTWJutkN1KHrMyCS2MTZF8w+CnQsyL/+iEt29Ox5JCZK7t4u3TqxxKnfhmE4CuCZuOgGslmL6UlXlvXHBZMB8rF8GHjoNx5OiR7EQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:25 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:25 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 10/14] iommu/arm-smmu-v3: Put the SVA mmu notifier in the smmu_domain Date: Wed, 8 May 2024 15:57:18 -0300 Message-ID: <10-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR03CA0025.namprd03.prod.outlook.com (2603:10b6:208:2d::38) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: fee9ea69-3209-45f3-c58b-08dc6f90b29d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: SFNUhcurUMcSTaMI3tJGiO4cjT+aUldiamWQUN+ePOpjjk6YjvIgyRH/3S4CgGR5sNVxsfDdfWf2q80FE324AFd7GcT3GCoF6nbLO3z3Vpf/t3qzR6I80CLRWC8lJixgF5DB+enxmBPLaGOW8LBpM2DGvetfeZl2oTzM1GO1ymYU5GzoSst8zD+pJJJaEZwakQL9D4dLV9g6lbE0/ai5+VG/Q+tWtLGEekp0tqnE0LRZ1bkz+4BQ3We0dAyvNyhUhfcfpuaTOQ/KSueB+KooALAAclBFfwic732cFf3NIsFyGhNE/nBm4zYE+B0z3kNMcdYotmGLlnZUOop+lL4NrX68vgER1PFeMiO7PUrrn8VYLsbsOu5UXdd6VdPflTX85msJd3OGzJDPDzGK9TcPysJ+ovfGVNHiaNEGs3wppoQZTLeKpEyXWEB/VA67byFBgPth6UNCoJZT2VxcMl1yUSHW0QFb6iyIKAh2miZp0tWGeDxtI6VE5CVZtxJ6jcWmfiaZsu4R9CQC5uy7I3UWb+laDNUeGaccnTGoguCOQe0YYWRdj7y24WcIZa8wrC+b/UGMSlV78cgV2nN/0IG1Shjlr8uq3jgAwRHAyH7OveSiGsT91FPIIwKyj4JX1pmbSXal7JG+5dUEyjKAUGf0iFyoNotrWdkngQfQLD+VuGbXUuRwR3dP10y5KmWm2cwPf/bpgeHEMHmww/76wune6EwoduxP7T2FLxyQk1kNhz1e1TNyz3YrYPcjw//qddxOJpzONOcA0pNHdXghZWVYsCmBu7pV0l4TgKBQZwi3cL6LlykG4e2JHyZIAwl4dzTIZX+SsYlxwbLJXaMI7Ed2nx56OfMDd6GEng5A5/7/0oyV4E24N7AbUifxTz4TExT8UNKhwgZAWH9BtfD2ifP7GLNYaFymj0N2CDL4uxaYe3lmrY94FR5/VycdREhX6n+emHDtVgg4duqZfMbLR2hV8wq1FDFsNF3+2fMlBVhy2svemd9odo5Cgg2LT8LmGwvmGNx+5yW7TjQV8yDsR02Dt5/AuQzSk9FlFONZQx7PNhP+X8gLGjzvYv+jqSaOFXb01thUJdX3PCAJOAhzR7E/YA9cMkmCGHBHxJa83BI4mGEe9GNXJIomPN6qTo5qTNBUBJH6q1FqHI/wml2P3SYggGWVjbP7VFVF2XFQ9jMa5XfGn7gZctHREB8AgxtL7m9001jVwIi8sOnA7zn5vv0aCdCmKbR6VOjX/UkenFMV/VOVHIbX6RagV+b7A1/793q/7z0vcnPNvq3e6Onk04yh52Utr64AOQnQyKLHRrpq4Mc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: q5ycyDKC0a000ZV66vy13T7JNpsb3g1RVmtpfBaZQUu9hFkl9U5fRMNVXGVzcDF4aBcCYeCd00lIjwFeGDIl1TRiftkzsZxT7HRjIMJa1l+0+jLdcof5KpNaxW5ekm9otmJOWvfm0f9opRpMnLeEurdyEC0w3fYofRLf9Nd2/TBO5WVz5YU7lO0k61RAYglYjNdCyorLgTlgyg80GQvkgCQrrdzD9V0+qbbRM/0RauSXJwcQjAQJxiCU+FDQInXLyxch9IWiswOYdfY4Fsdp/WC2xrY9aEgWCMuCUgkLPIF09cMgy8lUlwtjjZmATZ44Ig+frgZDw+jwWdE/CEOuB8DVQedElCyfoqmdKK2H2Hz6/RyUVrbMhnK7GoZiH9kqApQrqB5Mlb2FWUwMr1l+3oeVvxri+qmUL8P6cmE3l/hFYFI+oybi9pPCknh85a1K9RjOYdkp2LGNvzfFaDZn2FUM0+6ud0sEY9TviVwOawrXqtjTlXpl0GBDMxnJQWSut8cXtfzX/MvOZUxCKKhB318YAb4wirI+lJxMuq5ZTLciZ6NVJghCrbP4a9bcd4uAat7ZU18mEMrGKbKckeVw4+1G+/WmbpXoKAOhWwIrgYg5Mo7tqE94/cZDm3uYHAU0EmszOKV8UgJ24Foq6yAe07gKHMDbEAtmE/byHY06erlp+CqkxWFjZPeZUjPx1dWD2duz4RtgiISHD/lZKtwstjNGsUfz6OkSgCuU5P5/XpRbNjoaVK5tKDav38HXMqeC+G4INRQQHxBQOmiGgqVHohR8mo/+3bFHABisbEgiZY0bS+LwVUMq+Q+LlvxuC2fzT6xK9IKR6NvK+uQTiK/2hJKs0AHR/8VOFRHAA6zb4Tj7WxxT+47UaOR3+Ff8DpP9mb+20ken19OSWvkNW8D8NG6qIWJVaNnSvbt5VQw+dEquL9YNtt1YLxK6eybhSVIWKxuAIcmzpU9rOXOPSq2ULMfvxHbUY1thazgHGC4icKqbaifr4ejc5emRZWJAxZsy9apIqwB0XOGSFk+RDhyoPmUZvmhC5qwRoyv4m3sFj6Aup01YR20J2SUj936gjx8geSn3dlFUAWQUFA2MmUhIL8tQF8SmVvxsJ8t8WVmk/hUVy9FCSUt8A528FOSPPmtT48ynflY8wMaH7fBJsiingCr2wpiMHUOTqwwd3QFIaNjzHtQPvc/IRjISvTQ5oL5WHRC7KJNksbD56dBr4vkWG1RMYhqG+Y1k2MPmY5KsOdpewxpA1Pn/r+iLS1krAIcdOyo0Azeg+JXw9wwXhtDtI8ZTx9F70iot5kUXJXbEMZcX+lNE1OVUAmWXu1fsPp6n8BFzSzzZNLePJ2bdsm5FoT46CAROrC5Ej41nHYER7QhSTTB9ztnKj/BTwALnbIXQrtqLtbwHmSUp3mkrWDT41IV3I9kGZhKTPNoQUwDGTbAZNnZWzshd+Zg+pm/pKuc+qa0nl8iLdJ3+xws6x+4G6RyN+34j//fwUJtLLQA18q1Z8pkqW9GZY4g0gYKSk2i17YTM1UTm9Uz1xIMF8BetgVw0AR6bTOar0D9fsY2P0zUNc7TEWmoSbo/VcGO/HRxo X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fee9ea69-3209-45f3-c58b-08dc6f90b29d X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:24.5328 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2J4lWlhyGUBS0vkcg0js05Be7oTss/JZJ3E36gIpBvSmn0INX7+KeSHz9AhGl3Vj X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115746_369318_4216E3FB X-CRM114-Status: GOOD ( 22.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This removes all the notifier de-duplication logic in the driver and relies on the core code to de-duplicate and allocate only one SVA domain per mm per smmu instance. This naturally gives a 1:1 relationship between SVA domain and mmu notifier. It is a significant simplication of the flow, as we end up with a single struct arm_smmu_domain for each MM and the invalidation can then be shifted to properly use the masters list like S1/S2 do. Remove all of the previous mmu_notifier, bond, shared cd, and cd refcount logic entirely. The logic here is tightly wound together with the unusued BTM support. Since the BTM logic requires holding all the iommu_domains in a global ASID xarray it conflicts with the design to have a single SVA domain per PASID, as multiple SMMU instances will need to have different domains. Following patches resolve this by making the ASID xarray per-instance instead of global. However, converting the BTM code over to this methodology requires many changes. Thus, since ARM_SMMU_FEAT_BTM is never enabled, remove the parts of the BTM support for ASID sharing that interact with SVA as well. A followup series is already working on fully enabling the BTM support, that requires iommufd's VIOMMU feature to bring in the KVM's VMID as well. It will come with an already written patch to bring back the ASID sharing using a per-instance ASID xarray. https://lore.kernel.org/linux-iommu/20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com/ https://lore.kernel.org/linux-iommu/26-v6-228e7adf25eb+4155-smmuv3_newapi_p2_jgg@nvidia.com/ Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 398 ++++-------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 69 +-- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 13 +- 3 files changed, 89 insertions(+), 391 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 39ba6766513457..56871c67ba369a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -13,29 +13,9 @@ #include "arm-smmu-v3.h" #include "../../io-pgtable-arm.h" -struct arm_smmu_mmu_notifier { - struct mmu_notifier mn; - struct arm_smmu_ctx_desc *cd; - bool cleared; - refcount_t refs; - struct list_head list; - struct arm_smmu_domain *domain; -}; - -#define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) - -struct arm_smmu_bond { - struct mm_struct *mm; - struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; -}; - -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) - static DEFINE_MUTEX(sva_lock); -static void +static void __maybe_unused arm_smmu_update_s1_domain_cd_entry(struct arm_smmu_domain *smmu_domain) { struct arm_smmu_master_domain *master_domain; @@ -58,58 +38,6 @@ arm_smmu_update_s1_domain_cd_entry(struct arm_smmu_domain *smmu_domain) spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); } -/* - * Check if the CPU ASID is available on the SMMU side. If a private context - * descriptor is using it, try to replace it. - */ -static struct arm_smmu_ctx_desc * -arm_smmu_share_asid(struct mm_struct *mm, u16 asid) -{ - int ret; - u32 new_asid; - struct arm_smmu_ctx_desc *cd; - struct arm_smmu_device *smmu; - struct arm_smmu_domain *smmu_domain; - - cd = xa_load(&arm_smmu_asid_xa, asid); - if (!cd) - return NULL; - - if (cd->mm) { - if (WARN_ON(cd->mm != mm)) - return ERR_PTR(-EINVAL); - /* All devices bound to this mm use the same cd struct. */ - refcount_inc(&cd->refs); - return cd; - } - - smmu_domain = container_of(cd, struct arm_smmu_domain, cd); - smmu = smmu_domain->smmu; - - ret = xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, - XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); - if (ret) - return ERR_PTR(-ENOSPC); - /* - * Race with unmap: TLB invalidations will start targeting the new ASID, - * which isn't assigned yet. We'll do an invalidate-all on the old ASID - * later, so it doesn't matter. - */ - cd->asid = new_asid; - /* - * Update ASID and invalidate CD in all associated masters. There will - * be some overlap between use of both ASIDs, until we invalidate the - * TLB. - */ - arm_smmu_update_s1_domain_cd_entry(smmu_domain); - - /* Invalidate TLB entries previously associated with that context */ - arm_smmu_tlb_inv_asid(smmu, asid); - - xa_erase(&arm_smmu_asid_xa, asid); - return NULL; -} - static u64 page_size_to_cd(void) { static_assert(PAGE_SIZE == SZ_4K || PAGE_SIZE == SZ_16K || @@ -187,69 +115,6 @@ void arm_smmu_make_sva_cd(struct arm_smmu_cd *target, } EXPORT_SYMBOL_IF_KUNIT(arm_smmu_make_sva_cd); -static struct arm_smmu_ctx_desc *arm_smmu_alloc_shared_cd(struct mm_struct *mm) -{ - u16 asid; - int err = 0; - struct arm_smmu_ctx_desc *cd; - struct arm_smmu_ctx_desc *ret = NULL; - - /* Don't free the mm until we release the ASID */ - mmgrab(mm); - - asid = arm64_mm_context_get(mm); - if (!asid) { - err = -ESRCH; - goto out_drop_mm; - } - - cd = kzalloc(sizeof(*cd), GFP_KERNEL); - if (!cd) { - err = -ENOMEM; - goto out_put_context; - } - - refcount_set(&cd->refs, 1); - - mutex_lock(&arm_smmu_asid_lock); - ret = arm_smmu_share_asid(mm, asid); - if (ret) { - mutex_unlock(&arm_smmu_asid_lock); - goto out_free_cd; - } - - err = xa_insert(&arm_smmu_asid_xa, asid, cd, GFP_KERNEL); - mutex_unlock(&arm_smmu_asid_lock); - - if (err) - goto out_free_asid; - - cd->asid = asid; - cd->mm = mm; - - return cd; - -out_free_asid: - arm_smmu_free_asid(cd); -out_free_cd: - kfree(cd); -out_put_context: - arm64_mm_context_put(mm); -out_drop_mm: - mmdrop(mm); - return err < 0 ? ERR_PTR(err) : ret; -} - -static void arm_smmu_free_shared_cd(struct arm_smmu_ctx_desc *cd) -{ - if (arm_smmu_free_asid(cd)) { - /* Unpin ASID */ - arm64_mm_context_put(cd->mm); - mmdrop(cd->mm); - kfree(cd); - } -} - /* * Cloned from the MAX_TLBI_OPS in arch/arm64/include/asm/tlbflush.h, this * is used as a threshold to replace per-page TLBI commands to issue in the @@ -264,8 +129,8 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn, unsigned long start, unsigned long end) { - struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = + container_of(mn, struct arm_smmu_domain, mmu_notifier); size_t size; /* @@ -282,34 +147,22 @@ static void arm_smmu_mm_arch_invalidate_secondary_tlbs(struct mmu_notifier *mn, size = 0; } - if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) { - if (!size) - arm_smmu_tlb_inv_asid(smmu_domain->smmu, - smmu_mn->cd->asid); - else - arm_smmu_tlb_inv_range_asid(start, size, - smmu_mn->cd->asid, - PAGE_SIZE, false, - smmu_domain); - } + if (!size) + arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_domain->cd.asid); + else + arm_smmu_tlb_inv_range_asid(start, size, smmu_domain->cd.asid, + PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain_sva(smmu_domain, mm_get_enqcmd_pasid(mm), start, - size); + arm_smmu_atc_inv_domain(smmu_domain, start, size); } static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) { - struct arm_smmu_mmu_notifier *smmu_mn = mn_to_smmu(mn); - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; + struct arm_smmu_domain *smmu_domain = + container_of(mn, struct arm_smmu_domain, mmu_notifier); struct arm_smmu_master_domain *master_domain; unsigned long flags; - mutex_lock(&sva_lock); - if (smmu_mn->cleared) { - mutex_unlock(&sva_lock); - return; - } - /* * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. @@ -321,25 +174,26 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct *mm) struct arm_smmu_cd target; struct arm_smmu_cd *cdptr; - cdptr = arm_smmu_get_cd_ptr(master, mm_get_enqcmd_pasid(mm)); + cdptr = arm_smmu_get_cd_ptr(master, master_domain->ssid); if (WARN_ON(!cdptr)) continue; - arm_smmu_make_sva_cd(&target, master, NULL, smmu_mn->cd->asid); - arm_smmu_write_cd_entry(master, mm_get_enqcmd_pasid(mm), cdptr, + arm_smmu_make_sva_cd(&target, master, NULL, + smmu_domain->cd.asid); + arm_smmu_write_cd_entry(master, master_domain->ssid, cdptr, &target); } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain_sva(smmu_domain, mm_get_enqcmd_pasid(mm), 0, 0); - - smmu_mn->cleared = true; - mutex_unlock(&sva_lock); + arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_domain->cd.asid); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } static void arm_smmu_mmu_notifier_free(struct mmu_notifier *mn) { - kfree(mn_to_smmu(mn)); + struct arm_smmu_domain *smmu_domain = + container_of(mn, struct arm_smmu_domain, mmu_notifier); + + kfree(smmu_domain); } static const struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = { @@ -348,115 +202,6 @@ static const struct mmu_notifier_ops arm_smmu_mmu_notifier_ops = { .free_notifier = arm_smmu_mmu_notifier_free, }; -/* Allocate or get existing MMU notifier for this {domain, mm} pair */ -static struct arm_smmu_mmu_notifier * -arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_domain, - struct mm_struct *mm) -{ - int ret; - struct arm_smmu_ctx_desc *cd; - struct arm_smmu_mmu_notifier *smmu_mn; - - list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { - if (smmu_mn->mn.mm == mm) { - refcount_inc(&smmu_mn->refs); - return smmu_mn; - } - } - - cd = arm_smmu_alloc_shared_cd(mm); - if (IS_ERR(cd)) - return ERR_CAST(cd); - - smmu_mn = kzalloc(sizeof(*smmu_mn), GFP_KERNEL); - if (!smmu_mn) { - ret = -ENOMEM; - goto err_free_cd; - } - - refcount_set(&smmu_mn->refs, 1); - smmu_mn->cd = cd; - smmu_mn->domain = smmu_domain; - smmu_mn->mn.ops = &arm_smmu_mmu_notifier_ops; - - ret = mmu_notifier_register(&smmu_mn->mn, mm); - if (ret) { - kfree(smmu_mn); - goto err_free_cd; - } - - list_add(&smmu_mn->list, &smmu_domain->mmu_notifiers); - return smmu_mn; - -err_free_cd: - arm_smmu_free_shared_cd(cd); - return ERR_PTR(ret); -} - -static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_mn) -{ - struct mm_struct *mm = smmu_mn->mn.mm; - struct arm_smmu_ctx_desc *cd = smmu_mn->cd; - struct arm_smmu_domain *smmu_domain = smmu_mn->domain; - - if (!refcount_dec_and_test(&smmu_mn->refs)) - return; - - list_del(&smmu_mn->list); - - /* - * If we went through clear(), we've already invalidated, and no - * new TLB entry can have been formed. - */ - if (!smmu_mn->cleared) { - arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain_sva(smmu_domain, - mm_get_enqcmd_pasid(mm), 0, 0); - } - - /* Frees smmu_mn */ - mmu_notifier_put(&smmu_mn->mn); - arm_smmu_free_shared_cd(cd); -} - -static struct arm_smmu_bond *__arm_smmu_sva_bind(struct device *dev, - struct mm_struct *mm) -{ - int ret; - struct arm_smmu_bond *bond; - struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct iommu_domain *domain = iommu_get_domain_for_dev(dev); - struct arm_smmu_domain *smmu_domain; - - if (!(domain->type & __IOMMU_DOMAIN_PAGING)) - return ERR_PTR(-ENODEV); - smmu_domain = to_smmu_domain(domain); - if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) - return ERR_PTR(-ENODEV); - - if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); - - bond = kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return ERR_PTR(-ENOMEM); - - bond->mm = mm; - - bond->smmu_mn = arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret = PTR_ERR(bond->smmu_mn); - goto err_free_bond; - } - - list_add(&bond->list, &master->bonds); - return bond; - -err_free_bond: - kfree(bond); - return ERR_PTR(ret); -} - bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { unsigned long reg, fld; @@ -573,11 +318,6 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { - dev_err(master->dev, "cannot disable SVA, device is bound\n"); - mutex_unlock(&sva_lock); - return -EBUSY; - } arm_smmu_master_sva_disable_iopf(master); master->sva_enabled = false; mutex_unlock(&sva_lock); @@ -594,66 +334,51 @@ void arm_smmu_sva_notifier_synchronize(void) mmu_notifier_synchronize(); } -void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, - struct device *dev, ioasid_t id) -{ - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond = NULL, *t; - struct arm_smmu_master *master = dev_iommu_priv_get(dev); - - arm_smmu_remove_pasid(master, to_smmu_domain(domain), id); - - mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm == mm) { - bond = t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } - mutex_unlock(&sva_lock); -} - static int arm_smmu_sva_set_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct mm_struct *mm = domain->mm; - struct arm_smmu_bond *bond; struct arm_smmu_cd target; int ret; - if (mm_get_enqcmd_pasid(mm) != id || !master->cd_table.in_ste) + /* Prevent arm_smmu_mm_release from being called while we are attaching */ + if (!mmget_not_zero(domain->mm)) return -EINVAL; - mutex_lock(&sva_lock); - bond = __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(bond)) { - mutex_unlock(&sva_lock); - return PTR_ERR(bond); - } + /* + * This does not need the arm_smmu_asid_lock because SVA domains never + * get reassigned + */ + arm_smmu_make_sva_cd(&target, master, domain->mm, smmu_domain->cd.asid); + ret = arm_smmu_set_pasid(master, smmu_domain, id, &target); - arm_smmu_make_sva_cd(&target, master, mm, bond->smmu_mn->cd->asid); - ret = arm_smmu_set_pasid(master, to_smmu_domain(domain), id, &target); - if (ret) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - mutex_unlock(&sva_lock); - return ret; - } - mutex_unlock(&sva_lock); - return 0; + mmput(domain->mm); + return ret; } static void arm_smmu_sva_domain_free(struct iommu_domain *domain) { - kfree(to_smmu_domain(domain)); + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + /* + * Ensure the ASID is empty in the iommu cache before allowing reuse. + */ + arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_domain->cd.asid); + + /* + * Notice that the arm_smmu_mm_arch_invalidate_secondary_tlbs op can + * still be called/running at this point. We allow the ASID to be + * reused, and if there is a race then it just suffers harmless + * unnecessary invalidation. + */ + xa_erase(&arm_smmu_asid_xa, smmu_domain->cd.asid); + + /* + * Actual free is defered to the SRCU callback + * arm_smmu_mmu_notifier_free() + */ + mmu_notifier_put(&smmu_domain->mmu_notifier); } static const struct iommu_domain_ops arm_smmu_sva_domain_ops = { @@ -667,6 +392,8 @@ struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev, struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct arm_smmu_device *smmu = master->smmu; struct arm_smmu_domain *smmu_domain; + u32 asid; + int ret; smmu_domain = arm_smmu_domain_alloc(); if (IS_ERR(smmu_domain)) @@ -675,5 +402,22 @@ struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev, smmu_domain->domain.ops = &arm_smmu_sva_domain_ops; smmu_domain->smmu = smmu; + ret = xa_alloc(&arm_smmu_asid_xa, &asid, smmu_domain, + XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); + if (ret) + goto err_free; + + smmu_domain->cd.asid = asid; + smmu_domain->mmu_notifier.ops = &arm_smmu_mmu_notifier_ops; + ret = mmu_notifier_register(&smmu_domain->mmu_notifier, mm); + if (ret) + goto err_asid; + return &smmu_domain->domain; + +err_asid: + xa_erase(&arm_smmu_asid_xa, smmu_domain->cd.asid); +err_free: + kfree(smmu_domain); + return ERR_PTR(ret); } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 020e4efdf8fc06..15378c131a5bc7 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1439,22 +1439,6 @@ static void arm_smmu_free_cd_tables(struct arm_smmu_master *master) cd_table->cdtab = NULL; } -bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd) -{ - bool free; - struct arm_smmu_ctx_desc *old_cd; - - if (!cd->asid) - return false; - - free = refcount_dec_and_test(&cd->refs); - if (free) { - old_cd = xa_erase(&arm_smmu_asid_xa, cd->asid); - WARN_ON(old_cd != cd); - } - return free; -} - /* Stream table manipulation functions */ static void arm_smmu_write_strtab_l1_desc(__le64 *dst, struct arm_smmu_strtab_l1_desc *desc) @@ -2023,8 +2007,8 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } -static int __arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, - ioasid_t ssid, unsigned long iova, size_t size) +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) { struct arm_smmu_master_domain *master_domain; int i; @@ -2062,15 +2046,7 @@ static int __arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, if (!master->ats_enabled) continue; - /* - * Non-zero ssid means SVA is co-opting the S1 domain to issue - * invalidations for SVA PASIDs. - */ - if (ssid != IOMMU_NO_PASID) - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); - else - arm_smmu_atc_inv_to_cmd(master_domain->ssid, iova, size, - &cmd); + arm_smmu_atc_inv_to_cmd(master_domain->ssid, iova, size, &cmd); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; @@ -2082,19 +2058,6 @@ static int __arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } -static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, - unsigned long iova, size_t size) -{ - return __arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, - size); -} - -int arm_smmu_atc_inv_domain_sva(struct arm_smmu_domain *smmu_domain, - ioasid_t ssid, unsigned long iova, size_t size) -{ - return __arm_smmu_atc_inv_domain(smmu_domain, ssid, iova, size); -} - /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -2283,7 +2246,6 @@ struct arm_smmu_domain *arm_smmu_domain_alloc(void) mutex_init(&smmu_domain->init_mutex); INIT_LIST_HEAD(&smmu_domain->devices); spin_lock_init(&smmu_domain->devices_lock); - INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); return smmu_domain; } @@ -2325,7 +2287,7 @@ static void arm_smmu_domain_free_paging(struct iommu_domain *domain) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - arm_smmu_free_asid(&smmu_domain->cd); + xa_erase(&arm_smmu_asid_xa, smmu_domain->cd.asid); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; @@ -2343,11 +2305,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_device *smmu, u32 asid; struct arm_smmu_ctx_desc *cd = &smmu_domain->cd; - refcount_set(&cd->refs, 1); - /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret = xa_alloc(&arm_smmu_asid_xa, &asid, cd, + ret = xa_alloc(&arm_smmu_asid_xa, &asid, smmu_domain, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); cd->asid = (u16)asid; mutex_unlock(&arm_smmu_asid_lock); @@ -2806,6 +2766,9 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, /* The core code validates pasid */ + if (smmu_domain->smmu != master->smmu) + return -EINVAL; + if (!master->cd_table.in_ste) return -ENODEV; @@ -2827,9 +2790,14 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, return ret; } -void arm_smmu_remove_pasid(struct arm_smmu_master *master, - struct arm_smmu_domain *smmu_domain, ioasid_t pasid) +static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid, + struct iommu_domain *domain) { + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + + smmu_domain = to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); arm_smmu_clear_cd(master, pasid); if (master->ats_enabled) @@ -3107,7 +3075,6 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev) master->dev = dev; master->smmu = smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); ret = arm_smmu_insert_master(smmu, master); @@ -3289,12 +3256,6 @@ static int arm_smmu_def_domain_type(struct device *dev) return 0; } -static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid, - struct iommu_domain *domain) -{ - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); -} - static struct iommu_ops arm_smmu_ops = { .identity_domain = &arm_smmu_identity_domain, .blocked_domain = &arm_smmu_blocked_domain, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index a036610e50da14..100e6721e16131 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -587,9 +587,6 @@ struct arm_smmu_strtab_l1_desc { struct arm_smmu_ctx_desc { u16 asid; - - refcount_t refs; - struct mm_struct *mm; }; struct arm_smmu_l1_ctx_desc { @@ -712,7 +709,6 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; unsigned int ssid_bits; }; @@ -741,7 +737,7 @@ struct arm_smmu_domain { struct list_head devices; spinlock_t devices_lock; - struct list_head mmu_notifiers; + struct mmu_notifier mmu_notifier; }; /* The following are exposed for testing purposes. */ @@ -812,9 +808,8 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); -bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain_sva(struct arm_smmu_domain *smmu_domain, - ioasid_t ssid, unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size); #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); @@ -826,8 +821,6 @@ bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master); void arm_smmu_sva_notifier_synchronize(void); struct iommu_domain *arm_smmu_sva_domain_alloc(struct device *dev, struct mm_struct *mm); -void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, - struct device *dev, ioasid_t id); #else /* CONFIG_ARM_SMMU_V3_SVA */ static inline bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) { From patchwork Wed May 8 18:57:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A459C25B4F for ; Wed, 8 May 2024 18:58:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Enc2qwgOIiq+8niI8hAhQUjsouPrb6/8NxQ+lLAfqT4=; b=MR+Jpct68BYGYi F1mx4wofCCY8kOG0n1MTP+Q+vkh9rKn8gF3x06ZReIcCV9r/X7dsORfh8j+Zm4ZVcnscONkMxRMlb Zsxo2MkdT+CjFD6e0mdAJOYKhrD9hvQa8dmFVdB/UUCrhfGm7eDWUYzqSq6ErMMKVrJ+L7dKhgT7V KJ+p2EoIyO1HhHj9XIrA2t/GVUiobdBUECCSBx6+CIm4Fh6jzFItTEXC8j6ASGtk/BFUZ8CrSnZNM hs0GzwkIDTq2E9s0d/0vFojb8R44KTbeYK1l7n50jqWwalmKPjXxGjZqf6taqL3B8w6DBsX/3EHLj RiyHFrA05cl1yOyCafgw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUR-0000000GcSa-01sX; Wed, 08 May 2024 18:58:07 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU4-0000000Gc5t-2MCY for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iTGgWaVhKKJfzpRRJ54/8LvyjIZNaK6Rr/gXRA6uvNxF78Xj7YW4qVcu3CR42TJ4BmxqsxwzIbd87uqZeCudVr4xcsBorgFIBuDw/VL1oBo8oUhYObxsSZGcxcFv48jaC0ajm2xgRUwgG4OTWOZ6XY4r806w+o8eHMGXx3W+IrOEn97O1uu5P2PJqETHmWJSF57dzDkkKufCGKfqm9lx+I+TBM9jSrl2Axz5K26gxDNx06ixFHnK9rUFWGOFrzjxr1QS0OQXIinVMlVtyJBjXq+II3ScURtPvLoEgSloC1x8AUzRyZGGymtBa4Yt5bV4mqo1qrmdQ6Bd5safn9+ZRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NT7IIbn9fz06tsa0c2eTS+8dpxS2z/WQIjjoLDkyu24=; b=TM43J4JQoFjYULv3gsR8AAuJu8WO06MJkDttgzq/LjtTRUcD3TSBRkLizHnBs+sKINSm+RxoZrlKFB5KLjnkHHt57dbg7PC1UGlMChbluSHu2WMVX2TMJKI9yVx9mS34n/BGMsbjpVD2XvR5bLZ57ZuA661P/8P/RQoDLgN/fdmmyjQCoV3Fjoi8D4KbvQMkBtw5lqpbNlNETRZohntKTlv5gh734ptCc2RZ/Vwnku+UmPJwpJRTeH00FT7CMLG+CFJIVt+sSMkCugiaezvwMpk5N5PR4HSk9l3Es2Arrx2HLR2MzJZEM7W5q15TB1oix65AKgmwBNdnNiKSxc8brg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NT7IIbn9fz06tsa0c2eTS+8dpxS2z/WQIjjoLDkyu24=; b=jJyuQ9fkzbPx2qbT3p/idYfQY3mwIm5UcdfGwdkwhzZXe4Bsiv0rP2q+TFp8nNcXTgT/j1aTh3DET1ZUN4nwVsViqSUqRuDPdV4Tke2kZ/yorq6Me3oprppD6tdGskZYe6j5pmPsUOLEbLpP3OFFiaZvbSxE6dh9VHgDxVT+ePVfFWUCoWMeh3FSWQfkLybQjdkZ73s9rTUwS7lJ2LvVk6nU3+0jHb7M4hCjqQtU8MMPz94CUF6zKOh/Y357PBeQeaXEXGIYcxdjZeNPOCd7Gn3/Uso3M6YJ/+A2nkd/s+QKisoL1TSUxgPI1+2NM2LZf6X0ZQuD1+uIYZlSe6W9hQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:26 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:26 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 11/14] iommu/arm-smmu-v3: Allow IDENTITY/BLOCKED to be set while PASID is used Date: Wed, 8 May 2024 15:57:19 -0300 Message-ID: <11-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR03CA0033.namprd03.prod.outlook.com (2603:10b6:208:2d::46) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 65ff2d49-ee49-4c8a-670c-08dc6f90b324 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: 7JlDt9uVDSr9FOmQEV44TbTQWou1kVOrgjb1tgje9UkrQTm7nM0ikOd7Jh6k9/K3OcKxvHyuvTdEBsA7POFb8iI6qDuFIfb8zb2urLDxwsFtR3oSxIZqXY8QFeGFDImJIl7gBmfrrhMbNqInzC3VUkFcxrZSMUIB92Ue6/a/Jn0Vs+/gUmJ4T9eLzoj7y+2bNaHWwvFYItU9t5uCT86rWwos/P0RJ/mjk+TaVavsWixBx0a8iTjYv1afXkWVLueg4wqNdNnM1PW+7gwSCio2m2rn7abStzCSP101g72nCUFFaowtYYaVH2I/hA0DLNXpLln19lr3ejvgxF4bLg6RWF/XjYIDLu1UABeUYmUwBG0T0aZ1vDviUH0hy0CzP5MZSAH5ajjrTZoP5Mgq4Z5GPFOveQt0fmoT2fthu7Oi/ooY3SfbRonygz3eZk1DgphnoNY5rlbfCRaKQncT2R62SldUbF5b3ixEHlF/TrSAGPaPcr8bvSwKfNuwZ5Mwb7rfRK9/np/Ck/3CwHyatwLp8qvf5ydCH3FYcIwAicRGaGwuH1zzc8q+nxdIEUMFmyzktyS1+210yJVjSPh9Jils0FLEmnRAndSG5y8iy09vAFVy9f2npLBIoYSN/V491QNIJulBSwQx9ZioFqdNIA0PW9h1S5u8vcmyAYaC/3dGbV92IoXVRO6KoRBOqGEPwVji1cB3lMLbnFceYEK8m8g4CCz9gzZxqyD0XVAwR0gm4qafKXGojXDCT5TnszoSRyMXxUvh/Inq89jTMnJ8RUKNde0IlreQ2diK3DVeHN+dNLYbGRGkgN4+MhA/gqUZvvD3hkhfWyIHkRWwv798N5WO7yIf0z4JueW1GeBhWI8kIcdMSV62lSE1hX3zVcCFS6ZhR1ayzcKshmTmtGpud94d+clBr4T2Zy1l5BAXVMreAvSDhnIhKe1ddcLrJ7OJ1Pp/85+ykg4+uApCTySXourI9dx0Wto+mCswR+5xnTQClIB13q04vbNOIwQ8CtasJIax250k7yAV5BXtjUSMmW7uu0R3q10Heogqu+8Sc99OaczHzeMhkGd6Evsv1STRs95JCyw3KXstB0y0Hw+PPRnm+muUxrtYZogxdeYBvY9MY80tZZXcjTr4zH+Q/NwODifm54OEw4XeUT4cYT9nHsJ1nCe6iIzpiCQg5pF2+tw4APalSsWdKscgyt9r3izG+69Dmsq8/y5oiRblbkkbI9jLGd/AIk7R1M+m0/vo1mxn0XltgGE33yQmmmxm4SU8f2L7ZUxTi9OuwHkVZbXR6jbRcw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: aOV4llNp3mQBiGzPvHrL99JRP0s6aZ+1TGL9UPF3XOlpDup7lg2VsydV5qKBYAKtvVWKY7ZmIfu89kZbRr7C9t8/RF+/fxS5t2LO1E9kXoNyBWRW/vuRnmlYivqcAvvlwXfc5xMO8DcOiffk3yx2vlIzVb8qcb8O6G8SWH20dElZJZLbAHApEuMXyPRs4tYvwDlY4YXLoGw8xOkE2SlIWB/GyJO11CVWJsfu4f03iVaXXg5QT2r7m+ekpZcJTvPbAWZ9OdHsJfwBpWxpsuiY/o267+MOKNSY2B4dTHTcEPXMxpswEseT6u6yoZetCJhoPz1YFvSVfPoOAoDCS+cPnuKit/bor6MDE9yIXp1hjgYdc+U4nNypiS3TWEUCilcebJriZYZtbzj2BnZ3cxUDFVrc2vDZH0V2tPIz95+GSKvxhInJ9wMRhVwEf2Fo6v5ooT+IZPC1U667EKhgvZUiyAvEt1LevrnffFiNn95MAjaDVJVMEIRhfGfNReTNOct5QkMWIxEPAL5i0XYsXbx4uXa+TzSlgIZlG8m7VxrBk/AUlrO1kDp5IEh79JvSP8oRtCpd/pDmF/rxTZNPO+bm5zDnD6By5S3nz+pivWMlYSRFeAXSRhg7drbaumBUwO2xHQ8Mbzlgz8/9ks2t8/pVWh0FJ7VDQvH8QCDS/VF3nDcyf6Msv4ftDQ4KOEfU3u+abSYRVAS5/aVVsC8hQJXaV/jvz84aK1RZ7taJFFuCZCZe15GuPPygyWHwtKPwExqa50Fp9B5sthrH7TTkHDqepz2yinbmaPcgSTG/opxJGOvtYUFI7wHnRN0x6ayaKXfnKFEyDswpd0zA/hGoMVfzKeWilwSMxkg4wys06GLSK0c5nqkCde7Xz0NcYcplk5blolVgKciSN3AQtADYvKWdMHE28CS8iXxmYZDOaQgHzOEWlxjBzMMhyac69aesQpTEcyhBePeWvGzTt32lI7roKpd2cv5Ql3FoJv84A2gfJqevKxjQnuxR8/InNfS5d2a6pAAdRugbFYB86JDTHq2lxUTlYutRG2liybtpmN1ExCMHIVg0IdhvUDLrj3sEqi99esRJZXTImYdkmfi5rDO1YtBEgqWcCjEOu7nkgEBR4Fvt148O4Ft/A3Y2IEy+oVNjkkt5ruJLko9LcGpb0o5SqWFVZhZxDSeobyk8Hsla9DztijTmmDAtuvXYKcUXxJCJwf6wXJF6GxZKo0M1uAUp6KZjvS6lhGPi8zytxy0sCpiyzUtbN1axccW33n6b4gJ9HzbOD3n1vmYUBJk2acUXYtWeDgUCJmVP28+0XUS8C1isjmxJl4/72r8cjvmzuCqxy5DTrcHSujSU0SgcEEAmGYLCIy5aymBvIhRUQbM6N3bO5BM3a/Ye4caI/G5qbU8Tvlh7Tg9HMsymFJP59J9ug0225A60rAQXYMoEIk8t+4U6+rFc7jdH0/7hMiAWNW8f52EN+rsef1DKePzaYStpOytzCJnl2KjqgJKyw6rSOv5t0YtUAMiXJ5Bh+I2bm36Oh7qDNXGJYZJkqO1Pm9SJTeO09eVN2deT5CL0LStLS2IBkSRclIsPy7wwAimxcH3i X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65ff2d49-ee49-4c8a-670c-08dc6f90b324 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:25.4428 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZqbI2EUItWs1GapQavis6faYVbGjvjQL9q6SERpF6yoAXKdWpffpmjA3icHTU2ma X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115744_709694_50E70674 X-CRM114-Status: GOOD ( 26.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The HW supports this, use the S1DSS bits to configure the behavior of SSID=0 which is the RID's translation. If SSID's are currently being used in the CD table then just update the S1DSS bits in the STE, remove the master_domain and leave ATS alone. For iommufd the driver design has a small problem that all the unused CD table entries are set with V=0 which will generate an event if VFIO userspace tries to use the CD entry. This patch extends this problem to include the RID as well if PASID is being used. For BLOCKED with used PASIDs the F_STREAM_DISABLED (STRTAB_STE_1_S1DSS_TERMINATE) event is generated on untagged traffic and a substream CD table entry with V=0 (removed pasid) will generate C_BAD_CD. Arguably there is no advantage to using S1DSS over the CD entry 0 with V=0. As we don't yet support PASID in iommufd this is a problem to resolve later, possibly by using EPD0 for unused CD table entries instead of V=0, and not using S1DSS for BLOCKED. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 67 ++++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 4 +- 3 files changed, 52 insertions(+), 21 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c index a460b71f585789..d7e022bb9df530 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c @@ -164,7 +164,7 @@ static void arm_smmu_test_make_cdtable_ste(struct arm_smmu_ste *ste, .smmu = &smmu, }; - arm_smmu_make_cdtable_ste(ste, &master, true); + arm_smmu_make_cdtable_ste(ste, &master, true, STRTAB_STE_1_S1DSS_SSID0); } static void arm_smmu_v3_write_ste_test_bypass_to_abort(struct kunit *test) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 15378c131a5bc7..41d7a0664a445d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -991,6 +991,14 @@ void arm_smmu_get_ste_used(const __le64 *ent, __le64 *used_bits) STRTAB_STE_1_S1STALLD | STRTAB_STE_1_STRW | STRTAB_STE_1_EATS); used_bits[2] |= cpu_to_le64(STRTAB_STE_2_S2VMID); + + /* + * See 13.5 Summary of attribute/permission configuration fields + * for the SHCFG behavior. + */ + if (FIELD_GET(STRTAB_STE_1_S1DSS, le64_to_cpu(ent[1])) == + STRTAB_STE_1_S1DSS_BYPASS) + used_bits[1] |= cpu_to_le64(STRTAB_STE_1_SHCFG); } /* S2 translates */ @@ -1531,7 +1539,8 @@ EXPORT_SYMBOL_IF_KUNIT(arm_smmu_make_bypass_ste); VISIBLE_IF_KUNIT void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, - struct arm_smmu_master *master, bool ats_enabled) + struct arm_smmu_master *master, bool ats_enabled, + unsigned int s1dss) { struct arm_smmu_ctx_desc_cfg *cd_table = &master->cd_table; struct arm_smmu_device *smmu = master->smmu; @@ -1545,7 +1554,7 @@ void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, FIELD_PREP(STRTAB_STE_0_S1CDMAX, cd_table->s1cdmax)); target->data[1] = cpu_to_le64( - FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) | + FIELD_PREP(STRTAB_STE_1_S1DSS, s1dss) | FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | @@ -1556,6 +1565,11 @@ void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, FIELD_PREP(STRTAB_STE_1_EATS, ats_enabled ? STRTAB_STE_1_EATS_TRANS : 0)); + if ((smmu->features & ARM_SMMU_FEAT_ATTR_TYPES_OVR) && + s1dss == STRTAB_STE_1_S1DSS_BYPASS) + target->data[1] |= cpu_to_le64(FIELD_PREP( + STRTAB_STE_1_SHCFG, STRTAB_STE_1_SHCFG_INCOMING)); + if (smmu->features & ARM_SMMU_FEAT_E2H) { /* * To support BTM the streamworld needs to match the @@ -2733,7 +2747,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_make_s1_cd(&target_cd, master, smmu_domain); arm_smmu_write_cd_entry(master, IOMMU_NO_PASID, cdptr, &target_cd); - arm_smmu_make_cdtable_ste(&target, master, state.want_ats); + arm_smmu_make_cdtable_ste(&target, master, state.want_ats, + STRTAB_STE_1_S1DSS_SSID0); arm_smmu_install_ste_for_dev(master, &target); break; } @@ -2806,8 +2821,10 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid, mutex_unlock(&arm_smmu_asid_lock); } -static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, - struct device *dev, struct arm_smmu_ste *ste) +static void arm_smmu_attach_dev_ste(struct iommu_domain *domain, + struct device *dev, + struct arm_smmu_ste *ste, + unsigned int s1dss) { struct arm_smmu_master *master = dev_iommu_priv_get(dev); struct attach_state state = { @@ -2815,9 +2832,6 @@ static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, .ssid = IOMMU_NO_PASID, }; - if (arm_smmu_ssids_in_use(&master->cd_table)) - return -EBUSY; - /* * Do not allow any ASID to be changed while are working on the STE, * otherwise we could miss invalidations. @@ -2825,14 +2839,29 @@ static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, mutex_lock(&arm_smmu_asid_lock); /* - * The SMMU does not support enabling ATS with bypass/abort. When the - * STE is in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests - * and Translated transactions are denied as though ATS is disabled for - * the stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and - * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry). + * If the CD table is not in use we can use the provided STE, otherwise + * we use a cdtable STE with the provided S1DSS. */ - state.disable_ats = true; - arm_smmu_attach_prepare(master, domain, &state); + if (arm_smmu_ssids_in_use(&master->cd_table)) { + /* + * If a CD table has to be present then we need to run with ATS + * on even though the RID will fail ATS queries with UR. This is + * because we have no idea what the PASID's need. + */ + arm_smmu_attach_prepare(master, domain, &state); + arm_smmu_make_cdtable_ste(ste, master, state.want_ats, s1dss); + } else { + /* + * The SMMU does not support enabling ATS with bypass/abort. + * When the STE is in bypass (STE.Config[2:0] == 0b100), ATS + * Translation Requests and Translated transactions are denied + * as though ATS is disabled for the stream (STE.EATS == 0b00), + * causing F_BAD_ATS_TREQ and F_TRANSL_FORBIDDEN events + * (IHI0070Ea 5.2 Stream Table Entry). + */ + state.disable_ats = true; + arm_smmu_attach_prepare(master, domain, &state); + } arm_smmu_install_ste_for_dev(master, ste); arm_smmu_attach_commit(master, &state); mutex_unlock(&arm_smmu_asid_lock); @@ -2843,7 +2872,6 @@ static int arm_smmu_attach_dev_ste(struct iommu_domain *domain, * descriptor from arm_smmu_share_asid(). */ arm_smmu_clear_cd(master, IOMMU_NO_PASID); - return 0; } static int arm_smmu_attach_dev_identity(struct iommu_domain *domain, @@ -2853,7 +2881,8 @@ static int arm_smmu_attach_dev_identity(struct iommu_domain *domain, struct arm_smmu_master *master = dev_iommu_priv_get(dev); arm_smmu_make_bypass_ste(master->smmu, &ste); - return arm_smmu_attach_dev_ste(domain, dev, &ste); + arm_smmu_attach_dev_ste(domain, dev, &ste, STRTAB_STE_1_S1DSS_BYPASS); + return 0; } static const struct iommu_domain_ops arm_smmu_identity_ops = { @@ -2871,7 +2900,9 @@ static int arm_smmu_attach_dev_blocked(struct iommu_domain *domain, struct arm_smmu_ste ste; arm_smmu_make_abort_ste(&ste); - return arm_smmu_attach_dev_ste(domain, dev, &ste); + arm_smmu_attach_dev_ste(domain, dev, &ste, + STRTAB_STE_1_S1DSS_TERMINATE); + return 0; } static const struct iommu_domain_ops arm_smmu_blocked_ops = { diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 100e6721e16131..71ecced1db0226 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -761,8 +761,8 @@ void arm_smmu_make_abort_ste(struct arm_smmu_ste *target); void arm_smmu_make_bypass_ste(struct arm_smmu_device *smmu, struct arm_smmu_ste *target); void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, - struct arm_smmu_master *master, - bool ats_enabled); + struct arm_smmu_master *master, bool ats_enabled, + unsigned int s1dss); void arm_smmu_make_s2_domain_ste(struct arm_smmu_ste *target, struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, From patchwork Wed May 8 18:57:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 427CBC04FFE for ; Wed, 8 May 2024 18:58:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YypakDHMkABdkN6Pz4mtMtOogkuFXgnM/P4ZNh1FvA0=; b=QLPeZT2EKei2bz 2/72E5IeYIhsFLD0QXACdpdW6a7LUGs5W073icpQigLlJvchQv+npqvzn/QdbdVSpjCQTeVFs81Dy Xkcqae6OQsrrHOqvx9vQcTTm1CloCUPNbm05kXrAH2Nsfq1rvrNBSq3sk0Qr6Xi4NdCpH0mzP4W0g ebmTmkwybR3tsmAxcG2e/NKGqq88SikrpshRfKUpAR6p1/lRnWGYeGcHQOHbJQiGF3InkOjrR7+HA +ndqGsmehyTay4KX1HAMlmL2GdIlYroEfqjcpPP86mpHHASzpTdPjj1/K0Vmg+143V1Ix9pkoWVZ3 03IB/m3rFYE9sLw+hPmQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU7-0000000GcEF-3C8E; Wed, 08 May 2024 18:57:47 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU0-0000000Gc5t-1MGw for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X4FqwE/uXkyfFg5wKaTVvx1S3BGNQBKqbkA5tJm7NTR1YtcNkisggPUFJZMEL+aZcRMPw7Lb7a5TSEakZkdspegLjhcDUz+WsPOo8BQRhqRlK8q+5egEMmgz8nGR4paAG7lHViD2YRymbeLxFmTfDprHgHCFGFOT4DMU901HM/2rKIv71+VT7T04M8SJWt4BMx+CRQh5vAJ50MEfa03MqR5DxE6ez8PvzOLsQntxtc9Ph1sDrggG26+65AwWa/DG74sxPocv487IXCWiUXnnUB75xrjpTjXirZ9wUeLDJ2oMPlqqdFcAhHHEs6Z48d/lxxu4h6XRt3XLYZz8JiL+YQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xJVkKR6h/UPUFZGtWxN0hcTMSGKB62GiZmLTaZWpbcM=; b=JSo8mCjRjjjmTsAIBE6U4xVHqJs+/Z31CCxMkWl79d5g6L0doihLG4F/SGfuRb7eOr66Y4NjqPEFwch+xJQUhTAJpSLz9j1sbMmGtNTadOeVzTW9XvitsuY+fPQxt6WVyQLU1kSSRZ42n2dqpIssirdN7+5Wnhp/8ZnWPq1kLmdbBOWQsDNwdNkA3TaZg7mYh3RrNedDpyU4xqDq4zCB6YMIEP+4J2vNpEN42tKzDjVL34/SZXXBg3C+2Wf6UtJVd4mf0Ib0XrBPq8Uaiq43AtHD1xycmJnnq+AMAFF0NDaqU6vl+CEoZeSS/W1yMLjezDXrRvWAecvb9nG4OEVlgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xJVkKR6h/UPUFZGtWxN0hcTMSGKB62GiZmLTaZWpbcM=; b=TXddNuew9GyDf9yjNDCQp7ACyQlJFB+Lb1qXN7HOhF5/7kMvVno/P3a6gWW76r5jdqSy0gLhqyZ6IY9Bas04CT9kD5pjFtRLPdmSVPG+xOeilRtoMC7rnUG/txmJZ2vla+j8nxyxRNykxcTxaUHadFDbBea83OBbRZvbVKB0Vu0gGETSPaSZwkkif6EuYvQoePPjcozgzKwemSE8WcYCvI1wLpTDQBhuJIxTLp+25xft67b3EUbnXarpW2ZTvJv+OPiwMYi6gE2bKFH+KzUsJYbgo+rz+Mjw9oZTjGOeiv/izLjBk3H0BtVfeeyGGkX30ydyyEUadHlQrxQEsbhKUw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:26 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:26 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 12/14] iommu/arm-smmu-v3: Test the STE S1DSS functionality Date: Wed, 8 May 2024 15:57:20 -0300 Message-ID: <12-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0158.namprd03.prod.outlook.com (2603:10b6:208:32f::24) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: 179cffd4-feb3-4b39-56b3-08dc6f90b31e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: dJ6uk1waztZGBvM7ZSp4UhbiG/fNyakJ9qZaQPYNZBmj1j1JU90BpMfhFu8SpluA7ZHFtK1lEi1FUwPuZq0mvC3u4Wo51YgyPJAmRauRI4STrvo0D3mTtPt16yyiR6QGyvBi+mAjurZtCoXPLSNP2LzwWnRgxelnSDatQOAuet47KbRAjyudsjqaWLQrCBpVfvs1lw1QceOIRbPF10J60cvi5jyHWO/zL/v9LnAbkMXqPu+rG8DRskLCKVwCRztJyipvy8I9R9MNaA/olrLQND+9rn8rDVmbjzFuwtG2JB7sV1V7woqS75928fk02aBSLFdx7m+i1VUk2XtZLrJHtWt9+KAe/cOq63LRcvB+uf1O7W0ko5pW/yrPgSN0vAEQv33vsz5/2ZznjXM/F4Ujlx1LxkbfvFjsR7KD92v/jcLBKkxXuYBiFTYu7ZUfgMdGUu3/ioi8L7ih7fM+swNueKrVEWzMWzQ5yEqZ1g/ELmPzqLbc3L0Yp9SmT8iWdFWC/+VNtbqPj5lwSJ1Q9IXzu5nPHMluVxOfqntaK6ckpEekpXHvRFXcy3jPZv0C09tYPwmy3yVeCrRMwRu4suqXyDWQjwPmw+3kxE4ZtT+Axc4v/Bj6YIiYiS7azUNd//3C9ctXJE50TycfvtOzjSDmeAjEDgqC+fzoDipH/YqU+o6DR5MulV1axYUYYELuLZFnuMMOwn+B8b0Df2a75vbTaV1zgR1zst33vhp4GyhlX7RJSg7d2XUSbAeF+gfjtuy9GOFFVYZ5zySsvOdixLw614afTyflQzTDfMhHdMgdc34sPObFbi1r6MRrEkWeAIsKYbcpBQiba8881L6wxy6XjP1K/io/RaCDwCSJ9ZoI1rdIyox67Wwqy+mNsoo6n1bGRR8Nd85v2M+Dw57TqR1PPbId27jDHGLOaIL9sly0t661df+c6ScQWx3HWYTcV5iptUSS8aVn+VPxX7Go/Y03jBPiOmhCTtFHt9cG/2lXLyX0ShRUR/TQ9ies5nZ7/vMekdMNboqPcC/SSmwLMcon7lfTHItgqMuvg/9CJAQXdPYuCgwKk+xns2w/un6NncBoHeirpOeNtwoczc0b//nxrRFPj/KBZQJKDRhQFWaFar2rI5JETiaHh1j7YCGBEXdMCMy+uOAagD1ZWbaoNT0H/8797Z36OtErFQsCHZXhQQ/+aUikuRawNYxwhBjb0xgJ7Y84AB64GFWoXCNNKORw/Q797AwAvdsvw+7ubBaoPir2szrx2WlcD4P0WXB6kpt6fl+wMxMVpWOWRjTR7J6VTA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: j6lEEiX+uLdX3htcjXMxYq7bqggIxKMFTW2910Bf2SGUwg6l9nTTaBjuCsny6AzyH5tfRPP8NVMnKlvugvZBpDOs/61HFCNRMIee0GNZl0CMEHThtf6WOwIy2jjjXojxQirIiPD66lHv1GLq4HTSbNBN4BWh3C3tlNmhSNPA/29+FMPpCTKVUg9/Ym78aHUFSXGloC/+ZNyQoGIfdaL28ROYHecRBveRVBjahNcFLePWAlOMrvETMUf9zTdjgyFMFTatAfOa0+7pF7VDRNfAEyVatD4P0LbVtD9Nbly+ZE2b4HAdZvzfXDyl7l3oQeaBNJ/uBxh9452W7VqjSvAK36Soa1f2xnzdWfdQiCF8Ux+ydJKVHH4j26cKeJ61J14mjtGIV57e0Np2fv6ac5eNgmLYiyDJDAn9WOysX/o33qKN1Y5SkDaY08BH3VHJSzlDyYsg+ClW78T0ZDy2tv3DTwcJ9Qswnxwp5Ry8AKKRIFAgGTyRyYuNimhWdxqWOQ6eP4TfU0h/4b4fD2J9emxosm6ynRjBztBtMD/Kl9RZ/7pttl3a19NrUvxEB3f83JEOylhpdngrgDoYTzARR/QxIuLLQ0/WRvjvTR3iPF6FxwGyINNeF8yQKq5+l7RLknjSq0qy/HDewSZR+V4Df9r+rB67njutjIZW+pCFEwz8/NwT+zfb2AIZ5epr6855CTP3IvkFjG4H6rv2XCLqj5N7ABDtViT9SL9yS1RD9hQ/TEKWYtDB7M+MD5E81bTy+MCA1IhsyFNnjGWe4g/jpaKLiG/5EF0cW2iZ18+Pd3Sj+Kkt1fKnzpbGrhBu0higXyWcs5Ux60RTFbvN7kIloqAlghzSIsCYZK+uCSlAjbtp5jiMwnTVxURydQJc/VGF4Y3x/l0PzlEUI3ZJIM9nWFSoHdI8DEF/grc9esiez80BPlosKXkNBGxdoMt5sbsv8rh+xgh1RJgi+b/uAuxrRQeR6fXMdmw8CJyAjkzwftoab05U/h91BrM2a4YBr2BB+09DtGZup5Dcjhm4/PC/tfYdBitaKkZk3XgzeHS5aVwKLXfWcDSS/s9R11zUjI+hMO7RoPpoMOMbsdANzfeZxkYYvWQFaeRpWvmMxIE7N0nXJqjvYDS/9kqbRToVKnjX9P0pklZd6W+pzmP21C7a1QByPhDlHCgsPgfuG/QlrbyLrSCW2kp2oDHkDSxhgR6S3/4cSE53w/SxV/89DefXVbYwVtdbQ97+O7oT5EjAdsagrSPQWbT6bEK7OP+N3Dvw/6fDQuepGiVtMBMvb792yF+E+1LtJGvYm+0FuwTc/uJ/ZSFzX491NZA7r2HgiVXJgMiloIaS4Y3HJE71HWIpumiFa6M0HRfn7LKELZ+o5JQLobFDViQJ7ec3NzfgUb5+8L7gw8IlbDN0EvTUdAJrepYa+TQZzZGWIauy+BkJ/CJK0uUH91o8jYg3I22RKpWiUO9XpxvMcG03bYEzaXGnpIiKN1tms5vB/iyBcY3LoK2LzBvodpucBOGtDogjezlbVNKpWSHDS8g3Sy/vVYeG73Zh/qg1nD5/yRYNjbfx01SY9Bjre0lj8M2/wGFU2pPgUnpl X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 179cffd4-feb3-4b39-56b3-08dc6f90b31e X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:25.3682 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TesYD40LV5znsHE0BRvlAMDObT90tmAxX03rNO1et+HQAswp2rRmqJo1FNQHqYrC X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115740_531995_806626B6 X-CRM114-Status: GOOD ( 12.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org S1DSS brings in quite a few new transition pairs that are interesting. Test to/from S1DSS_BYPASS <-> S1DSS_SSID0, and BYPASS <-> S1DSS_SSID0. Test a contrived non-hitless flow to make sure that the logic works. Signed-off-by: Michael Shavit Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c | 113 +++++++++++++++++- 1 file changed, 108 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c index d7e022bb9df530..e0fce31eba54dd 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-test.c @@ -144,6 +144,14 @@ static void arm_smmu_v3_test_ste_expect_transition( KUNIT_EXPECT_MEMEQ(test, target->data, cur_copy.data, sizeof(cur_copy)); } +static void arm_smmu_v3_test_ste_expect_non_hitless_transition( + struct kunit *test, const struct arm_smmu_ste *cur, + const struct arm_smmu_ste *target, unsigned int num_syncs_expected) +{ + arm_smmu_v3_test_ste_expect_transition(test, cur, target, + num_syncs_expected, false); +} + static void arm_smmu_v3_test_ste_expect_hitless_transition( struct kunit *test, const struct arm_smmu_ste *cur, const struct arm_smmu_ste *target, unsigned int num_syncs_expected) @@ -155,6 +163,7 @@ static void arm_smmu_v3_test_ste_expect_hitless_transition( static const dma_addr_t fake_cdtab_dma_addr = 0xF0F0F0F0F0F0; static void arm_smmu_test_make_cdtable_ste(struct arm_smmu_ste *ste, + unsigned int s1dss, const dma_addr_t dma_addr) { struct arm_smmu_master master = { @@ -164,7 +173,7 @@ static void arm_smmu_test_make_cdtable_ste(struct arm_smmu_ste *ste, .smmu = &smmu, }; - arm_smmu_make_cdtable_ste(ste, &master, true, STRTAB_STE_1_S1DSS_SSID0); + arm_smmu_make_cdtable_ste(ste, &master, true, s1dss); } static void arm_smmu_v3_write_ste_test_bypass_to_abort(struct kunit *test) @@ -194,7 +203,8 @@ static void arm_smmu_v3_write_ste_test_cdtable_to_abort(struct kunit *test) { struct arm_smmu_ste ste; - arm_smmu_test_make_cdtable_ste(&ste, fake_cdtab_dma_addr); + arm_smmu_test_make_cdtable_ste(&ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); arm_smmu_v3_test_ste_expect_hitless_transition(test, &ste, &abort_ste, NUM_EXPECTED_SYNCS(2)); } @@ -203,7 +213,8 @@ static void arm_smmu_v3_write_ste_test_abort_to_cdtable(struct kunit *test) { struct arm_smmu_ste ste; - arm_smmu_test_make_cdtable_ste(&ste, fake_cdtab_dma_addr); + arm_smmu_test_make_cdtable_ste(&ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); arm_smmu_v3_test_ste_expect_hitless_transition(test, &abort_ste, &ste, NUM_EXPECTED_SYNCS(2)); } @@ -212,7 +223,8 @@ static void arm_smmu_v3_write_ste_test_cdtable_to_bypass(struct kunit *test) { struct arm_smmu_ste ste; - arm_smmu_test_make_cdtable_ste(&ste, fake_cdtab_dma_addr); + arm_smmu_test_make_cdtable_ste(&ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); arm_smmu_v3_test_ste_expect_hitless_transition(test, &ste, &bypass_ste, NUM_EXPECTED_SYNCS(3)); } @@ -221,11 +233,54 @@ static void arm_smmu_v3_write_ste_test_bypass_to_cdtable(struct kunit *test) { struct arm_smmu_ste ste; - arm_smmu_test_make_cdtable_ste(&ste, fake_cdtab_dma_addr); + arm_smmu_test_make_cdtable_ste(&ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); arm_smmu_v3_test_ste_expect_hitless_transition(test, &bypass_ste, &ste, NUM_EXPECTED_SYNCS(3)); } +static void arm_smmu_v3_write_ste_test_cdtable_s1dss_change(struct kunit *test) +{ + struct arm_smmu_ste ste; + struct arm_smmu_ste s1dss_bypass; + + arm_smmu_test_make_cdtable_ste(&ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); + arm_smmu_test_make_cdtable_ste(&s1dss_bypass, STRTAB_STE_1_S1DSS_BYPASS, + fake_cdtab_dma_addr); + + /* + * Flipping s1dss on a CD table STE only involves changes to the second + * qword of an STE and can be done in a single write. + */ + arm_smmu_v3_test_ste_expect_hitless_transition( + test, &ste, &s1dss_bypass, NUM_EXPECTED_SYNCS(1)); + arm_smmu_v3_test_ste_expect_hitless_transition( + test, &s1dss_bypass, &ste, NUM_EXPECTED_SYNCS(1)); +} + +static void +arm_smmu_v3_write_ste_test_s1dssbypass_to_stebypass(struct kunit *test) +{ + struct arm_smmu_ste s1dss_bypass; + + arm_smmu_test_make_cdtable_ste(&s1dss_bypass, STRTAB_STE_1_S1DSS_BYPASS, + fake_cdtab_dma_addr); + arm_smmu_v3_test_ste_expect_hitless_transition( + test, &s1dss_bypass, &bypass_ste, NUM_EXPECTED_SYNCS(2)); +} + +static void +arm_smmu_v3_write_ste_test_stebypass_to_s1dssbypass(struct kunit *test) +{ + struct arm_smmu_ste s1dss_bypass; + + arm_smmu_test_make_cdtable_ste(&s1dss_bypass, STRTAB_STE_1_S1DSS_BYPASS, + fake_cdtab_dma_addr); + arm_smmu_v3_test_ste_expect_hitless_transition( + test, &bypass_ste, &s1dss_bypass, NUM_EXPECTED_SYNCS(2)); +} + static void arm_smmu_test_make_s2_ste(struct arm_smmu_ste *ste, bool ats_enabled) { @@ -285,6 +340,48 @@ static void arm_smmu_v3_write_ste_test_bypass_to_s2(struct kunit *test) NUM_EXPECTED_SYNCS(2)); } +static void arm_smmu_v3_write_ste_test_s1_to_s2(struct kunit *test) +{ + struct arm_smmu_ste s1_ste; + struct arm_smmu_ste s2_ste; + + arm_smmu_test_make_cdtable_ste(&s1_ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); + arm_smmu_test_make_s2_ste(&s2_ste, true); + arm_smmu_v3_test_ste_expect_hitless_transition(test, &s1_ste, &s2_ste, + NUM_EXPECTED_SYNCS(3)); +} + +static void arm_smmu_v3_write_ste_test_s2_to_s1(struct kunit *test) +{ + struct arm_smmu_ste s1_ste; + struct arm_smmu_ste s2_ste; + + arm_smmu_test_make_cdtable_ste(&s1_ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); + arm_smmu_test_make_s2_ste(&s2_ste, true); + arm_smmu_v3_test_ste_expect_hitless_transition(test, &s2_ste, &s1_ste, + NUM_EXPECTED_SYNCS(3)); +} + +static void arm_smmu_v3_write_ste_test_non_hitless(struct kunit *test) +{ + struct arm_smmu_ste ste; + struct arm_smmu_ste ste_2; + + /* + * Although no flow resembles this in practice, one way to force an STE + * update to be non-hitless is to change its CD table pointer as well as + * s1 dss field in the same update. + */ + arm_smmu_test_make_cdtable_ste(&ste, STRTAB_STE_1_S1DSS_SSID0, + fake_cdtab_dma_addr); + arm_smmu_test_make_cdtable_ste(&ste_2, STRTAB_STE_1_S1DSS_BYPASS, + 0x4B4B4b4B4B); + arm_smmu_v3_test_ste_expect_non_hitless_transition( + test, &ste, &ste_2, NUM_EXPECTED_SYNCS(3)); +} + static void arm_smmu_v3_test_cd_expect_transition( struct kunit *test, const struct arm_smmu_cd *cur, const struct arm_smmu_cd *target, unsigned int num_syncs_expected, @@ -438,10 +535,16 @@ static struct kunit_case arm_smmu_v3_test_cases[] = { KUNIT_CASE(arm_smmu_v3_write_ste_test_abort_to_cdtable), KUNIT_CASE(arm_smmu_v3_write_ste_test_cdtable_to_bypass), KUNIT_CASE(arm_smmu_v3_write_ste_test_bypass_to_cdtable), + KUNIT_CASE(arm_smmu_v3_write_ste_test_cdtable_s1dss_change), + KUNIT_CASE(arm_smmu_v3_write_ste_test_s1dssbypass_to_stebypass), + KUNIT_CASE(arm_smmu_v3_write_ste_test_stebypass_to_s1dssbypass), KUNIT_CASE(arm_smmu_v3_write_ste_test_s2_to_abort), KUNIT_CASE(arm_smmu_v3_write_ste_test_abort_to_s2), KUNIT_CASE(arm_smmu_v3_write_ste_test_s2_to_bypass), KUNIT_CASE(arm_smmu_v3_write_ste_test_bypass_to_s2), + KUNIT_CASE(arm_smmu_v3_write_ste_test_s1_to_s2), + KUNIT_CASE(arm_smmu_v3_write_ste_test_s2_to_s1), + KUNIT_CASE(arm_smmu_v3_write_ste_test_non_hitless), KUNIT_CASE(arm_smmu_v3_write_cd_test_s1_clear), KUNIT_CASE(arm_smmu_v3_write_cd_test_s1_change_asid), KUNIT_CASE(arm_smmu_v3_write_cd_test_sva_clear), From patchwork Wed May 8 18:57:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 43971C04FFE for ; Wed, 8 May 2024 18:58:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FsXrWPfTM98wQe3SjASU3+viacYdXKbiIPvnVFVHLLA=; b=DG3xlW2zz4nvoB QPoKj1RA1LhR0sdksWfQ16R4rUkfGBzEjDnH8WlYcgM/qhomaYHKqwpIarIxhbxsEdb9xptpwBN5K er48EsTlr1pX2CzcVRPE4H6JovLPQ4AYDrmBrSkICu5LhDSegaeSdv5NlrL9DpSPTzJsj1nTctHgj tWgdHblmxICQ+Hcgzc1B+xOI+wsdVaYAlGkWViRPhvGtPG6EMA+/RK7wch7lnZNvVVLt3WV/4K/rB 2tluJ5nEb3JSIDA99ZlQY+LxYwxULOqxlTy9Kyg5VGlbpPSNhgCM8yhILbuws6efPdSAv2qYZPnka kzi7PZcZeWJPONEWxPcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUy-0000000GcqC-0lYb; Wed, 08 May 2024 18:58:40 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUJ-0000000Gc5t-35Vt for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:58:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kMidHUhtRzeJ/t7mCm/b8xyhLofsMviwCIzas3SGIbNihQJKJ/lFX3LBv+hroaeXMTOaOnGrXEOxVXf3JmzmK+vOmB73VUFCDn+7vuDc4nveCaUdHYkc7BwyIZqfDtZzFu59hdi4aGms9XT/L+UCZa7hw+bYkHD3ejPThK/EJOJnXM0zvZKtoPg7uisZl4e9gUDBmtlSIoX34EP9KNfDbXvGpPW4TuSDaorC/ABo1Jf1fbgVdMBn+xRiJHc3Zh8iZPlkz0aWoZoxmJp6IH+mi87DTzhvkdvp/MMokc9Ie4cn2f1Asrz6u2BycyYWs4h6O5vglMH3jUcwjOJOP0mhqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7Yd2J89cHeZ+kwbtskl9Z/l3Yrj8Lj9Itg60NrwJzs0=; b=B7bE3cDy83ibQ8fvlURu6NdQ8hys2k253SRVp3xozxfeT3oNJ9He3pgVS7RmUEg4wruLDWN2Vw7SItfbKC054Ir77nI0QBo4kjax17R6gFO72HU+makbqjVS3950sL9plHIO/0Gd5/Nyfw993pRI8e0EpF1w9fx3+VHFDZmzWz0MSLAhUfWamUYUSz6nay6/aFP0sCexVH46o4lk7fVzASA8/6ag4AsiJM42xCt7A9igQIIB2fuLcdZXHUU6K0HcOdbBUVcdFMpFlKKqwvG45FnCBl+3N8ahvJ6P8HERx/IG8JhX+4mGxO1fpEhVr96vV0WRgU5Zp+ifh//QZo3GoA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7Yd2J89cHeZ+kwbtskl9Z/l3Yrj8Lj9Itg60NrwJzs0=; b=Yj2F9ciDgG2BJfzG3HqSHPufMpUJeaBUbx4aHC9jq7MnkqCYNlIVrR+wTy39F0k6xrtZkCxmljM2WEQEnIXCq4/tSUBH/1FifUDrLfaXzwlRi6npjT7o4xlH8piyMM+J0wb/y19Lx21ZrT7Br6ow59OcdI3ersXHJNpRzn25JcKHTgGWXmZvAqj211+FVjHPtJCnqofI8RbOvhbqXz/89po2juN26FpVH2Tz2junLGsTF7oheK5tcJkCLf/ze0q2JPH4lRPgZetV3qszMeCPKagiPcxZywu/7qr7CFeLssggOFtwJRopVC3wdEfHXaU0Hl+RZ795fz/R2X7H7dc5BQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:29 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:29 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 13/14] iommu/arm-smmu-v3: Allow a PASID to be set when RID is IDENTITY/BLOCKED Date: Wed, 8 May 2024 15:57:21 -0300 Message-ID: <13-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR03CA0027.namprd03.prod.outlook.com (2603:10b6:208:2d::40) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: bf1c56e4-cbcc-44a0-86d4-08dc6f90b4be X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: qquxx29URg7maX3kZp5oEqG2VslmBtMISDTRQ5NS4T8tuBAa5j+zBw8IiPleUueAtEuZ6qRdFiF4fpfshyCToA37byl7n6rv90OWlQvSFWAWXHX0MGjzlm1s7y5rixS7zNcJEZJ/3tXqxMKDZwKbSa0iUSW2rzEVp8kKHYGBP5vHZPZvQFn7pOo2gKH3MbS7Xkh3VyVhg9hx0OldK6a1gFgWljUciz89eOcvcHZiQiha4j/GMHuHyPzODac63sGu6ju6Vyu6vd4/M3HMxRBtJhpjPKf5Z8OxxFttiyCXLQD/kkzFKyBq6JZ64WCYry1UbsW1uOmhLJUyh1nrCGvA7OxmrEj4DHIWL6cbab5nkWceUksffm1+NJNlnfzQcIrBCKfE9z5RbNDZSc5Rv8CXt1cciLS19gEYJYELGduXZeNpUczxFJpStFBAWcf/ptfbQalLaCa7UCYwZMDVydVEdYD63rrBILC77HxUAxX6G/zNAzEq6F//dmyOB5pBJritpquJAnYfA9kwqQoz6eAB/V1oUHFJLwcqgDL+1lKX65fOMoakCthtEZJ3Kn33o4XYDTvXIUa4gxcEGerF2XytQxeub52jnBdv+UVEnx7uC3RgFOeiyeDPY+UtHjkMGqdpBBBZu/cnbPwlw0Etc6Xe10YHmmi+Yyz7Gm3H4HjLOhOKjEbgNQ6zlj+vc0jyDjoSGk8lGsNDTjQeHAvyP7lPYZbh/Qd3QWJkj6qVGf/dk1GahYM3zWhI2wShdkzU1dNVfrFpKiqYjknaIW07Oa4777Py8KSlcgaELjbVlUOwJu9EVt56Jniq72dZM8RLmxmCvnnPzhkOY3uXGL5FVu49CFr89r8te2EnCepP2FWP1WiMH+HAftjQDAkySvwn3JXVDFtcC1cjmYEkYYlnhh7gn4TCy3BsJ7059+z7xdtxCi2fwpxZ273kWdg+vll8CXtVEpu0QPAvfdYlTFGt3vZ+9zF3JXLkQFQ9MVM0SS9B377DNUsm5Tsqs10/CfPp5xEhHWbdt05OOWgZXwDHhCuvqiGnz4nyaS+QX+7mvXaD+Aor0/5QJOlapLToUAPb0rnBnHPdeX2Czphmkbxbdpv2cPjO/CaTzmgcUQsGi5EBbvVhnu0lBX7K/jmd3nG+s3xon2nkAjUFpnfCwwJrGnGgtQfqFlA6MVNeHCt3iKOgGCvnTRjcwuj2fjmYdXUuBuq547fZWLxOvgDZUfwui64JrU4xla+I0eX1lsOIxoRNJN29X/jsq90kPtZvySTUf1OblBnGkTCIp+5X7tIaM3Mp/w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DpoS5/8Zng6VywqfMWlYCx6wmvoxHyn/g7H2fc9j5nW5AY7d/YquUrrhikCdy41UqVMj0bnLASWwdCjdEGjl0LhwzsxPE6UK62qvl7H4sl56QNI/lk7dEEQpqoiN1qb+ZBtjWOIIfNJ1PEjgO+5E6U0QAYxi4lC68zMyaq/wEcfO0p26XBF8XCLbjsXBE6gyj8YT21jkgr7zL0LmX8UyMkuDHfCZo6jgrZZDrEk3SsdPBGJxOsZsIRBTLYskX4ZA6HjeBbHz1SSH0ytCD4SgzhKe/dvzQX+w4PPLFW8S0miM+2R/ky2zF6bL1kg2iEOoaV4Mu1KWNFz1yRqRi68Bt7cgjNeWGSLSIS6guqxoAZ6pWUH6JSzwuUMtph6sL5oryGhAZFjuSlSPWP+8S3b82Tt8DOMGyBEieYc9yUInWJl7CrGXkU0WWwor4lSN6VV9rWg0brX8MQCnByO3/9Vu4IjENcHbMf1Bm0m087VRGl0Cj1VUNB+TSiP8B/mYeZ3Q0mrIDp/3W7OK9hLPAYfrHMxFdoRZ2H/EAotZWlFR16y2mxu+9VBsU8+w1FABX2xsHOAh1Icgz7v7M/8JCZMZg7InXyIkiKw5fEichznwZJHzYR94s/XUjUpjQTd8A7sfwTk14cvJFpZOAC9ZmqUYRqcx4YAcLYjyZBhMFLr2cfwA1IFKqfda6QGbmMIDiAhkGvBscwcRqy5HPcbuSer220HAVhylj4brtmFk0XvX9MkoFUZalkZT0kT4MnWtJl9tls8w3QdpuGPPY4fCQtADMdTLchP7E4NIb6K/tSLEXvNr/K0VgYUxvX5Y0QJNZ2a/PAPKQ0i6dNk6wc52oIVJ1yc9FPJOH7xdPJVSdIvo1nQuW7hvYuj6/HVyW0E7l5FX7mz3XfXGeeNHXt2nsq9b8sNtT0TudwIKHWCoanoy30rbHlW2ZnJrBbwaaC3077kC+L6vO/Af49SrUi2JTTBAApFPhnTEUeC0iGeHyVps5w7quaTOnKwxjmFOb0WbEDnoR5HYFBb8IM9LedBIsKP5n18PdIlZ4brCa912Ev4vQ5uGFcnUCd/EttBj/S15clhryaazAUcUVoe0p2UzVHVBZVaipAbmh8motdhSC9gHHFFtbZ8yY0cCapbviIVr3GcBxspuYXkJzORzcfLDUE0Fp2iOgdQhMwyGYa+WvH3bvMkRy6VwTwfcSWgKGDNETNWLXGV2sUm8ypS88mS0rw0in9nU4w/4itWSkULU0sbF2tEvcihAr8gh7apqzRkarmBLRJLD3kVGIki5gGgBRB60OFxIlCADpTSeQVUeeSjNandYZEsg0mQYYJ2hQYQD2L74CjAGgqBGMH9DPMUK19NkhaxOEf7S9SDnTlAb/zoBXIyyVQywX5nmgjg/1JnVLSKW4HyyHe4EcEu+dnQ1Icp3+nCWe2pu+T7XHVhHCMxGVDWt8QRI+U5XtqofJbv6GqOhEqv593OpjOiTblpPdIsJVD1TE7LlJOB5+33b02uOkrZXTSMCK2wGmPPX59VX/tqH5Jc8FF30zK+2mMdUcofyNyyxAh74o5b/fIE5FQO6u/hZnQK2iwahpfgA7wP5RRbY X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: bf1c56e4-cbcc-44a0-86d4-08dc6f90b4be X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:27.9222 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2fdFn9ZXcrkwJ0pg1MueKuLzrU7rdihs5Hx18hkiLidBgrm2LdGzgIa+ZC2mMWDs X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115759_963700_6C8CC5F1 X-CRM114-Status: GOOD ( 22.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the STE doesn't point to the CD table we can upgrade it by reprogramming the STE with the appropriate S1DSS. We may also need to turn on ATS at the same time. Keep track if the installed STE is pointing at the cd_table and the ATS state to trigger this path. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 49 ++++++++++++++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 +- 2 files changed, 49 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 41d7a0664a445d..ccc10ad3ab996d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2435,6 +2435,9 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master, master->cd_table.in_ste = FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(target->data[0])) == STRTAB_STE_0_CFG_S1_TRANS; + master->ste_ats_enabled = + FIELD_GET(STRTAB_STE_1_EATS, le64_to_cpu(target->data[1])) == + STRTAB_STE_1_EATS_TRANS; for (i = 0; i < master->num_streams; ++i) { u32 sid = master->streams[i].id; @@ -2765,10 +2768,36 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; } +static void arm_smmu_update_ste(struct arm_smmu_master *master, + struct iommu_domain *sid_domain, + bool want_ats) +{ + unsigned int s1dss = STRTAB_STE_1_S1DSS_TERMINATE; + struct arm_smmu_ste ste; + + if (master->cd_table.in_ste && master->ste_ats_enabled == want_ats) + return; + + if (sid_domain->type == IOMMU_DOMAIN_IDENTITY) + s1dss = STRTAB_STE_1_S1DSS_BYPASS; + else + WARN_ON(sid_domain->type != IOMMU_DOMAIN_BLOCKED); + + /* + * Change the STE into a cdtable one with SID IDENTITY/BLOCKED behavior + * using s1dss if necessary. The cd_table is already installed then + * the S1DSS is correct and this will just update the EATS. Otherwise + * it installs the entire thing. This will be hitless. + */ + arm_smmu_make_cdtable_ste(&ste, master, want_ats, s1dss); + arm_smmu_install_ste_for_dev(master, &ste); +} + int arm_smmu_set_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid, const struct arm_smmu_cd *cd) { + struct iommu_domain *sid_domain = iommu_get_domain_for_dev(master->dev); struct attach_state state = { /* * For now the core code prevents calling this when a domain is @@ -2784,8 +2813,10 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, if (smmu_domain->smmu != master->smmu) return -EINVAL; - if (!master->cd_table.in_ste) - return -ENODEV; + if (!master->cd_table.in_ste && + sid_domain->type != IOMMU_DOMAIN_IDENTITY && + sid_domain->type != IOMMU_DOMAIN_BLOCKED) + return -EINVAL; cdptr = arm_smmu_alloc_cd_ptr(master, pasid); if (!cdptr) @@ -2797,6 +2828,7 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, goto out_unlock; arm_smmu_write_cd_entry(master, pasid, cdptr, cd); + arm_smmu_update_ste(master, sid_domain, state.want_ats); arm_smmu_attach_commit(master, &state); @@ -2819,6 +2851,19 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid, arm_smmu_atc_inv_master(master, pasid); arm_smmu_remove_master_domain(master, &smmu_domain->domain, pasid); mutex_unlock(&arm_smmu_asid_lock); + + /* + * When the last user of the CD table goes away downgrade the STE back + * to a non-cd_table one. + */ + if (!arm_smmu_ssids_in_use(&master->cd_table)) { + struct iommu_domain *sid_domain = + iommu_get_domain_for_dev(master->dev); + + if (domain->type == IOMMU_DOMAIN_IDENTITY || + domain->type == IOMMU_DOMAIN_BLOCKED) + sid_domain->ops->attach_dev(sid_domain, dev); + } } static void arm_smmu_attach_dev_ste(struct iommu_domain *domain, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 71ecced1db0226..3c189a48e29757 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,8 @@ struct arm_smmu_master { /* Locked by the iommu core using the group mutex */ struct arm_smmu_ctx_desc_cfg cd_table; unsigned int num_streams; - bool ats_enabled; + bool ats_enabled : 1; + bool ste_ats_enabled : 1; bool stall_enabled; bool sva_enabled; bool iopf_enabled; From patchwork Wed May 8 18:57:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13659065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0C2EC04FFE for ; Wed, 8 May 2024 18:58:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1MfI/hxsjkpIP81OLr+kHlKgGRvtUKNTTkMrwmuIN/k=; b=VU4B+Yb2W7hjVT LHlMZe0Eu5fjPdEYAiM5ldAPkrvuobVL9VuBQ1yaBh7A+znn7paGbXhHq7OptqzXBccHQy2Maa8ln GI2FtCkWmQa1orKyhEtPs3uIjwOXPIvocGg86g1NMjC5HoYm/UOR8pHOMVlCpdaybkzOv/UED2phe TZ/jynTTdwdvEZXMRuw4HTKBAZ0453aC8dORo0p7XritH968q3yXbKcuJ6k44yCfmKnZhNs+n+QJA zSNmC5AUHFZvRYfSv7jA9pklQenQls+Wg7PsR6S3ETDIqp9u/i6SFrjs+/COalxRHfzEOCMrpdg2P 7P026bl/Dhvyk2pJW8Cw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mUB-0000000GcHh-4AS8; Wed, 08 May 2024 18:57:52 +0000 Received: from mail-dm6nam11on20600.outbound.protection.outlook.com ([2a01:111:f403:2415::600] helo=NAM11-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s4mU0-0000000Gc8t-31il for linux-arm-kernel@lists.infradead.org; Wed, 08 May 2024 18:57:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e8U6IqF7IWAm2YpN9Y7LPW2kDNIA+4mzRbdRC9JjByXqfF4SC/NTaLTAhDBnXanVaUchwzbZAr+Q68w1/3CilYqh+E8tLm47FLBg1FzD1F/MHN95gCZ4WipZHQdt8SZHxmORTHvOrARMXSwdMt2SAlR6BJFscYNh0jm2OPLeMmxCOUI2rOrXSvHmND8fIzD6iA9/qde2x4TdogEb0je2hN9JMfKOrOZj4kHM+w2S/nQH3yt2+LzztZMNgjBQxJ4y/uHE6+/jG2aCazAFmW+y6GWFfht5TcK2sp7Ge+Bpu9cuWbY9Jdadf4u41Tly7Ui9wxe4bO4kekk7nZBSksAoiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w0XpbKjKEw6H9M+bLyl+7sJR2dx4lSw7magdLibEy8U=; b=NP1PTGX8yqsKeEbRXttZ7FLlVD19RAz3MI5/lpj7y/XU2g1iMVuPkqNpdtp7YHD55PBtV/ZYqivY4Ed/tEIMnt1QRhPL0uzC91sOznkuWiywtX82WqqDGxVqj4sEA0AqW/9P63rkVW/Oh6237kzohhBJeS8paDnefhIttMydKm5UTCnqfARYNgAzH4Yt/81YDmiViPmR62wmzeaRIGXQx24CEA1JHxbV30ZG/6jyPnjVEk7jtIIjs0JNhQXL8tPlLcB4YAGWySWDWujYsPcC8IcsSbzd7wZ3ePlMGNsI0nm6Nv+ZSURHEFTPkM4bvlFJ89h85Lwa2J1Ys4Om9hF7ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w0XpbKjKEw6H9M+bLyl+7sJR2dx4lSw7magdLibEy8U=; b=nI9ejoMxqp6RRdCpdkmRPOX7hJ3fwXLXxSAqF1ZAqrdPqz/SoHPqn7RzWtJMfU4/i1W8K7xJhTzGjuq/gFqRFCyrZMQnMVixfH/jwImp+4pX7YIA3gU7HxWEauvsz3sXNjrLMwM7gQNNmlcbF6fhLgsA7vLIKma9PhPKiJoG3nN+KAjGh0fLvTwbr/sh/QZ8JDln3sLJfbk2egQpZv0HF99l05P6252oyjJ1Fz97vZv3osrjUAD/SVMBHV/wZLB+uRJKpJHXr5YMn32ol5iA6WZlKtlkQJgMEGzS4L/QhoVub4XwFBEubexJpu1nOaKbhLV2svo6e+utGTh7V9woeg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by CY8PR12MB7514.namprd12.prod.outlook.com (2603:10b6:930:92::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.46; Wed, 8 May 2024 18:57:25 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%3]) with mapi id 15.20.7544.045; Wed, 8 May 2024 18:57:25 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: [PATCH v7 14/14] iommu/arm-smmu-v3: Allow setting a S1 domain to a PASID Date: Wed, 8 May 2024 15:57:22 -0300 Message-ID: <14-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> In-Reply-To: <0-v7-9597c885796c+d2-smmuv3_newapi_p2b_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0261.namprd13.prod.outlook.com (2603:10b6:208:2ba::26) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|CY8PR12MB7514:EE_ X-MS-Office365-Filtering-Correlation-Id: ddb3c262-98a1-4ae7-d0e0-08dc6f90b293 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|7416005|1800799015|376005; X-Microsoft-Antispam-Message-Info: wzEyANIb6eEMQnWm1d2PSacIGDDpYM61TKNfnsd6KVE0CxNJ6hmhKNejcVO0oD+dLMZWzvLagP12rKu1WFQa4NRpNnSgBQJl5qxqU1HMqiHSHDdvNlA7NQOA81w78gyRO8Ip+OXsZkBiKFuccE0xjVrqdGOaSx6tBLmoT8Gq7CqXL0I+8zSUnIcWDkDJZMEx7xYvX4iYxKMPi76P1QwqOcONkJ1FHjv696tbUMrMJOl7w4p4Advm0Rif1vDcd0/8DlgWz20nNtEjQE0ArSsskk/BCKqPO3na+9EuSLBOEJvkZZHQILHfh2rgNaniKYagm4Y9JjlJ86NGJmbMQmlLAVkaX2dRFD+cnqxttzaDM21gyWpt0Sy6oYfeQYR4FsA8d90iybQT45tlqjS14hNmMxCV1HHKhJikwn09Q91cTMatKs7mpQXmboamFNaEMbvJNwZpiJVLGQFCB5F4YsQjKl/X9nLguCKJ0zc4cuBkM7SW/4fFhQJ4nPxqAQAs2DZmZ7LCY4w2iHcn4YAuztO8ZZLQU560SLpE6SDFojYKDwMJbHQcvDkF0xkMRf/T4nZMQTCloihn1GxNBaeJsirigC4oMBHXBrWjM/mPSxj8Bkvx5VOJsm9tKabKJDk71dOWqvYWVfwQR61VsUzKVOyyA7eCJEcqs4UDAUVyhe0z5mdNET8SesdkswkC8Z/AgDvV+wZg+EdklOaUA9eurtnBra6dcXU5LGn1FtIi88WqzUT32yrP0FDEigvKmJ0Qk49ANQYnPzR7RmJP61j4O365zQQCTiCIcT44QI5gXOZXdyK+wGEYr3QbbMnrm3a5F3ovLv78p2En5HyDX9x58vk3etc9uy7iqTaJeXHCzJMFNcX3zmFBnz2IvM6ZSoB0w71g6rpCNkEzpI2bT1+5oWTv2kRdCThB1fWrBgvNno2Nwu4fepMTmxui4m+Pwoo11McO5dgll0tPqVPMc9hQw6XWDEi3LuyjzT8jUlRHnvf8cmpNSiFFpalfjviQGPyaAoWcIpOgvURkjuherncrG1cB4wtFDWamevdRD1Y3aHARN4QueW2pw4pjnL0//mGZB5x3YQJkrvR7jDHXNenQvb6Lvt1pR1oqY+yJlOg0MxVBoNuVKOv9avQkNo/E9g8/XQgC+4MBTE5++u2Juroa+FLAwQu2DaR7hIrAO3MkfnHsfKtN/b8yuPH8zTQO5b9lRVUHqTWM8DeC/dPLs/61UeNqHoUXtHoqjnC98sx+3pxZ9JKuUqY5YNk/PAfEkTqhxcmKrYuYBZG6TMWCsNPs0QpFLQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(7416005)(1800799015)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: yyhEIrg5reKg3VkfpslBMrkYk/u2vYvd62G7QilZCCXdNH+LuSk5upMrzt6ooq3TU6o53wnJfnZfSb1r9xyl5zf4CH/pec0064+siCAEhrKa+f2Hc6ozBcp7LkOgHjzz0OurO6eYWUYsXdCoUM6FDZ2k0DvHBRFeCjXGXYfxKQ0v4PoFFB6VzPYtHgQqowzWxxVG+eyKYupAeg5ZoYZ4ZkifikMwHG5Qxf7ok+jQn3BJxHFlDzb6QaV0PcytrNS4l0dvIvv8pyJWmF8dOu8cwCZlUAtvaAmp4RcFwWnHxvPduWu5W9xSo5/d1dEphnCT6Ku+y6kgqzUfXZCZdXVeFtFahdHajvT3EFBqn0RMAz+u5/vPoKDUcI94UiEw4oM0HHL8wB4DofAd8v8z3lkmLNZXUI69TRA6ZjXN7VtRSkK571mTjIAJzLePm8AyJ/LL57x6fL9+1W2tcoRqxRFyxBm7cjSh1fXqhb8cuTX97Ryze2rza7EeejguTaPFourbVz5jvRc0gjXNLQ7i0F0t//wX7T1UL9OJ7AwiIO5bDhsgT2Uk9ds6FdgqlmcGVwD09xvRTxwND2JKcXN0z2cDSRCOC8fjRN/n2PL6ZoLGGkftO2VQ8kY6u1FpXErvU+F0GhhG/PE3cOAdjsdixSao5aMg8Bq/0CQ8MNnffa8GQak46xdwvTflzA2QPX1/O391W5ivPKtupjc+X5nk7yAM/FNNSWxZMNnIFWdszqrVB04+uQ3BXgBc2L6LYrgeDMNZykS/MZFUpG8h6UdGDAzPEFqVQflDStGsc019ARVwMa8f8Fo38rsaEVUzoW4+TjeBI/SDXxTUvbGc/I/JbPhS/JRGsAy1PvEOGwoHfD5NKXcDpqJZPDYkQTvpXaae4mWV9/DDw7h4f4fiRwG7U+zgNuYVWIdUDasdTC0yGNgpOuec2BQh84b0svz/LO/1sjuACcynb/LLw+0TC9Pv2mOoIwECeXh+gm9Zd1BZYDjK6bPy9bINx4EPyh5zRqgpKLJhbvglsDgmkQ8vvtxN2eUTMYV8Nd81PlU8V+CuxZ5zE3WKMf62IoQlIi9/BGGiIhDLkoGHf6prOZvMv+fRQdwWIOmzSJSSrjmAACqK1STCr101OhWb/WpZQwxYpb1Ko6ZLYw7n1vN312pWzisWc05+a6oppft1ZDM7i6ZRnVkQRS3Njs0LMEvy3/NVgG1GomDSWStl2rcfEyXr5srdCFj9to7aRco42JhAIqvOg9IslaO0SC5aGXAwVO4yONQ/NfUZg2BkD4qDVrXSoPBxcYH3BqjEoRQhXD+XwcQ9f3tcij1IcXm9OHEys0vMv/WZmqfj5MRSENspfoWEM8YHMc5bk8Wtu5WVXPdPyH+DLirO6y2VeirPxFlFx+7ypEHEavcsJHTu5Ultd5k7q9HRRtGsm3Dyazp4qH8aYZZ5C/x95Sde1gSZcIftBQ0e9Sp4+8p4BO6bL0jnwfnBpif6i/9gvgQDjKvNJ1L/eKSjyYNYAIKVVhT8unyFYGqLfy1uJTgcNbaVYQz8ScruMs0S+O+KjVbXMUs5NuKRKw47Rgbq5sSIOCsKzaTwL6H/J2jWT4/v X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ddb3c262-98a1-4ae7-d0e0-08dc6f90b293 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:57:24.3794 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Nj9fbZ9Uc6gDIGbACiR1MBaakdBgdunUA05PUv8p0wNkXeZP4nxUmKWpfMkNRMIp X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7514 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240508_115740_869771_BFEA2632 X-CRM114-Status: GOOD ( 17.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The SVA cleanup made the SSID logic entirely general so all we need to do is call it with the correct cd table entry for a S1 domain. This is slightly tricky because of the ASID and how the locking works, the simple fix is to just update the ASID once we get the right locks. Tested-by: Nicolin Chen Tested-by: Shameer Kolothum Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 41 ++++++++++++++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index ccc10ad3ab996d..9506073678bbcb 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2768,6 +2768,36 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return 0; } +static int arm_smmu_s1_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t id) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + struct arm_smmu_device *smmu = master->smmu; + struct arm_smmu_cd target_cd; + int ret = 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) + ret = arm_smmu_domain_finalise(smmu_domain, smmu); + else if (smmu_domain->smmu != smmu) + ret = -EINVAL; + mutex_unlock(&smmu_domain->init_mutex); + if (ret) + return ret; + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) + return -EINVAL; + + /* + * We can read cd.asid outside the lock because arm_smmu_set_pasid() + * will fix it + */ + arm_smmu_make_s1_cd(&target_cd, master, smmu_domain); + return arm_smmu_set_pasid(master, to_smmu_domain(domain), id, + &target_cd); +} + static void arm_smmu_update_ste(struct arm_smmu_master *master, struct iommu_domain *sid_domain, bool want_ats) @@ -2795,7 +2825,7 @@ static void arm_smmu_update_ste(struct arm_smmu_master *master, int arm_smmu_set_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid, - const struct arm_smmu_cd *cd) + struct arm_smmu_cd *cd) { struct iommu_domain *sid_domain = iommu_get_domain_for_dev(master->dev); struct attach_state state = { @@ -2827,6 +2857,14 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, if (ret) goto out_unlock; + /* + * We don't want to obtain to the asid_lock too early, so fix up the + * caller set ASID under the lock in case it changed. + */ + cd->data[0] &= ~cpu_to_le64(CTXDESC_CD_0_ASID); + cd->data[0] |= cpu_to_le64( + FIELD_PREP(CTXDESC_CD_0_ASID, smmu_domain->cd.asid)); + arm_smmu_write_cd_entry(master, pasid, cdptr, cd); arm_smmu_update_ste(master, sid_domain, state.want_ats); @@ -3352,6 +3390,7 @@ static struct iommu_ops arm_smmu_ops = { .owner = THIS_MODULE, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = arm_smmu_attach_dev, + .set_dev_pasid = arm_smmu_s1_set_dev_pasid, .map_pages = arm_smmu_map_pages, .unmap_pages = arm_smmu_unmap_pages, .flush_iotlb_all = arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 3c189a48e29757..c32ac37c0b0e00 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -801,7 +801,7 @@ void arm_smmu_write_cd_entry(struct arm_smmu_master *master, int ssid, int arm_smmu_set_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid, - const struct arm_smmu_cd *cd); + struct arm_smmu_cd *cd); void arm_smmu_remove_pasid(struct arm_smmu_master *master, struct arm_smmu_domain *smmu_domain, ioasid_t pasid);