From patchwork Thu Apr 25 14:31:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tanmay Jagdale X-Patchwork-Id: 13643450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1D91FC4345F for ; Thu, 25 Apr 2024 14:32:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:CC :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=PuKIWFUSSLyeF828GJC+Hf2svf8YRybBnRdzkPEiLrA=; b=TUqdstEH01L+RN nXtj0HiEqXSG+JGT5VIPmQ+4OKSbFFB2DqPQbrapX6YR01f/9+6GZb5HNp5um9o0WpDdBm71EMjYC GEpx/9tDh2InTh7oHhvgd1ZIIG47BLOEbb9m9oJgco8YtvH79tMdCKJrCwL4g0sEyx6FEfscB29Hj 3w54+blj32yRFfxigUpM4oN265n6qsLb1K3F53w8JtX0pStJRwzp4gUNUe8Sv89/axCfwYMjRFnjJ MgdE974/99zX9YsPwL/H0MyJKcngj50GI3yWPDzECNxwasU2kVfT+kqYIa29Yl7G8vCDQI3HQ4au/ QHBvFlLtKSyKO1/4d4rg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s008a-00000008qIh-3KtQ; Thu, 25 Apr 2024 14:31:48 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s008Z-00000008qHe-0vFb for linux-arm-kernel@bombadil.infradead.org; Thu, 25 Apr 2024 14:31:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Content-Transfer-Encoding :MIME-Version:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To:Content-ID: Content-Description:In-Reply-To:References; bh=jYw/vx1k9Nqv/Lp9uCDaV7RJmIlPtnQxIERYqC1PzH0=; b=FKepQuxHBdBfd0Z9sVWTItLWAw 8AByVZwbw0bkgsw2y4Lr9367FZTsGlPVYuZLMeMDY7aNei137D5rbptdE6VlBJLolDhoPOC/jzFZb UmxdKTn2Ga1kqgtValsJvAo/hBev22bY8QAbgTLVYSRp6oQAT9uGAB04IEhCLFDUpPiI3Y5Hii0TD mbjfaxIZvoTfs91kU0nkD701iUAFGKgw+UuT9dYVpDybStP9/iDDGiDE6ahtVBJy/eilDzCyu8cl7 PTDz/yjADmeHWUdWYGpNBC9r7ma0h4kY5XtD3WCnYkXBw1cNMJ3mqR3xqMNPPo/RxMNMZTC4gLlME sVaS7j6w==; Received: from mx0a-0016f401.pphosted.com ([67.231.148.174] helo=mx0b-0016f401.pphosted.com) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s008V-0000000EwPq-1LKU for linux-arm-kernel@lists.infradead.org; Thu, 25 Apr 2024 14:31:46 +0000 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 43PA4K4g007365; Thu, 25 Apr 2024 07:31:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding:content-type; s=pfpt0220; bh=jYw/vx1k 9Nqv/Lp9uCDaV7RJmIlPtnQxIERYqC1PzH0=; b=iFlL/d07xHnLlF6tVzlghQjF f73rphVOU1ed0SOutDmxchgprcWpy8g8YBo1UYrNqevJOqjdy9XFF/e7J35YXSz3 Ji7PgmwOYmS3MmWULKSRcrskG0LWJ6E/T2yMkMhCIIwzdF/eBWpUsYJLmFjhcHs+ 0RaVEGrYPflOV/VN5Gtu9Ei7a0/rAFeZIReB+43sDYfu6pT3bb88DFr9TYjdTC7S gSJu5LXVHyjxx7VkQ7V8WicH0P1RSdwGl755foM1iUWnKLlUJw16BWUW/Y1t102X HQje/weHBCW8iBgRS+/MOtAyYhN0LpdAc7wWcNMHjPNxEIZWvUjLq+5WFCenGQ== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3xqn3n99b1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 25 Apr 2024 07:31:22 -0700 (PDT) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 25 Apr 2024 07:31:10 -0700 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Thu, 25 Apr 2024 07:31:10 -0700 Received: from cn10ka.sclab.marvell.com (unknown [10.106.49.40]) by maili.marvell.com (Postfix) with ESMTP id E7ABA3F704A; Thu, 25 Apr 2024 07:31:09 -0700 (PDT) From: Tanmay Jagdale To: , , , , , , , , CC: , , , , , , Tanmay Jagdale Subject: [PATCH V3 2/2] iommu/arm-smmu-v3: Ensure that a set of associated commands are inserted in the same ECMDQ Date: Thu, 25 Apr 2024 07:31:07 -0700 Message-ID: <20240425143107.52318-1-tanmay@marvell.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: olgSPLhUeyvLkQl6fzW8Zrn6MteoYZlz X-Proofpoint-GUID: olgSPLhUeyvLkQl6fzW8Zrn6MteoYZlz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-25_14,2024-04-25_01,2023-05-22_02 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240425_153143_817614_A99BEACB X-CRM114-Status: GOOD ( 18.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Zhen Lei The SYNC command only ensures that the command that precedes it in the same ECMDQ must be executed, but cannot synchronize the commands in other ECMDQs. If an unmap involves multiple commands, some commands are executed on one core, and the other commands are executed on another core. In this case, after the SYNC execution is complete, the execution of all preceded commands can not be ensured. Prevent the process that performs a set of associated commands insertion from being migrated to other cores ensures that all commands are inserted into the same ECMDQ. Signed-off-by: Zhen Lei Signed-off-by: Tanmay Jagdale --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 42 +++++++++++++++++---- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 8e088ca4e8e1..d53b808de03f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -248,6 +248,18 @@ static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) return 0; } +static void arm_smmu_preempt_disable(struct arm_smmu_device *smmu) +{ + if (smmu->ecmdq_enabled) + preempt_disable(); +} + +static void arm_smmu_preempt_enable(struct arm_smmu_device *smmu) +{ + if (smmu->ecmdq_enabled) + preempt_enable(); +} + /* High-level queue accessors */ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) { @@ -1229,12 +1241,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_master *master, }; cmds.num = 0; + + arm_smmu_preempt_disable(smmu); for (i = 0; i < master->num_streams; i++) { cmd.cfgi.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); } static int arm_smmu_alloc_cd_leaf_table(struct arm_smmu_device *smmu, @@ -1979,31 +1994,38 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) { - int i; + int i, ret; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; + struct arm_smmu_device *smmu = master->smmu; arm_smmu_atc_inv_to_cmd(IOMMU_NO_PASID, 0, 0, &cmd); cmds.num = 0; + + arm_smmu_preempt_disable(smmu); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(master->smmu, &cmds, &cmd); + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); + ret = arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); + + return ret; } int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size) { - int i; + int i, ret; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; + struct arm_smmu_device *smmu = smmu_domain->smmu; - if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) + if (!(smmu->features & ARM_SMMU_FEAT_ATS)) return 0; /* @@ -2027,6 +2049,7 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, cmds.num = 0; + arm_smmu_preempt_disable(smmu); spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_for_each_entry(master, &smmu_domain->devices, domain_head) { if (!master->ats_enabled) @@ -2034,12 +2057,15 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); + ret = arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); + + return ret; } /* IO_PGTABLE API */ @@ -2104,6 +2130,7 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, cmds.num = 0; + arm_smmu_preempt_disable(smmu); while (iova < end) { if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { /* @@ -2135,6 +2162,7 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, iova += inv_range; } arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); } static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size,