From patchwork Thu Apr 25 14:41:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tanmay Jagdale X-Patchwork-Id: 13643455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8E273C4345F for ; Thu, 25 Apr 2024 14:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PuKIWFUSSLyeF828GJC+Hf2svf8YRybBnRdzkPEiLrA=; b=E5yFAG17pHtCdK ect6p+GhgTF+ZZ3d/a9QZC8O8EtKaxoo7dh5LcG3maJfsWBSze+ivSLGjZIrgBZKWRcx8wg8f+iym vyD1LBX8Ok89bYR1lGpWwQeXG6DzrhHm8n5uT2FM9gWFT8QqP7vVigSA7GaSiO4zlfqQnDYlqALYe v4UAORSqe3jpekWgEUBXJMeorTC+llDQGapY0brI6GK9T4poJsIsXVT5UgL3faRR12uKCJmVj/5fX fK15NOVw0qm/PDadFtGRIUHHmRNSIP7pd0/AktH1WOYnpcTDJCGB+2ZkDF/AYzSJrJPpbOU2XM9RJ gH9DJcndzXn3Mk+f2ezA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s00JH-00000008tYy-1Hmb; Thu, 25 Apr 2024 14:42:51 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s00J9-00000008tV1-1Z7E for linux-arm-kernel@bombadil.infradead.org; Thu, 25 Apr 2024 14:42:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Content-Transfer-Encoding :MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From: Sender:Reply-To:Content-ID:Content-Description; bh=jYw/vx1k9Nqv/Lp9uCDaV7RJmIlPtnQxIERYqC1PzH0=; b=khgtvgVQrlvCF7S71Babu1a8sE mt006dlswkkkhMJR4OKfZtMnHmqLwFB+kWDvfhr4c6W1Hd6s0+IMeDokfbK1YXWT9KMvf0YkboBz4 WSRtBrFSr9HdSTAns4KRFk1vtPPpGWL5UyMp2VGeLLETq27eB62PP9St8WouxjZdordJ5b2j6LdHM QgXPeXKzx/NDCvkyhCndTyndBthYmtm7n5VE4I00V41TPXqQtQp0x2pl8LEQ+PZl3VBqwE+fIQ7YQ 4e4JqnHutBgjHLWBE8Jg16qpYVEMM6SOx/Go2UXMCQvxpd0EcsZFhsY4C6WQWAhuZruvj6KFtqqdu qziJk90g==; Received: from mx0a-0016f401.pphosted.com ([67.231.148.174] helo=mx0b-0016f401.pphosted.com) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s00J6-0000000ExEp-0WOT for linux-arm-kernel@lists.infradead.org; Thu, 25 Apr 2024 14:42:42 +0000 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 43PA4Kst007359; Thu, 25 Apr 2024 07:42:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=jYw/vx1k9Nqv/Lp9uCDaV7RJmIlPtnQxIERYqC1PzH0=; b=L14 jPM1h9DgTAjTheFhZhK+rlCkoeiyhZ3hcSnE88+SysQlHikYQ2DsN/elhSOGLjjP KtLs6r4l5Y8rFBV2L2kIHHfHPoEYBG89d8z6jXPLJgDkDqTAxLXHHYV9WEMDOGSz TEid6Ul8loPHurpScZfRxW2zLrDjW5uJuPt1JgGTgou8E+k80UA4e/XYlbsACxTD b+ejE1V3v2UEh6P7Fm2JO5k6YdSLgOzwIPpFXBKx9Ooe1lG7KQa53BTcY5rbeNTB OTL2Mkm5yzxdVdvZWe9BdN9/ffU8cQsUcitqyEJuVuUCGnCtvLste4FaM75VSSAW 9BmbEdKRM290bvSlkBQ== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3xqn3n9b37-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 25 Apr 2024 07:42:25 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 25 Apr 2024 07:42:24 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Thu, 25 Apr 2024 07:42:24 -0700 Received: from cn10ka.sclab.marvell.com (unknown [10.106.49.40]) by maili.marvell.com (Postfix) with ESMTP id 7DF713F705F; Thu, 25 Apr 2024 07:42:24 -0700 (PDT) From: Tanmay Jagdale To: , , , , , , , , CC: , , , , , , Tanmay Jagdale Subject: [PATCH V3 2/2] iommu/arm-smmu-v3: Ensure that a set of associated commands are inserted in the same ECMDQ Date: Thu, 25 Apr 2024 07:41:52 -0700 Message-ID: <20240425144152.52352-3-tanmay@marvell.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240425144152.52352-1-tanmay@marvell.com> References: <20240425144152.52352-1-tanmay@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 6t4kvIEkRZw-ys-5Yz0J_HM1ajFbssZn X-Proofpoint-GUID: 6t4kvIEkRZw-ys-5Yz0J_HM1ajFbssZn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-25_14,2024-04-25_01,2023-05-22_02 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240425_154240_612964_17A4F2C6 X-CRM114-Status: GOOD ( 18.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Zhen Lei The SYNC command only ensures that the command that precedes it in the same ECMDQ must be executed, but cannot synchronize the commands in other ECMDQs. If an unmap involves multiple commands, some commands are executed on one core, and the other commands are executed on another core. In this case, after the SYNC execution is complete, the execution of all preceded commands can not be ensured. Prevent the process that performs a set of associated commands insertion from being migrated to other cores ensures that all commands are inserted into the same ECMDQ. Signed-off-by: Zhen Lei Signed-off-by: Tanmay Jagdale --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 42 +++++++++++++++++---- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 8e088ca4e8e1..d53b808de03f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -248,6 +248,18 @@ static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) return 0; } +static void arm_smmu_preempt_disable(struct arm_smmu_device *smmu) +{ + if (smmu->ecmdq_enabled) + preempt_disable(); +} + +static void arm_smmu_preempt_enable(struct arm_smmu_device *smmu) +{ + if (smmu->ecmdq_enabled) + preempt_enable(); +} + /* High-level queue accessors */ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) { @@ -1229,12 +1241,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_master *master, }; cmds.num = 0; + + arm_smmu_preempt_disable(smmu); for (i = 0; i < master->num_streams; i++) { cmd.cfgi.sid = master->streams[i].id; arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); } static int arm_smmu_alloc_cd_leaf_table(struct arm_smmu_device *smmu, @@ -1979,31 +1994,38 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) { - int i; + int i, ret; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; + struct arm_smmu_device *smmu = master->smmu; arm_smmu_atc_inv_to_cmd(IOMMU_NO_PASID, 0, 0, &cmd); cmds.num = 0; + + arm_smmu_preempt_disable(smmu); for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(master->smmu, &cmds, &cmd); + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); + ret = arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); + + return ret; } int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, unsigned long iova, size_t size) { - int i; + int i, ret; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; + struct arm_smmu_device *smmu = smmu_domain->smmu; - if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) + if (!(smmu->features & ARM_SMMU_FEAT_ATS)) return 0; /* @@ -2027,6 +2049,7 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, cmds.num = 0; + arm_smmu_preempt_disable(smmu); spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_for_each_entry(master, &smmu_domain->devices, domain_head) { if (!master->ats_enabled) @@ -2034,12 +2057,15 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, for (i = 0; i < master->num_streams; i++) { cmd.atc.sid = master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } } spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); - return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); + ret = arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); + + return ret; } /* IO_PGTABLE API */ @@ -2104,6 +2130,7 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, cmds.num = 0; + arm_smmu_preempt_disable(smmu); while (iova < end) { if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) { /* @@ -2135,6 +2162,7 @@ static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, iova += inv_range; } arm_smmu_cmdq_batch_submit(smmu, &cmds); + arm_smmu_preempt_enable(smmu); } static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size,