From patchwork Mon Aug 12 20:52:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mostafa Saleh X-Patchwork-Id: 13761046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5B92DC52D7C for ; Mon, 12 Aug 2024 21:07:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:Cc:To:From:Subject:Message-ID:References:Mime-Version: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6XoN+HaFS5yfD0CNRqQpWijMjbiOviXEPumC1NW+kKU=; b=gzg8bhW3AYDVtX9E8a/O+oacUg gM4LYKLNLkEHqRyxdDA6Kce6Si6xQfotzlNvlk3c99ut99QDLjXLL0jIf9A1AhGX38x0I8JIPE2e8 GkcP6T0QJ7o0i01qWR9UGU8om8/NZ7lss+kgQIc5GXI4USigne6bLDp8wD8fL2yD1LWrQBp10h/MT SAtfXO6ZaimSIN9GMd1v0MVf8ps4xSmu+rsXYpUGUQ15DE7qyYFCORVaWcFUkXkOFXMxSeXSQaOar GvQ7NQoxNUySCaiZP/D+xnDXnq/w3hodGZBQLlErAq4hqv6IGEDGR0VUXFkKRr6oVPBXy8qc3ixoK ZlnpPedA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sdcGP-00000001ZtC-2zKk; Mon, 12 Aug 2024 21:07:38 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sdc2T-00000001WKy-41G1 for linux-arm-kernel@lists.infradead.org; Mon, 12 Aug 2024 20:53:15 +0000 Received: by mail-wm1-x34a.google.com with SMTP id 5b1f17b1804b1-4280f233115so32329115e9.2 for ; Mon, 12 Aug 2024 13:53:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723495991; x=1724100791; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=6XoN+HaFS5yfD0CNRqQpWijMjbiOviXEPumC1NW+kKU=; b=mjed5z0zkuYI/C6uRz6OvJiV8KO2A/qQuN96O0RDPoDwhT/4n0O5PVRbS3H3TuhW+S TL6Eei+Pgfpmyv1OI+/xgdMjKMHqBK0sHMflp8hmeKz8L1Pp9g0FXJPu41SBpXCPztjR tKrWYaXyCIjMPhuLYvhpOkwrUVQFqsoNHi4/NIz4ftR45+5qwmP/zFbRW+HN1IScZC9d QjSg3zgcZb8ZAIYpKOGvwtmmE7BK+iSs4YVFWYsOH9c1wCROSGArZfQkX45TimoyZpBL 9U+q3SC6uG6s/UFMR9jSLJSxXMyVaJqKEAvU/czyd8iEvDFT7jq92L0PyFcaLx0Brzmd /a6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723495991; x=1724100791; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=6XoN+HaFS5yfD0CNRqQpWijMjbiOviXEPumC1NW+kKU=; b=aVTRmVaWPEOgl1xEmMujkG7JFG+hIuD+/5JNeHvJX3YAEZlorDhh4cCDGivOO1Fck3 8A9vwLyIoVuKJJ/p4o/6ZRSlXoCpEmRsokfNqDDFihJ/kgBpJeQFLjAeInIcLxE96CgX A1w50LNMb7ddu04GcrElkjJ/uPMXUW5qUWJ+40lYY4pd8Iq/LXrH4X2KROf8SKxmCWzs kCnV0BE0Bh0WhfLlfS4t+j+E5iP2e0mtc45wClPL3rfdwZXIjZLvj90d/PY7DBrU+iCG G2qXuKSnrfbHhqgxOwTyGlU4h9lsR1Pz5LiI9iigbE2ImpmTEjK9CwImoXLjO7+DpfQt zKcw== X-Forwarded-Encrypted: i=1; AJvYcCVfhPnX3ptpuRevsdb7eCzwCO/iJXQGZe380ARBiWFUea8Kec0lkNUpkNC4Da9aA2eCzq64wdei5zdvtyjuhUsW57CmCDb9b5yUCY9sLrv2GxTKknM= X-Gm-Message-State: AOJu0Yzxb/sYcmN7XRyCgEldwPQ54bv2e9zM++dZC0ywbmMepYPqDayF pqwV5CA9wls/faAx/suEaHenQXkPbiulQCQrYG2p15aRxASP7vqEjC4+7i7Ky3G6Rb3DjRJKpEh q4GkEXgc0MQ== X-Google-Smtp-Source: AGHT+IFttRPjHqcCMA/MGAc53YWP+S1ssedfHB7twOrXF8ji7+KWw2xkkyaVB0s2fmulVPed+0z3LTjiheg2UQ== X-Received: from mostafa.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:333c]) (user=smostafa job=sendgmr) by 2002:a7b:c304:0:b0:426:59d7:dcc4 with SMTP id 5b1f17b1804b1-429d48b0061mr30705e9.7.1723495991252; Mon, 12 Aug 2024 13:53:11 -0700 (PDT) Date: Mon, 12 Aug 2024 20:52:54 +0000 In-Reply-To: <20240812205255.97781-1-smostafa@google.com> Mime-Version: 1.0 References: <20240812205255.97781-1-smostafa@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240812205255.97781-2-smostafa@google.com> Subject: [PATCH 1/2] iommu/arm-smmu-v3: Match Stall behaviour for S2 From: Mostafa Saleh To: linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, will@kernel.org, robin.murphy@arm.com, joro@8bytes.org Cc: jgg@ziepe.ca, nicolinc@nvidia.com, mshavit@google.com, Mostafa Saleh X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240812_135314_082194_1E5E0CA4 X-CRM114-Status: GOOD ( 13.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org S2S must be set when stall model is forced "ARM_SMMU_FEAT_STALL_FORCE". But at the moment the driver ignores that, instead of doing the minimum and only set S2S for “ARM_SMMU_FEAT_STALL_FORCE” we can just match what S1 does which also set it for “ARM_SMMU_FEAT_STALL” and the master has requested stalls. This makes the driver more consistent when running on different SMMU instances with different supported stages. Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 5 +++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a31460f9f3d4..8d573d9ca93c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1562,6 +1562,11 @@ void arm_smmu_make_cdtable_ste(struct arm_smmu_ste *target, (cd_table->cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) | FIELD_PREP(STRTAB_STE_0_S1CDMAX, cd_table->s1cdmax)); + /* S2S is ignored if stage-2 exists but not enabled. */ + if (master->stall_enabled && + smmu->features & ARM_SMMU_FEAT_TRANS_S2) + target->data[0] |= FIELD_PREP(STRTAB_STE_2_S2S, 1); + target->data[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1DSS, s1dss) | FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 14bca41a981b..0dc7ad43c64c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -267,6 +267,7 @@ struct arm_smmu_ste { #define STRTAB_STE_2_S2AA64 (1UL << 51) #define STRTAB_STE_2_S2ENDI (1UL << 52) #define STRTAB_STE_2_S2PTW (1UL << 54) +#define STRTAB_STE_2_S2S (1UL << 57) #define STRTAB_STE_2_S2R (1UL << 58) #define STRTAB_STE_3_S2TTB_MASK GENMASK_ULL(51, 4) From patchwork Mon Aug 12 20:52:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mostafa Saleh X-Patchwork-Id: 13761047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 55F93C52D7C for ; Mon, 12 Aug 2024 21:08:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=DuUqrCWF/xHWcFuXKlPjIWucc9/NhZLN0OsUooYWUjg=; b=H/eEnHlO97PA0mOtLFaAvoUNt2 jhFPFOwJGf07mvQNLw610cL3KDkMuXtQYKIbmr1ZyIEwz92OgI0FRZ9C3w76XOtMu8ih6FUWxICYU GzoaGoBrUmoYFD98UJUvaVYKe8SUrl2tCE+vp96aZZW/nnF5yAkbaPylOKcHUo3zt2HwZrPlBXrnF gka7RE+Fp/AufxJAyk2LNjmj4uudwihULqlNH202qvtG+kIanpxYcFxF16T33eQ4qqY+yFwRFzuLv 5sGIvk6M0oqMD2f7JrRPagZ4biMs5YySn2r1chp6DMontO9CM3YeHyKLojwWDN38W/EI/FBiM8rwK lf3eDzIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sdcGz-00000001a0D-25E1; Mon, 12 Aug 2024 21:08:13 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sdc2W-00000001WMp-1ZJi for linux-arm-kernel@lists.infradead.org; Mon, 12 Aug 2024 20:53:17 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-665a6dd38c8so103124837b3.1 for ; Mon, 12 Aug 2024 13:53:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723495994; x=1724100794; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DuUqrCWF/xHWcFuXKlPjIWucc9/NhZLN0OsUooYWUjg=; b=bdMfrft1MAfsZdyvOXKw5oZ3L+PtgLqNW5b+5wfzbZ3kYHZArGiyRQmolI74RPDNAl 4gN7cF2M+pWsZBJOyTnH278BzmKR8pOWtMGspersr5gK2Mmw7qIeprtnj4FFA4dlsUHk Qp6IPtknuPRDRuSmuCggAXJCatXFPCjvJB9DLenVAfDlCN/5ZRBcnHr3Dzt8+zRZAjJq x1uYPHc6W/DkAswkHpvN1WydSmtnij0zR7aUOPYuVpS6IQyIsEtFW9FTDEwg9wDeVECF bs3xZsnrUJr7yQT16Dq+XnOhSObeLYxYgeMeE7qXpDKaVVoYdbyuEveGHF2PYdPgUHvh PnaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723495994; x=1724100794; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DuUqrCWF/xHWcFuXKlPjIWucc9/NhZLN0OsUooYWUjg=; b=ZcTiAvs4+PuOzxgobj7JDq5AbGu+m+41yIlgazm+AbAy7M36nfSkabQ2zCQ7iwc0r9 Bx5DZK8Ez2Y5yXJvn71fPRI0BpymQPZqfoYNg9XubdKuw8Md8GgU1YOTweg1a75RovPE WbqWS4qm7YVOBpq+bIB4QE2NSqzp8y1fEY7Opi/wAqAfI7KsC2XMKTxtqhHFNw1EA5Hl H7piBVwgw78sBEw53nrvhiEd8NPqhdk1i2W21RJz8bxvGQMzIgriofQUhfRyg/WfNyuY 1PcvnIisO7YMtA7eC5Vh3NmC/fvYqAUZ5bxeE91yCGZpamg1VHWIH23FGmW09x5AxYQD CANQ== X-Forwarded-Encrypted: i=1; AJvYcCVYNczobS5Uk1WFYL1+p0tTaDv16Dqn1TBpDmHyP3Hfera0xTSqHZJCxNEXYPI7/qvGZV2o0g3m78rVoP9XI89Ue3yJO8m3WVcNFyvOy9uaeRe0hic= X-Gm-Message-State: AOJu0YwXCy85Yysv/MOcDywSJ7axvhQ9+JiM+nJZKrg7x5t99YOnKssI Sp/it+q43z/6U2ZJAgueLoB2cSJow+8MSBKuI4ALAjrqGXckws1usj/FcIf8M3yHiVYGnd0qViA 6dE9BlowCKw== X-Google-Smtp-Source: AGHT+IENy8kvftAeMsPnXaFLGnElxbpTX8pP3lf5fKRnJiwZqy3MmYbMOT7IN0Eau0daUlvYKEg/llpPkpn9+g== X-Received: from mostafa.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:333c]) (user=smostafa job=sendgmr) by 2002:a25:aa46:0:b0:e0b:d229:af01 with SMTP id 3f1490d57ef6-e113ce919bdmr1758276.6.1723495994572; Mon, 12 Aug 2024 13:53:14 -0700 (PDT) Date: Mon, 12 Aug 2024 20:52:55 +0000 In-Reply-To: <20240812205255.97781-1-smostafa@google.com> Mime-Version: 1.0 References: <20240812205255.97781-1-smostafa@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240812205255.97781-3-smostafa@google.com> Subject: [PATCH 2/2] iommu/arm-smmu-v3: Report stalled S2 events From: Mostafa Saleh To: linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, will@kernel.org, robin.murphy@arm.com, joro@8bytes.org Cc: jgg@ziepe.ca, nicolinc@nvidia.com, mshavit@google.com, Mostafa Saleh X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240812_135316_466936_11038688 X-CRM114-Status: GOOD ( 20.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Previously, S2 stall was disabled and in case there was an event it wouldn't be reported on the assumption that it's always pinned by VFIO. However, now since we can enable stall, devices that use S2 outside VFIO should be able to report the stalls similar to S1. Also, to keep the old behaviour were S2 events from nested domains were not reported as they are pinned (from VFIO) add a new flag to track this. Signed-off-by: Mostafa Saleh --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 18 +++++++++++++----- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 8d573d9ca93c..ffa865529d73 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1733,6 +1733,7 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) u32 sid = FIELD_GET(EVTQ_0_SID, evt[0]); struct iopf_fault fault_evt = { }; struct iommu_fault *flt = &fault_evt.fault; + struct arm_smmu_domain *smmu_domain; switch (FIELD_GET(EVTQ_0_ID, evt[0])) { case EVT_ID_TRANSLATION_FAULT: @@ -1744,10 +1745,6 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) return -EOPNOTSUPP; } - /* Stage-2 is always pinned at the moment */ - if (evt[1] & EVTQ_1_S2) - return -EFAULT; - if (!(evt[1] & EVTQ_1_STALL)) return -EOPNOTSUPP; @@ -1782,6 +1779,15 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) goto out_unlock; } + /* It is guaranteed that smmu_domain exists as EVTQ_1_STALL is checked. */ + smmu_domain = to_smmu_domain(iommu_get_domain_for_dev(master->dev)); + + /* nesting domain is always pinned at the moment */ + if (smmu_domain->enable_nesting) { + ret = -EINVAL; + goto out_unlock; + } + iommu_report_device_fault(master->dev, &fault_evt); out_unlock: mutex_unlock(&smmu->streams_mutex); @@ -3373,8 +3379,10 @@ static int arm_smmu_enable_nesting(struct iommu_domain *domain) mutex_lock(&smmu_domain->init_mutex); if (smmu_domain->smmu) ret = -EPERM; - else + else { smmu_domain->stage = ARM_SMMU_DOMAIN_S2; + smmu_domain->enable_nesting = true; + } mutex_unlock(&smmu_domain->init_mutex); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 0dc7ad43c64c..f66efeec2cf8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -745,6 +745,8 @@ struct arm_smmu_domain { spinlock_t devices_lock; struct mmu_notifier mmu_notifier; + + bool enable_nesting; }; /* The following are exposed for testing purposes. */