From patchwork Tue Oct 15 06:57:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13835789 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F10E18BB84; Tue, 15 Oct 2024 06:58:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975501; cv=none; b=XNu2h6rFaHH/JJTq5JTCzHaR2BPFezxAOI0DgQ3aA68rXmdmBqMfhW5lTtOn3NBaSmEtwp++2foYUhTiLtTOj6Q2ihOSP7+SDjHZZmgPxfZPObnAi5lqmUYHh4evsCu0HWEg5HOzq9huUVnwYV5efz0CTTRL/xzJyCkrwFS9mtc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975501; c=relaxed/simple; bh=qS+Toc7PoXNztvG/RjDoMwseOURx1foo6t3tLhCVIwc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jhkgjD0WSuzdC7moY4XVLf8cdOsDb3ldAspERORLo+tV79tua6KRq1G1qmFjptXsmysjuPrRKIB+B+3xgpdWQZ78x4pzjoBAeyJd0v4LNsJRpT96NXKnjK5nJgjhAdGzmJjNoBMXvp6OuPORnZ8fSj15X0Qru2ucBtjhkK49J/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DEMDn77Q; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DEMDn77Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728975501; x=1760511501; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qS+Toc7PoXNztvG/RjDoMwseOURx1foo6t3tLhCVIwc=; b=DEMDn77Q9TLHrlxK2B51qveI+3uCOVFs97fdkwMwvdegr+/tc2Ixh1m7 kLoaJpOjftxIJP/loKb5rJgC6rwsU3oWA7iw4ZnNTxg4Zk3BkD7wk0YzI tkRYPcbnp6ZaOSh4DGCVafgMe2LV6WGUv72+vTN+qqoUPCnUldEHtVHoB Qhw/r0NubcOFwJBd+7rurm8+HXQnQUX+mBod7n6VEGWM7gPGY/3N6GARv ABOWGwvyp8BdRhLHnH0g5U14xNe7GicdvJ6B1YyH/tWyGF8S1NTSMr9H+ nPqu+XCzIfkRMeu58YUw8B6xNKOPUZAR+fCisIqN8zPo0zl7hBuSwSrRi w==; X-CSE-ConnectionGUID: xTGs+c9MSyShSQ4ARa+3ow== X-CSE-MsgGUID: r3krmHe2TseOjmubiRoJjQ== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28447813" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28447813" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:20 -0700 X-CSE-ConnectionGUID: tRe05deZRmuyQuXMemuphQ== X-CSE-MsgGUID: M+zrl6rURpGmI9kqJpisHQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="81776909" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:16 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Jonathan Cameron , Davidlohr Bueso , Gregory Price , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero , Ben Cheatham Subject: [PATCH 1/5] cxl: Rename ACPI_CEDT_CFMWS_RESTRICT_TYPE2/TYPE3 Date: Tue, 15 Oct 2024 14:57:09 +0800 Message-Id: <20241015065713.308671-2-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241015065713.308671-1-ying.huang@intel.com> References: <20241015065713.308671-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 According to the description of the "Window Restrictions" field of "CFMWS Structure" in the CXL spec v3.1 section 9.18.1.3: CXL Fixed Memory Window Structure (CFMWS), the bit 0 of "Window Restrictions" is formerly known as "CXL Type 2 Memory" and renamed to "Device Coherent", while the bit 1 is formerly known as "CXL Type 3 Memory" and renamed to "Host-only Coherent". Because type 3 memory can only be host-only coherent before, while it can be host-only coherent or device coherent with "Back-Invalidate" now. To avoid confusion about type 2/3 memory and device/host-only coherent in Linux kernel, the patch renames corresponding bit definition from ACPI_CEDT_CFMWS_RESTRICT_TYPE2/TYPE3 to ACPI_CEDT_CFMWS_RESTRICT_DEVCOH/HOSTONLYCOH. This makes the kernel code consistent with the spec too. The patch also renames the corresponding cxl_decoder flags CXL_DECODER_F_TYPE2/TYPE3 to CXL_DECODER_F_DEVCOH/HOSTONLYCOH. No functionality change is expected. Signed-off-by: "Huang, Ying" Suggested-by: Jonathan Cameron Reviewed-by: Davidlohr Bueso Reviewed-by: Gregory Price Cc: Dan Williams Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Cc: Ben Cheatham --- drivers/cxl/acpi.c | 8 ++++---- drivers/cxl/core/port.c | 8 ++++---- drivers/cxl/cxl.h | 14 +++++++------- include/acpi/actbl1.h | 10 +++++----- tools/testing/cxl/test/cxl.c | 18 +++++++++--------- 5 files changed, 29 insertions(+), 29 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 82b78e331d8e..3115f246273b 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -115,10 +115,10 @@ static unsigned long cfmws_to_decoder_flags(int restrictions) { unsigned long flags = CXL_DECODER_F_ENABLE; - if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_TYPE2) - flags |= CXL_DECODER_F_TYPE2; - if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_TYPE3) - flags |= CXL_DECODER_F_TYPE3; + if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_DEVCOH) + flags |= CXL_DECODER_F_DEVCOH; + if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH) + flags |= CXL_DECODER_F_HOSTONLYCOH; if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_VOLATILE) flags |= CXL_DECODER_F_RAM; if (restrictions & ACPI_CEDT_CFMWS_RESTRICT_PMEM) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index a5e6f3d23cfb..35b6ad4ea0f9 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -125,8 +125,8 @@ static DEVICE_ATTR_RO(name) CXL_DECODER_FLAG_ATTR(cap_pmem, CXL_DECODER_F_PMEM); CXL_DECODER_FLAG_ATTR(cap_ram, CXL_DECODER_F_RAM); -CXL_DECODER_FLAG_ATTR(cap_type2, CXL_DECODER_F_TYPE2); -CXL_DECODER_FLAG_ATTR(cap_type3, CXL_DECODER_F_TYPE3); +CXL_DECODER_FLAG_ATTR(cap_type2, CXL_DECODER_F_DEVCOH); +CXL_DECODER_FLAG_ATTR(cap_type3, CXL_DECODER_F_HOSTONLYCOH); CXL_DECODER_FLAG_ATTR(locked, CXL_DECODER_F_LOCK); static ssize_t target_type_show(struct device *dev, @@ -326,14 +326,14 @@ static struct attribute *cxl_decoder_root_attrs[] = { static bool can_create_pmem(struct cxl_root_decoder *cxlrd) { - unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_PMEM; + unsigned long flags = CXL_DECODER_F_HOSTONLYCOH | CXL_DECODER_F_PMEM; return (cxlrd->cxlsd.cxld.flags & flags) == flags; } static bool can_create_ram(struct cxl_root_decoder *cxlrd) { - unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_RAM; + unsigned long flags = CXL_DECODER_F_HOSTONLYCOH | CXL_DECODER_F_RAM; return (cxlrd->cxlsd.cxld.flags & flags) == flags; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0fc96f8bf15c..a34e4256aa5f 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -315,13 +315,13 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev, * Additionally indicate whether decoder settings were autodetected, * user customized. */ -#define CXL_DECODER_F_RAM BIT(0) -#define CXL_DECODER_F_PMEM BIT(1) -#define CXL_DECODER_F_TYPE2 BIT(2) -#define CXL_DECODER_F_TYPE3 BIT(3) -#define CXL_DECODER_F_LOCK BIT(4) -#define CXL_DECODER_F_ENABLE BIT(5) -#define CXL_DECODER_F_MASK GENMASK(5, 0) +#define CXL_DECODER_F_RAM BIT(0) +#define CXL_DECODER_F_PMEM BIT(1) +#define CXL_DECODER_F_DEVCOH BIT(2) +#define CXL_DECODER_F_HOSTONLYCOH BIT(3) +#define CXL_DECODER_F_LOCK BIT(4) +#define CXL_DECODER_F_ENABLE BIT(5) +#define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { CXL_DECODER_DEVMEM = 2, diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h index 199afc2cd122..2b2111035669 100644 --- a/include/acpi/actbl1.h +++ b/include/acpi/actbl1.h @@ -551,11 +551,11 @@ struct acpi_cedt_cfmws_target_element { /* Values for Restrictions field above */ -#define ACPI_CEDT_CFMWS_RESTRICT_TYPE2 (1) -#define ACPI_CEDT_CFMWS_RESTRICT_TYPE3 (1<<1) -#define ACPI_CEDT_CFMWS_RESTRICT_VOLATILE (1<<2) -#define ACPI_CEDT_CFMWS_RESTRICT_PMEM (1<<3) -#define ACPI_CEDT_CFMWS_RESTRICT_FIXED (1<<4) +#define ACPI_CEDT_CFMWS_RESTRICT_DEVCOH (1) +#define ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH (1<<1) +#define ACPI_CEDT_CFMWS_RESTRICT_VOLATILE (1<<2) +#define ACPI_CEDT_CFMWS_RESTRICT_PMEM (1<<3) +#define ACPI_CEDT_CFMWS_RESTRICT_FIXED (1<<4) /* 2: CXL XOR Interleave Math Structure */ diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 90d5afd52dd0..3982d292d286 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -209,7 +209,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_VOLATILE, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 4UL, @@ -224,7 +224,7 @@ static struct { }, .interleave_ways = 1, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_VOLATILE, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -239,7 +239,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 4UL, @@ -254,7 +254,7 @@ static struct { }, .interleave_ways = 1, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -269,7 +269,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 4UL, @@ -284,7 +284,7 @@ static struct { }, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_VOLATILE, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M, @@ -301,7 +301,7 @@ static struct { .interleave_arithmetic = ACPI_CEDT_CFMWS_ARITHMETIC_XOR, .interleave_ways = 0, .granularity = 4, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -317,7 +317,7 @@ static struct { .interleave_arithmetic = ACPI_CEDT_CFMWS_ARITHMETIC_XOR, .interleave_ways = 1, .granularity = 0, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 8UL, @@ -333,7 +333,7 @@ static struct { .interleave_arithmetic = ACPI_CEDT_CFMWS_ARITHMETIC_XOR, .interleave_ways = 2, .granularity = 0, - .restrictions = ACPI_CEDT_CFMWS_RESTRICT_TYPE3 | + .restrictions = ACPI_CEDT_CFMWS_RESTRICT_HOSTONLYCOH | ACPI_CEDT_CFMWS_RESTRICT_PMEM, .qtg_id = FAKE_QTG_ID, .window_size = SZ_256M * 16UL, From patchwork Tue Oct 15 06:57:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13835790 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88C411AB52D; Tue, 15 Oct 2024 06:58:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975505; cv=none; b=MJcHsBV0TqliqpmHTJ+671pEUDs9ZS6Vfc/tJ0gkhILPlQRebfrHQM9aAm7ez0E82Cym4Mvk58HAr9dHl6X6xiuV7Dguen/7KCIxRZU70DzC9TnpmLN5AMgxAHu9HMGNv7LtG2BWi61OkLXB/olzB195JM/hz/bRzn6NJTHjhSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975505; c=relaxed/simple; bh=nxme6BbZVCGNJxz3mRXMUQP2VLpNyphZdCA3VzVUPQc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=L7VPho3U4uT21Khh5ZWq1RkT/BiBpXajQ4x8YNI7xhUxvbsVZ7vMoX+8tNNt+a5kmfuo3pF3Mxx5LjeKGQ2GetIjvFM+VXZztoLPFDLGK1j4iqZIhUR8Wzh1nwuaGklq4V8DWKmQBPw+jE6VxetnMunOjO4c/epEMk94Bk8iPEc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=i3+p+QDs; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="i3+p+QDs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728975504; x=1760511504; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nxme6BbZVCGNJxz3mRXMUQP2VLpNyphZdCA3VzVUPQc=; b=i3+p+QDsIWhfX9IeIcu/bH/xoND17DwrawTFaZlZ0De57s74Bf2OSoh7 4yBI7wmEhcHoer1ymxuCvm10F+Moo6x8vDiA+ALAA1dGkn4uXb3NCHpxN qaDORJ/e7VFyo1vylrAejraXAaxX4WmA4gSjM6xqgKfzhaDO+gV1FXenp bQjDowyTNrL0RyBRZ2BeYuhC5oN67Bknsp/Su7nM50gJnuj26VDrx9kUC BXtoZeKZ8esqXts1pBBEK22foEZM3RhiYH240C1feip6zbK9u9W6XqsHE qf45icF+ykGMsMvBvBktY4BW4Kgezzu7ijUPum8T9S5n6Hyf4I4FnLbjf A==; X-CSE-ConnectionGUID: nn6aYHQjQ5mCX33bTAUkFA== X-CSE-MsgGUID: jkZkg7JIQ+yHLuRI/Pq4tA== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28447832" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28447832" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:23 -0700 X-CSE-ConnectionGUID: bSY7ES7aSSWHbiWmxl3bow== X-CSE-MsgGUID: DyRc54RxT82BNZh/yJmr6Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="81776912" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:20 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Davidlohr Bueso , Gregory Price , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero , Ben Cheatham Subject: [PATCH 2/5] cxl: Rename CXL_DECODER_HOSTONLYMEM/DEVMEM Date: Tue, 15 Oct 2024 14:57:10 +0800 Message-Id: <20241015065713.308671-3-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241015065713.308671-1-ying.huang@intel.com> References: <20241015065713.308671-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Previously, CXL type3 devices (memory expanders) use host only coherence (HDM-H), while CXL type2 devices (accelerators) use dev coherence (HDM-D). So the name of the target device type of a cxl decoder is CXL_DECODER_HOSTONLYMEM for type3 devices and CXL_DECODER_DEVMEM for type2 devices. However, this isn't true anymore. CXL type3 devices can use dev coherence + back invalidation (HDM-DB) too. To avoid confusion between the device type and coherence, the patch renames CXL_DECODER_HOSTONLYMEM/DEVMEM to CXL_DECODER_EXPANDER/ACCEL. We don't expect any functionality change in this patch. Signed-off-by: "Huang, Ying" Reviewed-by: Davidlohr Bueso Reviewed-by: Gregory Price Cc: Dan Williams Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Cc: Ben Cheatham --- drivers/cxl/acpi.c | 2 +- drivers/cxl/core/hdm.c | 16 ++++++++-------- drivers/cxl/core/port.c | 6 +++--- drivers/cxl/core/region.c | 2 +- drivers/cxl/cxl.h | 4 ++-- tools/testing/cxl/test/cxl.c | 6 +++--- 6 files changed, 18 insertions(+), 18 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 3115f246273b..21486e471305 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -382,7 +382,7 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws, cxld = &cxlrd->cxlsd.cxld; cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions); - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->hpa_range = (struct range) { .start = cfmws->base_hpa, .end = cfmws->base_hpa + cfmws->window_size - 1, diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 3df10517a327..57b54ecdb000 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -572,7 +572,7 @@ static void cxld_set_interleave(struct cxl_decoder *cxld, u32 *ctrl) static void cxld_set_type(struct cxl_decoder *cxld, u32 *ctrl) { u32p_replace_bits(ctrl, - !!(cxld->target_type == CXL_DECODER_HOSTONLYMEM), + !!(cxld->target_type == CXL_DECODER_EXPANDER), CXL_HDM_DECODER0_CTRL_HOSTONLY); } @@ -771,7 +771,7 @@ static int cxl_setup_hdm_decoder_from_dvsec( if (!len) return -ENOENT; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->commit = NULL; cxld->reset = NULL; cxld->hpa_range = info->dvsec_range[which]; @@ -847,9 +847,9 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; else - cxld->target_type = CXL_DECODER_DEVMEM; + cxld->target_type = CXL_DECODER_ACCEL; guard(rwsem_write)(&cxl_region_rwsem); if (cxld->id != cxl_num_decoders_committed(port)) { @@ -876,16 +876,16 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, * more precision. */ if (cxlds->type == CXL_DEVTYPE_CLASSMEM) - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; else - cxld->target_type = CXL_DECODER_DEVMEM; + cxld->target_type = CXL_DECODER_ACCEL; } else { /* To be overridden by region type at commit time */ - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; } if (!FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl) && - cxld->target_type == CXL_DECODER_HOSTONLYMEM) { + cxld->target_type == CXL_DECODER_EXPANDER) { ctrl |= CXL_HDM_DECODER0_CTRL_HOSTONLY; writel(ctrl, hdm + CXL_HDM_DECODER0_CTRL_OFFSET(which)); } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 35b6ad4ea0f9..e80b0af3d812 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -135,9 +135,9 @@ static ssize_t target_type_show(struct device *dev, struct cxl_decoder *cxld = to_cxl_decoder(dev); switch (cxld->target_type) { - case CXL_DECODER_DEVMEM: + case CXL_DECODER_ACCEL: return sysfs_emit(buf, "accelerator\n"); - case CXL_DECODER_HOSTONLYMEM: + case CXL_DECODER_EXPANDER: return sysfs_emit(buf, "expander\n"); } return -ENXIO; @@ -1768,7 +1768,7 @@ static int cxl_decoder_init(struct cxl_port *port, struct cxl_decoder *cxld) /* Pre initialize an "empty" decoder */ cxld->interleave_ways = 1; cxld->interleave_granularity = PAGE_SIZE; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->hpa_range = (struct range) { .start = 0, .end = -1, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 7bb79f3f318c..036356bc4204 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2531,7 +2531,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(-EBUSY); } - return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM); + return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); } static ssize_t create_pmem_region_store(struct device *dev, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index a34e4256aa5f..f95101994238 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -324,8 +324,8 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev, #define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { - CXL_DECODER_DEVMEM = 2, - CXL_DECODER_HOSTONLYMEM = 3, + CXL_DECODER_ACCEL = 2, + CXL_DECODER_EXPANDER = 3, }; /* diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 3982d292d286..352a62c745c6 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -724,7 +724,7 @@ static void default_mock_decoder(struct cxl_decoder *cxld) cxld->interleave_ways = 1; cxld->interleave_granularity = 256; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->commit = mock_decoder_commit; cxld->reset = mock_decoder_reset; } @@ -798,7 +798,7 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld) cxld->interleave_ways = 2; eig_to_granularity(window->granularity, &cxld->interleave_granularity); - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->flags = CXL_DECODER_F_ENABLE; cxled->state = CXL_DECODER_STATE_AUTO; port->commit_end = cxld->id; @@ -831,7 +831,7 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld) } else cxlsd->target[0] = dport; cxld = &cxlsd->cxld; - cxld->target_type = CXL_DECODER_HOSTONLYMEM; + cxld->target_type = CXL_DECODER_EXPANDER; cxld->flags = CXL_DECODER_F_ENABLE; iter->commit_end = 0; /* From patchwork Tue Oct 15 06:57:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13835791 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F8C71B4F3E; Tue, 15 Oct 2024 06:58:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975508; cv=none; b=dDoLBu7deY6aj6/TrD6v7MK8mEg/ivtc/LoiTTs7j6Xd2QK0vNbXNE3997+YvvAByazjZsJRx8yh20ZNWPpQjj0QBUv//33phhz3mtmbm6LMrU0zHH/Z3GmJsp29V85FVJ3O2oS86r3WUfjqq5SO7dEGbl2Chq/UUsXq34VR1Zw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975508; c=relaxed/simple; bh=pvBwtu4JQxUWjKxLWnVdsVP0pKJHDkT2XbcNeyPqdwM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=t42ley1W8c8OxP13OCbBPg24QvjogZ7NJvg93BoX+sAB4KArxzd97sS1YcF0ydH0osmgn+bigoWU3RORfammNrqbgoJ542VM6T8gNcGOYr6xlJDQRjvWlZ8UdgDjUNWaYnHGsOuKjj8Zc9lz3jqgB8wUVltTyqGG0ICi2PSwZ90= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iQlOSu9K; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iQlOSu9K" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728975507; x=1760511507; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pvBwtu4JQxUWjKxLWnVdsVP0pKJHDkT2XbcNeyPqdwM=; b=iQlOSu9K1R04SJWp1c0MkVrnOWXu2HsSrCjkQ8O9gFG/AmCp3eRpLWTl O/uIyyt0jTDLnvZ1qA+h2u2xQk5UKMlGhr0PHmn8k6RmfgZ0pNuU3HEIK qwg3+7gc3agWwvRdxhdiwAl75CVCcnbbSwh4JN/8RAw13Y94Fw4GTiBDF 3HCWYyWU2EchTmuysXKb71599KuTfmxoPq2DocmUbg49WQlgjW0g+26N+ 2Pr+Tw42owx7e741me/8Evia1WUMxIS7L97FykcyO5JTHhna4DR4v8+3Y PR0Sn3DgbQFqgRFZL3UbwKnpDbVw3ISb7Q8bUnJHfM12tqJuYv+I/Kt25 w==; X-CSE-ConnectionGUID: /WhJb/cWQn6S3JEq1zSWnw== X-CSE-MsgGUID: 8hUg0ANSSmyxq31h68C3Pw== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28447850" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28447850" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:27 -0700 X-CSE-ConnectionGUID: Ia9b77BnQTGPsPhfFK+cIw== X-CSE-MsgGUID: CKFDbMAfToW98mATEuauYw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="81776917" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:23 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Gregory Price , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero , Ben Cheatham Subject: [PATCH 3/5] cxl: Separate coherence from target type Date: Tue, 15 Oct 2024 14:57:11 +0800 Message-Id: <20241015065713.308671-4-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241015065713.308671-1-ying.huang@intel.com> References: <20241015065713.308671-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Previously, target type (expander or accelerator) and coherence (HOSTONLY (HDM-H) or DEV (HDM-D/DB)) are synonym. So current kernel uses target type to designate coherence too. However, it's possible for expanders to use HDM-DB now. So, the patch separates coherence from target type. Accordingly, the patch sets the HOSTONLY field of decoder ctrl register (CXL_HDM_DECODER0_CTRL_HOSTONLY) according to the coherence instead of the target type. Because we cannot determine the coherence of decoders via target type, the patch lets accelerator/expander device drivers specify coherence explicitly via newly added coherence field in struct cxl_dev_state. The coherence of each end points in a region needs to be same. So, the patch records the coherence of the first added end point in struct region. Then, it checks whether the coherence of all other end points is same. Signed-off-by: "Huang, Ying" Reviewed-by: Gregory Price Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Cc: Ben Cheatham --- drivers/cxl/core/hdm.c | 22 +++++++++++++++------- drivers/cxl/core/mbox.c | 1 + drivers/cxl/core/port.c | 1 + drivers/cxl/core/region.c | 37 ++++++++++++++++++++++++++++++++++--- drivers/cxl/cxl.h | 10 ++++++++++ drivers/cxl/cxlmem.h | 12 ++++++++++++ 6 files changed, 73 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 57b54ecdb000..478fb6691759 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -569,10 +569,10 @@ static void cxld_set_interleave(struct cxl_decoder *cxld, u32 *ctrl) *ctrl |= CXL_HDM_DECODER0_CTRL_COMMIT; } -static void cxld_set_type(struct cxl_decoder *cxld, u32 *ctrl) +static void cxld_set_coherence(struct cxl_decoder *cxld, u32 *ctrl) { u32p_replace_bits(ctrl, - !!(cxld->target_type == CXL_DECODER_EXPANDER), + !!(cxld->coherence == CXL_DECODER_HOSTONLYCOH), CXL_HDM_DECODER0_CTRL_HOSTONLY); } @@ -667,7 +667,7 @@ static int cxl_decoder_commit(struct cxl_decoder *cxld) /* common decoder settings */ ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(cxld->id)); cxld_set_interleave(cxld, &ctrl); - cxld_set_type(cxld, &ctrl); + cxld_set_coherence(cxld, &ctrl); base = cxld->hpa_range.start; size = range_len(&cxld->hpa_range); @@ -846,10 +846,13 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, cxld->flags |= CXL_DECODER_F_ENABLE; if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; - if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) { cxld->target_type = CXL_DECODER_EXPANDER; - else + cxld->coherence = CXL_DECODER_HOSTONLYCOH; + } else { cxld->target_type = CXL_DECODER_ACCEL; + cxld->coherence = CXL_DECODER_DEVCOH; + } guard(rwsem_write)(&cxl_region_rwsem); if (cxld->id != cxl_num_decoders_committed(port)) { @@ -879,13 +882,18 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, cxld->target_type = CXL_DECODER_EXPANDER; else cxld->target_type = CXL_DECODER_ACCEL; + if (cxlds->coherence == CXL_DEVCOH_HOSTONLY) + cxld->coherence = CXL_DECODER_HOSTONLYCOH; + else + cxld->coherence = CXL_DECODER_DEVCOH; } else { - /* To be overridden by region type at commit time */ + /* To be overridden by region type/coherence at commit time */ cxld->target_type = CXL_DECODER_EXPANDER; + cxld->coherence = CXL_DECODER_HOSTONLYCOH; } if (!FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl) && - cxld->target_type == CXL_DECODER_EXPANDER) { + cxld->coherence == CXL_DECODER_HOSTONLYCOH) { ctrl |= CXL_HDM_DECODER0_CTRL_HOSTONLY; writel(ctrl, hdm + CXL_HDM_DECODER0_CTRL_OFFSET(which)); } diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 5175138c4fb7..fb98cd1a8b61 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1450,6 +1450,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mds->cxlds.reg_map.host = dev; mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; + mds->cxlds.coherence = CXL_DEVCOH_HOSTONLY; mds->ram_perf.qos_class = CXL_QOS_CLASS_INVALID; mds->pmem_perf.qos_class = CXL_QOS_CLASS_INVALID; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e80b0af3d812..9ebbffcea26a 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1769,6 +1769,7 @@ static int cxl_decoder_init(struct cxl_port *port, struct cxl_decoder *cxld) cxld->interleave_ways = 1; cxld->interleave_granularity = PAGE_SIZE; cxld->target_type = CXL_DECODER_EXPANDER; + cxld->coherence = CXL_DECODER_HOSTONLYCOH; cxld->hpa_range = (struct range) { .start = 0, .end = -1, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 036356bc4204..21b877d8582f 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1005,9 +1005,10 @@ static int cxl_rr_alloc_decoder(struct cxl_port *port, struct cxl_region *cxlr, } /* - * Endpoints should already match the region type, but backstop that - * assumption with an assertion. Switch-decoders change mapping-type - * based on what is mapped when they are assigned to a region. + * Endpoints should already match the region type/coherence, but + * backstop that assumption with an assertion. Switch-decoders change + * mapping-type/coherence based on what is mapped when they are assigned + * to a region. */ dev_WARN_ONCE(&cxlr->dev, port == cxled_to_port(cxled) && @@ -1016,6 +1017,13 @@ static int cxl_rr_alloc_decoder(struct cxl_port *port, struct cxl_region *cxlr, dev_name(&cxled_to_memdev(cxled)->dev), dev_name(&cxld->dev), cxld->target_type, cxlr->type); cxld->target_type = cxlr->type; + dev_WARN_ONCE(&cxlr->dev, + port == cxled_to_port(cxled) && + cxld->coherence != cxlr->coherence, + "%s:%s mismatch decoder coherence %d -> %d\n", + dev_name(&cxled_to_memdev(cxled)->dev), + dev_name(&cxld->dev), cxld->coherence, cxlr->coherence); + cxld->coherence = cxlr->coherence; cxl_rr->decoder = cxld; return 0; } @@ -1925,6 +1933,29 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -ENXIO; } + /* Set the coherence of region to that of the first endpoint */ + if (cxlr->coherence == CXL_DECODER_INVALIDCOH) { + unsigned long flags = cxlrd->cxlsd.cxld.flags; + enum cxl_decoder_coherence coherence = cxled->cxld.coherence; + + cxlr->coherence = coherence; + if ((coherence == CXL_DECODER_HOSTONLYCOH && + !(flags & CXL_DECODER_F_HOSTONLYCOH)) || + (coherence == CXL_DECODER_DEVCOH && + !(flags & CXL_DECODER_F_DEVCOH))) { + dev_dbg(&cxlr->dev, +"%s:%s endpoint coherence: %d isn't supported by root decoder: %#lx\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + coherence, flags); + return -ENXIO; + } + } else if (cxled->cxld.coherence != cxlr->coherence) { + dev_dbg(&cxlr->dev, "%s:%s coherence mismatch: %d vs %d\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + cxled->cxld.coherence, cxlr->coherence); + return -ENXIO; + } + if (!cxled->dpa_res) { dev_dbg(&cxlr->dev, "%s:%s: missing DPA allocation.\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev)); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f95101994238..1927a1849d82 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -328,6 +328,12 @@ enum cxl_decoder_type { CXL_DECODER_EXPANDER = 3, }; +enum cxl_decoder_coherence { + CXL_DECODER_INVALIDCOH, + CXL_DECODER_HOSTONLYCOH, + CXL_DECODER_DEVCOH, +}; + /* * Current specification goes up to 8, double that seems a reasonable * software max for the foreseeable future @@ -344,6 +350,7 @@ enum cxl_decoder_type { * @interleave_ways: number of cxl_dports in this decode * @interleave_granularity: data stride per dport * @target_type: accelerator vs expander (type2 vs type3) selector + * @coherence: host only vs device coherence selector * @region: currently assigned region for this decoder * @flags: memory type capabilities and locking * @commit: device/decoder-type specific callback to commit settings to hw @@ -356,6 +363,7 @@ struct cxl_decoder { int interleave_ways; int interleave_granularity; enum cxl_decoder_type target_type; + enum cxl_decoder_coherence coherence; struct cxl_region *region; unsigned long flags; int (*commit)(struct cxl_decoder *cxld); @@ -517,6 +525,7 @@ struct cxl_region_params { * @id: This region's id. Id is globally unique across all regions * @mode: Endpoint decoder allocation / access mode * @type: Endpoint decoder target type + * @coherence: Endpoint decoder coherence * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown * @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge * @flags: Region state flags @@ -530,6 +539,7 @@ struct cxl_region { int id; enum cxl_decoder_mode mode; enum cxl_decoder_type type; + enum cxl_decoder_coherence coherence; struct cxl_nvdimm_bridge *cxl_nvb; struct cxl_pmem_region *cxlr_pmem; unsigned long flags; diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 2a25d1957ddb..f9156c578bde 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -394,6 +394,16 @@ enum cxl_devtype { CXL_DEVTYPE_CLASSMEM, }; +/* + * enum cxl_devcoherence - the coherence of the cxl device + * @CXL_DEVCOH_DEV - HDM-D (type 2) or HDM-DB (type 2/3) + * @CXL_DEVCOH_HOSTONLY - HDM-H (type 3) + */ +enum cxl_devcoherence { + CXL_DEVCOH_DEV, + CXL_DEVCOH_HOSTONLY, +}; + /** * struct cxl_dpa_perf - DPA performance property entry * @dpa_range: range for DPA address @@ -427,6 +437,7 @@ struct cxl_dpa_perf { * @ram_res: Active Volatile memory capacity configuration * @serial: PCIe Device Serial Number * @type: Generic Memory Class device or Vendor Specific Memory device + * @coherence: Device or Host only coherence * @cxl_mbox: CXL mailbox context */ struct cxl_dev_state { @@ -442,6 +453,7 @@ struct cxl_dev_state { struct resource ram_res; u64 serial; enum cxl_devtype type; + enum cxl_devcoherence coherence; struct cxl_mailbox cxl_mbox; }; From patchwork Tue Oct 15 06:57:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13835792 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC7E718BB8B; Tue, 15 Oct 2024 06:58:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975511; cv=none; b=VtMkKpEsQLeVFDdRHD3cWmsae+JLa3Gyhhv8DYwkcr4FvCQm5ZUmI4JeQKvoLQZKp07UgD9t+APP7NBQUW+Sqc2Bo6rfB56TyW8ABYIadiBbPbuOB0y/Xv8sB4+ErE+O26TypivLtgOHhkUYfpGZsu5mH4b4fypNGtiZe1G5ScU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975511; c=relaxed/simple; bh=kfEWIK4MkSwe8J7QnC0SyLg4DySWos7YcTLodpSGzF4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dRnQ5b+NrfMzwDohIFOcwJLIMcKonH8PvcpyJXU5Mx2eAw/mUbxFF5JRzgMTOjAMudQDPNw9VMXqp3TjccIB5F185pvyjdj4vbyC34xKMZi/GOUhlc1nQAAizW5qHvSMy2qn6rsiDyZAJ0kcHuoD3VgxC+iORwjveMbGZyqtUeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FSaoNfVh; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FSaoNfVh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728975510; x=1760511510; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kfEWIK4MkSwe8J7QnC0SyLg4DySWos7YcTLodpSGzF4=; b=FSaoNfVhzvjqUj7zL7/RUeTgUxJ7/LYtdOuVBN+d5Pen+TeFG7RKTo48 VylfYU2zHgW4KIjCA4XHOOWmu5VWNWpaUhWd4QOs04xIIe/k5vhBbaaBd fh/t1o2bPnSED+YHHSil/17mS95OXd3HK6vO1BxwAQvauk2T70iqb2Ot/ H+6VtEXnyK3wZqr+dmkRqzPd4DoH2HfHKa0z27hjMbpZIzZvY1iMcxmI2 EO/8x78zZ7NFD5WdVtBtmN09ZGmKacAf9qPs/V1XQX6R9Adz+Lm02pUo8 Ifw8+vRERY5j5p+Kp35fMC08E9n9yd4ILGc9lF9EFUkhb/H3hsnAorL0x Q==; X-CSE-ConnectionGUID: ScUbsDBRRYyapZMOGQprow== X-CSE-MsgGUID: WASINfLtSUmB05w42J1NPw== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28447865" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28447865" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:30 -0700 X-CSE-ConnectionGUID: WKEUq0feTha0qIlAlJPguA== X-CSE-MsgGUID: AML8i+ePQx6G445dIA2iRQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="81776920" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:26 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Gregory Price , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero , Ben Cheatham Subject: [PATCH 4/5] cxl: Set type of region to that of the first endpoint Date: Tue, 15 Oct 2024 14:57:12 +0800 Message-Id: <20241015065713.308671-5-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241015065713.308671-1-ying.huang@intel.com> References: <20241015065713.308671-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Current kernel hard-codes the type of region to type 3 expander now, because this is the only supported device type. As a preparation to support type 2 accelerators, the patch sets the type of region to that of the first endpoint. Then, the patch checks whether the type of region is same as the type of other endpoints of the region. Because what we really need is to make sure the type of all endpoints of a region is same. And, the patch lets expander/accelerator device drivers specify the target type of endpoint devices via struct cxl_dev_state. Signed-off-by: "Huang, Ying" Reviewed-by: Gregory Price Cc: Dan Williams Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Cc: Ben Cheatham --- drivers/cxl/acpi.c | 1 - drivers/cxl/core/hdm.c | 28 +++++++++++++--------------- drivers/cxl/core/port.c | 2 ++ drivers/cxl/core/region.c | 13 +++++++------ drivers/cxl/cxl.h | 1 + 5 files changed, 23 insertions(+), 22 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 21486e471305..29c2a44b122c 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -382,7 +382,6 @@ static int __cxl_parse_cfmws(struct acpi_cedt_cfmws *cfmws, cxld = &cxlrd->cxlsd.cxld; cxld->flags = cfmws_to_decoder_flags(cfmws->restrictions); - cxld->target_type = CXL_DECODER_EXPANDER; cxld->hpa_range = (struct range) { .start = cfmws->base_hpa, .end = cfmws->base_hpa + cfmws->window_size - 1, diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 478fb6691759..c9accf8be71f 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -841,18 +841,25 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, .end = base + size - 1, }; + if (cxled) { + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + + if (cxlds->type == CXL_DEVTYPE_CLASSMEM) + cxld->target_type = CXL_DECODER_EXPANDER; + else + cxld->target_type = CXL_DECODER_ACCEL; + } + /* decoders are enabled if committed */ if (committed) { cxld->flags |= CXL_DECODER_F_ENABLE; if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; - if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) { - cxld->target_type = CXL_DECODER_EXPANDER; + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_HOSTONLY, ctrl)) cxld->coherence = CXL_DECODER_HOSTONLYCOH; - } else { - cxld->target_type = CXL_DECODER_ACCEL; + else cxld->coherence = CXL_DECODER_DEVCOH; - } guard(rwsem_write)(&cxl_region_rwsem); if (cxld->id != cxl_num_decoders_committed(port)) { @@ -874,21 +881,12 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); struct cxl_dev_state *cxlds = cxlmd->cxlds; - /* - * Default by devtype until a device arrives that needs - * more precision. - */ - if (cxlds->type == CXL_DEVTYPE_CLASSMEM) - cxld->target_type = CXL_DECODER_EXPANDER; - else - cxld->target_type = CXL_DECODER_ACCEL; if (cxlds->coherence == CXL_DEVCOH_HOSTONLY) cxld->coherence = CXL_DECODER_HOSTONLYCOH; else cxld->coherence = CXL_DECODER_DEVCOH; } else { - /* To be overridden by region type/coherence at commit time */ - cxld->target_type = CXL_DECODER_EXPANDER; + /* To be overridden by region coherence at commit time */ cxld->coherence = CXL_DECODER_HOSTONLYCOH; } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 9ebbffcea26a..d1bc6aed6509 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -139,6 +139,8 @@ static ssize_t target_type_show(struct device *dev, return sysfs_emit(buf, "accelerator\n"); case CXL_DECODER_EXPANDER: return sysfs_emit(buf, "expander\n"); + default: + break; } return -ENXIO; } diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 21b877d8582f..d709738ada61 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1926,7 +1926,10 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -ENXIO; } - if (cxled->cxld.target_type != cxlr->type) { + /* Set the type of region to that of the first endpoint */ + if (cxlr->type == CXL_DECODER_INVALID) { + cxlr->type = cxled->cxld.target_type; + } else if (cxled->cxld.target_type != cxlr->type) { dev_dbg(&cxlr->dev, "%s:%s type mismatch: %d vs %d\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), cxled->cxld.target_type, cxlr->type); @@ -2476,7 +2479,6 @@ static int cxl_region_calculate_adistance(struct notifier_block *nb, * @cxlrd: root decoder * @id: memregion id to create, or memregion_free() on failure * @mode: mode for the endpoint decoders of this region - * @type: select whether this is an expander or accelerator (type-2 or type-3) * * This is the second step of region initialization. Regions exist within an * address space which is mapped by a @cxlrd. @@ -2486,8 +2488,7 @@ static int cxl_region_calculate_adistance(struct notifier_block *nb, */ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, int id, - enum cxl_decoder_mode mode, - enum cxl_decoder_type type) + enum cxl_decoder_mode mode) { struct cxl_port *port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent); struct cxl_region *cxlr; @@ -2498,7 +2499,7 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, if (IS_ERR(cxlr)) return cxlr; cxlr->mode = mode; - cxlr->type = type; + cxlr->type = CXL_DECODER_INVALID; dev = &cxlr->dev; rc = dev_set_name(dev, "region%d", id); @@ -2562,7 +2563,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(-EBUSY); } - return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); + return devm_cxl_add_region(cxlrd, id, mode); } static ssize_t create_pmem_region_store(struct device *dev, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1927a1849d82..f4cbe5c292ea 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -324,6 +324,7 @@ resource_size_t cxl_rcd_component_reg_phys(struct device *dev, #define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { + CXL_DECODER_INVALID = 0, CXL_DECODER_ACCEL = 2, CXL_DECODER_EXPANDER = 3, }; From patchwork Tue Oct 15 06:57:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 13835793 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2017F1C7B99; Tue, 15 Oct 2024 06:58:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975514; cv=none; b=HopGh/1pSHki04uY/sbl4KBLNAuEzFIMbvzME8ODJWCBlrNsPpxM/gBDvOyEeZEUK2V7MEGQVSyVbop4owGrNxGzuMsqS+r1RQIzQwnjr+NKLOMiR9PTZo1jjBg2OgajfHX6DNi6H26RNl4bq1vOq+iCHzKAiutVPuIRpYeCRnU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728975514; c=relaxed/simple; bh=rdfJ2s+ybBBsiZWkLx6WCDUJESH8OwrCvCQaJyX1YaQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cAyKMWMYO5TK1/9FkkxNxJLYYRPgmB5zqq50UNLEF9HuI7ywChf1b+XFCyCcSTZJyhqwlcGjD6BuPZhA4He17ga0c69l96hWoVRr1P6oH6GSbY5+PKCFCtLX9DsPY3fTgIpcaUqtA8C8bVmVFuu84HYETNimFmeXxNAy3xhLnNM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bYi5aUPq; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bYi5aUPq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728975514; x=1760511514; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rdfJ2s+ybBBsiZWkLx6WCDUJESH8OwrCvCQaJyX1YaQ=; b=bYi5aUPqVRP46V+ZB2r3ZoK3ZSoAh+gTaLORphU6YtGtc2EPfKrBYXZu iEXFEFzOlt1VAvBYate0TOTI+N4sEldtISts8dKb3UE+RuPH+CPAJMZRu EXLXtelhycCDBiQWoCQQJCWmwWGasJY0YjbKtev5gR1lTMMajk4Rv9wt/ +Zp0cU/RTpoZ163G/lbEMSi1nN0N6UOnYJZ7HmsucvxNJg1O37zA6a2P2 lfDkUo3/4jHlYlf4ZymMfsqpbxgGtGichuylTxcVbgXU5VQ8LSoKdFyIH /nBMlkIrIa5eLX0JPljDj9Pce/stc0qCh7l1ag19uAGq3kpGPls32w7Wi Q==; X-CSE-ConnectionGUID: UpVmXI9SR3yYLwFu3iFprA== X-CSE-MsgGUID: CySwF2NbS5is1dYqmyn+Lw== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="28447876" X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="28447876" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:33 -0700 X-CSE-ConnectionGUID: LPAqurBaQ/yaUiD/0Ni2Iw== X-CSE-MsgGUID: /2ss5ox9QNSaykbqey4WHA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,204,1725346800"; d="scan'208";a="81776923" Received: from yhuang6-mobl2.sh.intel.com ([10.238.3.32]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Oct 2024 23:58:30 -0700 From: Huang Ying To: Dan Williams , Dave Jiang Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Huang Ying , Gregory Price , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , Ira Weiny , Alejandro Lucero , Ben Cheatham Subject: [PATCH 5/5] cxl: Avoid to create dax regions for type2 accelerators Date: Tue, 15 Oct 2024 14:57:13 +0800 Message-Id: <20241015065713.308671-6-ying.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20241015065713.308671-1-ying.huang@intel.com> References: <20241015065713.308671-1-ying.huang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The memory range of a type2 accelerator should be managed by the type2 accelerator specific driver instead of the common dax region drivers, as discussed in [1]. [1] https://lore.kernel.org/linux-cxl/66469ff1b8fbc_2c2629427@dwillia2-xfh.jf.intel.com.notmuch/ So, the patch skips dax regions creation for type2 accelerator device memory regions. Based on: https://lore.kernel.org/linux-cxl/168592159835.1948938.1647215579839222774.stgit@dwillia2-xfh.jf.intel.com/ Signed-off-by: "Huang, Ying" Co-developed-by: Dan Williams Signed-off-by: Dan Williams Reviewed-by: Gregory Price Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Cc: Alejandro Lucero Cc: Ben Cheatham --- drivers/cxl/core/region.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index d709738ada61..708be236c9a2 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3473,6 +3473,14 @@ static int cxl_region_probe(struct device *dev) p->res->start, p->res->end, cxlr, is_system_ram) > 0) return 0; + /* + * Accelerator regions have specific usage, skip + * device-dax registration. + */ + if (cxlr->type == CXL_DECODER_ACCEL) + return 0; + + /* Expander routes to device-dax */ return devm_cxl_add_dax_region(cxlr); default: dev_dbg(&cxlr->dev, "unsupported region mode: %d\n",