From patchwork Sun Mar 24 23:18:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13601011 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F56615E802; Sun, 24 Mar 2024 23:18:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711322296; cv=none; b=N3H+8yxH9MBqABwxOs5HWu95i7n8Te+rH7YomDJStGSQo2wMDsQY9ww3KjvKb8yaAdG+0O/PlEp3f/o49IxGnmhVC5Tg/IK5HrCDe2LuwKnh6wanmVYt4g+xb8N3HEcxMulUpbg9U8NU2BYC0YqTMMFYAGNPhTMJ7a+Or53801A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711322296; c=relaxed/simple; bh=M+medqUQU7lSqKA8l3jnSvTpXlTcFSndgvbsEVhVGSE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gEInjHlyXJNSh4UNelLiGLayVL26Q3iXe/7SIEoaFHl+ToAIhiY2bAa4+SdrFSk2NAaYvsPVoEGolMzR3Tmb+pm6h1xJrd2HYEJTtmQyH06yt0kLuNVw2kJerocA19YHqBmfeenuQLFdNPX4MlpXtv/BKZHV1ck6Wl1Qy3KPHGA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=R910GRuI; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="R910GRuI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1711322294; x=1742858294; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=M+medqUQU7lSqKA8l3jnSvTpXlTcFSndgvbsEVhVGSE=; b=R910GRuIV+e9EdIJftqoKllO0TrIJGfS5bZz5obDCEVCVbZKXtZE446i BClirswQyUcJ6qko32wB7r0pS1W5BUDzJuJ20QgPtYD1OzWsxi6EOm+aL Gg0Hdi8eSQzwOyXNG8G9OeFFYa8yeCt+d2XqVQLyPMGA1ML3oW6Wv7wUh dC1qYHZbmuUlhAxzsX0TckAzS+zhkV0qRUbqKA6GizkUwGHYwN32P+EHm +3A5e770zm5XMMWu+0wSpVNT0xLZFwaF10Wegjv/oU52kZbc7taK2N2YC n7qNiyE8xVhDaY3JdHUAIv2ey+RV3dAd32qy+NkPFqbeAlDI9nPojcKe7 A==; X-IronPort-AV: E=McAfee;i="6600,9927,11023"; a="10078005" X-IronPort-AV: E=Sophos;i="6.07,152,1708416000"; d="scan'208";a="10078005" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2024 16:18:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,152,1708416000"; d="scan'208";a="15869353" Received: from iweiny-mobl.amr.corp.intel.com (HELO localhost) ([10.213.186.165]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2024 16:18:12 -0700 From: ira.weiny@intel.com Date: Sun, 24 Mar 2024 16:18:09 -0700 Subject: [PATCH 06/26] cxl/port: Add Dynamic Capacity mode support to endpoint decoders Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240324-dcd-type2-upstream-v1-6-b7b00d623625@intel.com> References: <20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com> In-Reply-To: <20240324-dcd-type2-upstream-v1-0-b7b00d623625@intel.com> To: Dave Jiang , Fan Ni , Jonathan Cameron , Navneet Singh Cc: Dan Williams , Davidlohr Bueso , Alison Schofield , Vishal Verma , Ira Weiny , linux-btrfs@vger.kernel.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org X-Mailer: b4 0.13-dev-2d940 X-Developer-Signature: v=1; a=ed25519-sha256; t=1711322284; l=4932; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=E2n+KEwFlMb8AhAZBsqU2WmNuZ+3LaZb4X01l2hoYBk=; b=tTH4ZAwHdfKGAQPOCI5pNgXdEob7CTitZZvEt7eLXnGvKVR1KOpNg6o1Z+oNzN8TsI8B/c66v cIINC79d5udBywpso7Bk2MnApBWLBCH3y2qzBGkSKytjp5XCEaL7iyg X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= From: Navneet Singh Endpoint decoders which are used to map Dynamic Capacity must be configured to point to the correct Dynamic Capacity (DC) Region. The decoder mode currently represents the partition the decoder points to such as ram or pmem. Expand the mode to include DC regions [partitions]. Signed-off-by: Navneet Singh Co-developed-by: Ira Weiny Signed-off-by: Ira Weiny --- Changes for v1: [iweiny: eliminate added gotos] [iweiny: Mark DC support for 6.10 kernel] --- Documentation/ABI/testing/sysfs-bus-cxl | 21 +++++++++++---------- drivers/cxl/core/hdm.c | 19 +++++++++++++++++++ drivers/cxl/core/port.c | 16 ++++++++++++++++ 3 files changed, 46 insertions(+), 10 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index fff2581b8033..8b3efaf6563c 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -316,23 +316,24 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/mode -Date: May, 2022 -KernelVersion: v6.0 +Date: May, 2022, June 2024 +KernelVersion: v6.0, v6.10 (dcY) Contact: linux-cxl@vger.kernel.org Description: (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it translates from a host physical address range, to a device local address range. Device-local address ranges are further split - into a 'ram' (volatile memory) range and 'pmem' (persistent - memory) range. The 'mode' attribute emits one of 'ram', 'pmem', - 'mixed', or 'none'. The 'mixed' indication is for error cases - when a decoder straddles the volatile/persistent partition - boundary, and 'none' indicates the decoder is not actively - decoding, or no DPA allocation policy has been set. + into a 'ram' (volatile memory) range, 'pmem' (persistent + memory) range, or Dynamic Capacity (DC) range. The 'mode' + attribute emits one of 'ram', 'pmem', 'dcY', 'mixed', or + 'none'. The 'mixed' indication is for error cases when a + decoder straddles the volatile/persistent partition boundary, + and 'none' indicates the decoder is not actively decoding, or + no DPA allocation policy has been set. 'mode' can be written, when the decoder is in the 'disabled' - state, with either 'ram' or 'pmem' to set the boundaries for the - next allocation. + state, with 'ram', 'pmem', or 'dcY' to set the boundaries for + the next allocation. What: /sys/bus/cxl/devices/decoderX.Y/dpa_resource diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 66b8419fd0c3..e22b6f4f7145 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -255,6 +255,14 @@ static void devm_cxl_dpa_release(struct cxl_endpoint_decoder *cxled) __cxl_dpa_release(cxled); } +static int dc_mode_to_region_index(enum cxl_decoder_mode mode) +{ + if (mode < CXL_DECODER_DC0 || CXL_DECODER_DC7 < mode) + return -EINVAL; + + return mode - CXL_DECODER_DC0; +} + static int __cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, resource_size_t base, resource_size_t len, resource_size_t skipped) @@ -411,6 +419,7 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled, struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); struct cxl_dev_state *cxlds = cxlmd->cxlds; struct device *dev = &cxled->cxld.dev; + int rc; guard(rwsem_write)(&cxl_dpa_rwsem); if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) @@ -433,6 +442,16 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled, return -ENXIO; } break; + case CXL_DECODER_DC0 ... CXL_DECODER_DC7: + rc = dc_mode_to_region_index(mode); + if (rc < 0) + return rc; + + if (resource_size(&cxlds->dc_res[rc]) == 0) { + dev_dbg(dev, "no available dynamic capacity\n"); + return -ENXIO; + } + break; default: dev_dbg(dev, "unsupported mode: %d\n", mode); return -EINVAL; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e59d9d37aa65..80c0651794eb 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -208,6 +208,22 @@ static ssize_t mode_store(struct device *dev, struct device_attribute *attr, mode = CXL_DECODER_PMEM; else if (sysfs_streq(buf, "ram")) mode = CXL_DECODER_RAM; + else if (sysfs_streq(buf, "dc0")) + mode = CXL_DECODER_DC0; + else if (sysfs_streq(buf, "dc1")) + mode = CXL_DECODER_DC1; + else if (sysfs_streq(buf, "dc2")) + mode = CXL_DECODER_DC2; + else if (sysfs_streq(buf, "dc3")) + mode = CXL_DECODER_DC3; + else if (sysfs_streq(buf, "dc4")) + mode = CXL_DECODER_DC4; + else if (sysfs_streq(buf, "dc5")) + mode = CXL_DECODER_DC5; + else if (sysfs_streq(buf, "dc6")) + mode = CXL_DECODER_DC6; + else if (sysfs_streq(buf, "dc7")) + mode = CXL_DECODER_DC7; else return -EINVAL;