From patchwork Mon Feb 6 01:02:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C998CC63797 for ; Mon, 6 Feb 2023 01:02:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 634EF6B0075; Sun, 5 Feb 2023 20:02:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E4626B0078; Sun, 5 Feb 2023 20:02:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AD326B007B; Sun, 5 Feb 2023 20:02:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3B9E66B0075 for ; Sun, 5 Feb 2023 20:02:38 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 10DC9C0542 for ; Mon, 6 Feb 2023 01:02:38 +0000 (UTC) X-FDA: 80435066796.04.DB33B31 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf24.hostedemail.com (Postfix) with ESMTP id 0416F180008 for ; Mon, 6 Feb 2023 01:02:35 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dthLYZQH; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645356; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fcoNHcMUcJJwPs1KEbhTpNNlrtjbqY612S+nrL6Mvp4=; b=mX6i2z1BbzAbZfA1g9/6viFw2zYAQ22mBazY/dODSQ6HlFN5pEUmFYtD01G/A2FQgqiqyB qLbv1eEIFvAQwO5XvfTK2+EzIdptqRy6UnKX3esNCFZy+ZJ2MdOrx5xkITHEUo3bRO4TtX f4l82TInMOUxEWkjzjXmrIP7mCRwGjs= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dthLYZQH; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645356; a=rsa-sha256; cv=none; b=Rt+kAf9a3+eySbl9reK9e27mktrI6SQgz/VMHQzm0XGm5Voe3zNfoRR8psP3l95bNJx0mC jd2wo6wrAhk9LBGo/tqMBL+ab1qXnmMD2h57iEf8SDmPdVmp/BPOvU3dgFxn4cSXuI2bEz hyjGoqJW77BTRYM/w4V9NnkP2aaN6iU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645356; x=1707181356; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+oQ5qejLH1X6RwG1ebhyZ7IMLgkFgjFEswwe1eorwQo=; b=dthLYZQHJ2m72UA9oqCiebnaIgjimLRTAufyejT/fw2nNSS9PwsOYeMC VCeGy9JvCRBHAdG6Z9Enw4L3KI4N2/sm3mFKhB/J6i71faDhmDTcMiur0 krgqQuOMgJ8clBl7uIBmHm8k17ULHlufOsluqSO1UHPGN892csto1dMPo yzkTyZcx7aeUSiPWvyWfmswsHGy2Cf5HxCGhIt8rdAUXjqtFNBcGaJkeZ 71tae0J1k/+Qqvjvv20X13v0/0rzY1PyZDiuJpQQ4d+T1XMFM22Gs2a5p MRpXYT3xKVxuOlYgiouPDUgYbMIdrPt2otiof5N38sQRvfFUjZSz2WNcJ Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="331243775" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="331243775" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:35 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="643855715" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="643855715" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:35 -0800 Subject: [PATCH 01/18] cxl/Documentation: Update references to attributes added in v6.0 From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:02:35 -0800 Message-ID: <167564535494.847146.12120939572640882946.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0416F180008 X-Stat-Signature: 4cxms88kw14o9qwtck41uiwey1o3nhjx X-Rspam-User: X-HE-Tag: 1675645355-980553 X-HE-Meta: U2FsdGVkX1/1cWaHakiqBq1EI8bJNUO8m104udpKSp7o1243P/bSiVzbVemOCBjx0N6lmBr1vY43Wm8qgCEEl9R4cbGe83IwCmgL0950DtIWd1VyZOK4FPLXt236W7pRrUWFUeTcn47lp4rDwkXcmFKbn5oiy5HRwtb/keed86gbglX0UVBmpqTE9D1xmjSKHN3SFfN8d3xPnm0uTqMmx5WmR4dk0DU3+GIoYpKWO6AnFOsJGPEat0YZG33qodfvKAEKQZzPmc4YzNk7TuP03ZiNF0q89y/LtrsTJUNTBxC0LFqTrmKhKHh+EGpNcaL1uAUHaQ4ZiM0aPUIfXLDv0LDITvkLPnn5BVKQ3+LZcKOBBvZXdQbc5zLYkwljZJNuW2fxwDVapMqpl2KAXtIckFvhF9TGRIdWnhuWOk4ZF9SnpUJnl5BDhotL0e+31DIxtzIr4PVhj/sFHSuw3/oAmTEfemgal1T2OBmoz9ILl1YZ9kRmadGfOvN2++j+OuFF2b0bUYZGgqhAnP1zhARubU8j1u234V9c17NPV7ZhSuz8VYeYg1dy4F+exWT7fGigj1dXAD5ooRaLiNkMWw6bM8FCgpm1UfovBYhN0UUjHBIdygw/pxuEvf3wvhPZNZux9R0wDn4erBEYqLmONssneTfSeKl2F6et8vAs8bIxIyiHIx5j5FKTdPRMJUR/MIJZiRLo/Ov9hpR6IuaeoGv5PfyRjtLV2OnDy9WVtzuebbtyfR7liHrq49qMjbVw3MuML/1hKKpvStZVrM7he5vJFCmatRTvPCfofzjDyMUaYI79cCohfKV27Dw1RWff060h4/EgTDK7EM+6YCgaQiY1dzyQWsua7bCl0e40BLP9BxQGXa3grpmamY1oRK/6uprOwzmvUCWaMJljBCbdrIuk0BTqoKsTOWnbMxfPEVC6u23AXDuNDl2FGmTPwaMk4jYPvWAfr+tQuwjaap7sGoE gscTvCTN cwrhfcxHAOhuCb04iLeAvFHZj9dP/aYSdp7L+Om7zDkrMRIuT5Js1dNUhA4O6BYyi0Nak/rRMeYlVCj511PmMt94UH5PThRPvOYIWLcVPfX3yknfHuV7gU7CDYA6T94/zt5ItAgCuMYfhY/7yi8+JfMKyBk9yYHkRhMnlxIHUpKldlm077qdFCaVjUCD/BpiQDfqzGOV1NpGTeV3bhEbNTLdfxyBKT/rXDv6f8TL8q+x4kOwNeYE7U0fznEiOmatIHZbXQVyTkc6Bacqnw2RupAHWfqBAxTjbiqC4al0Eqn/U+KfwzUJdvVxXHgKePH0jKZP+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Prior to Linus deciding that the kernel that following v5.19 would be v6.0, the CXL ABI documentation already referenced v5.20. In preparation for updating these entries update the kernel version to v6.0. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Gregory Price Reviewed-by: Davidlohr Bueso Reviewed-by: Ira Weiny Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- Documentation/ABI/testing/sysfs-bus-cxl | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 329a7e46c805..5be032313e29 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -198,7 +198,7 @@ Description: What: /sys/bus/cxl/devices/endpointX/CDAT Date: July, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) If this sysfs entry is not present no DOE mailbox was @@ -209,7 +209,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/mode Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it @@ -229,7 +229,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/dpa_resource Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) When a CXL decoder is of devtype "cxl_decoder_endpoint", @@ -240,7 +240,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/dpa_size Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it @@ -260,7 +260,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/interleave_ways Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) The number of targets across which this decoder's host @@ -275,7 +275,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/interleave_granularity Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) The number of consecutive bytes of host physical address @@ -287,7 +287,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/create_pmem_region Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process @@ -303,7 +303,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/delete_region Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (WO) Write a string in the form 'regionZ' to delete that region, @@ -312,7 +312,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/uuid Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write a unique identifier for the region. This field must @@ -322,7 +322,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/interleave_granularity Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Set the number of consecutive bytes each device in the @@ -333,7 +333,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/interleave_ways Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Configures the number of devices participating in the @@ -343,7 +343,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/size Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) System physical address space to be consumed by the region. @@ -360,7 +360,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/resource Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) A region is a contiguous partition of a CXL root decoder @@ -372,7 +372,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/target[0..N] Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write an endpoint decoder object name to 'targetX' where X @@ -391,7 +391,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/commit Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write a boolean 'true' string value to this attribute to From patchwork Mon Feb 6 01:02:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22CC1C636CD for ; Mon, 6 Feb 2023 01:02:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C00EB6B0078; Sun, 5 Feb 2023 20:02:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB0E86B007B; Sun, 5 Feb 2023 20:02:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A796D6B007D; Sun, 5 Feb 2023 20:02:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 99AE96B0078 for ; Sun, 5 Feb 2023 20:02:43 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 71BF91A03FE for ; Mon, 6 Feb 2023 01:02:43 +0000 (UTC) X-FDA: 80435067006.13.CF7C188 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf24.hostedemail.com (Postfix) with ESMTP id 59DD0180002 for ; Mon, 6 Feb 2023 01:02:41 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="JBq5/8tj"; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1+dl7gdU5BIw/whwSe9Fk7DKYVHBtFbRO6x5tt6HwSw=; b=eBMLkeU8kQZHEMQzgJloIOP18LuSKw/3OkUoiv6UDWsGtciXvOgDtFIn1JlJcqlNHCCF88 fsJTvqHIodpufSKsrAHU9yITVifvzqC7jL1AdHYvLFDsik5k5/VPeM5yFnHpILdsj0pbMQ tvlFqxPyJTDI4RIW1Hgc51lUdMiBs+c= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="JBq5/8tj"; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645361; a=rsa-sha256; cv=none; b=hcEJbqhbuLe111VgIOh1D0lOyOUCHnNY2epx1Wv/b0r+7zPqwH2JT3tjII1l7d2K2KChdi 7c8ASMZT+iAdUmTh2VZ00ifEu+N7vzHh5PIj6ApWdnHqlBGR4Q+X9KSvdEApFCde7sfiO8 vnW00SzXWDwl4uKc47ycxoCrHCAD/+8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645361; x=1707181361; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UOKI+/A98jKePPsZHuytj7Ccuyq4WgZ+jxkAtz+DXQ4=; b=JBq5/8tjjChrlSIbXtpzVmElLFRJbt4qZKfb8xkCnrhAN51PLeBfh7Gu 1BgG10LCNGDvEtwjZPdFaMsbk1euVlUZlwjo0RNz4QKHgQR68jMY1FcvW 9GtjYhal95kUFr42WM0sjKU1z3rK0OIUmw/VV2k3XYFlsZ+qTO1XrRx+x PFGOUSDRLWkDPeAwoES1EAp08ZTN9MLYMleqpqXIu+OyNjApidWZWXkm6 OKdk0rEACHO38vXFuwIwMSklmK1b0HAYksJn3Mh3Dnj9qQcsiQEF/TqWO z+ev0j4zrORXdVfoDKIGhhULlZZV5GUQ2fJvPS6l2dy5AVd1kJmAN2O5C w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="331243786" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="331243786" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:40 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="643855730" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="643855730" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:40 -0800 Subject: [PATCH 02/18] cxl/region: Add a mode attribute for regions From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:02:40 -0800 Message-ID: <167564536041.847146.11330354943211409793.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 59DD0180002 X-Stat-Signature: poesurmdm7waru5at49og1e7wr5airnt X-Rspam-User: X-HE-Tag: 1675645361-756393 X-HE-Meta: U2FsdGVkX1+g4iac/r0No74yhNHzGUVDCA7NyZ4laCI9+luMYbt3XXz+yIiNcueCOK9epT70Pvnci34qaD/o6YxdUZQ81lZy1N2/g7vJ+phQvsn0Mh7I46Qq9Hl9ymOw1FvAu4dR4WNdMYFG05dbwmdfsJGrk2Ck2POfDPWv4F56ohyQEwLaBhpHovOriLqcOdLLc4hMRvV75Qjyjc0gP+iT57ZsJljDiexDIGbLbG77jA4hf7jp+G5d65dWyjXhK23rmprG/9Rp9hJTVam4LSJjfHQOnxBuHqyBtpevkXk/0ZxOmLllou5dMmRNF3hRf9HZStARCeUvbMM4chG0sdnyLhSPzhGO9My7rxJnKUd/flNBuj6KCTMg3ZaHJd3yPCFPvUayQ1xRo0hbyYiKNG9J04brYEzGVdtwxbxFeMt5thqe+0NbPmpFjz76HHKC5wBob0enPONbVi5Z6QfS6x8laXe8/EHbvpeylx6IUtneTh5ToMPykoBc2a4/FDdwxToIAxV4f3L4YuDZ9Vm8xeOZK3lrAeJYkBYgLNQJOmQ1vQqJlaLZzC9qjoQMlhhx8XbOFrEXrI6S1gXYEi+QJ9Ca/nienUKlfWdN6gdPO+fZFOvh6fXw4UtdZYLw/g1/Tpqmkrqvm1K0Q/Xw+5wlTqzSTfCcLPuSmWifVDsvABnsWHpWs3PzKwwNiRqYdg97xKsRUheI4LAbtJ/lIdPcWFiVux67PMwgsDof/K9Zu7jdTTwQoKlx7MFemoSvia783HbfT6rMjHPFrEhyidxDNyxbzSZKW1Vb/f1GPc6aZjy6rwza5gJx+rDd23Pi73yBU7yIwkO/mXEzBPAV3ZP3G4P7CJpEuoZ4urEp+OQFkqACBBFLBDQGVykSF61iQg++9bgbnAqtwRMayPGv6honZCtaxKQhBzNg5kEyRzlgvQb1wrpI/Vro1VVAIdwmEIn1VefDBFpeve94jjzbRSq DJouCHem qqZq0zJD/Kp1ShzLOkbuWxu50YnQPH8PgkzHQWBL3sGfgVJYPR1qbcWh2HO64iMbTwSwBlSW5n3K/92IR/tW1jM6U5GSrmEWBDP295PQldJTS82R6g4RoQ8lfZFGehXbD7ibf3h5E22PcmKrt6HIf1WsM3QWTWDVM/rc/A+wPBhk/GBkCteR2pye1LYMRXRquNbtWzTB2mo3rNlSczIo/mQeeq1L/syjrw2qnN6Cbb4q8AOeBn5Hj3+r3gciy72AGAXMtGk7VnpnV+jh5+Oa5+9Lqng== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for a new region type, "ram" regions, add a mode attribute to clarify the mode of the decoders that can be added to a region. Share the internals of mode_show() (for decoders) with the region case. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Gregory Price Reviewed-by: Ira Weiny Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- Documentation/ABI/testing/sysfs-bus-cxl | 11 +++++++++++ drivers/cxl/core/port.c | 12 +----------- drivers/cxl/core/region.c | 10 ++++++++++ drivers/cxl/cxl.h | 14 ++++++++++++++ 4 files changed, 36 insertions(+), 11 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 5be032313e29..058b0c45001f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -358,6 +358,17 @@ Description: results in the same address being allocated. +What: /sys/bus/cxl/devices/regionZ/mode +Date: January, 2023 +KernelVersion: v6.3 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) The mode of a region is established at region creation time + and dictates the mode of the endpoint decoder that comprise the + region. For more details on the possible modes see + /sys/bus/cxl/devices/decoderX.Y/mode + + What: /sys/bus/cxl/devices/regionZ/resource Date: May, 2022 KernelVersion: v6.0 diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 410c036c09fa..8566451cb22f 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -180,17 +180,7 @@ static ssize_t mode_show(struct device *dev, struct device_attribute *attr, { struct cxl_endpoint_decoder *cxled = to_cxl_endpoint_decoder(dev); - switch (cxled->mode) { - case CXL_DECODER_RAM: - return sysfs_emit(buf, "ram\n"); - case CXL_DECODER_PMEM: - return sysfs_emit(buf, "pmem\n"); - case CXL_DECODER_NONE: - return sysfs_emit(buf, "none\n"); - case CXL_DECODER_MIXED: - default: - return sysfs_emit(buf, "mixed\n"); - } + return sysfs_emit(buf, "%s\n", cxl_decoder_mode_name(cxled->mode)); } static ssize_t mode_store(struct device *dev, struct device_attribute *attr, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 60828d01972a..17d2d0c12725 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -458,6 +458,15 @@ static ssize_t resource_show(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RO(resource); +static ssize_t mode_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + + return sysfs_emit(buf, "%s\n", cxl_decoder_mode_name(cxlr->mode)); +} +static DEVICE_ATTR_RO(mode); + static int alloc_hpa(struct cxl_region *cxlr, resource_size_t size) { struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); @@ -585,6 +594,7 @@ static struct attribute *cxl_region_attrs[] = { &dev_attr_interleave_granularity.attr, &dev_attr_resource.attr, &dev_attr_size.attr, + &dev_attr_mode.attr, NULL, }; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index aa3af3bb73b2..ca76879af1de 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -320,6 +320,20 @@ enum cxl_decoder_mode { CXL_DECODER_DEAD, }; +static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) +{ + static const char * const names[] = { + [CXL_DECODER_NONE] = "none", + [CXL_DECODER_RAM] = "ram", + [CXL_DECODER_PMEM] = "pmem", + [CXL_DECODER_MIXED] = "mixed", + }; + + if (mode >= CXL_DECODER_NONE && mode <= CXL_DECODER_MIXED) + return names[mode]; + return "mixed"; +} + /** * struct cxl_endpoint_decoder - Endpoint / SPA to DPA decoder * @cxld: base cxl_decoder_object From patchwork Mon Feb 6 01:02:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A405C636CD for ; Mon, 6 Feb 2023 01:02:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B2966B0074; Sun, 5 Feb 2023 20:02:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 262336B007B; Sun, 5 Feb 2023 20:02:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12A416B007D; Sun, 5 Feb 2023 20:02:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 045E76B0074 for ; Sun, 5 Feb 2023 20:02:49 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CA44D1608FC for ; Mon, 6 Feb 2023 01:02:48 +0000 (UTC) X-FDA: 80435067216.25.702AA3E Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf24.hostedemail.com (Postfix) with ESMTP id 9ABF8180004 for ; Mon, 6 Feb 2023 01:02:46 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=e7t2nDwo; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645366; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SYcqHqXqWBmUxiNkmsR7zsutAFtrAKEoBuJf+0yeadY=; b=AOGibk8HqybjL56BWLnuVu5YOLhGLYK2gm7OsGwNKpdFuVxeCpwK/WjIhoQ/WqRPAlUJ36 Nb/L54j3szK0SzOx97KM/fKHQGlruv9fTuMsQ6N3p3j41llzVg0KhtE7yma6yhgFzOgQEI /jFSMiHNrkOpwWauFlK/VNtvk4Xpf9g= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=e7t2nDwo; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645366; a=rsa-sha256; cv=none; b=b3qTRZIeA/olRTFxt6U9R20DF0KJymXe10PuBo1iTcA6vOCaxjUP/8VRObL65+bTGICbcc UWGV+Qgk6q2io6nofqAq327mxmFgCjWxu4aLwlt8dJtZNqll8KqSl6dzXPKG/BG+xzNB2r dmU3fV13Rtn9dIOk41QytgWEI0T/6vg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645366; x=1707181366; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yb8ZrYfywa+KhatYaUiHluR1JdOlK0Jjr6L/f1PwJLc=; b=e7t2nDwoNb+uIEsh993w0izEa8mmYBvCqYCqwTBSq4PDXkOy/vxOB6oU wpd21Z4fF/iH0hBdxWocp3LViKoQDIMX3lXnJpj6gZHRjHB2ubl8TVBvt XOFMcPKyyp/DKBzzlyosbg4HVpjX5vTvG8J2AQ/rwbV6SD9NGB/gGYd3B jhrca1sILgvOUYmSpDrd8X0UREgYGTEQW31ik5RgQ6vlTqETADsRN2UJb wukBZvCO+XMkiZBJUYGJP6yelijxsi6DtaUixEbC8US5XT7NxlxKtr7me Cr2EKOXvWCKh2m9Y+xAZxE8M9+w/hjY41ON/80xpfzZveEE8EacckMD1V A==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="331243791" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="331243791" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:46 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="643855738" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="643855738" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:46 -0800 Subject: [PATCH 03/18] cxl/region: Support empty uuids for non-pmem regions From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:02:45 -0800 Message-ID: <167564536587.847146.12703125206459604597.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 9ABF8180004 X-Stat-Signature: ysdraiedzhzqciz471ckpb8spjxy7tr4 X-Rspam-User: X-HE-Tag: 1675645366-840354 X-HE-Meta: U2FsdGVkX1+RMsA9PMwFvKORcmWnkGQT60XA+cIG3bzamcUg+3hdoymbVXfz+tkxtTdN90BvILeOBFlqA8VfFn7xN6YxVtXEY4oAyJt+q6NTX/cDBEp3wdYBepTb1pZH1if1eRJaFjalcoC35v0unvPcUHZ22HI8C7abbmcTamoClZ9L9rvFgCtKQbNkXeJ/lRfKy+nk8L2y7IbmVm40qqU0h5BqTzVPFCL2lZAl1iahO46wII2MvBdMZT8LkB9SinbOW4hM5hkMjHzK5HaMKYDnyJIBkpfZl4FJO8fYRZueVPsJ+zOA81oflxK1bIvhR8ifF4UDiCtW23c25lws2vbqxF4VIK8ORi/KlGxWbaoG9FiTqPw/qyn1rJgmAr9a/cwyq7D0Ies7huDpleP+u7PHIPrPkntf01KznE+RPnLRoLumBXqkKwlxhR+URDQYsZS8roZ/SImdiEWkZZgQBbGyw4nF3b7Hz06xw02tszbJ618QU2wBDQxtDTa5Wm58ZZlbBLHwp6FoX7TXTNtvZ3cia9ITUFUMfAx7jcTpC1uotUVP/XtO5pYd1x1aFuBIZDC8wPTOhNkLGo3/CILGffaXn+Ffc+tBHFHHbX3iSr5AUIbXx7c3wEWBEXhNj9ASPQ8TIQ+mZNglhSfDlhulVBqUgKDFMJRHyeJFvSbkxqmYX+9eTTf87mbr1rb+o9sNoR5GWDZ2mno8erWW8hJfgbk6iM33EwtDXcPPgwzeMawYBW4L97mrJkdQ9d+hJ6Rs9L53ZRbhuwVLLw4WpmhgIYgx5P2QVRFEWBBg/x/IdAFnBxHaJYa9VGptjN+QywKElfYphbglsDex1llqHtisTH3NJGtcT2sdbugnKkATcfhMEk14WK8bMC1y6vuL9RfIkE78AI8rtMNBMDMcp2f3PNOfS3qW54v0Ck2/Zx4iayXKCdTQJ9chseE8WucGmb6klwfBm2MhbKAYgClLPNI ha9lW1bg LcH6T/ZkM3sXy0lh5gGwENaRb3EvWX6hH5BVj5iYhQNPhnASxR7a0AicBHvfkPNTkCHprA/bSAhLzIVx7oN5cAMuYt07oYBv5L7HB9thZ9Kvc5Qr4W1dhCUYPOEIGQ18td/QBY9Hpp/oBIlpMClabtDlIykddzApDY24sxlR/1G2ZLXn6HX3h+LzaYZUao3EA9+GhHMefvlnTmjdb6Whe+vgNDWDms1XWZyiDNe9Q4AdVdc2sgqLXah7iwBVvbyebPvyK8SPFT2m9YeWlK/OwE7N1PvKcTlm3BvJH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Shipping versions of the cxl-cli utility expect all regions to have a 'uuid' attribute. In preparation for 'ram' regions, update the 'uuid' attribute to return an empty string which satisfies the current expectations of 'cxl list -R'. Otherwise, 'cxl list -R' fails in the presence of regions with the 'uuid' attribute missing. Force the attribute to be read-only as there is no facility or expectation for a 'ram' region to recall its uuid from one boot to the next. Signed-off-by: Dan Williams Reviewed-by: Vishal Verma --- Documentation/ABI/testing/sysfs-bus-cxl | 3 ++- drivers/cxl/core/region.c | 7 +++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 058b0c45001f..4c4e1cbb1169 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -317,7 +317,8 @@ Contact: linux-cxl@vger.kernel.org Description: (RW) Write a unique identifier for the region. This field must be set for persistent regions and it must not conflict with the - UUID of another region. + UUID of another region. For volatile ram regions this + attribute is a read-only empty string. What: /sys/bus/cxl/devices/regionZ/interleave_granularity diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 17d2d0c12725..c9e7f05caa0f 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -45,7 +45,10 @@ static ssize_t uuid_show(struct device *dev, struct device_attribute *attr, rc = down_read_interruptible(&cxl_region_rwsem); if (rc) return rc; - rc = sysfs_emit(buf, "%pUb\n", &p->uuid); + if (cxlr->mode != CXL_DECODER_PMEM) + rc = sysfs_emit(buf, "\n"); + else + rc = sysfs_emit(buf, "%pUb\n", &p->uuid); up_read(&cxl_region_rwsem); return rc; @@ -301,7 +304,7 @@ static umode_t cxl_region_visible(struct kobject *kobj, struct attribute *a, struct cxl_region *cxlr = to_cxl_region(dev); if (a == &dev_attr_uuid.attr && cxlr->mode != CXL_DECODER_PMEM) - return 0; + return 0444; return a->mode; } From patchwork Mon Feb 6 01:02:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D17AFC636CC for ; Mon, 6 Feb 2023 01:02:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71E596B0075; Sun, 5 Feb 2023 20:02:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6CEA56B007B; Sun, 5 Feb 2023 20:02:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 596F96B007D; Sun, 5 Feb 2023 20:02:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4BBD86B0075 for ; Sun, 5 Feb 2023 20:02:54 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 195E11A03FE for ; Mon, 6 Feb 2023 01:02:54 +0000 (UTC) X-FDA: 80435067468.29.73DFEE3 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf24.hostedemail.com (Postfix) with ESMTP id 19C73180002 for ; Mon, 6 Feb 2023 01:02:51 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FTdgOzxi; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645372; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Cve9O0QR++TCuVy7x01qT9ZK650rmv2E/f3LeUgFdo4=; b=wd2ZKqklzcF+sYObbfhBWzvm7jGFILC44XuqHkepctvFfGMI4buzseTCVW1EdbcviOIoUG 879o1kxgLv9tzdGIjpSxD1l4RtrJ7CnrXtC52d0y2ofT4aToK5zBg0YqvLuwPie1SwCsZG oIoQtXk8A6zlt3fVeUZ21F9BaBLwzmk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FTdgOzxi; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645372; a=rsa-sha256; cv=none; b=aSGM+xJ9eSWSsGtpxTiN8oCIDpFCumKgsB4qMEqmFmsiX/3EJAs4KSWm4TAZkfqdGdS3l8 /0ymGa/u+TKPYlLgiioEa8/6nbowIp4IUBK/oBeqR7Itu6iwqui+1gBHBRQCeoVIk3rK8m 5TLPg5ig4sIikI+kf88wyOG4Y0lfliY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645372; x=1707181372; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yszGLPkFMXedVGllSqe6JjeO5D0jX4yLcgUfwilZAzI=; b=FTdgOzxiK/n8EQrct1NKXny2jtk1ViZwiVEpgp9OZkPYE4CXUjZBJSUj VZqMuRxyEbtehyZ11M36/P/r7n4tsbt9qu/15yhULYXKG+d0NwadZA5uE SM+sw2ibRu3KiXXkjtyL6zw95dwgLgx/KWw2mmruy7mpKlYo5yO/RQ931 Z1TkR8raZZ56FFpisyDfzoUnARRtGBkVagXqkbGJE0jhqf899RhjZMwSl PAz64pP9vJ7oILUYmdcQ/JfH3g6hgKj8YfEyejrG04X1WngqADCNrvHQe 8P1ktKEjs18+ZoWNCYjnZ9s2rjiUuqdvRREd76PUOnofiVCkEWeAWO4Ow Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="331243799" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="331243799" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:51 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="643855741" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="643855741" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:51 -0800 Subject: [PATCH 04/18] cxl/region: Validate region mode vs decoder mode From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:02:51 -0800 Message-ID: <167564537131.847146.9020072654741860107.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 19C73180002 X-Stat-Signature: xf3sfe6g3eqabkp9qjaekezn5gfpnm9k X-Rspam-User: X-HE-Tag: 1675645371-952182 X-HE-Meta: U2FsdGVkX187drxW97Ey7ZUAWDxv0oqI2xO+NF7kRcLBkkjtlVheKBamklIUX8h2T9VGNyyvBhYYTpbtj97mGEANjWU5K9EIl/N5Huk4Zck40tzI5BWn2bYPXDXcnXkSP3qLF7t+gHW/fUOsWRfLzIT1p1oLM52mQjSwp+FU6QidiLy8q2N3/2k7hvR+VMKS4LTV9nkp1OluGblzCY6RPwObLZGBrstCsLPxOqXVx6sf1buVMMEmtE70jWDpTyNS1hDNqc3xnPkNGVkkb04V7RFyqdwCp1cHmIUt3LYMFQHNcCMPpAUQveHW724oWDgWEfPjhz5l2B9jAxtHEYj7IxYw0AOtzsUMJVafBBbIjJoyJbCML5S4u2I3HqhaH4o5R+f0neP6XhXahgYdtZ7rffGF/H3cb8R7esuL23LfBznIScGafoCjfBIgk9rA5kjzSQ8dyCqL49t3GQ0YCP9nRjj6JyZtntkKBIY+cxvyDbHgRTFZIW4KdcHsVynSTCRAk+2ZYnQe7JRYRIOkI3Y6oa8gs4/nAriGrGzNFpOpcL3BkUT0qhn6NwDolx5W4th2ZWQEVnbi0KGoI2ASyxH176JH8lRpFdogXEPuVl4HdpyA+HoSQ9tw1Ju+QeUdRf57m6L1YoYk17eeFUZOd22Lx8fnwk2SWnK6yfYVyCuuRSINNNM4+h6V8pV2yy3lhyC3nVXiDUB7v38W2MmRPo4gMKW3zDhJSTcVAK+t+g1BZmcfeS8XLoyEahm7vi6+wViFajZHPZYlFl/W0Sr11YHMLaqWfB9f5gKScLu+AhQsi3A+XIeiqh/XUb30QbSC2Tnf+J3+IBkVhHysYvZXrr/JIQTXAH/sDS0Zh8oBdKf/uE9FoVBwluX7kVj930qvwzhSTAfjZihxBSWdrXwitvOa0W+cgnXoXlVpX8lcSwuy+7Oi0fZ3AUpppvBoLRRFQ69tnEFbAMHE0Ufse5BpEwU 4JO7QURG XMbpMwgA3mgqT+8OdfxCnbfFX/nQjC/xFE9Rdp1KCPo+ZlKAkeTckfNr+xScZpZsOEacJwvceFpg2PXZNcC2CN8/I0PD+5NwcZG3h2f2pv3avBLknBEZizFi/bvAvEsJ20VV2RBPfwAvnbGeiD31ZoP/24mjMmlOai9ge/B4Pw1YdEPEfu1vmumgE763XUxAVc1Zn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for a new region mode, do not, for example, allow 'ram' decoders to be assigned to 'pmem' regions and vice versa. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/cxl/core/region.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index c9e7f05caa0f..53d6dbe4de6d 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1217,6 +1217,12 @@ static int cxl_region_attach(struct cxl_region *cxlr, struct cxl_dport *dport; int i, rc = -ENXIO; + if (cxled->mode != cxlr->mode) { + dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", + dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); + return -EINVAL; + } + if (cxled->mode == CXL_DECODER_DEAD) { dev_dbg(&cxlr->dev, "%s dead\n", dev_name(&cxled->cxld.dev)); return -ENODEV; From patchwork Mon Feb 6 01:02:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BF3CC636CD for ; Mon, 6 Feb 2023 01:03:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2D8A96B007B; Sun, 5 Feb 2023 20:03:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 289196B007D; Sun, 5 Feb 2023 20:03:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 151996B007E; Sun, 5 Feb 2023 20:03:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 066376B007B for ; Sun, 5 Feb 2023 20:03:00 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D3D5E1406A1 for ; Mon, 6 Feb 2023 01:02:59 +0000 (UTC) X-FDA: 80435067678.06.64B4D0D Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf24.hostedemail.com (Postfix) with ESMTP id A7C1B180016 for ; Mon, 6 Feb 2023 01:02:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dL0oz9mO; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645378; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wEOiU6QpO3ymx0xLdLYjdGFirFQKBSl8KjCe46IoVio=; b=aKT26rCUYMHbOWl+aEr+p5E02FrC3FOrDarxe3rZOye5+u7F1C0xCjuwuKWWDT7clA+fTQ lHG6JKaDNB3QiGEbmJh3AxxNI1EhJ1OklDlt9dMxVLNR1DW6HpBqfZCjYk3YBi7W/T9ju2 pd8XmdlutRBGdBDjldo0ijrjS8mY0Hs= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dL0oz9mO; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645378; a=rsa-sha256; cv=none; b=dudUvqZwpNabL20wANTnS9PT6xFMfBqusbQMIXWlIehGJPDkTvQS+sy8Bt/mv853UFsQC7 4EweFM+NphXBaH7Dh8v9i8uhwPm9yV/qKWFuPS7OWhrJqcmr9hWkQtyNm1rrb2uaE2/o8M U0OHfHvxTnu4o/f2pSvKQ9qXijqBWiI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645377; x=1707181377; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2NFt5YIQN4Pv/UJUgAAJbE/vCys4h+IOB5VcXpizhjY=; b=dL0oz9mOolNG2TuoGvxn4nrTmr9p9Q4GzxTt7u9Vs/Ne4ZpTmlBMojLr JZnB6iS3wbppxMKVcJIIENUdQ9hLZiG5wJ25P2qpjTXvfdhSbv9TJLzsh AeHfczri/tiaxuaxrFzo4Uzqx5tWOScpxW/TK7Mgb8mR22LZ24Jh23Hzv oXr6KQpGa4AyWmadad0Htu63v3XB5lX5btTEDAR/CYsJTc/dD5YzxJkK8 Spln6wehYlxKj1At75mrrNR9BFkVxfhMq8Ounrmj+83HPFuWTScHMq5pe DMoXKZzCWo+u/rmGI+qU9CiCJ8Ayrc5ifVjRnxsIGSanhOe3A7pvTj2V8 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="331243808" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="331243808" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:57 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="643855748" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="643855748" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:57 -0800 Subject: [PATCH 05/18] cxl/region: Add volatile region creation support From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:02:56 -0800 Message-ID: <167564537678.847146.4066579806086171091.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A7C1B180016 X-Stat-Signature: agxx5ks9katwwz5jh99oxu5jd1eg1jo3 X-Rspam-User: X-HE-Tag: 1675645377-867982 X-HE-Meta: U2FsdGVkX18nyy6ICw0qRNfN3XcJgfgClnVkyQmiluTfbs49cjW6GcPgRvIhrFRIBF6XQLI0nextwoZHa+R7XxW+3rIE0IGxaLLQc0cmqtiWjwL+BdL09Bp96mSqh2BG+6PK8qJAvSJwOihJlZBEHfSJFitPWil5UGbyQopWoEUehBscrhUvWK2JNwMXLKJjZOSgcahW0zbAFYxy6R6KMnG8RNE/D8v/IrBsLWtnFDGmTixksK8U2qjdiawl9XoySL8B7JyYui8aCxC+ysokZrm7nJ/2PHbitjt2RGbVQ8Gm22E0JaR29ykCES06zyA4+4KcmAW37bbQW1tmka83WvUtAnHADZxhYu5KTZNpO8XEVyocGBsRCoVkodbgv9U/Fuast0WBKZiibu3EpGabVSxLo0IrkzN2lH04Pqe+O2vKbbJOErT/09xU62pjEaFEDPEPqweif4ovMbH0jbhLqmI3usAD2f/665xQpFAFvoRoIFT7qs214bgiTn2UCotffx3gQtkXf0WkN8Tone0vsESpBaRUFyJTySBT/XLmFVctuQS5PvalzjbTViNgIngfV2M/imy10TjAga5A6FYKF7NZfHZ1UOAwWXwn/SN1fT52DzrndY8vZszxEm/ed4aVQORh0tF5l5iIw+R+NU2MSLa+eF1BlyfTXr1Zu17xFbgfOJ7+jPPkYGYtKHscSuIYrHxzJWtSgynURvAOiJCY8VWPtCTCpIFDibp3/JQUzHDtI9FFaeKo94eZsaQPk2d9D8K9tP26SF09+R9sdTk5mx24S7wch+vSdk5mnkzoo0Sex91uY47Rf7RXNQyQMmpAblgjvwjQuteiF4GnBNKEnJn6dh6pQqNoBEiCQ/0gSQr0BZ9TdzLI9Qvxieq9yms0Z46NYLjnkHo+BVavzLJA8BgHa1tUwVeX2dSn5lXFZa9/eAXqxX4191T+qfyDvljuWC8UAo2QbE8DIMU7YJL bYdqn4f8 lhQCditBjf9yrKNAS4UnnoIUTu8v9sjKOIU70CBTXV14AFV2q2Mtzo1gJ/c6i8XjFJUEPDGAiCQdywFpO3sKH9lW8Mcx4B/xM673sgYd4wyRaSIWM0N11f4q9Tb1fVRs6pLC86ijBif4CN1mf2OxrYC45FGTRQmlpo0w8NA1nTyadiQoHs2fpAQ0TYd0bDnu6XE0Wg6lbGCujPWtWSbrtUR7lz8KMScih1TlpHskPqxj5D7MSH5Di178Ldh64TWKQ/w9CyoLT9Zc1+/eRXjH3/s/2PA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Expand the region creation infrastructure to enable 'ram' (volatile-memory) regions. The internals of create_pmem_region_store() and create_pmem_region_show() are factored out into helpers __create_region() and __create_region_show() for the 'ram' case to reuse. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- Documentation/ABI/testing/sysfs-bus-cxl | 22 +++++----- drivers/cxl/core/core.h | 1 drivers/cxl/core/port.c | 14 ++++++ drivers/cxl/core/region.c | 71 +++++++++++++++++++++++++------ 4 files changed, 82 insertions(+), 26 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 4c4e1cbb1169..3acf2f17a73f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -285,20 +285,20 @@ Description: interleave_granularity). -What: /sys/bus/cxl/devices/decoderX.Y/create_pmem_region -Date: May, 2022 -KernelVersion: v6.0 +What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region +Date: May, 2022, January, 2023 +KernelVersion: v6.0 (pmem), v6.3 (ram) Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process - of defining a new persistent memory region (interleave-set) - within the decode range bounded by root decoder 'decoderX.Y'. - The value written must match the current value returned from - reading this attribute. An atomic compare exchange operation is - done on write to assign the requested id to a region and - allocate the region-id for the next creation attempt. EBUSY is - returned if the region name written does not match the current - cached value. + of defining a new persistent, or volatile memory region + (interleave-set) within the decode range bounded by root decoder + 'decoderX.Y'. The value written must match the current value + returned from reading this attribute. An atomic compare exchange + operation is done on write to assign the requested id to a + region and allocate the region-id for the next creation attempt. + EBUSY is returned if the region name written does not match the + current cached value. What: /sys/bus/cxl/devices/decoderX.Y/delete_region diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 8c04672dca56..5eb873da5a30 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -11,6 +11,7 @@ extern struct attribute_group cxl_base_attribute_group; #ifdef CONFIG_CXL_REGION extern struct device_attribute dev_attr_create_pmem_region; +extern struct device_attribute dev_attr_create_ram_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 8566451cb22f..47e450c3a5a9 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -294,6 +294,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, SET_CXL_REGION_ATTR(create_pmem_region) + SET_CXL_REGION_ATTR(create_ram_region) SET_CXL_REGION_ATTR(delete_region) NULL, }; @@ -305,6 +306,13 @@ static bool can_create_pmem(struct cxl_root_decoder *cxlrd) return (cxlrd->cxlsd.cxld.flags & flags) == flags; } +static bool can_create_ram(struct cxl_root_decoder *cxlrd) +{ + unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_RAM; + + return (cxlrd->cxlsd.cxld.flags & flags) == flags; +} + static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute *a, int n) { struct device *dev = kobj_to_dev(kobj); @@ -313,7 +321,11 @@ static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute * if (a == CXL_REGION_ATTR(create_pmem_region) && !can_create_pmem(cxlrd)) return 0; - if (a == CXL_REGION_ATTR(delete_region) && !can_create_pmem(cxlrd)) + if (a == CXL_REGION_ATTR(create_ram_region) && !can_create_ram(cxlrd)) + return 0; + + if (a == CXL_REGION_ATTR(delete_region) && + !(can_create_pmem(cxlrd) || can_create_ram(cxlrd))) return 0; return a->mode; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 53d6dbe4de6d..8dea49c021b8 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1685,6 +1685,15 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, struct device *dev; int rc; + switch (mode) { + case CXL_DECODER_RAM: + case CXL_DECODER_PMEM: + break; + default: + dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); + return ERR_PTR(-EINVAL); + } + cxlr = cxl_region_alloc(cxlrd, id); if (IS_ERR(cxlr)) return cxlr; @@ -1713,12 +1722,38 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(rc); } +static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf) +{ + return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +} + static ssize_t create_pmem_region_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + return __create_region_show(to_cxl_root_decoder(dev), buf); +} - return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +static ssize_t create_ram_region_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return __create_region_show(to_cxl_root_decoder(dev), buf); +} + +static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, + enum cxl_decoder_mode mode, int id) +{ + int rc; + + rc = memregion_alloc(GFP_KERNEL); + if (rc < 0) + return ERR_PTR(rc); + + if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { + memregion_free(rc); + return ERR_PTR(-EBUSY); + } + + return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); } static ssize_t create_pmem_region_store(struct device *dev, @@ -1727,29 +1762,37 @@ static ssize_t create_pmem_region_store(struct device *dev, { struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); struct cxl_region *cxlr; - int id, rc; + int rc, id; rc = sscanf(buf, "region%d\n", &id); if (rc != 1) return -EINVAL; - rc = memregion_alloc(GFP_KERNEL); - if (rc < 0) - return rc; + cxlr = __create_region(cxlrd, CXL_DECODER_PMEM, id); + if (IS_ERR(cxlr)) + return PTR_ERR(cxlr); + return len; +} +DEVICE_ATTR_RW(create_pmem_region); - if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { - memregion_free(rc); - return -EBUSY; - } +static ssize_t create_ram_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + struct cxl_region *cxlr; + int rc, id; - cxlr = devm_cxl_add_region(cxlrd, id, CXL_DECODER_PMEM, - CXL_DECODER_EXPANDER); + rc = sscanf(buf, "region%d\n", &id); + if (rc != 1) + return -EINVAL; + + cxlr = __create_region(cxlrd, CXL_DECODER_RAM, id); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); - return len; } -DEVICE_ATTR_RW(create_pmem_region); +DEVICE_ATTR_RW(create_ram_region); static ssize_t region_show(struct device *dev, struct device_attribute *attr, char *buf) From patchwork Mon Feb 6 01:03:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 165BDC636CD for ; Mon, 6 Feb 2023 01:03:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7A246B007D; Sun, 5 Feb 2023 20:03:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2A746B007E; Sun, 5 Feb 2023 20:03:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CBA36B0080; Sun, 5 Feb 2023 20:03:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7E96C6B007D for ; Sun, 5 Feb 2023 20:03:06 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4A5121207B5 for ; Mon, 6 Feb 2023 01:03:06 +0000 (UTC) X-FDA: 80435067972.06.CA675F2 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf16.hostedemail.com (Postfix) with ESMTP id 2D8EA180006 for ; Mon, 6 Feb 2023 01:03:03 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Y3OfPSwr; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645384; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N9GYVMVjWRUDVkd7PeIA4dsq7aLV5XQKEqWsb7dne8Y=; b=XI7Ah3Pt7Kd/Q3x2ip5sbEkZUrIcQ3Ud2ai3XZPqlHcKDfQ6M61nJ5QzmfzR/HjV2R8Jn8 9goJsAekiP1Bh18CzIx8roXyCVFBh8bsVwjudEkKpkrGZwnsJGeoRrvqq0/P7IoACHP3km t8JMAnnbD6SawCH6kdFNjmAOhVb4mZQ= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Y3OfPSwr; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645384; a=rsa-sha256; cv=none; b=JjVg4R3nUyZQL/OoEb8sZ19tcF4IpRA2OPlVNL3pL0PUE8iLqhP+FBdUS/x3Iuku/aOfvf ShrjuFT1LKBudUEUeI0kDyzfF8P21L31xFSeGsxZY6UGq0RaT6fnTl9feKc0hOjcqcQnJ2 VUjktbjTTny04nTnAVmjMYlGcEThOrI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645384; x=1707181384; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gxV5boq5z7i+vQyYxApsVbkxX+I+JHzPNbJoiDtgews=; b=Y3OfPSwr83yDD55RMdg4AXe7mkDrdYz3NkJ2BvLg01lVednVIZyMt616 O8Fn+wtlw/ccYV7B06kz9ROxoygljbnm7S9dYjenLyX0QW0GF1Md4rdXj PsWW727MEx3zr0h7yO6zlVV7RPi4gsE66ygCuCr7TH0Awp47WbDrqifqU YunMWoSOLNofEevrOVpM/HKO2TfCOdCAvq/Up2+vqbaHGtK0KitB+LpP3 U1NN2dHhU2kEua/p5LLEyGgw4DovbkbdOlDQZ7bpYTmjdvbiO+21HGMSp 2w/1JbKUeW6z7UU14fD0eTiy4ABI6aztsXaag3KDNuT7zB4FtdWlBN08R w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="312763181" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="312763181" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:02 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="616291294" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616291294" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:02 -0800 Subject: [PATCH 06/18] cxl/region: Refactor attach_target() for autodiscovery From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:02 -0800 Message-ID: <167564538227.847146.16305045998592488364.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: zrdawy4hifgwz7sg7u5nbbe7qh4x5pcn X-Rspamd-Queue-Id: 2D8EA180006 X-HE-Tag: 1675645383-877410 X-HE-Meta: U2FsdGVkX19aS5d/Z/Frk2JI4WlM5Xmdy28gbe6Nwu/F4MYxhnHinSxWrzKKiLx2b7sKZQ1YCd813AfeBj7dxlnzGLU2NXKmQQ+QbEjAON4OPp2Glrz7ANhsygwTngmCTdJxELVMwPP81hd9Oo1E3ZOVAhFLUOQ2y4Wxv4zLV3lLZI5WIqkaVlT98MCxutE6dnjwIjKzQTGjMvKnzXngm+4uYR29pQs6nIU7o1+RK0L0Feh93x0gPe+94LVGzXkMdlLpV++Vxid+8jTir12YyRgDYIrTJ5Ot3/2KHTa5Dg+dLTPk9A+JbyxFnW0HEHwe/pbIDLVEMGeR2onvZW0MwUkM3kZEYrqMEFxjebQe1rpEJQynyZMkNuho/yzIPNK+k4AUxOrtEJ3sm+vGLQ/dqmcHuicW31FHlfd6oKKrJh/yDxafShaeplTjt4scUYbUqMCa0GjCBLnHKPzunV+/xYU9bL8Jm7LdTwu7L8DZCiNjXEGdlI2ACwKkz6PXebDCC9wquHyYnh0ALY7oA5oSryePR70Lt4mTKDFWm1c4mUsobx6eTatry2v8mAWrfVdNnigWjXLBLU7EI+t9pOdnI0xGuKhtrHJWPTHtAYph9SV5RDARjYKdDunQJ8LD0cgJ9kudSrFUlf1lAdHAh5gouRQYsYT7+CN/6NrLRr9ktDJwEk2Oy3XtJB5s66vDktYdRxUm6SOGJWf0Kz7f3s429vOULxdviA5aTJvrvxxydLWQImTPj07JeI3LQDyVmZuaqSJw9xCv56SLVKz3s7EJ/BYvORkKjdL/dOyR7K7RIOmimfzWeZ3e1BAoF6fmBblEK5qRGY6zKtinE+VNIO/u2dwS7uF33TNPecRb8OG6dQlWUX4bngbav4g9Za0/HctEkUrF4bFJI0Sl9c0D0evQKloeziBYoIMN2TbHAb+tPj1WYbUIxYKQRiPTihPAeH+Sw3/8I+OteKDGmOpzZIq JBg3s1v5 NkAcb3UDHDVkvwGv6gF5X+2Z7WHhidarXjO35baYT6IOwmhnKFoSMmyoid0oJW9NPXZJBBwRbo+fSZeVitt9omJPxNvmqa04rj5FhJ2ugmiE1VZaC7Zfgdmd3XyNQZjqVv9vOtIJo9slVyehIdTxM4mTkVZbmudi/AW/UaWyD6XonVXXlI8xzfLVVlTcCyiYlPqWqclzuD04Ui2k6NsRO4olm2scq4XnbB7xPNVdwvBudyyw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Region autodiscovery is the process of kernel creating 'struct cxl_region' object to represent active CXL memory ranges it finds already active in hardware when the driver loads. Typically this happens when platform firmware establishes CXL memory regions and then publishes them in the memory map. However, this can also happen in the case of kexec-reboot after the kernel has created regions. In the autodiscovery case the region creation process starts with a known endpoint decoder. Refactor attach_target() into a helper that is suitable to be called from either sysfs, for runtime region creation, or from cxl_port_probe() after it has enumerated all endpoint decoders. The cxl_port_probe() context is an async device-core probing context, so it is not appropriate to allow SIGTERM to interrupt the assembly process. Refactor attach_target() to take @cxled and @state as arguments where @state indicates whether waiting from the region rwsem is interruptible or not. No behavior change is intended. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/cxl/core/region.c | 47 +++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 19 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 8dea49c021b8..97eafdd75675 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1418,31 +1418,25 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled) up_write(&cxl_region_rwsem); } -static int attach_target(struct cxl_region *cxlr, const char *decoder, int pos) +static int attach_target(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, int pos, + unsigned int state) { - struct device *dev; - int rc; - - dev = bus_find_device_by_name(&cxl_bus_type, NULL, decoder); - if (!dev) - return -ENODEV; - - if (!is_endpoint_decoder(dev)) { - put_device(dev); - return -EINVAL; - } + int rc = 0; - rc = down_write_killable(&cxl_region_rwsem); + if (state == TASK_INTERRUPTIBLE) + rc = down_write_killable(&cxl_region_rwsem); + else + down_write(&cxl_region_rwsem); if (rc) - goto out; + return rc; + down_read(&cxl_dpa_rwsem); - rc = cxl_region_attach(cxlr, to_cxl_endpoint_decoder(dev), pos); + rc = cxl_region_attach(cxlr, cxled, pos); if (rc == 0) set_bit(CXL_REGION_F_INCOHERENT, &cxlr->flags); up_read(&cxl_dpa_rwsem); up_write(&cxl_region_rwsem); -out: - put_device(dev); return rc; } @@ -1480,8 +1474,23 @@ static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos, if (sysfs_streq(buf, "\n")) rc = detach_target(cxlr, pos); - else - rc = attach_target(cxlr, buf, pos); + else { + struct device *dev; + + dev = bus_find_device_by_name(&cxl_bus_type, NULL, buf); + if (!dev) + return -ENODEV; + + if (!is_endpoint_decoder(dev)) { + rc = -EINVAL; + goto out; + } + + rc = attach_target(cxlr, to_cxl_endpoint_decoder(dev), pos, + TASK_INTERRUPTIBLE); +out: + put_device(dev); + } if (rc < 0) return rc; From patchwork Mon Feb 6 01:03:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B56DC636CC for ; Mon, 6 Feb 2023 01:03:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2151B6B007E; Sun, 5 Feb 2023 20:03:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C5886B0080; Sun, 5 Feb 2023 20:03:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 066C26B0081; Sun, 5 Feb 2023 20:03:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EDB536B007E for ; Sun, 5 Feb 2023 20:03:10 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BADA8C0542 for ; Mon, 6 Feb 2023 01:03:10 +0000 (UTC) X-FDA: 80435068140.21.3A7783C Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf16.hostedemail.com (Postfix) with ESMTP id 9C05C180016 for ; Mon, 6 Feb 2023 01:03:08 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=V6ixechA; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645389; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QwPbb25Pw5aw7WOOgs3j6gqEoHoJGh6BBlNnw9kgFiU=; b=pUqalFNeTsP0ENSrAFN+J4gmi3V/mhGpgMqIosLhydb+oHNH8XopXtN/xHX1c2ps7/FFbe lSElP8vg0Yx/sU2ggPCz72ljVcGGf0xkh4LyIaF34foQtiEZ5v+xhZWAWmwL5x5b5x9uYP aAnkitcQviqCdbtp1rwq62qRKlFb0kE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=V6ixechA; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645389; a=rsa-sha256; cv=none; b=SMSXNAwaf4fcNY7HfcalazJrpclCndbfcRLi8f2KQbLQZ9jc3vD7kYUUeUfr074VdXNFHx yV/f5NTLxMSljDBXSYQP24xOXt496dC19YIyBgv4H5O/fGlRiMQc3MciTrmfGSGRP5YCaf Y85fL+9ngcO5IzFP+PiHrsT60NAb6BA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645388; x=1707181388; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pd/OzsallZam53iFg2v2xJwCJFvmTCwZnqxjvamzJrs=; b=V6ixechAb0aRGXZJn8LGYRHnpWFwokMukFUkZSNgO3O286d93Qz+2pRd It6QuUCPzNPz0q8zDU3uyInALTW715NE1d5UHEvYJ4dQY1uG8EQKL8aRY D2URgHZpvHKBYmAywnZ42EzXD6oR/gwarEtJqq8oqWb7/oAIZbdA2nX5m d4j+HK6AKaUT45dMwcqzy3kbiHPeNqSGj8o/nLBmmj+DPjH6Of5/UkeLC q/VWl5vpnBB9fLyF438ixhX8JGihlu+UBioMRKxbQIKGe6FBUeg8yE7+G sJQ8F6LSVC2AxjxNeqII65+G5I1vbFsw9zSKaHKOXwU8kf2HLN5riSvFO w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="312763187" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="312763187" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:08 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="616291299" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616291299" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:08 -0800 Subject: [PATCH 07/18] cxl/region: Move region-position validation to a helper From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:07 -0800 Message-ID: <167564538779.847146.8356062886811511706.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: ybz61a9qyzx8x67u87u9mxdg1fxx9jh1 X-Rspamd-Queue-Id: 9C05C180016 X-HE-Tag: 1675645388-695973 X-HE-Meta: U2FsdGVkX18QtXlAMiYTlZtlGdHZWAuG8TASpk/PlSN993BBzYa7n5r8JkY9hg90D6oaxbCnT+gcaqXEvhq6L0RPTVnaX2uaqljWgOjKwCpQ5eLdxIQsOxMeI90fy2WnYkswVv5fRwqkEctTCOMqjcQZ2jzHjqn4RAJL3IOcoG5VMOD9HTSntzDV8SV5rWPv0vwZCKljSePR2ubPgcowjPPBH1S6CGPxn7wfq+e6B2cUP6x1/BKTDrRWmFxm0ysJKEw0Tpf3r51wYPik9MLIp+QE1OlOQkIKKg56ppZXcCwydzCcvHQBJO1p+ALekvo0fRiEjF7dVI9nFv7ouOLbrgLhmN8T4tbYZUI5BJ4tG7r/iZyLBGrZVnDw48plj31bGO4OetYyYBKgqT6bmsmdr+Sr9f6LQE4mZJnInipHELBNQB2WER3pDSe+31nKMngJbjEmGkgSR186GkvreGkD7hPYkBjasBV8JUfXQ3BsgoXzDu5sMHHVzUE6tsBdSQCiY2TkjEM5uBNwloQIW7N7D5LGSjwvmFsdq1UFBZ6cStplgOx3b6yxNs7Rj8JBGotDGyKCQm6+iRIW4V5TnyO5KbvbWzjzF2R7C8lbzwtL39PhmL1UZUVUuNiIx3JnWI6czUlIpicoPqFduaxsLHCwlp4q2yfroFfXQXtRiSzFEgB4g4ryBwESLbfoiIT4njpq1UBtsQwhN3iEcvkySyLz0+dhrlh758fsQO+sdDB0QrbYtg0NeC5BCQ1y1BXr/e0ZPZdkW06uZ3akDd4TF5H6qso0iJ2N7fDwDFgqW6n1Ys8aW++4+TkslbY9Mk0ANDPoWPwexRVlCL7wm/2GhryHAL1U25lCgbdTEEIC40ZorNpmvjb7Y/p0cncoVX733gXT8YYlJH7tyA+wuk/ySxXa3lkVew0b2hSevaqISkYjJPfv+lsTBoBLpKl+bHsvZ2L9hu5dGGyNef0msV8nXy8 iqcxTHz2 /1AdQXqCq1gyM5K/v3OCI/J4sBshnJBhztxhEZS5l3ty0ocDYAIZ1Q/nIDlhU1DdzbSsKgHCeXsyqTh7TnQRQTYbRaYjTvfCrMH33VcC3b+ejGTjQbZI3SlIRS56zHd9iZxMz9vLmURXJkLKFJ/odnJM7c9un/2IuEefyetsAx6Sa8Ko8z8A8PuvqJNmTlJ0BiX/8mbGWjY3aC73wkSp5VPkXrRYXq74vZ7LJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for region autodiscovery, that needs all devices discovered before their relative position in the region can be determined, consolidate all position dependent validation in a helper. Recall that in the on-demand region creation flow the end-user picks the position of a given endpoint decoder in a region. In the autodiscovery case the position of an endpoint decoder can only be determined after all other endpoint decoders that claim to decode the region's address range have been enumerated and attached. So, in the autodiscovery case endpoint decoders may be attached before their relative position is known. Once all decoders arrive, then positions can be determined and validated with cxl_region_validate_position() the same as user initiated on-demand creation. Signed-off-by: Dan Williams Reviewed-by: Vishal Verma --- drivers/cxl/core/region.c | 119 +++++++++++++++++++++++++++++---------------- 1 file changed, 76 insertions(+), 43 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 97eafdd75675..c82d3b6f3d1f 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1207,35 +1207,13 @@ static int cxl_region_setup_targets(struct cxl_region *cxlr) return 0; } -static int cxl_region_attach(struct cxl_region *cxlr, - struct cxl_endpoint_decoder *cxled, int pos) +static int cxl_region_validate_position(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, + int pos) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); - struct cxl_port *ep_port, *root_port, *iter; struct cxl_region_params *p = &cxlr->params; - struct cxl_dport *dport; - int i, rc = -ENXIO; - - if (cxled->mode != cxlr->mode) { - dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", - dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); - return -EINVAL; - } - - if (cxled->mode == CXL_DECODER_DEAD) { - dev_dbg(&cxlr->dev, "%s dead\n", dev_name(&cxled->cxld.dev)); - return -ENODEV; - } - - /* all full of members, or interleave config not established? */ - if (p->state > CXL_CONFIG_INTERLEAVE_ACTIVE) { - dev_dbg(&cxlr->dev, "region already active\n"); - return -EBUSY; - } else if (p->state < CXL_CONFIG_INTERLEAVE_ACTIVE) { - dev_dbg(&cxlr->dev, "interleave config missing\n"); - return -ENXIO; - } + int i; if (pos < 0 || pos >= p->interleave_ways) { dev_dbg(&cxlr->dev, "position %d out of range %d\n", pos, @@ -1274,6 +1252,71 @@ static int cxl_region_attach(struct cxl_region *cxlr, } } + return 0; +} + +static int cxl_region_attach_position(struct cxl_region *cxlr, + struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder *cxled, + const struct cxl_dport *dport, int pos) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_port *iter; + int rc; + + if (cxlrd->calc_hb(cxlrd, pos) != dport) { + dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + dev_name(&cxlrd->cxlsd.cxld.dev)); + return -ENXIO; + } + + for (iter = cxled_to_port(cxled); !is_cxl_root(iter); + iter = to_cxl_port(iter->dev.parent)) { + rc = cxl_port_attach_region(iter, cxlr, cxled, pos); + if (rc) + goto err; + } + + return 0; + +err: + for (iter = cxled_to_port(cxled); !is_cxl_root(iter); + iter = to_cxl_port(iter->dev.parent)) + cxl_port_detach_region(iter, cxlr, cxled); + return rc; +} + +static int cxl_region_attach(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, int pos) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_region_params *p = &cxlr->params; + struct cxl_port *ep_port, *root_port; + struct cxl_dport *dport; + int rc = -ENXIO; + + if (cxled->mode != cxlr->mode) { + dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", + dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); + return -EINVAL; + } + + if (cxled->mode == CXL_DECODER_DEAD) { + dev_dbg(&cxlr->dev, "%s dead\n", dev_name(&cxled->cxld.dev)); + return -ENODEV; + } + + /* all full of members, or interleave config not established? */ + if (p->state > CXL_CONFIG_INTERLEAVE_ACTIVE) { + dev_dbg(&cxlr->dev, "region already active\n"); + return -EBUSY; + } else if (p->state < CXL_CONFIG_INTERLEAVE_ACTIVE) { + dev_dbg(&cxlr->dev, "interleave config missing\n"); + return -ENXIO; + } + ep_port = cxled_to_port(cxled); root_port = cxlrd_to_port(cxlrd); dport = cxl_find_dport_by_dev(root_port, ep_port->host_bridge); @@ -1284,13 +1327,6 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -ENXIO; } - if (cxlrd->calc_hb(cxlrd, pos) != dport) { - dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n", - dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), - dev_name(&cxlrd->cxlsd.cxld.dev)); - return -ENXIO; - } - if (cxled->cxld.target_type != cxlr->type) { dev_dbg(&cxlr->dev, "%s:%s type mismatch: %d vs %d\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), @@ -1314,12 +1350,13 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -EINVAL; } - for (iter = ep_port; !is_cxl_root(iter); - iter = to_cxl_port(iter->dev.parent)) { - rc = cxl_port_attach_region(iter, cxlr, cxled, pos); - if (rc) - goto err; - } + rc = cxl_region_validate_position(cxlr, cxled, pos); + if (rc) + return rc; + + rc = cxl_region_attach_position(cxlr, cxlrd, cxled, dport, pos); + if (rc) + return rc; p->targets[pos] = cxled; cxled->pos = pos; @@ -1343,10 +1380,6 @@ static int cxl_region_attach(struct cxl_region *cxlr, err_decrement: p->nr_targets--; -err: - for (iter = ep_port; !is_cxl_root(iter); - iter = to_cxl_port(iter->dev.parent)) - cxl_port_detach_region(iter, cxlr, cxled); return rc; } From patchwork Mon Feb 6 01:03:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46560C636CD for ; Mon, 6 Feb 2023 01:03:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E234B6B007D; Sun, 5 Feb 2023 20:03:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DD3006B0080; Sun, 5 Feb 2023 20:03:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C9B7F6B0081; Sun, 5 Feb 2023 20:03:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BD1276B007D for ; Sun, 5 Feb 2023 20:03:25 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 84423160927 for ; Mon, 6 Feb 2023 01:03:25 +0000 (UTC) X-FDA: 80435068770.29.C1F62E1 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf02.hostedemail.com (Postfix) with ESMTP id 7B1338000D for ; Mon, 6 Feb 2023 01:03:23 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=QNceDMpx; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf02.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645403; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DyqHyyZT2lU9emCKE6Q5ouwqYz48kTemZ1q+V82QNnw=; b=hL6F9Xotcw3Z8U7IS/NZyFfX6ODp/MP5FmPNPffiwgAvhZFjwoleePKWsnn94A7qW5vhgl OllWw+AP5WFTRC5v3Bbl989vxVGBPfu+vVDyaexZ82hkXTqFZQY0jBVHo1gUUTNLV2rjhs mkAIWaLjA1L09mxiVxZ4tuK0KGG93u8= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=QNceDMpx; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf02.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645403; a=rsa-sha256; cv=none; b=6XvBcj6GiAFhA+3hQVh0tlgVFCvfAFW6LPHLjuvvd68FZN1s1vDyjrD3I0tI2avZRg+7p3 q967TWsUorExj9n/OO/YKILGwJRXTfVWVLU0+7xvhX3+LZA0BbqPgsVst/goun8CTfD2NR XNArWrDOkUCOElcB3pThelZqvrpW2vM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645403; x=1707181403; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AeIQeRHiJX+pQkHKAuoYi54yr8qg0sN4RL762t1p8Jc=; b=QNceDMpxKiEMBZ+4me7vlQjZkeOPixd9VRv31S72dgMKE2lFbuzQwe5B mdJnzXE5dh+jdKkElO0PoTDbKX6aUOmHYQAvIF1u28CRD76cwNLl+Qykd xIEFFXxSaRMSpl0fZBnlsAUq9HDKqIgWRUYdZIIzFc0UDaOp8IDCOjg2v o1zMxWweCY9SJWDQaBJQ5MsVW4G3bE3yazj4TwfiyZIhdR2W6R5Co3yub SvURQVuDs3XaAlDDem0LiX7KI5J6cf90KgbOtKrBM+g2x1TQ2TIGkPwH8 xM/MClSaN4sEzifokUN2kIHuxlKItnqg0qMPEbhdIf+j6Q6M8gap5UxXi A==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="312763199" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="312763199" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:13 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="616291316" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616291316" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:13 -0800 Subject: [PATCH 08/18] kernel/range: Uplevel the cxl subsystem's range_contains() helper From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Kees Cook , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:13 -0800 Message-ID: <167564539327.847146.788601375229324484.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: 7B1338000D X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: eeb567nyu7gfzgu48p1cwyo1yykq4xm9 X-HE-Tag: 1675645403-150389 X-HE-Meta: U2FsdGVkX18LpE7nptaiUo+FlshJjgzDkMesDlXoMGP5WhRqVSGqiHBuBGK7gsGNQKu3LWG/TNZliJCAwKnMmzAU6NwksQVIeTfz6eDNqkyanw5foByJOcJevK7iyhfWz8WWLSGo+64L8R9xVlfEfyD7Vn+xi2oQOKM4apdh22fWmQGHSMIsPuf8AcE/PuBMPcLRj6TaHUntzOboM7R4p5pGBWeVd6p22MQY7ZcHKlgyBL8xVrc6oKadeLVUmCi2i/4h++YUa3mXEPe4hbzDEe2z3Vjgg5e+x3jFGUH8RGfaH08OHQLIG+/SCdMKLcUulTwsz2t4qLLZbBIj5tvQDEsGS/JnTwon+9H0UGz/mAj77DEL662NAriQnc31szw+E1uGkD+4og7xGf6AUtvOaYBdAqfOMxj4O1ND757S6hTLnWHw6s3OAo8Vvg8ECH37/ZhbUx9lKdDdwoBnCtCjYzuZXoNMtVwn+My+0IWZdJPDEQK3+2s+IjR+qQcpK5M+fPUQRJzPNUDn3GGEVEn0nCtGKPNMCGzAmTcxSl1WJoEX68g4sSr+hagW44YQ9ziPLOufS9BnC5ut1GWFwaa0vzvNFlSLyPrm5QywHDDV+z/mmjqTqAIlG9twDxsiH+83BdqTf4WahnC+My6bVVhysj+WgUEE98HbMoL6ouX+Cs3GxYOeYV/BVZ2McNn7Q8wcQFNl6F4xiGG8fZuP3BFZkdkQ3oENz+9lLkIH73q2KxdASqkUChWH+Qi2+PtF2demcDsAGRkjeKRE4Sg3hUy5A8SEA1/dF+7xFIA3EWkEXR+KQ0wAM5TIOlrvKhaSGeamH9rmV9s5/y0eEzIXSArMXzjWkAYurwX3sHMrMz06tafPVzQi3CJ/tH8I2R2XclVBWsJcdRwZghUHwDXtiikMBk/ykCcBjs+F/SzGWB4hAwk6O34yipSptJ+Uhnspz5kYJxJawEZTonEGQoxRpX/ tQm2QgVE vwEqm/BI1aBjNeRcYGnrJdnQSqimjjjDBof/U3+yVyQnAVVz0tbisXmyKJS64qlsf7gpW8g5xQBH8NoTtllfFxS9a6w9RVzycL/uLvTM98tMolY+wcA6L5nVUWzVFzA6pO16BllocGPzlVBxKhwOGdGWgsYRHq2JmwsViX5isC3PmcKSZyXZ/1DRgkCyiC5RFxKv9INwXCKVhdEQ4IBIEJXXj//aKUDg9wb/g/5nyW1S5b5c1t2GzBLLNWoJV64R1fZLf7CvHNkXzqhY0/ld9Rd/LF2TPmQ5cUiJP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In support of the CXL subsystem's use of 'struct range' to track decode address ranges, add a common range_contains() implementation with identical semantics as resource_contains(); The existing 'range_contains()' in lib/stackinit_kunit.c is namespaced with a 'stackinit_' prefix. Cc: Kees Cook Signed-off-by: Dan Williams Reviewed-by: Gregory Price Reviewed-by: Ira Weiny Reviewed-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Vishal Verma --- drivers/cxl/core/pci.c | 5 ----- include/linux/range.h | 5 +++++ lib/stackinit_kunit.c | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 1d1492440287..9ed2120dbf8a 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -214,11 +214,6 @@ static int devm_cxl_enable_mem(struct device *host, struct cxl_dev_state *cxlds) return devm_add_action_or_reset(host, clear_mem_enable, cxlds); } -static bool range_contains(struct range *r1, struct range *r2) -{ - return r1->start <= r2->start && r1->end >= r2->end; -} - /* require dvsec ranges to be covered by a locked platform window */ static int dvsec_range_allowed(struct device *dev, void *arg) { diff --git a/include/linux/range.h b/include/linux/range.h index 274681cc3154..7efb6a9b069b 100644 --- a/include/linux/range.h +++ b/include/linux/range.h @@ -13,6 +13,11 @@ static inline u64 range_len(const struct range *range) return range->end - range->start + 1; } +static inline bool range_contains(struct range *r1, struct range *r2) +{ + return r1->start <= r2->start && r1->end >= r2->end; +} + int add_range(struct range *range, int az, int nr_range, u64 start, u64 end); diff --git a/lib/stackinit_kunit.c b/lib/stackinit_kunit.c index 4591d6cf5e01..05947a2feb93 100644 --- a/lib/stackinit_kunit.c +++ b/lib/stackinit_kunit.c @@ -31,8 +31,8 @@ static volatile u8 forced_mask = 0xff; static void *fill_start, *target_start; static size_t fill_size, target_size; -static bool range_contains(char *haystack_start, size_t haystack_size, - char *needle_start, size_t needle_size) +static bool stackinit_range_contains(char *haystack_start, size_t haystack_size, + char *needle_start, size_t needle_size) { if (needle_start >= haystack_start && needle_start + needle_size <= haystack_start + haystack_size) @@ -175,7 +175,7 @@ static noinline void test_ ## name (struct kunit *test) \ \ /* Validate that compiler lined up fill and target. */ \ KUNIT_ASSERT_TRUE_MSG(test, \ - range_contains(fill_start, fill_size, \ + stackinit_range_contains(fill_start, fill_size, \ target_start, target_size), \ "stack fill missed target!? " \ "(fill %zu wide, target offset by %d)\n", \ From patchwork Mon Feb 6 01:03:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129234 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7515FC636D4 for ; Mon, 6 Feb 2023 01:03:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CDB56B0081; Sun, 5 Feb 2023 20:03:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BB916B0083; Sun, 5 Feb 2023 20:03:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E49136B0082; Sun, 5 Feb 2023 20:03:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D6CFC6B0080 for ; Sun, 5 Feb 2023 20:03:27 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9C083C07E9 for ; Mon, 6 Feb 2023 01:03:27 +0000 (UTC) X-FDA: 80435068854.04.BEFA16A Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf02.hostedemail.com (Postfix) with ESMTP id A3F8C80004 for ; Mon, 6 Feb 2023 01:03:25 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=MAv+KLKh; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf02.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cqplDjwh2oExsglW+Yu6+lwtbgBNMUJgi9SjSFWIj54=; b=37oMTTOZf3ErHG0hLLr+F1HxcV+00QbihArCJgw1ds6AnHU5yfi05pxvqJfwHtpdtHSg56 GZdNa+5ns0HDVmI22yKc2Ang0sglhbwxlrhi6QB8uhXssFAMhXntAVGUBSbm9x2//COtXN JpBertiD5yvOeAyabNkPe3QiMm0Y/3I= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=MAv+KLKh; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf02.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645405; a=rsa-sha256; cv=none; b=JxQxLirmTklAjLkix65+dHhN79ZAXXQh99OhpBqKAxTGW+iJQRCjQr1VeJ8S5f1QFWIhPZ e7NAVLZOcxDbRH+go5Sh+Ys6mwGm1Brc47Q8b2rru0QDFVjrpB4ngXhyUf18Z1jpeK+RiY MdGlrs+jxdD91nTbcvPICTXcsWw/VEo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645405; x=1707181405; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1wW75nbCrf3Mo9olc/rXs6RitxwA0HhPGRVwUk3MkpQ=; b=MAv+KLKhgRC94QyOseM45DcqExeIqgt1/jGSmn7oU4HqPixgkYVg6cJO 9mINmwY/1ASf6UYfhsZgbSMC8yaRSS2yFibBrphmvMFYT9KOmzO36ilyn Hp5GQFa7sgQkgHueZvEWYQDlk2L9M1SzHClbJggVbZ1kddZU0IFpJyNdy ZKNjJjJp4/rHiSwuV5kgnWy2FQIksxbAa1MNIhujYAyzQX2IYKOFt/kLO h6CXzBePnnHtPlPLrBZGytpWtZLEMIiSnfFgGTbj6zk7VVIFN9QRax7Ee y91d2glGMo5hUY3LW3x6aG/KQcr4FdlbCH5XX/b5lO1mPoAPBZyu94Oie g==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="312763210" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="312763210" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:19 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="616291323" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616291323" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:19 -0800 Subject: [PATCH 09/18] cxl/region: Enable CONFIG_CXL_REGION to be toggled From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:18 -0800 Message-ID: <167564539875.847146.16213498614174558767.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: A3F8C80004 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: ffjx4f415n77n53hz1si5otmsxadqob3 X-HE-Tag: 1675645405-807700 X-HE-Meta: U2FsdGVkX18On5iVF2MmyZ0b2QrZEmQ4P9W4ZStrrRDz6liomkKuFp8n9qz0pUBvzPtdQV9oxIrOKxhd0ecjz/m4KlJh+A2vrPB8EVJSjNoT51qCs6mCjZvhx3J2dVVHKrxnDQlLsCketHbFHDQ1IpDSjM4ozYkscfXt4ffIII4z0Mvv/apsqemE4qvBXi34VOJ5R2k2SmGzNmySz4ToTA7gF38AugjZv7BFxxA5+A3OD354ZsEXDeGvIy34DKBMLVIK7OCI6L6OKi0XV472WpKY9okz0rS0VU/5oB7MZY2TTiuOggrXa3vuNoGtcz66GbfYX9+ux6hrtixGKvMFtAra83xYUZ1IgNg6W3u/XgR8gMB+ipreV6NQs9XkLOwkhnRgcU97SRCPjrAxp6TAkrpc0he9S0f/gK1Br30hzoBCEbVo2Oga0bOD2w+cHHWnqF3ABg/gG0LUAD0VRCL966lr8TS+3zCBsM04L3Gxh6nc+kn2S6m6EInmaPih078XNMHCQn0xw8qy3jF06YAAF4FAtseDuHgpYBv9KK2alK8uYTH7J5i47rlZaMFoeLZtZSBIolPKNIXNfV0AJ5z7iO3U6sfv8juXkK9ops8ZTrEqclY6kVmO6ZHM+rdPERPZGF4MAXNL4WzciUFTpbFUlV49K/sFkIfStGKG6Sge9PtBKzw3NR3ghi2aZ+YnG2lwpv4tgcdizoE/sUuD1aFco7xbwLEZtrHvIMXKV8/gmJGp6yUPk3LX80YQlPozh5oytJVuSy8RM4J6EljIcuiQ9K6dDdFutM78/TvKcqxoD1bJUXJbW1SBHaLF0eBOreqJxMv3B+JsP7g6uV4j/kQUnSpp8ROqg0TPtc7lWA16DeNVEMcLdAihuSyaU74OMXj6d4O8/HZjktMRkLuZ7MUdFPdvYUTWh2CaPW6CjDafE+OaRnsgAG0O61QcbTagDDoJA2Wf1GTqvqtml8v33dU veK4NNe/ 4AYFlZSlilV6HISKfwhAbzq2iEe30aVjX3vg6HXXXb733RFsFHn7W1gOiAzi+mQtbARV4NjD3W9SdfqYqhQo281xxtEDdcUIXjK91dwWq88uHhCgy3a01GUpm+y/UtZ8FHEHyprSDfqBZsTiL1HZ+VOzJfHLN4a5yFifb3EosSUkMmbk36R8ETMBl1OB+JTPJ7UKfdtrpH6INNaCfD/kXXqjXPV89f0nAqpPXkZUNLL2wDTo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add help text and a label so the CXL_REGION config option can be toggled. This is mainly to enable compile testing without region support. Signed-off-by: Dan Williams Reviewed-by: Gregory Price Reviewed-by: Jonathan Cameron Reviewed-by: Vishal Verma --- drivers/cxl/Kconfig | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 0ac53c422c31..163c094e67ae 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -104,12 +104,22 @@ config CXL_SUSPEND depends on SUSPEND && CXL_MEM config CXL_REGION - bool + bool "CXL: Region Support" default CXL_BUS # For MAX_PHYSMEM_BITS depends on SPARSEMEM select MEMREGION select GET_FREE_REGION + help + Enable the CXL core to enumerate and provision CXL regions. A CXL + region is defined by one or more CXL expanders that decode a given + system-physical address range. For CXL regions established by + platform-firmware this option enables memory error handling to + identify the devices participating in a given interleaved memory + range. Otherwise, platform-firmware managed CXL is enabled by being + placed in the system address map and does not need a driver. + + If unsure say 'y' config CXL_REGION_INVALIDATION_TEST bool "CXL: Region Cache Management Bypass (TEST)" From patchwork Mon Feb 6 01:03:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E609C636CC for ; Mon, 6 Feb 2023 01:03:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 413746B0080; Sun, 5 Feb 2023 20:03:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E8CE6B0082; Sun, 5 Feb 2023 20:03:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 150D06B0085; Sun, 5 Feb 2023 20:03:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F3D3B6B0080 for ; Sun, 5 Feb 2023 20:03:27 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CC08616090A for ; Mon, 6 Feb 2023 01:03:27 +0000 (UTC) X-FDA: 80435068854.02.B64310F Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf11.hostedemail.com (Postfix) with ESMTP id CA91240007 for ; Mon, 6 Feb 2023 01:03:25 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=X5XV5SF7; spf=pass (imf11.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D/bJOO1mS1zFW+X1GhlTLkQPeNBZ6+P3WTqVqsv2v7g=; b=OW+ljQzc34drVxKKMxq42nLvXzakySro6wJ40Y3dEngfPNEsxw64X3esoqtPET4rOt+CA4 jK2KYM6oT5IvcOWznoXpYevZTiJS8rPTS8GKQYVGpIpE+SF+zC0aUZBoR+kDbFmqCCA8QY Km+mX+0XBTutjWk8grA2/mgQBv/utlo= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=X5XV5SF7; spf=pass (imf11.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645406; a=rsa-sha256; cv=none; b=JEsTVjP7fD2H43tm4le0cqMQG+DRkkpJdynBIAbk46gM8aYVLi8+H63SrZq+sbgSbvpmgh jFE8UGiVh5t/UOemriaXEt164Hh/aeym/L6YdumXzzoYooyXG3vGBmNEi2jNW9taesvi+9 AVgXfwMlbbzTXZMRyxsxNmkLyjQanbk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645405; x=1707181405; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=p3cJbdfBQzbXgLwCpMurOEl3oekEj08Qxoo3cX7rCec=; b=X5XV5SF7dVYfv53QwVVGbEhdhn4OT71WXRkFOX8nhVyBJ2+Ja407c0Bs TFKgzn6X1Aiw0yC4T3+vxvVIYnymDbCjHNVXEaCuiLZQTseU+f7nx0+wr V8G0r1jiorHVAWHcEeVVvAPXjmN/nafADqKP+Rbqp6EVY5hj1FciAP1Z7 ViP6u/gWCxZuNH8IdaOxDR0NPI/lOOuQoQ5Hn0w8vtQ8LwTzf+40w3wZv Dn+an039/64w6HArMtcFMxi5ifEC9lQiw16tG1/7gX4Yu2o+6yBuJTKGb lP0ZIhtaLU7jfCBAzDGK1bHjtEAu9U7MXOSQa6/ZjMziR+i73UwfnDBHz g==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="312763216" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="312763216" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:24 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="616291328" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616291328" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:24 -0800 Subject: [PATCH 10/18] cxl/region: Fix passthrough-decoder detection From: Dan Williams To: linux-cxl@vger.kernel.org Cc: stable@vger.kernel.org, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:24 -0800 Message-ID: <167564540422.847146.13816934143225777888.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CA91240007 X-Rspam-User: X-Stat-Signature: ke8jrx7k9esapnq4sf5n65i3x4ruw893 X-HE-Tag: 1675645405-536569 X-HE-Meta: U2FsdGVkX1+rmNcdKMDqYDrUd5H9BAnS86ljFzDMbkM0Pv6/J1rggvrrXDlICNkLMznEl2oeHC2dmao0T1zrMLlK4BTKS+VGcYGccCclv2zaGwtUMYkg1J7sS37UTmoSK18WXQInYLs0aAvpXptDVznclApWQq95lUd/mdv5bYn2nDY5CrBA1/YhG2ThliUFrf93UXvsnxDMIK6NUyhwDiK7uODKVNYFSoDqGV5FHuA3urfCKiq485g5T0YbXdd9NL4S/09m2lcVFRCnunG8Fi43wmYzT8n4LIQoai3T+eh+HwvCro7Qh703dgDDoaf4sK2nPLNN8mzueKspWHjxL727RQRR9YMbC6vK6JLL15YP1k6h6gfd6SejeoYHFiuKxd8BJ7YIeUu5dNwF5FjcvL2ijyB4EDIwpqxR0oIYIvcpaSpvuHmpdKnNKxFZ4tCUrqg6IYUaXDjoyWZWdRGCjoUSiGfyFPSyrdn4M3uKPho7F6+pBlG60ip09uxU/WAe1rs5vrwJroq8hBmH8Fzj/HlKSwUdL1rOWItGi3ddzEXmJABKFr0t5dbnSpsiEE0ckxRO17DCz1XbHHmhFRuiJYjI6TLJU1mORtb+jYciH8dNmG+XTXDJQ0uq+RjzQn/ehEGW82TM/RUhBrZFaMl4fw/vdXwNmzH97uFramSCTCNacgL2MDHQBefhOBsLuB061dlSqsjRnHrMwua306p2urYqBx8K7Cfv+faMvFwJRqYz+xwV2IOAU9HD020jKhUZ9jFuDNBLDxboJ/Xt+gETEFLqXsP6Jezjz5Daeajg5uhq16PqexNp3XLx/WsFBlnRL8e78KS4YNHh5WTB/3OOBqlEzxT09swDRffF22Z0AsKrK2SnOE+7tgOZwCInK8MpRp5wrCzoY8hrDjOPLCz41LY08xI+t2/2Z5GjDqj5nMaKH20cqEByUvN3q/QkucIojMsHd8j5RstIZBOROj/ jcjUj+Hw vxrhrstzUYWvCRzlsaNQp4aNgxswvDV6G87LPo79Wivh2VHCijW2oLsgrPFuU7h/GIhHmygfshLYWkA4npXG+2x3wC7TFLfS3VI6TQSONUQPCW/9WlJ25d/ps/JEW5gAZLWDGCkoCf+4hGB4Nov5w453BSPx1bvb1HPMDKqi7Sph78GFRFqsoprJ5fnTVfyfg7k8KEfgitmbw0MQiVSJPK5gs27wbKAnwFNf8QVdDv+APf7I6vSo1a9jBGA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A passthrough decoder is a decoder that maps only 1 target. It is a special case because it does not impose any constraints on the interleave-math as compared to a decoder with multiple targets. Extend the passthrough case to multi-target-capable decoders that only have one target selected. I.e. the current code was only considering passthrough *ports* which are only a subset of the potential passthrough decoder scenarios. Fixes: e4f6dfa9ef75 ("cxl/region: Fix 'distance' calculation with passthrough ports") Cc: Signed-off-by: Dan Williams Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/cxl/core/region.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index c82d3b6f3d1f..34cf95217901 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1019,10 +1019,10 @@ static int cxl_port_setup_targets(struct cxl_port *port, int i, distance; /* - * Passthrough ports impose no distance requirements between + * Passthrough decoders impose no distance requirements between * peers */ - if (port->nr_dports == 1) + if (cxl_rr->nr_targets == 1) distance = 0; else distance = p->nr_targets / cxl_rr->nr_targets; From patchwork Mon Feb 6 01:03:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12B85C636D4 for ; Mon, 6 Feb 2023 01:03:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B6786B0082; Sun, 5 Feb 2023 20:03:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 966336B0083; Sun, 5 Feb 2023 20:03:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B9666B0085; Sun, 5 Feb 2023 20:03:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6A4426B0082 for ; Sun, 5 Feb 2023 20:03:34 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2D87A1207BC for ; Mon, 6 Feb 2023 01:03:34 +0000 (UTC) X-FDA: 80435069148.04.34FE69B Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf12.hostedemail.com (Postfix) with ESMTP id D21734000D for ; Mon, 6 Feb 2023 01:03:31 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CphY8uKN; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf12.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645412; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9o5uw/5XrtFGjoX9/jZ37eeOniaL5aEi21ZMoLNKLdE=; b=T/IkTnzParnvq7RdYNn3nYNA7nEjXWdLLTo1o0sjhknu9lphdcF4HFxTDMbiciN+hGcunh 2MzA0dvLrZxYMpUjcR3pyHtMnx5numjXoYKEpyDgBT9yMhnVL+KXCN8/Ia5BVwi4EZclYz ryFO2BiJZ0KO3fqY5L8ZzLSzLVDohYU= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CphY8uKN; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf12.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645412; a=rsa-sha256; cv=none; b=8RzWpPEngt2krLuYELrz9qfWQWC2HGKb9ofSxvzwG5tFVWViY3anpK3t0HkgHSpZW4YOlt aTKRekt+2Kl8WuDgwDVUnLnI9q/3aoOFGpEgatXI/WdwweJjZO3G6RzJGnghJjWVGfJxdq DdC3Jzngl+UKsJ1yw8MCoDuxp144V3A= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645412; x=1707181412; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=npXpg/Au3yCYgiHvmbPs27wvuVW7voTuMbho/HBxtfI=; b=CphY8uKNHrzGEgiwP5auZT2P8gZLCjoD6OIGYKPT0Hd7bNdqbIRoJNeg TJsO//tUhlgSSpizhfwyHCQHc0aDoWHExydGEZaTN3dniYif/xb4GSshf Jqz3OR0PC9hqdi1D/5btduszm+ssFZhU15bB16fhUPdnzH/iRTE6ZIkAE BT7DT+K2fCGg2fIwpHQDaoBEh1P63AcdHzTXvcQEDVQTCwT94VUmP3xUG NU84QMaSrRqn5c/asRQG0jtpi8GEcT3Y7zQDx+0pigDN4JHEXF7WFNqh6 Prdla2mDSghlU28aMg/idZCEeDpsLDt+ruLx+T+yPLJNrCEcqTsKN7Rbh Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="312763223" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="312763223" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:30 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="616291339" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616291339" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:30 -0800 Subject: [PATCH 11/18] cxl/region: Add region autodiscovery From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:29 -0800 Message-ID: <167564540972.847146.17096178433176097831.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D21734000D X-Stat-Signature: ac8fsgc3z96guhjy6dzj4q96who5dtwr X-HE-Tag: 1675645411-778515 X-HE-Meta: U2FsdGVkX1+69ss+JHiwTro3yMybvUt8cZn3ZyppwKAd93k4dJ7PsaIk5b1QMpGp0nvBR6IO/GwBYI29qG9sqBRd3t2smUtpf8gqyhy5Wg3N7+Z7cIAmI6Jn9fB0ymwV2CB+sRWVIAvZ0AffM2+QyT2RWiXDh51t0o8uZRAlfW5SjBVB+AVwa3Tv/C29X1s3ntbRFE9OVQHv4lslJx10TYvleKJVpiGUP5zSM3JRCJUr98npmWoRXz4lJMXvIwEju9D6vv/paoe30UUJLIy/hfv8/KN8T/81KESMB5qbME3Sqm0lgvt6qDZ6FqK1XSGAKtaShHESZw1UKTmmbc8zRJkUxNDHGWRrxGB5QcvcL/ki9rgubIBCES0Sd7wC/tDTjVXRN1jercA7cpiFPjmhhVqDQv95mau6z3g3Dm32XfhLm1Y0YUjQTMOO2pwWJiTCMBSftmBddbTwwZhEB3mL7of+8sRgOwdTJPqBw4/Ku9papErzq9PJtCPVRCYmRZo2o2r4fnRRIF2aalSZdx6Szl7pdlv7xctYNbLdIgMpo95ZTWvpRgR83l2vOakyMo+Xp+xg3LWocCp06epVe6J3jlntDLw4kiuFhWVvhCRSUV32LU/QsPjvOco2uDOK9NRtyRoF87UYMg9aC7jchRlPnAVecv14GVw3sz3GgG3tUuUMwy/MhpRIojTJxQAbhGPWHVohsATF1pNFWEsoPPkH1U/CzLGMsafJOsGOMSdwf8gWMIpjC+nKWaKEtfhBQvVWxVxcJec98SmGzmjr1bHv5xk+CsN8oo+/OYUKlFdGshOhZPXMiNjCyX8eIbTUJSbFhpq67wfhfDuZEFrOPd84vD109dpfpGZefO5xf64KzWJ5dnqQiSbvXLCAeuBBokYlagxNqRoUB8IR/xNaz0rOSMPiiD9ELeRqzPV0s9512+A8dD9MGO4qBoKMjEAbjuPPWOGr6kcE05S3Tc0Anty UnNXLgR0 nQBXT+f/Id60A4tHRPYzNiVxaKOgVFRe2ATOMIGdbS3Y8hl/j8SN2Q1PSGk7RWX/9XTSen0thuh6C/aQA3zzVOi0ZrbuMWv+xDXHbF9A28pnuUFvLK9GQpTH31Isd9o1wLyXruP2M8JosZyURt9d1l2dyyffORj9iJgGDYbz+JvmCZ8cp4gC/3iqfxWvfGlYyiYck X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Region autodiscovery is an asynchrounous state machine advanced by cxl_port_probe(). After the decoders on an endpoint port are enumerated they are scanned for actively enabled instances. Each active decoder is flagged for auto-assembly CXL_DECODER_F_AUTO and attached to a region. If a region does not already exist for the address range setting of the decoder one is created. That creation process may race with other decoders of the same region being discovered since cxl_port_probe() is asynchronous. A new 'struct cxl_root_decoder' lock, @range_lock, is introduced to mitigate that race. Once all decoders have arrived, "p->nr_targets == p->interleave_ways", they are sorted by their relative decode position. The sort algorithm involves finding the point in the cxl_port topology where one leg of the decode leads to deviceA and the other deviceB. At that point in the topology the target order in the 'struct cxl_switch_decoder' indicates the relative position of those endpoint decoders in the region. >From that point the region goes through the same setup and validation steps as user-created regions, but instead of programming the decoders it validates that driver would have written the same values to the decoders as were already present. Signed-off-by: Dan Williams --- drivers/cxl/core/hdm.c | 5 drivers/cxl/core/port.c | 2 drivers/cxl/core/region.c | 496 ++++++++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxl.h | 16 + drivers/cxl/port.c | 26 ++ 5 files changed, 531 insertions(+), 14 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index dcc16d7cb8f3..174cddfec6e8 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -674,7 +674,7 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) up_read(&cxl_dpa_rwsem); port->commit_end--; - cxld->flags &= ~CXL_DECODER_F_ENABLE; + cxld->flags &= ~(CXL_DECODER_F_ENABLE | CXL_DECODER_F_AUTO); return 0; } @@ -719,7 +719,7 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, /* decoders are enabled if committed */ if (committed) { - cxld->flags |= CXL_DECODER_F_ENABLE; + cxld->flags |= CXL_DECODER_F_ENABLE | CXL_DECODER_F_AUTO; if (ctrl & CXL_HDM_DECODER0_CTRL_LOCK) cxld->flags |= CXL_DECODER_F_LOCK; if (FIELD_GET(CXL_HDM_DECODER0_CTRL_TYPE, ctrl)) @@ -783,6 +783,7 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, return rc; } *dpa_base += dpa_size + skip; + return 0; } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 47e450c3a5a9..8130430ffbcf 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -446,6 +446,7 @@ bool is_endpoint_decoder(struct device *dev) { return dev->type == &cxl_decoder_endpoint_type; } +EXPORT_SYMBOL_NS_GPL(is_endpoint_decoder, CXL); bool is_root_decoder(struct device *dev) { @@ -1622,6 +1623,7 @@ struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, } cxlrd->calc_hb = calc_hb; + mutex_init(&cxlrd->range_lock); cxld = &cxlsd->cxld; cxld->dev.type = &cxl_decoder_root_type; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 34cf95217901..6fe8c70790df 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -520,7 +521,12 @@ static void cxl_region_iomem_release(struct cxl_region *cxlr) if (device_is_registered(&cxlr->dev)) lockdep_assert_held_write(&cxl_region_rwsem); if (p->res) { - remove_resource(p->res); + /* + * Autodiscovered regions may not have been able to insert their + * resource. + */ + if (p->res->parent) + remove_resource(p->res); kfree(p->res); p->res = NULL; } @@ -1101,12 +1107,35 @@ static int cxl_port_setup_targets(struct cxl_port *port, return rc; } - cxld->interleave_ways = iw; - cxld->interleave_granularity = ig; - cxld->hpa_range = (struct range) { - .start = p->res->start, - .end = p->res->end, - }; + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { + if (cxld->interleave_ways != iw || + cxld->interleave_granularity != ig || + cxld->hpa_range.start != p->res->start || + cxld->hpa_range.end != p->res->end || + ((cxld->flags & CXL_DECODER_F_ENABLE) == 0)) { + dev_err(&cxlr->dev, + "%s:%s %s expected iw: %d ig: %d %pr\n", + dev_name(port->uport), dev_name(&port->dev), + __func__, iw, ig, p->res); + dev_err(&cxlr->dev, + "%s:%s %s got iw: %d ig: %d state: %s %#llx:%#llx\n", + dev_name(port->uport), dev_name(&port->dev), + __func__, cxld->interleave_ways, + cxld->interleave_granularity, + (cxld->flags & CXL_DECODER_F_ENABLE) ? + "enabled" : + "disabled", + cxld->hpa_range.start, cxld->hpa_range.end); + return -ENXIO; + } + } else { + cxld->interleave_ways = iw; + cxld->interleave_granularity = ig; + cxld->hpa_range = (struct range) { + .start = p->res->start, + .end = p->res->end, + }; + } dev_dbg(&cxlr->dev, "%s:%s iw: %d ig: %d\n", dev_name(port->uport), dev_name(&port->dev), iw, ig); add_target: @@ -1117,7 +1146,17 @@ static int cxl_port_setup_targets(struct cxl_port *port, dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), pos); return -ENXIO; } - cxlsd->target[cxl_rr->nr_targets_set] = ep->dport; + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { + if (cxlsd->target[cxl_rr->nr_targets_set] != ep->dport) { + dev_dbg(&cxlr->dev, "%s:%s: %s expected %s at %d\n", + dev_name(port->uport), dev_name(&port->dev), + dev_name(&cxlsd->cxld.dev), + dev_name(ep->dport->dport), + cxl_rr->nr_targets_set); + return -ENXIO; + } + } else + cxlsd->target[cxl_rr->nr_targets_set] = ep->dport; inc = 1; out_target_set: cxl_rr->nr_targets_set += inc; @@ -1159,6 +1198,13 @@ static void cxl_region_teardown_targets(struct cxl_region *cxlr) struct cxl_ep *ep; int i; + /* + * In the auto-discovery case skip automatic teardown since the + * address space is already active + */ + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) + return; + for (i = 0; i < p->nr_targets; i++) { cxled = p->targets[i]; cxlmd = cxled_to_memdev(cxled); @@ -1191,8 +1237,8 @@ static int cxl_region_setup_targets(struct cxl_region *cxlr) iter = to_cxl_port(iter->dev.parent); /* - * Descend the topology tree programming targets while - * looking for conflicts. + * Descend the topology tree programming / validating + * targets while looking for conflicts. */ for (ep = cxl_ep_load(iter, cxlmd); iter; iter = ep->next, ep = cxl_ep_load(iter, cxlmd)) { @@ -1287,6 +1333,182 @@ static int cxl_region_attach_position(struct cxl_region *cxlr, return rc; } +static int cxl_region_attach_auto(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, int pos) +{ + struct cxl_region_params *p = &cxlr->params; + + if (!(cxled->cxld.flags & CXL_DECODER_F_AUTO)) { + dev_err(&cxlr->dev, + "%s: unable to add decoder to autodetected region\n", + dev_name(&cxled->cxld.dev)); + return -EINVAL; + } + + if (pos >= 0) { + dev_dbg(&cxlr->dev, "%s: expected auto position, not %d\n", + dev_name(&cxled->cxld.dev), pos); + return -EINVAL; + } + + if (p->nr_targets >= p->interleave_ways) { + dev_err(&cxlr->dev, "%s: no more target slots available\n", + dev_name(&cxled->cxld.dev)); + return -ENXIO; + } + + /* + * Temporarily record the endpoint decoder into the target array. Yes, + * this means that userspace can view devices in the wrong position + * before the region activates, and must be careful to understand when + * it might be racing region autodiscovery. + */ + pos = p->nr_targets; + p->targets[pos] = cxled; + cxled->pos = pos; + p->nr_targets++; + + return 0; +} + +static struct cxl_port *next_port(struct cxl_port *port) +{ + if (!port->parent_dport) + return NULL; + return port->parent_dport->port; +} + +static int decoder_match_range(struct device *dev, void *data) +{ + struct cxl_endpoint_decoder *cxled = data; + struct cxl_switch_decoder *cxlsd; + + if (!is_switch_decoder(dev)) + return 0; + + cxlsd = to_cxl_switch_decoder(dev); + return range_contains(&cxlsd->cxld.hpa_range, &cxled->cxld.hpa_range); +} + +static void find_positions(const struct cxl_switch_decoder *cxlsd, + const struct cxl_port *iter_a, + const struct cxl_port *iter_b, int *a_pos, + int *b_pos) +{ + int i; + + for (i = 0, *a_pos = -1, *b_pos = -1; i < cxlsd->nr_targets; i++) { + if (cxlsd->target[i] == iter_a->parent_dport) + *a_pos = i; + else if (cxlsd->target[i] == iter_b->parent_dport) + *b_pos = i; + if (*a_pos >= 0 && *b_pos >= 0) + break; + } +} + +static int cmp_decode_pos(const void *a, const void *b) +{ + struct cxl_endpoint_decoder *cxled_a = *(typeof(cxled_a) *)a; + struct cxl_endpoint_decoder *cxled_b = *(typeof(cxled_b) *)b; + struct cxl_memdev *cxlmd_a = cxled_to_memdev(cxled_a); + struct cxl_memdev *cxlmd_b = cxled_to_memdev(cxled_b); + struct cxl_port *port_a = cxled_to_port(cxled_a); + struct cxl_port *port_b = cxled_to_port(cxled_b); + struct cxl_port *iter_a, *iter_b; + + /* Exit early if any prior sorting failed */ + if (cxled_a->pos < 0 || cxled_b->pos < 0) + return 0; + + /* + * Walk up the hierarchy to find a shared port, find the decoder + * that maps the range, compare the relative position of those + * dport mappings. + */ + for (iter_a = port_a; iter_a; iter_a = next_port(iter_a)) { + struct cxl_port *next_a, *next_b, *port; + struct cxl_switch_decoder *cxlsd; + + next_a = next_port(iter_a); + for (iter_b = port_b; iter_b; iter_b = next_port(iter_b)) { + int a_pos, b_pos, result; + struct device *dev; + unsigned int seq; + + next_b = next_port(iter_b); + if (next_a != next_b) + continue; + if (!next_a) + goto out; + port = next_a; + dev = device_find_child(&port->dev, cxled_a, + decoder_match_range); + if (!dev) { + struct range *range = &cxled_a->cxld.hpa_range; + + dev_err(port->uport, + "failed to find decoder that maps %#llx-:%#llx\n", + range->start, range->end); + cxled_a->pos = -1; + return 0; + } + + cxlsd = to_cxl_switch_decoder(dev); + do { + seq = read_seqbegin(&cxlsd->target_lock); + find_positions(cxlsd, iter_a, iter_b, &a_pos, + &b_pos); + } while (read_seqretry(&cxlsd->target_lock, seq)); + + if (a_pos < 0 || b_pos < 0) { + dev_err(port->uport, + "failed to find shared decoder for %s and %s\n", + dev_name(cxlmd_a->dev.parent), + dev_name(cxlmd_b->dev.parent)); + cxled_a->pos = -1; + result = 0; + } else { + result = a_pos - b_pos; + dev_dbg(port->uport, "%s: %s comes %s %s\n", + dev_name(&cxlsd->cxld.dev), + dev_name(cxlmd_a->dev.parent), + result < 0 ? "before" : "after", + dev_name(cxlmd_b->dev.parent)); + } + + put_device(dev); + + return result; + } + } +out: + dev_err(cxlmd_a->dev.parent, "failed to find shared port with %s\n", + dev_name(cxlmd_b->dev.parent)); + cxled_a->pos = -1; + return 0; +} + +static int cxl_region_sort_targets(struct cxl_region *cxlr) +{ + struct cxl_region_params *p = &cxlr->params; + int i, rc = 0; + + sort(p->targets, p->nr_targets, sizeof(p->targets[0]), cmp_decode_pos, + NULL); + + for (i = 0; i < p->nr_targets; i++) { + struct cxl_endpoint_decoder *cxled = p->targets[i]; + + if (cxled->pos < 0) + rc = -ENXIO; + cxled->pos = i; + } + + dev_dbg(&cxlr->dev, "region sort %s\n", rc ? "failed" : "successful"); + return rc; +} + static int cxl_region_attach(struct cxl_region *cxlr, struct cxl_endpoint_decoder *cxled, int pos) { @@ -1350,6 +1572,50 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -EINVAL; } + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { + int i; + + rc = cxl_region_attach_auto(cxlr, cxled, pos); + if (rc) + return rc; + + /* await more targets to arrive... */ + if (p->nr_targets < p->interleave_ways) + return 0; + + /* + * All targets are here, which implies all PCI enumeration that + * affects this region has been completed. Walk the topology to + * sort the devices into their relative region decode position. + */ + rc = cxl_region_sort_targets(cxlr); + if (rc) + return rc; + + for (i = 0; i < p->nr_targets; i++) { + cxled = p->targets[i]; + ep_port = cxled_to_port(cxled); + dport = cxl_find_dport_by_dev(root_port, + ep_port->host_bridge); + rc = cxl_region_attach_position(cxlr, cxlrd, cxled, + dport, i); + if (rc) + return rc; + } + + rc = cxl_region_setup_targets(cxlr); + if (rc) + return rc; + + /* + * If target setup succeeds in the autodiscovery case + * then the region is already committed. + */ + p->state = CXL_CONFIG_COMMIT; + + return 0; + } + rc = cxl_region_validate_position(cxlr, cxled, pos); if (rc) return rc; @@ -1500,8 +1766,8 @@ static int detach_target(struct cxl_region *cxlr, int pos) return rc; } -static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos, - size_t len) +static ssize_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos, + size_t len) { int rc; @@ -2079,6 +2345,191 @@ static int devm_cxl_add_pmem_region(struct cxl_region *cxlr) return rc; } +static int match_decoder_by_range(struct device *dev, void *data) +{ + struct range *r1, *r2 = data; + struct cxl_root_decoder *cxlrd; + + if (!is_root_decoder(dev)) + return 0; + + cxlrd = to_cxl_root_decoder(dev); + r1 = &cxlrd->cxlsd.cxld.hpa_range; + return range_contains(r1, r2); +} + +static int match_region_by_range(struct device *dev, void *data) +{ + struct cxl_region_params *p; + struct cxl_region *cxlr; + struct range *r = data; + int rc = 0; + + if (!is_cxl_region(dev)) + return 0; + + cxlr = to_cxl_region(dev); + p = &cxlr->params; + + down_read(&cxl_region_rwsem); + if (p->res && p->res->start == r->start && p->res->end == r->end) + rc = 1; + up_read(&cxl_region_rwsem); + + return rc; +} + +/* Establish an empty region covering the given HPA range */ +static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder *cxled) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_port *port = cxlrd_to_port(cxlrd); + struct range *hpa = &cxled->cxld.hpa_range; + struct cxl_region_params *p; + struct cxl_region *cxlr; + struct resource *res; + int rc; + + do { + cxlr = __create_region(cxlrd, cxled->mode, + atomic_read(&cxlrd->region_id)); + } while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY); + + if (IS_ERR(cxlr)) { + dev_err(cxlmd->dev.parent, + "%s:%s: %s failed assign region: %ld\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + __func__, PTR_ERR(cxlr)); + return cxlr; + } + + down_write(&cxl_region_rwsem); + p = &cxlr->params; + if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) { + dev_err(cxlmd->dev.parent, + "%s:%s: %s autodiscovery interrupted\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + __func__); + rc = -EBUSY; + goto err; + } + + set_bit(CXL_REGION_F_AUTO, &cxlr->flags); + + res = kmalloc(sizeof(*res), GFP_KERNEL); + if (!res) { + rc = -ENOMEM; + goto err; + } + + *res = DEFINE_RES_MEM_NAMED(hpa->start, range_len(hpa), + dev_name(&cxlr->dev)); + rc = insert_resource(cxlrd->res, res); + if (rc) { + /* + * Platform-firmware may not have split resources like "System + * RAM" on CXL window boundaries see cxl_region_iomem_release() + */ + dev_warn(cxlmd->dev.parent, + "%s:%s: %s %s cannot insert resource\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + __func__, dev_name(&cxlr->dev)); + } + + p->res = res; + p->interleave_ways = cxled->cxld.interleave_ways; + p->interleave_granularity = cxled->cxld.interleave_granularity; + p->state = CXL_CONFIG_INTERLEAVE_ACTIVE; + + rc = sysfs_update_group(&cxlr->dev.kobj, get_cxl_region_target_group()); + if (rc) + goto err; + + dev_dbg(cxlmd->dev.parent, "%s:%s: %s %s res: %pr iw: %d ig: %d\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), __func__, + dev_name(&cxlr->dev), p->res, p->interleave_ways, + p->interleave_granularity); + + /* ...to match put_device() in cxl_add_to_region() */ + get_device(&cxlr->dev); + up_write(&cxl_region_rwsem); + + return cxlr; + +err: + up_write(&cxl_region_rwsem); + devm_release_action(port->uport, unregister_region, cxlr); + return ERR_PTR(rc); +} + +void cxl_add_to_region(struct cxl_endpoint_decoder *cxled) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct range *hpa = &cxled->cxld.hpa_range; + struct cxl_decoder *cxld = &cxled->cxld; + struct cxl_root_decoder *cxlrd; + struct cxl_region_params *p; + struct cxl_region *cxlr; + struct cxl_port *root; + bool attach = false; + struct device *dev; + + root = find_cxl_root(&cxlmd->dev); + if (!root) { + dev_err(cxlmd->dev.parent, "%s: failed to map CXL root\n", + dev_name(&cxlmd->dev)); + return; + } + + dev = device_find_child(&root->dev, &cxld->hpa_range, + match_decoder_by_range); + if (!dev) { + dev_err(cxlmd->dev.parent, + "%s:%s no CXL window for range %#llx:%#llx\n", + dev_name(&cxlmd->dev), dev_name(&cxld->dev), + cxld->hpa_range.start, cxld->hpa_range.end); + goto out; + } + cxlrd = to_cxl_root_decoder(dev); + + mutex_lock(&cxlrd->range_lock); + dev = device_find_child(&cxlrd->cxlsd.cxld.dev, hpa, + match_region_by_range); + if (!dev) + cxlr = construct_region(cxlrd, cxled); + else + cxlr = to_cxl_region(dev); + mutex_unlock(&cxlrd->range_lock); + + if (IS_ERR(cxlr)) + goto out; + + attach_target(cxlr, cxled, -1, TASK_UNINTERRUPTIBLE); + + down_read(&cxl_region_rwsem); + p = &cxlr->params; + attach = p->state >= CXL_CONFIG_COMMIT; + up_read(&cxl_region_rwsem); + + if (attach) { + int rc = device_attach(&cxlr->dev); + + /* + * If device_attach() fails the range may still be + * active via the platform-firmware memory map + */ + if (rc < 0) + dev_err(&cxlr->dev, "failed to enable, range: %pr\n", + p->res); + } + + put_device(&cxlr->dev); +out: + put_device(&root->dev); +} +EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, CXL); + static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) { if (!test_bit(CXL_REGION_F_INCOHERENT, &cxlr->flags)) @@ -2103,6 +2554,15 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) return 0; } +static int is_system_ram(struct resource *res, void *arg) +{ + struct cxl_region *cxlr = arg; + struct cxl_region_params *p = &cxlr->params; + + dev_dbg(&cxlr->dev, "%pr has System RAM: %pr\n", p->res, res); + return 1; +} + static int cxl_region_probe(struct device *dev) { struct cxl_region *cxlr = to_cxl_region(dev); @@ -2136,6 +2596,18 @@ static int cxl_region_probe(struct device *dev) switch (cxlr->mode) { case CXL_DECODER_PMEM: return devm_cxl_add_pmem_region(cxlr); + case CXL_DECODER_RAM: + /* + * The region can not be manged by CXL if any portion of + * it is already online as 'System RAM' + */ + if (walk_iomem_res_desc(IORES_DESC_NONE, + IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY, + p->res->start, p->res->end, cxlr, + is_system_ram) > 0) + return 0; + dev_dbg(dev, "TODO: hookup devdax\n"); + return 0; default: dev_dbg(&cxlr->dev, "unsupported region mode: %d\n", cxlr->mode); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ca76879af1de..9b3765c5c81a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -261,6 +261,8 @@ resource_size_t cxl_rcrb_to_component(struct device *dev, * cxl_decoder flags that define the type of memory / devices this * decoder supports as well as configuration lock status See "CXL 2.0 * 8.2.5.12.7 CXL HDM Decoder 0 Control Register" for details. + * Additionally indicate whether decoder settings were autodetected, + * user customized. */ #define CXL_DECODER_F_RAM BIT(0) #define CXL_DECODER_F_PMEM BIT(1) @@ -268,6 +270,7 @@ resource_size_t cxl_rcrb_to_component(struct device *dev, #define CXL_DECODER_F_TYPE3 BIT(3) #define CXL_DECODER_F_LOCK BIT(4) #define CXL_DECODER_F_ENABLE BIT(5) +#define CXL_DECODER_F_AUTO BIT(6) #define CXL_DECODER_F_MASK GENMASK(5, 0) enum cxl_decoder_type { @@ -380,6 +383,7 @@ typedef struct cxl_dport *(*cxl_calc_hb_fn)(struct cxl_root_decoder *cxlrd, * @region_id: region id for next region provisioning event * @calc_hb: which host bridge covers the n'th position by granularity * @platform_data: platform specific configuration data + * @range_lock: sync region autodiscovery by address range * @cxlsd: base cxl switch decoder */ struct cxl_root_decoder { @@ -387,6 +391,7 @@ struct cxl_root_decoder { atomic_t region_id; cxl_calc_hb_fn calc_hb; void *platform_data; + struct mutex range_lock; struct cxl_switch_decoder cxlsd; }; @@ -436,6 +441,13 @@ struct cxl_region_params { */ #define CXL_REGION_F_INCOHERENT 0 +/* + * Indicate whether this region has been assembled by autodetection or + * userspace assembly. Prevent endpoint decoders outside of automatic + * detection from being added to the region. + */ +#define CXL_REGION_F_AUTO 1 + /** * struct cxl_region - CXL region * @dev: This region's device @@ -699,6 +711,7 @@ struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct device *dev); #ifdef CONFIG_CXL_REGION bool is_cxl_pmem_region(struct device *dev); struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); +void cxl_add_to_region(struct cxl_endpoint_decoder *cxled); #else static inline bool is_cxl_pmem_region(struct device *dev) { @@ -708,6 +721,9 @@ static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) { return NULL; } +static inline void cxl_add_to_region(struct cxl_endpoint_decoder *cxled) +{ +} #endif /* diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 5453771bf330..012a0c6f8476 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -30,6 +30,23 @@ static void schedule_detach(void *cxlmd) schedule_cxl_memdev_detach(cxlmd); } +static int discover_region(struct device *dev, void *data) +{ + const unsigned long flags = CXL_DECODER_F_ENABLE | CXL_DECODER_F_AUTO; + struct cxl_endpoint_decoder *cxled; + + if (!is_endpoint_decoder(dev)) + return 0; + + cxled = to_cxl_endpoint_decoder(dev); + if ((cxled->cxld.flags & flags) != flags) + return 0; + + cxl_add_to_region(cxled); + + return 0; +} + static int cxl_port_probe(struct device *dev) { struct cxl_port *port = to_cxl_port(dev); @@ -78,6 +95,15 @@ static int cxl_port_probe(struct device *dev) return rc; } + if (!is_cxl_endpoint(port)) + return 0; + + /* + * Now that all endpoint decoders are successfully enumerated, + * try to assemble regions from committed decoders + */ + device_for_each_child(dev, NULL, discover_region); + return 0; } From patchwork Mon Feb 6 01:03:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 736AEC636CD for ; Mon, 6 Feb 2023 01:03:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 178DD6B0075; Sun, 5 Feb 2023 20:03:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1021E6B0078; Sun, 5 Feb 2023 20:03:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE5396B0083; Sun, 5 Feb 2023 20:03:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DEC1F6B0075 for ; Sun, 5 Feb 2023 20:03:39 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C18F61406B9 for ; Mon, 6 Feb 2023 01:03:39 +0000 (UTC) X-FDA: 80435069358.05.15990D2 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 91A231C0024 for ; Mon, 6 Feb 2023 01:03:37 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nqrrCVRX; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645417; a=rsa-sha256; cv=none; b=iasE8yrUl1HhtgwfHtSr2BL9Wv1MPmqgin8N+eQaTIsohKZCHyGhDbrSkeaTs3FLbsaYzg iskUQ0sbAP8ZOkUW+WN1cTLm6NxdjSJoSzSA92kdemUv/ZdaLlTV4tZOgWWSZq/85rwX9C CaI4DT+gweRcqgQ4b83ZopNUIuen3LA= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nqrrCVRX; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645417; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1C/Z6EBP9muvAR3HbqgsSQpSfqYixv6+xKfJ8Vt04KE=; b=HnjUpHiXi8AUuDzOqrSD1ssH8MsqubcLVTjnbJbDpMozD7XFYYBVz/g16qrHcpmGn7Wq+X tt27TQ5DUuS8o3vdJvngyMJof76wuNtsjTaxUw4DDXXjrUc46+/cL1j0eziClmAueNf09M u/FnyN5Y5u7MAooIeOy6F3MgRfhK6+4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645417; x=1707181417; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=t66eRnuPtljjvIsyyQm2MInJpJjgFLhSqg1tPwrvkw4=; b=nqrrCVRXSRlAy5prcHR6X2Bs9xdFf1/Xngyijzx/9l7jJ++iBU+2GYZY xwOs8KdQJfnXHQ5wcgyEuVtL+xKxPxG3eZc6G6UMPQJ0ugZls6+GsC9fG TBeXdNa8O/U9qvTrck63a0by7/vR4y6sIpGjUUH3EIv9vpXRHZf+I3COP LhEDdiwORZL6QCs1W75UG6S+/bY2NyN+q6eiCev9xSnSt2FWpGwjI2oL9 3iiBNtLTfRPzylXELypon/A3gkTuXJ6/ytdfixz+YU/dzJ4k8L+s/ka62 YrDHXA9tYifjz+j3e2mCt4tYNSoEfgLq79y+zouC9JD3IGCJKubFy89RF A==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442657" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442657" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:36 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006645" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006645" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:35 -0800 Subject: [PATCH 12/18] tools/testing/cxl: Define a fixed volatile configuration to parse From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:35 -0800 Message-ID: <167564541523.847146.12199636368812381475.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 91A231C0024 X-Rspamd-Server: rspam01 X-Stat-Signature: ksp1rn4s6tqm8raigcnp8pqdz3urxzoo X-HE-Tag: 1675645417-315850 X-HE-Meta: U2FsdGVkX19vncSISbhqbPHZvIEtVpFj4Km2hwE0xKeSMEWghvMyiEAQUUwORVleQ78UDi34/4VOHpYXDw2tcZOIscCEVNLOTlOd3/Sxt6W2DncgyeB8UBWl2S7wC7uYYcj3DpiWYM2fbhDxrng+8ppOcbHGbTr+e2Xtmy+IoadaOKJ5Po6bhPuo8xwvn7xrZ3lB+m6Gq057KUGWKozLdar4ubKD6WSuT6nwg7U9z+/brHhgRaMrTSbt1pQRgIceQFJcut4GuC13V6luiWUH816hIlOgJTGgGhDHL7mBLLPZr9ZLkg0AZTgqS/1sJa9q9srGgOQfsaeHH7o6xLK9G0jnxeIFyrNhzMboq/DdJx22JTaAC2fGhCeaQgSFehnl8PfLc4hpF3FASlb9Y1w2gIs+VSnOUaCAZ2syu6BtJCyy67ThDQNoNcR3I7HdnWKcsQ8PGm13n9uJFYvb3QRkNMsllmyHmI23ECLH+YGeN5tCCZ84t/Dupp25ely3D+OJzCZlxsR1W54w4M5FiQs4HlBP8EY7KaILyRKylclqqNyl64XMJxYGeaPZ6y24HhbtCxw+TAt7iJ9r1RwCQd2eWi2XAjGcwv9WrsBap17QzV/8jEcZJp6Zu4rTwrgKueMj4TLOkt/BpEmjvdhUDZw0ufc8QEsTOQDAhl67qZp0WBP1VYPQQkInhewdF8kAK0lynrbS6RS0oWldtTWF6JXoEp1PMVrMjJkugdgyKZxgBF37W4ZHhJq0VV/TFqU8SQRh/qTsBRf+KmF4j5XgbpZHRfh3o+D277EHMdjT/ldLTV5Idxq6DPUmRW2N0c6oU4AqHl6QSrS8hpM5wDsSb8G6FBGUcDa24RGg1mdvOTE+Nh6PJBMVL1L8pLonRhHkfGjuCUhss+NW6n4QzOqF8i/qrc4cQ1b4xIjICTGMgho/Kq+6roD0T72EUzIoJ2/iBVcPo8xF5ucZcK8CbZ11Vbn TJ59JxYq htqlBusbRKp3RQiwQ2X8wysGB5+dGdnVO9tDjYrK2mnFAihzg8jj9Op4UK+1ErVIc8tNwawdodlmwQ+ixrMMqEE/tgUkunh+qbd5QiXSpi/A4kisoXHuKZ6/IdOJES4LlLZXbfOQz4yw5shc8YzwILZ7rlaihrQR58OriQkQ7g7GdQZjouFna1ZlCfNyPV/4MOIN8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Take two endpoints attached to the first switch on the first host-bridge in the cxl_test topology and define a pre-initialized region. This is a x2 interleave underneath a x1 CXL Window. $ modprobe cxl_test $ # cxl list -Ru { "region":"region3", "resource":"0xf010000000", "size":"512.00 MiB (536.87 MB)", "interleave_ways":2, "interleave_granularity":4096, "decode_state":"commit" } Signed-off-by: Dan Williams --- drivers/cxl/core/core.h | 3 - drivers/cxl/core/hdm.c | 3 + drivers/cxl/core/port.c | 4 + drivers/cxl/cxl.h | 4 + drivers/cxl/cxlmem.h | 3 + tools/testing/cxl/test/cxl.c | 146 +++++++++++++++++++++++++++++++++++++++--- 6 files changed, 148 insertions(+), 15 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 5eb873da5a30..479f01da6d35 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -57,9 +57,6 @@ resource_size_t cxl_dpa_size(struct cxl_endpoint_decoder *cxled); resource_size_t cxl_dpa_resource_start(struct cxl_endpoint_decoder *cxled); extern struct rw_semaphore cxl_dpa_rwsem; -bool is_switch_decoder(struct device *dev); -struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev); - int cxl_memdev_init(void); void cxl_memdev_exit(void); void cxl_mbox_init(void); diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 174cddfec6e8..28fa4d835259 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -279,7 +279,7 @@ static int __cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, return 0; } -static int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, +int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, resource_size_t base, resource_size_t len, resource_size_t skipped) { @@ -295,6 +295,7 @@ static int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, return devm_add_action_or_reset(&port->dev, cxl_dpa_release, cxled); } +EXPORT_SYMBOL_NS_GPL(devm_cxl_dpa_reserve, CXL); resource_size_t cxl_dpa_size(struct cxl_endpoint_decoder *cxled) { diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 8130430ffbcf..8d0895cbae93 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -458,6 +458,7 @@ bool is_switch_decoder(struct device *dev) { return is_root_decoder(dev) || dev->type == &cxl_decoder_switch_type; } +EXPORT_SYMBOL_NS_GPL(is_switch_decoder, CXL); struct cxl_decoder *to_cxl_decoder(struct device *dev) { @@ -485,6 +486,7 @@ struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev) return NULL; return container_of(dev, struct cxl_switch_decoder, cxld.dev); } +EXPORT_SYMBOL_NS_GPL(to_cxl_switch_decoder, CXL); static void cxl_ep_release(struct cxl_ep *ep) { @@ -528,7 +530,7 @@ static const struct device_type cxl_port_type = { bool is_cxl_port(struct device *dev) { - return dev->type == &cxl_port_type; + return dev && dev->type == &cxl_port_type; } EXPORT_SYMBOL_NS_GPL(is_cxl_port, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 9b3765c5c81a..4c6ee6c96f23 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -452,6 +452,7 @@ struct cxl_region_params { * struct cxl_region - CXL region * @dev: This region's device * @id: This region's id. Id is globally unique across all regions + * @fixed: At least one decoder in this region was locked down at init * @mode: Endpoint decoder allocation / access mode * @type: Endpoint decoder target type * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown @@ -462,6 +463,7 @@ struct cxl_region_params { struct cxl_region { struct device dev; int id; + bool fixed; enum cxl_decoder_mode mode; enum cxl_decoder_type type; struct cxl_nvdimm_bridge *cxl_nvb; @@ -643,8 +645,10 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct cxl_decoder *to_cxl_decoder(struct device *dev); struct cxl_root_decoder *to_cxl_root_decoder(struct device *dev); +struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev); struct cxl_endpoint_decoder *to_cxl_endpoint_decoder(struct device *dev); bool is_root_decoder(struct device *dev); +bool is_switch_decoder(struct device *dev); bool is_endpoint_decoder(struct device *dev); struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, unsigned int nr_targets, diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index ab138004f644..82a430cfc3ff 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -79,6 +79,9 @@ static inline bool is_cxl_endpoint(struct cxl_port *port) } struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds); +int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, + resource_size_t base, resource_size_t len, + resource_size_t skipped); static inline struct cxl_ep *cxl_ep_load(struct cxl_port *port, struct cxl_memdev *cxlmd) diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 920bd969c554..ea761afa446f 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -703,6 +703,141 @@ static int mock_decoder_reset(struct cxl_decoder *cxld) return 0; } +static void default_mock_decoder(struct cxl_decoder *cxld) +{ + cxld->hpa_range = (struct range){ + .start = 0, + .end = -1, + }; + + cxld->interleave_ways = 1; + cxld->interleave_granularity = 256; + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->commit = mock_decoder_commit; + cxld->reset = mock_decoder_reset; +} + +static int first_decoder(struct device *dev, void *data) +{ + struct cxl_decoder *cxld; + + if (!is_switch_decoder(dev)) + return 0; + cxld = to_cxl_decoder(dev); + if (cxld->id == 0) + return 1; + return 0; +} + +static void mock_init_hdm_decoder(struct cxl_decoder *cxld) +{ + struct acpi_cedt_cfmws *window = mock_cfmws[0]; + struct platform_device *pdev = NULL; + struct cxl_endpoint_decoder *cxled; + struct cxl_switch_decoder *cxlsd; + struct cxl_port *port, *iter; + const int size = SZ_512M; + struct cxl_memdev *cxlmd; + struct cxl_dport *dport; + struct device *dev; + bool hb0 = false; + u64 base; + int i; + + if (is_endpoint_decoder(&cxld->dev)) { + cxled = to_cxl_endpoint_decoder(&cxld->dev); + cxlmd = cxled_to_memdev(cxled); + WARN_ON(!dev_is_platform(cxlmd->dev.parent)); + pdev = to_platform_device(cxlmd->dev.parent); + + /* check is endpoint is attach to host-bridge0 */ + port = cxled_to_port(cxled); + do { + if (port->uport == &cxl_host_bridge[0]->dev) { + hb0 = true; + break; + } + if (is_cxl_port(port->dev.parent)) + port = to_cxl_port(port->dev.parent); + else + port = NULL; + } while (port); + port = cxled_to_port(cxled); + } + + /* + * The first decoder on the first 2 devices on the first switch + * attached to host-bridge0 mock a fake / static RAM region. All + * other decoders are default disabled. Given the round robin + * assignment those devices are named cxl_mem.0, and cxl_mem.4. + * + * See 'cxl list -BMPu -m cxl_mem.0,cxl_mem.4' + */ + if (!hb0 || pdev->id % 4 || pdev->id > 4 || cxld->id > 0) { + default_mock_decoder(cxld); + return; + } + + base = window->base_hpa; + cxld->hpa_range = (struct range) { + .start = base, + .end = base + size - 1, + }; + + cxld->interleave_ways = 2; + eig_to_granularity(window->granularity, &cxld->interleave_granularity); + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->flags = CXL_DECODER_F_ENABLE | CXL_DECODER_F_AUTO; + port->commit_end = cxld->id; + devm_cxl_dpa_reserve(cxled, 0, size / cxld->interleave_ways, 0); + cxld->commit = mock_decoder_commit; + cxld->reset = mock_decoder_reset; + + /* + * Now that endpoint decoder is set up, walk up the hierarchy + * and setup the switch and root port decoders targeting @cxlmd. + */ + iter = port; + for (i = 0; i < 2; i++) { + dport = iter->parent_dport; + iter = dport->port; + dev = device_find_child(&iter->dev, NULL, first_decoder); + /* + * Ancestor ports are guaranteed to be enumerated before + * @port, and all ports have at least one decoder. + */ + if (WARN_ON(!dev)) + continue; + cxlsd = to_cxl_switch_decoder(dev); + if (i == 0) { + /* put cxl_mem.4 second in the decode order */ + if (pdev->id == 4) + cxlsd->target[1] = dport; + else + cxlsd->target[0] = dport; + } else + cxlsd->target[0] = dport; + cxld = &cxlsd->cxld; + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->flags = CXL_DECODER_F_ENABLE; + iter->commit_end = 0; + /* + * Switch targets 2 endpoints, while host bridge targets + * one root port + */ + if (i == 0) + cxld->interleave_ways = 2; + else + cxld->interleave_ways = 1; + cxld->interleave_granularity = 256; + cxld->hpa_range = (struct range) { + .start = base, + .end = base + size - 1, + }; + put_device(dev); + } +} + static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) { struct cxl_port *port = cxlhdm->port; @@ -748,16 +883,7 @@ static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) cxld = &cxled->cxld; } - cxld->hpa_range = (struct range) { - .start = 0, - .end = -1, - }; - - cxld->interleave_ways = min_not_zero(target_count, 1); - cxld->interleave_granularity = SZ_4K; - cxld->target_type = CXL_DECODER_EXPANDER; - cxld->commit = mock_decoder_commit; - cxld->reset = mock_decoder_reset; + mock_init_hdm_decoder(cxld); if (target_count) { rc = device_for_each_child(port->uport, &ctx, From patchwork Mon Feb 6 01:03:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1FDFC636CC for ; Mon, 6 Feb 2023 01:03:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 792CC6B0074; Sun, 5 Feb 2023 20:03:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 743786B0078; Sun, 5 Feb 2023 20:03:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60BF26B0083; Sun, 5 Feb 2023 20:03:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4C7146B0074 for ; Sun, 5 Feb 2023 20:03:44 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2FDC916090A for ; Mon, 6 Feb 2023 01:03:44 +0000 (UTC) X-FDA: 80435069568.30.A3CC35E Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 25BD21C000F for ; Mon, 6 Feb 2023 01:03:41 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=eP4cAodk; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645422; a=rsa-sha256; cv=none; b=UQxXx7EIAnSR+1gFy72lG7uy8e923CaV551MFhD8xYhxb9v3HWrheR1nl1cKJcp89ulMo/ a54i0EYz+84fUCfpYDiRFShgXx/2vYinhjYjGJy2YlSYt3/I2uWZXKEenFq2GgZ00OVoUa P0HXxNqe4kjrDNexJewuD4vJD6pRnmY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=eP4cAodk; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Clt0rrKEtsA377yEcSYmcHAsRruSX8GZOpR5+rVGnW4=; b=6su7OKNcpLHZTN/VOxIwmR1Ro+JzSn1uP9MP5j89bOYvSYhFdDInEE4+9JaDZ7ECD+mOct l4h+zr4gsfJi7gd65YbgWeqpV1KqjaloOJuaa7myHnSjcnFABdDkoH7wUcUPJ9xqLLmhKp y4ZEFkXj+uAEX4Q2Mea9YeVnIEkg7j4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645422; x=1707181422; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=efc+FneV0jKpVBpxm5XWbyZ/alUdKgjj2AHpNBEopmM=; b=eP4cAodkDv9olDacOZruUJcfdxGqC4rI3/oXKsdQ9WA1ghgZBEzLp6qv JS8iCtDqBIDD1qHzA2jkCTPuX1VYfZwJsHCaPpSaaO0hidgw10jVE2145 SvxbHlm44ZgOIt19CRU4lneXSPw4I1V31vewUly8kPIt5FJQP+CZvgmS9 aruP7uE4GUbavns8r0xdoV0JtEqlvkXqm6V1Q0Ii50wl5dXolomth4JiV cYdjD9Z5fRN2K2HMKhvNMQ8V0vh0xg5U+0cNFn8D4AUBcNXOS4SAoL0WA LT2FQHTmQ7xOBSgxa5Y212jgt2DF1sAQfl8hYjyR0Usi+fGs3a7foNc5w w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442668" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442668" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:41 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006670" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006670" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:41 -0800 Subject: [PATCH 13/18] dax/hmem: Move HMAT and Soft reservation probe initcall level From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:41 -0800 Message-ID: <167564542109.847146.10113972881782419363.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 25BD21C000F X-Rspamd-Server: rspam01 X-Stat-Signature: uz8xxi4ajtcttywag5nqbe6z65qsfpj5 X-HE-Tag: 1675645421-933436 X-HE-Meta: U2FsdGVkX1+etovQOXpkuwn1VjB17osceqexPW+zcXcXaQL4xsA1yrMilHjvKfFOpj3EsHmFni46t79hmVcJL9PqTnuOhN65dpEU6KZufKr4JZ2Kyo85fLp6GirYnjCfad+mB0vK2E0JaV804LSYmHXbS6a1g8e2mPOU4TjHIDmLv3bhhzb6xoxahLa3CIoFuYD9IrZKCxLlxFjaf47NsdJH5vi/k4ospiWHRWRrNWxhmB0NEqn9lNONAf2tco+1IkWVHeijisdkPfbtqf6p0cJRu+Zys3Bj+RdzdrlAauyVkhiVD0TMlAa5vjxKCXIgD6KMPCgQ9+HfcfQ0U+Gf62xDtZxD9RYSsunMyUKMo8kQUZ7llwdOmuryCrmRBg+d75+UzobgItUeuVnwtBtb08/YjO4NDGtSYB4/OZJS8pgHpxxJLr+YalgCDJeCO7C/yQSsCBwjsWhmA/q97L3RIWgDPmLMQsHq9qsLr/JM+tPRw6/Mt/iF9/sTr1sfHjB1IteaENnRPOJ2OrMrUOH590jW/QH6ftftdqXy/+N8hOcAg2dVAPNrc4+h3pH4aGSLNU2RGr3z9PEHkfuwDRSu0QMRIvQW8jIfrgvxom2dQyi0ENPENq74SPAyksmwoHCfOl/HSJZGwW84WFFlUmNChBbuCgedp5FF9NH0mlLph3YqqjedMYB0WbJ/9NbYegtuTh3aLCFe4WlQvrHBvqzSZrLaO+0ws6mM7sral+03soi1vM0F6h+5zfnXGPW9EiV+CnRaU9i782CSTkoDOWDTJqOU8Itic0jGPHZrvVdW1QXUpm3u0sL7XsqFyffq8WZ2Gbo8p0W57v2xy7+vDWZXfmXlcGHDU+qF2jLzEPBGCvn+l4Ox9uuhFafMHE46UmHsiWojYGCwrKAsZTTwCHH/MSJYgjTZEV0ru/CkMvsdr5/i2hK2tEjVofP677LuwHuGpxt3GVQxjJNWQSeSTvO 5pCQSYLV xEfMyjeGkZLlEZcggoxChozxmQ/t4UOee0QFuxE5vogjpbvvGTtgFLsRoXaMYajzuleQW0tpQrlQIWk1udU4fT9k7ALamyhs+KqLAlrG2htwj8y4mfCuwVCyUYbHP6Tqhnb2j2yQWnNOVu5nouM6cpBF9Q9g/yTmUhO7Yaq9kBT4Mg7lk9z8AytvN7w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for moving more filtering of "hmem" ranges into the dax_hmem.ko module, update the initcall levels. HMAT range registration moves to subsys_initcall() to be done before Soft Reservation probing, and Soft Reservation probing is moved to device_initcall() to be done before dax_hmem.ko initialization if it is built-in. Signed-off-by: Dan Williams --- drivers/acpi/numa/hmat.c | 2 +- drivers/dax/hmem/Makefile | 3 ++- drivers/dax/hmem/device.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index 605a0c7053be..ff24282301ab 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -869,4 +869,4 @@ static __init int hmat_init(void) acpi_put_table(tbl); return 0; } -device_initcall(hmat_init); +subsys_initcall(hmat_init); diff --git a/drivers/dax/hmem/Makefile b/drivers/dax/hmem/Makefile index 57377b4c3d47..d4c4cd6bccd7 100644 --- a/drivers/dax/hmem/Makefile +++ b/drivers/dax/hmem/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_DEV_DAX_HMEM) += dax_hmem.o +# device_hmem.o deliberately precedes dax_hmem.o for initcall ordering obj-$(CONFIG_DEV_DAX_HMEM_DEVICES) += device_hmem.o +obj-$(CONFIG_DEV_DAX_HMEM) += dax_hmem.o device_hmem-y := device.o dax_hmem-y := hmem.o diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index 903325aac991..20749c7fab81 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -104,4 +104,4 @@ static __init int hmem_init(void) * As this is a fallback for address ranges unclaimed by the ACPI HMAT * parsing it must be at an initcall level greater than hmat_init(). */ -late_initcall(hmem_init); +device_initcall(hmem_init); From patchwork Mon Feb 6 01:03:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19DDBC636CD for ; Mon, 6 Feb 2023 01:03:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A79326B0078; Sun, 5 Feb 2023 20:03:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2A866B0083; Sun, 5 Feb 2023 20:03:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 918496B0085; Sun, 5 Feb 2023 20:03:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8393C6B0078 for ; Sun, 5 Feb 2023 20:03:50 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 612291C2E96 for ; Mon, 6 Feb 2023 01:03:50 +0000 (UTC) X-FDA: 80435069820.29.A09F16A Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 52AE51C0010 for ; Mon, 6 Feb 2023 01:03:48 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=gx6W2Yk7; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645428; a=rsa-sha256; cv=none; b=Gz6v2FywrcNJoSZCPEh5/mBk2tvV8rMCu5EM0e/O7QIfOJYkrpe7B/d5FvAuthMZrnSX32 kSwKCoIiPnmar9sA4cxWTUwDE++UFm0+79I5VkqGwf20zvdpd9d23DnpODh+IJyKJ9VnxZ tvl5LZaL2bUHtapLiR2XVbEmkFmaEtM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=gx6W2Yk7; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645428; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PDQ4VDXVLgXkGdPLT1+87HpZN513nvknBCEh5IvZ+p4=; b=Z+Q/mP1kZeU3REznatznNKhYpRFBkkQHEk15eQbCtIGL73XD0BBMPXt8SGCLrwEmqgnanO yq3+eCb8pk8JfjvZkSwnxDfI0ApVxbClx4YkRqLPzrvx7HxS5OAQnVNF+UM26/Gulq672Y WgA9ndWBJPHA77x8xxgBkJJTKrEn2dk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645428; x=1707181428; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ME8XRCvN/v2Itlb7KzpHyRZDGDqzMok/X5JUlaNFKjM=; b=gx6W2Yk7XxUKIxbtRO/Apx7OhpEQTJ0eQl6sXp0DGths2dlKfgaZ1P4r 0D2q2vibkUfC6QgEdK5GRqChWGVIMrOzisFzB9Zf8wxbjcID73o89BksI Ksz0ca6bY8e7LUixQLwHvXt0ggbUdx4c6OqVRG4RVs6ovr/1XSy3wYUfN zx16ZDXa1ABvplS9N76bKv7VbSQbAKOrQZSc32Wa7acroDOhOcnZufNsC g6zjwEf6PxeGynkPREXyfhGKqfVWEANJrNgCyulAPx4UavrwVeADpR1QV hVYJbz6AaJgT9rIXsUO9vIc9NrjnWKLc55Z0SlV8z72qCoOZkBmi2iMQ3 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442677" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442677" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:48 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006679" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006679" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:47 -0800 Subject: [PATCH 14/18] dax/hmem: Drop unnecessary dax_hmem_remove() From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:46 -0800 Message-ID: <167564542679.847146.17174404738816053065.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 52AE51C0010 X-Rspamd-Server: rspam01 X-Stat-Signature: zsyhotekjs1cjohrh5pwso9zdkwirpyu X-HE-Tag: 1675645428-130711 X-HE-Meta: U2FsdGVkX1+8+OdFegyN3153owIuMc/UATABVr2tJuUye0yhwbQ+EWWi7ZxkgyIDhyGGjB2JmdNCfNtIhKoYEne+UqTGpXiWTb7TUV+GFC9ZmU+tc4dlRZ07jyPDBjQBkfFo1sfSrFPbsnEVy94O8T1lI/6aVLxVRU0iSL7gQBRgDa0K1L6aVkbAgs9CZqvPjjBJc3DEOpU8uNBzOwjkN6lGTXrHdQXnJ/UtGiG3MADL8uJUaRxLREM6dEV63cply6Py49BRstRIcQKZ5021Qz79Mk5IZnP4XQ1konxRZrlHUI5kYzVhFjGKGG6kAQd4vRHcsSR0YApd14qoWR5OHzxsOO1yQ3cwZ1a8U1hAnpfSWs78XHsA6eKbEmLuMgIS4Yv5Cywh6sfKb+0ZzOy7/03p7xtQPK3WoKajzvXkZ6TN7buXyZuqrg734pIo8XZpPziH+5fapihh9S2Ua6mtuoa65RNkvx9h3I3cqTgrm92uN/BHCkHZ+/XGdWi0qrAGRpThtjN1OLIvd787r8piKhosFSkZVrpIX6jXZY57Z1OaqKORTRWOBkwB4911XTtR5VMo9k6ziLWeRROvHRY8jLk6Qt/7kzdmhJDW/69vfHHm2rBYH9mXXFfG6wRJcQzcMnsGDdHSwSjvROr68IlG9+XEAW0P2st+D+ZY9lOmfzNjefbQ5p0E1hnatZ5/ulViOTh4aZ89B/7Jqju2J2CV3RsQop7AQgwsAx1l5vEecxgeiQ6hb6/4BlVtcEmF7/NuWOIGpESntMdnACOnn5LGFtptG8JgZwbsvkNDy0dpsmh+MxfRdoo4SnOg1SeAuhmAB6ozikO0i422xEWxm0YZxEVptlE1R9FipGbZUzRypn5Me2S6mR7K04WbEe401algVEP5/oMwO7f85zvxiRsF99AXfJJr9G3nNxNs926IzsAXVrN2wb9YPCz+SO6nYrqe+L6idPue5msKs5IMIMM 3BmT8Emg ZGMxmGy2oRuoyWQNHYj2BWc5bp0MgpXjXVQxn0iE0fb8pUIOv9BNeR8pBueSfzJMMSYQaCo9acf2m2wvYCnn1pU8UYbwEktZlhN3EcbOFqom/80DYxDNYiV4EQDs7yMX4A/1MjpWadJOF2j++CMySDokXxhFPqTEBhczliu7uR88PAz+MVBQPkfwFWm4Pndqp5GqlXbD5Lj2+LCWpHg1n+Z0mDHj55FNOXR8M9a1xwcj7wS0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Empty driver remove callbacks can just be elided. Signed-off-by: Dan Williams Reviewed-by: Gregory Price Reviewed-by: Jonathan Cameron --- drivers/dax/hmem/hmem.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 1bf040dbc834..c7351e0dc8ff 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -44,15 +44,8 @@ static int dax_hmem_probe(struct platform_device *pdev) return 0; } -static int dax_hmem_remove(struct platform_device *pdev) -{ - /* devm handles teardown */ - return 0; -} - static struct platform_driver dax_hmem_driver = { .probe = dax_hmem_probe, - .remove = dax_hmem_remove, .driver = { .name = "hmem", }, From patchwork Mon Feb 6 01:03:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59433C636CD for ; Mon, 6 Feb 2023 01:03:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE6836B0083; Sun, 5 Feb 2023 20:03:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E96296B0085; Sun, 5 Feb 2023 20:03:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D84D16B0087; Sun, 5 Feb 2023 20:03:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C6A006B0083 for ; Sun, 5 Feb 2023 20:03:56 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A78EEAAB0D for ; Mon, 6 Feb 2023 01:03:56 +0000 (UTC) X-FDA: 80435070072.01.0D22DA3 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 84FC01C0012 for ; Mon, 6 Feb 2023 01:03:54 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="av4WjYS/"; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645434; a=rsa-sha256; cv=none; b=uE9sufAesefDNZFDNqT9OakEPWmwJAtxB+GRDsF+nnawR2F2iMWPKm1z8eKKxNiCv0HE8p 6CAuyx3xujZPseV48rqfvGS7pftfV3ackMAgWI7PhqYGmruceHOYOESqfahkfukoOCUs0F QZlvzriFB7DCMZ+cYLYvStQlBfIv7bI= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="av4WjYS/"; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645434; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KxV56fPJ4XnWvUyoBXg9UQfQNZA5DT3iJgCZKSpkNFg=; b=dLmq3W8Yy62AXLh1IXjK6CeLI3xJ3o8IwjZkO40cVJ5oVejeYpo9CDLe2/PebQfI7Q0Vj7 XyWzfez3hBWgPj1HgX1ZRBIzBI0VTXn4m/9YZ9vpt9H0/crpPdPXBEB3eoBpmXilPz44S5 GjAzinx2iUaNCcFfU4Kac2gxdUgJSCc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645434; x=1707181434; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UGjDlEaM3SmHqxTQJem08iZ8rEDpl61Vdevr59NhbRE=; b=av4WjYS/GIFWEccOeYzYOEsQ0c5yprUOqTWRziN3M+HEke2oc95HsbpP rurWoUn1GLC7IenNpl30oNu5tbai5dJRi1+ZUtMJSGJbMTMpdEDCmaSXU q7iJ/H//ICzTYR2MUa2JwY308x1cw6MbTqIuldYvbZTWe5Vwi/VScIlyZ 64IL8CuOWd6g0qYaGHWfjuwPRoyKBgkqm7iHf3PVvm9YlGGrlTZiMG9M9 H5S+htDW2oEDcdSLB7S37L+0J+gCj7svwN6sTjgaaYnR128/SpcOyuU1m hC5Li1DWuRhg1I3RdYPK1t7cTit+9l8nurCdcSvtQ4M6SceXs4cDial9K w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442687" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442687" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:54 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006720" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006720" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:53 -0800 Subject: [PATCH 15/18] dax/hmem: Convey the dax range via memregion_info() From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:53 -0800 Message-ID: <167564543303.847146.11045895213318648441.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 84FC01C0012 X-Rspamd-Server: rspam01 X-Stat-Signature: rcww4hb89y4ce38aztzrbmjz7k13uxnm X-HE-Tag: 1675645434-571586 X-HE-Meta: U2FsdGVkX193o2nBrzSS2JCKa5Oa9YrnE9rhzOEywvavnvBmi3k8qA9Mj15+X4TELKG7PzViN1M9srE9oScptnGzDph3zXQ6iUqd1lvaTJUqkh5u+dX4QA1ugtOC7qNaFFhJdLPXc4XwnEsOuLUezZV2UaXZC8KJM6/9Yye7U8IyP0/EZ382LL27SOTSc8AspwuBPSoh+sLyK5WDUFUi0XO/fKs/1DBry11N0HLL5L4cqPD97pQRU0EiPFAN9yRp+DuS3zQeMTgwkA6lhqAe3IRsSV42l1OrPuLWVLZckY2fjV3ceFkXtlWeJpuTyEUN7gAllwMklbHC2GjzF8t3xeLra5HxDaksxpcioFMI7kOixzQ1gx7PUo2ISyc5NOjuMm3BOuX0XQbi5ISo7Ktx9h3FX8jYh2boRMPWcsA95cr4VrZm4bvL41Yg1FDa25GIKBcxg3HrHYJR4qGY/lz88TaToS6nHc070itEGxgOA/nrdzf23VlgLUJJZoDibLaSomLJ0ZG3MYS45psbM1WtCCnFOn1ixQjiDH22vhb+F5tWfLuL1oE9KmY6merQAR5CWt7k0TqZyRkBkfHqISKV4QRU1YzL4JhcTMYARJJYEI/cJUBAvWeCMw2xoGaSiY8gC5Q+u98S3+JYR54Zv6kozNVtncKr16EnF+fU4LQr2nuTZPLrPa6NM2nhGFd0H4wetxeYDyWDzTd7dgd8yDp8zqgFzF9EiGLUys2Cnr76E+PYR4jKaavaWtEEfUnN614/0GDBzLx2cn7ZnSOi13o0XICUVLT5AxypYE/kty2aOhD+kvMBasJFzVf685j9s8oAjDu9IZO/Pko6Aba4opDHWFPUwVu+zgwSflKJGOTo2B1YGf9apXYTGPgPc8GbJTRS5iLsZf4c2Jj6N/Y0OUC2kiWUvlzxDl0ZDiKiCSnOrKB/UBISVExu85ZkceM/X5BZD7R1CwSHRfEOfbiqn4m 7QcxsGqt yAINfvSvThXUmEoYkjfAeB+G1/OgFbbrJ2XCXs6LYYlMYul47nOmr/piDWwqKhKAfdaoriKzN/gCfnAv++5bAVMdpteGQuw9lDhvVXhABfIuJxu67o4VViM7Oe7yFRpRJ6UTR8lRutwIpiABASzFyYK649fzJQ2qi3iRK8veufgMCkqEvUT8ZNYRx1vthQg2Z+7oyGvndlK5Zb8+X7SMyqWONhO9pUjnjxgIjzBzkQP4jHGw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for hmem platform devices to be unregistered, stop using platform_device_add_resources() to convey the address range. The platform_device_add_resources() API causes an existing "Soft Reserved" iomem resource to be re-parented under an inserted platform device resource. When that platform device is deleted it removes the platform device resource and all children. Instead, it is sufficient to convey just the address range and let request_mem_region() insert resources to indicate the devices active in the range. This allows the "Soft Reserved" resource to be re-enumerated upon the next probe event. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- drivers/dax/hmem/device.c | 37 ++++++++++++++----------------------- drivers/dax/hmem/hmem.c | 14 +++----------- include/linux/memregion.h | 2 ++ 3 files changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index 20749c7fab81..b1b339bccfe5 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -15,15 +15,8 @@ static struct resource hmem_active = { .flags = IORESOURCE_MEM, }; -void hmem_register_device(int target_nid, struct resource *r) +void hmem_register_device(int target_nid, struct resource *res) { - /* define a clean / non-busy resource for the platform device */ - struct resource res = { - .start = r->start, - .end = r->end, - .flags = IORESOURCE_MEM, - .desc = IORES_DESC_SOFT_RESERVED, - }; struct platform_device *pdev; struct memregion_info info; int rc, id; @@ -31,55 +24,53 @@ void hmem_register_device(int target_nid, struct resource *r) if (nohmem) return; - rc = region_intersects(res.start, resource_size(&res), IORESOURCE_MEM, - IORES_DESC_SOFT_RESERVED); + rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, + IORES_DESC_SOFT_RESERVED); if (rc != REGION_INTERSECTS) return; id = memregion_alloc(GFP_KERNEL); if (id < 0) { - pr_err("memregion allocation failure for %pr\n", &res); + pr_err("memregion allocation failure for %pr\n", res); return; } pdev = platform_device_alloc("hmem", id); if (!pdev) { - pr_err("hmem device allocation failure for %pr\n", &res); + pr_err("hmem device allocation failure for %pr\n", res); goto out_pdev; } - if (!__request_region(&hmem_active, res.start, resource_size(&res), + if (!__request_region(&hmem_active, res->start, resource_size(res), dev_name(&pdev->dev), 0)) { - dev_dbg(&pdev->dev, "hmem range %pr already active\n", &res); + dev_dbg(&pdev->dev, "hmem range %pr already active\n", res); goto out_active; } pdev->dev.numa_node = numa_map_to_online_node(target_nid); info = (struct memregion_info) { .target_node = target_nid, + .range = { + .start = res->start, + .end = res->end, + }, }; rc = platform_device_add_data(pdev, &info, sizeof(info)); if (rc < 0) { - pr_err("hmem memregion_info allocation failure for %pr\n", &res); - goto out_resource; - } - - rc = platform_device_add_resources(pdev, &res, 1); - if (rc < 0) { - pr_err("hmem resource allocation failure for %pr\n", &res); + pr_err("hmem memregion_info allocation failure for %pr\n", res); goto out_resource; } rc = platform_device_add(pdev); if (rc < 0) { - dev_err(&pdev->dev, "device add failed for %pr\n", &res); + dev_err(&pdev->dev, "device add failed for %pr\n", res); goto out_resource; } return; out_resource: - __release_region(&hmem_active, res.start, resource_size(&res)); + __release_region(&hmem_active, res->start, resource_size(res)); out_active: platform_device_put(pdev); out_pdev: diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index c7351e0dc8ff..5025a8c9850b 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -15,25 +15,17 @@ static int dax_hmem_probe(struct platform_device *pdev) struct memregion_info *mri; struct dev_dax_data data; struct dev_dax *dev_dax; - struct resource *res; - struct range range; - - res = platform_get_resource(pdev, IORESOURCE_MEM, 0); - if (!res) - return -ENOMEM; mri = dev->platform_data; - range.start = res->start; - range.end = res->end; - dax_region = alloc_dax_region(dev, pdev->id, &range, mri->target_node, - PMD_SIZE, 0); + dax_region = alloc_dax_region(dev, pdev->id, &mri->range, + mri->target_node, PMD_SIZE, 0); if (!dax_region) return -ENOMEM; data = (struct dev_dax_data) { .dax_region = dax_region, .id = -1, - .size = region_idle ? 0 : resource_size(res), + .size = region_idle ? 0 : range_len(&mri->range), }; dev_dax = devm_create_dev_dax(&data); if (IS_ERR(dev_dax)) diff --git a/include/linux/memregion.h b/include/linux/memregion.h index bf83363807ac..c01321467789 100644 --- a/include/linux/memregion.h +++ b/include/linux/memregion.h @@ -3,10 +3,12 @@ #define _MEMREGION_H_ #include #include +#include #include struct memregion_info { int target_node; + struct range range; }; #ifdef CONFIG_MEMREGION From patchwork Mon Feb 6 01:03:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8979BC636D4 for ; Mon, 6 Feb 2023 01:04:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B4016B0081; Sun, 5 Feb 2023 20:04:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 261136B0085; Sun, 5 Feb 2023 20:04:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1023D6B0087; Sun, 5 Feb 2023 20:04:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 012BF6B0081 for ; Sun, 5 Feb 2023 20:04:02 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BC6AA803F1 for ; Mon, 6 Feb 2023 01:04:02 +0000 (UTC) X-FDA: 80435070324.17.676E6B4 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 8413C1C000F for ; Mon, 6 Feb 2023 01:04:00 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LIqmGTc6; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645440; a=rsa-sha256; cv=none; b=yUi7eLT/wwyKjUWiZDfncUfz+GQftDqx1ZbDDoUFwc4mJ4AD1N5jlr+8gyZopp0g5+GUF2 TwVIVhquX61TCHfTkjNx5CzxXUHExiQckMzUfsn8VQEI2i0PDb3aZiwyC449eqBbIRYLjP fgi4eYoimMlSKqcX7/ksP3xUD5Y3sVU= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LIqmGTc6; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645440; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IThRJ8AAz29y2tyZz3TJNf+mt/0mN2lPr0PQdXvH1JE=; b=haGseEVpAc0PXZrS6kN3T4hBnMFvhY7Zk1TGIQh/fW9sk+g7obox7KE9QXjvvC0LkyqvSs IGmJeJSGJ1NrjNESUlqoCysuyQPGVxR90m2aKAMd3kFRvVdEvzkHrJxsVwUkkAr8tZ2b0/ QqdUghy/axccXV8yAICee+Ix6GODwxw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645440; x=1707181440; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=udm2GNyy9Pfhnt9VZa/fhxu0/423zpDc7iWLFcwUZ5U=; b=LIqmGTc6B/dR6Ei3RnkaMQXOvqRkdICSn5dXYt3Pfu2QZqu/OFa1fX2K vgtweA5YADEuqqgL2TbP/PMDuiPFkHk5+YP/qD1kjsVOHkY7y07J2XoSX nER0LCGnqIgejM4WlFkNr5+S28nqFfE1WWo3GEQaEV68NoOSACoWzpDcw VduUNRY6VJE1TBKuhuih5bpM/Wg87/0zhopHB0e3ejTKHxy5Zy7Mc9B+b 9+8WXLagwqFplSxUqokwwPMZnhb1Tvh5qdC63kEzSjh03nNf9n8owSYlL 90D08XQd6UPkTVicwG2bc8ZgNto0IYtukD+8Yo/cQmLvhodVMP5k1SCBz A==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442697" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442697" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:00 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006734" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006734" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:03:59 -0800 Subject: [PATCH 16/18] dax/hmem: Move hmem device registration to dax_hmem.ko From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:03:59 -0800 Message-ID: <167564543923.847146.9030380223622044744.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 8413C1C000F X-Rspamd-Server: rspam01 X-Stat-Signature: ijx8nx8s3xtx9br98np94w1ze6m7g3n8 X-HE-Tag: 1675645440-129595 X-HE-Meta: U2FsdGVkX1/TUKPY6RflvWsdNsPce6HA7z7mCHe5k2IzRM4ydmKp8Tdsnzer9QXhd7ZMf6R1kw7wSxUVPfFyUT3J5i7upFealkvZbCUCJebX7B3YyE3IeYf0qeVDD0+2dx4cFJYFxTHBec6E/dRjVmbFEe2kw0N7zWqVK+YNkHMmpUeXAuGKd5LGJSGksiJVZ9LFl42GA41Z27TbnRg14zdRc+7Bz2On4fj2Q+Un7GcqkbM/2UoYuM2p/NhL7DIjOOqHg0tfmlBaYCHngE91SzJrHKoZsbSUBegp/v4xM7SPG9r3HTnOb6UTurpZp8ZMRMiFeQdH4VlhLu/j4vKTeHbtnPpZOxuh+2gNrfplUF1b9+trPxihXRfFCOosZuiIaHO7aNDy9FBq8rbJUrW2h3aKvXl5srBBEMDNQCJwAlMu8OQK8wsz3lWd0+S/F9gYiIDXhQ8TAQEFa9HWeU5+U8GvUbtdciVH2GQmpSIMecfbNuNbzXoGgu12nBAtwy2UhTQXcbnb8yhbnBnp+7t4sXARjyonk+sLNuzC/ZGyousqDorBPwAw/n1o0F0m+IJyUd+vH+IHz74EWapZPDkFBu67X+KoZvo3p9ZLOgQzVUNcjGrC2+J5T1PigNWvJyeAGcqkTKX8UqdlUHfcgtspvUAbbUsa3k8NkkL3G9z0vWs6/r4taDUa1cnWgzvynw2DmkXtYJYPDASZjzKWVbPasDG1BKuT8Uv6tAP7J33LhCgu8db48nhKT/tDMyUBY+3hOFK7maLaJDx0OY59CAXOuiYvngp2FcN0rafp3Gm5C/rym8T5yLT+Xx8tek4ztSbNmIgtKd7hXg6GXxJpgqN+ijvkJsMfuFYAnveHYwG3t6z6loGy2OVl6kPTANrOnvgDo/mZIvnyODznCnBlRSFA+0jT4EQuRiV2ZztP1wm+NhyGMsycUIGBUs4IjeST+euMxNo0rMUDh/b/jWj830k rI4gDkPY f2E2+bUJkbkK3SuxvAOapjHzOKFO2WBi6PpNn9XADiZoox+s4Q5YtJG9HC3eHGail3tmgGppXkBGflCiIKiQ4zLjIl4MXE9FnKnpolh1b33/uj8Mm+XcRbDCb4QAOLJj7dO21AryylFCJFOqd36/KZHqBupO8YSKSHYDU0sdhILOpcifevL1CXty9ol+BNM/KRITqHBabOP3O9eI2tKODjx5oir0ouYCi/eGQdpCUYI+z5iA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for the CXL region driver to take over the responsibility of registering device-dax instances for CXL regions, move the registration of "hmem" devices to dax_hmem.ko. Previously the builtin component of this enabling (drivers/dax/hmem/device.o) would register platform devices for each address range and trigger the dax_hmem.ko module to load and attach device-dax instances to those devices. Now, the ranges are collected from the HMAT and EFI memory map walking, but the device creation is deferred. A new "hmem_platform" device is created which triggers dax_hmem.ko to load and register the platform devices. Signed-off-by: Dan Williams --- drivers/acpi/numa/hmat.c | 2 - drivers/dax/Kconfig | 2 - drivers/dax/hmem/device.c | 91 +++++++++++++++++++-------------------- drivers/dax/hmem/hmem.c | 105 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/dax.h | 7 ++- 5 files changed, 155 insertions(+), 52 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index ff24282301ab..bba268ecd802 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -718,7 +718,7 @@ static void hmat_register_target_devices(struct memory_target *target) for (res = target->memregions.child; res; res = res->sibling) { int target_nid = pxm_to_node(target->memory_pxm); - hmem_register_device(target_nid, res); + hmem_register_resource(target_nid, res); } } diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index 5fdf269a822e..d13c889c2a64 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -46,7 +46,7 @@ config DEV_DAX_HMEM Say M if unsure. config DEV_DAX_HMEM_DEVICES - depends on DEV_DAX_HMEM && DAX=y + depends on DEV_DAX_HMEM && DAX def_bool y config DEV_DAX_KMEM diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index b1b339bccfe5..f9e1a76a04a9 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -8,6 +8,8 @@ static bool nohmem; module_param_named(disable, nohmem, bool, 0444); +static bool platform_initialized; +static DEFINE_MUTEX(hmem_resource_lock); static struct resource hmem_active = { .name = "HMEM devices", .start = 0, @@ -15,71 +17,66 @@ static struct resource hmem_active = { .flags = IORESOURCE_MEM, }; -void hmem_register_device(int target_nid, struct resource *res) +int walk_hmem_resources(struct device *host, walk_hmem_fn fn) +{ + struct resource *res; + int rc = 0; + + mutex_lock(&hmem_resource_lock); + for (res = hmem_active.child; res; res = res->sibling) { + rc = fn(host, (int) res->desc, res); + if (rc) + break; + } + mutex_unlock(&hmem_resource_lock); + return rc; +} +EXPORT_SYMBOL_GPL(walk_hmem_resources); + +static void __hmem_register_resource(int target_nid, struct resource *res) { struct platform_device *pdev; - struct memregion_info info; - int rc, id; + struct resource *new; + int rc; - if (nohmem) + new = __request_region(&hmem_active, res->start, resource_size(res), "", + 0); + if (!new) { + pr_debug("hmem range %pr already active\n", res); return; + } - rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, - IORES_DESC_SOFT_RESERVED); - if (rc != REGION_INTERSECTS) - return; + new->desc = target_nid; - id = memregion_alloc(GFP_KERNEL); - if (id < 0) { - pr_err("memregion allocation failure for %pr\n", res); + if (platform_initialized) return; - } - pdev = platform_device_alloc("hmem", id); + pdev = platform_device_alloc("hmem_platform", 0); if (!pdev) { - pr_err("hmem device allocation failure for %pr\n", res); - goto out_pdev; - } - - if (!__request_region(&hmem_active, res->start, resource_size(res), - dev_name(&pdev->dev), 0)) { - dev_dbg(&pdev->dev, "hmem range %pr already active\n", res); - goto out_active; - } - - pdev->dev.numa_node = numa_map_to_online_node(target_nid); - info = (struct memregion_info) { - .target_node = target_nid, - .range = { - .start = res->start, - .end = res->end, - }, - }; - rc = platform_device_add_data(pdev, &info, sizeof(info)); - if (rc < 0) { - pr_err("hmem memregion_info allocation failure for %pr\n", res); - goto out_resource; + pr_err_once("failed to register device-dax hmem_platform device\n"); + return; } rc = platform_device_add(pdev); - if (rc < 0) { - dev_err(&pdev->dev, "device add failed for %pr\n", res); - goto out_resource; - } + if (rc) + platform_device_put(pdev); + else + platform_initialized = true; +} - return; +void hmem_register_resource(int target_nid, struct resource *res) +{ + if (nohmem) + return; -out_resource: - __release_region(&hmem_active, res->start, resource_size(res)); -out_active: - platform_device_put(pdev); -out_pdev: - memregion_free(id); + mutex_lock(&hmem_resource_lock); + __hmem_register_resource(target_nid, res); + mutex_unlock(&hmem_resource_lock); } static __init int hmem_register_one(struct resource *res, void *data) { - hmem_register_device(phys_to_target_node(res->start), res); + hmem_register_resource(phys_to_target_node(res->start), res); return 0; } diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 5025a8c9850b..e7bdff3132fa 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -3,6 +3,7 @@ #include #include #include +#include #include "../bus.h" static bool region_idle; @@ -43,8 +44,110 @@ static struct platform_driver dax_hmem_driver = { }, }; -module_platform_driver(dax_hmem_driver); +static void release_memregion(void *data) +{ + memregion_free((long) data); +} + +static void release_hmem(void *pdev) +{ + platform_device_unregister(pdev); +} + +static int hmem_register_device(struct device *host, int target_nid, + const struct resource *res) +{ + struct platform_device *pdev; + struct memregion_info info; + long id; + int rc; + + rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, + IORES_DESC_SOFT_RESERVED); + if (rc != REGION_INTERSECTS) + return 0; + + id = memregion_alloc(GFP_KERNEL); + if (id < 0) { + dev_err(host, "memregion allocation failure for %pr\n", res); + return -ENOMEM; + } + rc = devm_add_action_or_reset(host, release_memregion, (void *) id); + if (rc) + return rc; + + pdev = platform_device_alloc("hmem", id); + if (!pdev) { + dev_err(host, "device allocation failure for %pr\n", res); + return -ENOMEM; + } + + pdev->dev.numa_node = numa_map_to_online_node(target_nid); + info = (struct memregion_info) { + .target_node = target_nid, + .range = { + .start = res->start, + .end = res->end, + }, + }; + rc = platform_device_add_data(pdev, &info, sizeof(info)); + if (rc < 0) { + dev_err(host, "memregion_info allocation failure for %pr\n", + res); + goto out_put; + } + + rc = platform_device_add(pdev); + if (rc < 0) { + dev_err(host, "%s add failed for %pr\n", dev_name(&pdev->dev), + res); + goto out_put; + } + + return devm_add_action_or_reset(host, release_hmem, pdev); + +out_put: + platform_device_put(pdev); + return rc; +} + +static int dax_hmem_platform_probe(struct platform_device *pdev) +{ + return walk_hmem_resources(&pdev->dev, hmem_register_device); +} + +static struct platform_driver dax_hmem_platform_driver = { + .probe = dax_hmem_platform_probe, + .driver = { + .name = "hmem_platform", + }, +}; + +static __init int dax_hmem_init(void) +{ + int rc; + + rc = platform_driver_register(&dax_hmem_platform_driver); + if (rc) + return rc; + + rc = platform_driver_register(&dax_hmem_driver); + if (rc) + platform_driver_unregister(&dax_hmem_platform_driver); + + return rc; +} + +static __exit void dax_hmem_exit(void) +{ + platform_driver_unregister(&dax_hmem_driver); + platform_driver_unregister(&dax_hmem_platform_driver); +} + +module_init(dax_hmem_init); +module_exit(dax_hmem_exit); MODULE_ALIAS("platform:hmem*"); +MODULE_ALIAS("platform:hmem_platform*"); MODULE_LICENSE("GPL v2"); MODULE_AUTHOR("Intel Corporation"); diff --git a/include/linux/dax.h b/include/linux/dax.h index 2b5ecb591059..bf6258472e49 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -262,11 +262,14 @@ static inline bool dax_mapping(struct address_space *mapping) } #ifdef CONFIG_DEV_DAX_HMEM_DEVICES -void hmem_register_device(int target_nid, struct resource *r); +void hmem_register_resource(int target_nid, struct resource *r); #else -static inline void hmem_register_device(int target_nid, struct resource *r) +static inline void hmem_register_resource(int target_nid, struct resource *r) { } #endif +typedef int (*walk_hmem_fn)(struct device *dev, int target_nid, + const struct resource *res); +int walk_hmem_resources(struct device *dev, walk_hmem_fn fn); #endif From patchwork Mon Feb 6 01:04:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80BEBC636CD for ; Mon, 6 Feb 2023 01:04:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C6B46B007B; Sun, 5 Feb 2023 20:04:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 176FE6B007E; Sun, 5 Feb 2023 20:04:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03E696B0087; Sun, 5 Feb 2023 20:04:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EA0E76B007B for ; Sun, 5 Feb 2023 20:04:08 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C6575AAB19 for ; Mon, 6 Feb 2023 01:04:08 +0000 (UTC) X-FDA: 80435070576.01.67D0A68 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 86BE21C0015 for ; Mon, 6 Feb 2023 01:04:06 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GK1gs7gj; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645446; a=rsa-sha256; cv=none; b=5UQUN47RVOnxkAbId0aSMsnwLo20sfksRHWRkGpfNEDjOonnY695vHRySq/XXc+4uYH9aR lmX+vT/dA2IebRLTlKvuAzfwywbfbNS9tX+Sak3jGX/fogKIegBJwvMChy8fn2tXtTBpQI jJhOFmqGcQiYYRX3N/0NssvxMcGcbN4= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GK1gs7gj; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645446; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2XqSYO6eKicwcLlCsbrsut86hBWoVutw8KDqDZ9u4tI=; b=0ZybmgsIizjVbdWPDFLwvB/d6DEm6UreyEK+Vq33R/9E+vq3fT+PhwKj3qBp0wG9pcPcD2 wElFbd2KWfdXzAxP2XccDbA9eKhYwNZOF52KMCBzv6TnPxOyR7vzU/DqfxzGFFN/z8Roib UKPWZeGW0y7JPowEgL0n2kJzyOyEuHM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645446; x=1707181446; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hrvnto86ChTGsi4fHJ4Lqo9DG2au7l7fsMPus7NStq0=; b=GK1gs7gjNinPobEQFOc3k1Nm8HHd2JJM9E/4bB4HdriIxfEMKG5XNFt4 Jd5gEST7yqgEKCU9g485QmHjb1eas/X6CWWiF5J3NKnaaQCQ6Y0Usl0tH 5Jxi0PPg6cdsoLbfgzzLaWATrN8ECoQ8FpYmIqw26RJjxotTkde0r6sEu JDf59wAnKEN9RTg3MmJZaKr7n5oYDm+Khup3md3v11wwwjcp6n7sO4dTd h2AGgKld6nS6NzNNLBqKLsmE5jpj7Qjwg0tdkFQvfRrdmFsKiG9UjglUx prwWkl53zzNWPOhyljA7QUOC3JSDkk1lx1AhNnLbaYpAHvjzovuJo/qNG Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442703" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442703" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:06 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006766" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006766" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:05 -0800 Subject: [PATCH 17/18] dax: Assign RAM regions to memory-hotplug by default From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Michal Hocko , David Hildenbrand , Dave Hansen , linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:04:05 -0800 Message-ID: <167564544513.847146.4645646177864365755.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 86BE21C0015 X-Rspamd-Server: rspam01 X-Stat-Signature: 45u41n7otj1oad75bx3j841k615w71i8 X-HE-Tag: 1675645446-859271 X-HE-Meta: U2FsdGVkX1/crWvcmhmD6SFMdV6SlwDZOzOpJsRVzAaaveb59n/75yxUNpgscuvpMhcd/pgU9t9GdEi5KnvIllcmLs0DEsGxrfGuqL5uty7yrwrf7sm0zjkySnwsAmCkVtxp5GHK1FgCXjUZwl/8i1AdI2O+GwK+hbBc5qHSaSbt3Zb/usOXmcBMISBft4bNDXDj+zyupxZbnCJMuaNm4qnhFtg1W6N7KbRvEf4qhkwC1UePBXJCrWsCBdrWxxemx2cctrCgJNsdgO90z2WsaP1S2JLjXllizjn6bW2ZTOKl59RQ02NcyXwTS+yz1bGMftojJh3kBuFHEZjAsTE0x3EJ+bvNB5Xju3vVZAkIbNgBEAH7wob4IdaT8yYndXZxW29ITgKYjYOkisbpwBKVAVEjU1WA1aC6IuEPIZ7Lex2oBoUEXD19ojo+L6XtwIF7MjNoqZzufJEyOmXBVTs2AekDpLsm0fjRcYOzhF7jzWcLxeo912ebGA91BwpaU+VINCl/p3y4x8Z9X4SXCMzU00WjonDc98Xq3bS9RzGij98ieX9nPbQ8HIA+ZO8ambufOsv7ybbc0eeD62Kh8eSJfPOk7GhTAQ5tcv4lLuFcB9wqaXFkabKbRlzIwVxjc7ght3FVSHS3vphXg+tYMWfFLq0C3ak15L2Wo2Bk8fJgjWZJTnX3koisSfVYHchDx+HfUgGRacnyFxl9k5ILbzJ5eXnl4FhJRJAObRGPFfpX2v/Lr7/gs/kpW32f04Xgi+sgXPKe0cULTdICF09/8f7U/W/ceY4sVIYqNsESZ6h4Pz0IyWpGaitUTwacSetctAuFOZJm6NN6wUrWps0XpKd5loGe1dWkwo80QTOe3sCemMP6aGoOIiRWyvNro43IoY3ZJSjjAWRUyD0b4JYALXHOJbVoF8RAHd8B9xx/151NzIdymCadt7WeSRiXtKV29AEWG/BFMRrU6Eg2X2sCGOZ q4E+5N65 bx7RwVn04cY/1ytPZaEdcD/7IvpLNNd5PlAwBvgiMrckluw39no1fgRcpF2+Jr1+5cMuoc0uxM4T6c/rulXtacWfJgiAb+GLRZtsG+XdN7gv1f46mRyFC95Yt1YKbxCYkvnGRB7eALMtQb9QHRr/dh0bZHsLiGq9Kr6QVwe4XC1tpz0jBI/cYToJ9JN75UKDAmCoJoWjKrrp07ezSfXJa1IiKzgbHDfSErp8KAmUobHH5TpZct6CnINW3+2NesHO7WWb6jfFYbry6GRg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The default mode for device-dax instances is backwards for RAM-regions as evidenced by the fact that it tends to catch end users by surprise. "Where is my memory?". Recall that platforms are increasingly shipping with performance-differentiated memory pools beyond typical DRAM and NUMA effects. This includes HBM (high-bandwidth-memory) and CXL (dynamic interleave, varied media types, and future fabric attached possibilities). For this reason the EFI_MEMORY_SP (EFI Special Purpose Memory => Linux 'Soft Reserved') attribute is expected to be applied to all memory-pools that are not the general purpose pool. This designation gives an Operating System a chance to defer usage of a memory pool until later in the boot process where its performance properties can be interrogated and administrator policy can be applied. 'Soft Reserved' memory can be anything from too limited and precious to be part of the general purpose pool (HBM), too slow to host hot kernel data structures (some PMEM media), or anything in between. However, in the absence of an explicit policy, the memory should at least be made usable by default. The current device-dax default hides all non-general-purpose memory behind a device interface. The expectation is that the distribution of users that want the memory online by default vs device-dedicated-access by default follows the Pareto principle. A small number of enlightened users may want to do userspace memory management through a device, but general users just want the kernel to make the memory available with an option to get more advanced later. Arrange for all device-dax instances not backed by PMEM to default to attaching to the dax_kmem driver. From there the baseline memory hotplug policy (CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE / memhp_default_state=) gates whether the memory comes online or stays offline. Where, if it stays offline, it can be reliably converted back to device-mode where it can be partitioned, or fronted by a userspace allocator. So, if someone wants device-dax instances for their 'Soft Reserved' memory: 1/ Build a kernel with CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=n or boot with memhp_default_state=offline, or roll the dice and hope that the kernel has not pinned a page in that memory before step 2. 2/ Write a udev rule to convert the target dax device(s) from 'system-ram' mode to 'devdax' mode: daxctl reconfigure-device $dax -m devdax -f Cc: Michal Hocko Cc: David Hildenbrand Cc: Dave Hansen Signed-off-by: Dan Williams Reviewed-by: Gregory Price --- drivers/dax/Kconfig | 2 +- drivers/dax/bus.c | 53 ++++++++++++++++++++--------------------------- drivers/dax/bus.h | 12 +++++++++-- drivers/dax/device.c | 3 +-- drivers/dax/hmem/hmem.c | 12 ++++++++++- drivers/dax/kmem.c | 1 + 6 files changed, 46 insertions(+), 37 deletions(-) diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index d13c889c2a64..1163eb62e5f6 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -50,7 +50,7 @@ config DEV_DAX_HMEM_DEVICES def_bool y config DEV_DAX_KMEM - tristate "KMEM DAX: volatile-use of persistent memory" + tristate "KMEM DAX: map dax-devices as System-RAM" default DEV_DAX depends on DEV_DAX depends on MEMORY_HOTPLUG # for add_memory() and friends diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 1dad813ee4a6..012d576004e9 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -56,6 +56,25 @@ static int dax_match_id(struct dax_device_driver *dax_drv, struct device *dev) return match; } +static int dax_match_type(struct dax_device_driver *dax_drv, struct device *dev) +{ + enum dax_driver_type type = DAXDRV_DEVICE_TYPE; + struct dev_dax *dev_dax = to_dev_dax(dev); + + if (dev_dax->region->res.flags & IORESOURCE_DAX_KMEM) + type = DAXDRV_KMEM_TYPE; + + if (dax_drv->type == type) + return 1; + + /* default to device mode if dax_kmem is disabled */ + if (dax_drv->type == DAXDRV_DEVICE_TYPE && + !IS_ENABLED(CONFIG_DEV_DAX_KMEM)) + return 1; + + return 0; +} + enum id_action { ID_REMOVE, ID_ADD, @@ -216,14 +235,9 @@ static int dax_bus_match(struct device *dev, struct device_driver *drv) { struct dax_device_driver *dax_drv = to_dax_drv(drv); - /* - * All but the 'device-dax' driver, which has 'match_always' - * set, requires an exact id match. - */ - if (dax_drv->match_always) + if (dax_match_id(dax_drv, dev)) return 1; - - return dax_match_id(dax_drv, dev); + return dax_match_type(dax_drv, dev); } /* @@ -1413,13 +1427,10 @@ struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data) } EXPORT_SYMBOL_GPL(devm_create_dev_dax); -static int match_always_count; - int __dax_driver_register(struct dax_device_driver *dax_drv, struct module *module, const char *mod_name) { struct device_driver *drv = &dax_drv->drv; - int rc = 0; /* * dax_bus_probe() calls dax_drv->probe() unconditionally. @@ -1434,26 +1445,7 @@ int __dax_driver_register(struct dax_device_driver *dax_drv, drv->mod_name = mod_name; drv->bus = &dax_bus_type; - /* there can only be one default driver */ - mutex_lock(&dax_bus_lock); - match_always_count += dax_drv->match_always; - if (match_always_count > 1) { - match_always_count--; - WARN_ON(1); - rc = -EINVAL; - } - mutex_unlock(&dax_bus_lock); - if (rc) - return rc; - - rc = driver_register(drv); - if (rc && dax_drv->match_always) { - mutex_lock(&dax_bus_lock); - match_always_count -= dax_drv->match_always; - mutex_unlock(&dax_bus_lock); - } - - return rc; + return driver_register(drv); } EXPORT_SYMBOL_GPL(__dax_driver_register); @@ -1463,7 +1455,6 @@ void dax_driver_unregister(struct dax_device_driver *dax_drv) struct dax_id *dax_id, *_id; mutex_lock(&dax_bus_lock); - match_always_count -= dax_drv->match_always; list_for_each_entry_safe(dax_id, _id, &dax_drv->ids, list) { list_del(&dax_id->list); kfree(dax_id); diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index fbb940293d6d..8cd79ab34292 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -11,7 +11,10 @@ struct dax_device; struct dax_region; void dax_region_put(struct dax_region *dax_region); -#define IORESOURCE_DAX_STATIC (1UL << 0) +/* dax bus specific ioresource flags */ +#define IORESOURCE_DAX_STATIC BIT(0) +#define IORESOURCE_DAX_KMEM BIT(1) + struct dax_region *alloc_dax_region(struct device *parent, int region_id, struct range *range, int target_node, unsigned int align, unsigned long flags); @@ -25,10 +28,15 @@ struct dev_dax_data { struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data); +enum dax_driver_type { + DAXDRV_KMEM_TYPE, + DAXDRV_DEVICE_TYPE, +}; + struct dax_device_driver { struct device_driver drv; struct list_head ids; - int match_always; + enum dax_driver_type type; int (*probe)(struct dev_dax *dev); void (*remove)(struct dev_dax *dev); }; diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 5494d745ced5..ecdff79e31f2 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -475,8 +475,7 @@ EXPORT_SYMBOL_GPL(dev_dax_probe); static struct dax_device_driver device_dax_driver = { .probe = dev_dax_probe, - /* all probe actions are unwound by devm, so .remove isn't necessary */ - .match_always = 1, + .type = DAXDRV_DEVICE_TYPE, }; static int __init dax_init(void) diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index e7bdff3132fa..5ec08f9f8a57 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -11,15 +11,25 @@ module_param_named(region_idle, region_idle, bool, 0644); static int dax_hmem_probe(struct platform_device *pdev) { + unsigned long flags = IORESOURCE_DAX_KMEM; struct device *dev = &pdev->dev; struct dax_region *dax_region; struct memregion_info *mri; struct dev_dax_data data; struct dev_dax *dev_dax; + /* + * @region_idle == true indicates that an administrative agent + * wants to manipulate the range partitioning before the devices + * are created, so do not send them to the dax_kmem driver by + * default. + */ + if (region_idle) + flags = 0; + mri = dev->platform_data; dax_region = alloc_dax_region(dev, pdev->id, &mri->range, - mri->target_node, PMD_SIZE, 0); + mri->target_node, PMD_SIZE, flags); if (!dax_region) return -ENOMEM; diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index 4852a2dbdb27..918d01d3fbaa 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -239,6 +239,7 @@ static void dev_dax_kmem_remove(struct dev_dax *dev_dax) static struct dax_device_driver device_dax_kmem_driver = { .probe = dev_dax_kmem_probe, .remove = dev_dax_kmem_remove, + .type = DAXDRV_KMEM_TYPE, }; static int __init dax_kmem_init(void) From patchwork Mon Feb 6 01:04:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AB27C636CD for ; Mon, 6 Feb 2023 01:04:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D86366B0087; Sun, 5 Feb 2023 20:04:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D36B86B0088; Sun, 5 Feb 2023 20:04:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BFE956B0089; Sun, 5 Feb 2023 20:04:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AF7866B0087 for ; Sun, 5 Feb 2023 20:04:18 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 83C64807B5 for ; Mon, 6 Feb 2023 01:04:18 +0000 (UTC) X-FDA: 80435070996.14.27D5FD7 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf14.hostedemail.com (Postfix) with ESMTP id 503C8100017 for ; Mon, 6 Feb 2023 01:04:15 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ZvQqh8VQ; spf=pass (imf14.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645455; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WoGSHwnT/W3+RJx6SbbGQTVic1beRgCu7pka0JR1tBA=; b=IHJ/dFv3l6p/Np+4mvsY5WkOwOfPjxRga/Fm2rFhhe+brm7zABa0fB9JcGI9qdVc3dn0kb E4Lh4zhn/IEheKQItZ1lza6XaZLaZXE0K9UuQnz1CeS0Q3CW1iAbqat84Zvsxmwmt51sSq vKvAGKRh+3TiK49Uv8NV9LhwXCHkSk4= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ZvQqh8VQ; spf=pass (imf14.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645455; a=rsa-sha256; cv=none; b=8Uvh4OE5VgpNUFtwWpCNwHELO+5IgTTGHzFU1m3qOlJrnX2Re8CiUh9GWZe/EiMNEoTF5d DIUMfkCGlM1doXrBNSDZcECbHl0oJtftKKdMd9Cw6F+j1ojkdUsVOas3OLy8gg2ljy8dNG PXhonCCqOi8adCknxk+EfrYDM6vbWtc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645455; x=1707181455; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wPmMkCp6DCx1PzgRrsiFSVETspBvehWCurZIdpJuvvk=; b=ZvQqh8VQIBF9nB8ExPcINE5S9etEydLDb4qrzZz6vKhHA/vIMMU79fwN qX0qoxIwpauUf+RuNxjBClgIse2gE7ty+k8jjy4XSUr0HSvjpSH+Mw6Jm VVww5e6aB1V9fvEELz28X07FBxwyFcU92iE5xPLNZB3i8GOHbUcola4AI xxGIGH2Cx0LemrUKwou9mVbI6AmyWrzyL89y1dUgHEowJNrsx1CIMSWfA AdwG/2im5Nl0n2qc47hlX7MDx03ZkaIbrCk329iec2H7kVUiPWTV2JZec OaxNGZItl53dF+DO8CPRm2lclYCyDqEfA8G7QD72GovblWakQblGKRR8T A==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="329123120" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="329123120" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:12 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="698692901" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="698692901" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:11 -0800 Subject: [PATCH 18/18] cxl/dax: Create dax devices for CXL RAM regions From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:04:11 -0800 Message-ID: <167564545116.847146.4741351262959589920.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Stat-Signature: 1fhqpp4q1g9of3soxxwgscizt1qmjqu6 X-Rspam-User: X-Rspamd-Queue-Id: 503C8100017 X-Rspamd-Server: rspam06 X-HE-Tag: 1675645455-12597 X-HE-Meta: U2FsdGVkX19GNXstQd7REny9a4vrsBTGhTThN9cu4tA6pw8u7VKYerxPzE7WFAmcGQfh3BRCiEOeMk6dC0VWowKhbR9ya9DIlHlON67jU2dz0kRXqGbLlcRW0gf5ZeRBzEZm2CI42J88KJLB5juafkKZhgwTpAVX5qflOjSK4yZG/qFgwXE5aUvZR6pS9cryQb3KEbYZasgbgsE+htOxZhkXgDv9HtrT/ThDf3gaJiF/QP8VBrsK0UQESDdEKEJDP7ldjHBTw6/PuqG9pCLb9t2/Q6Ak78nkT4qkEUlGgtxzdimClxXrGMtscx06IlO5/XJjthDJdMTb0U+EnTlB9ObPoBougciDNulxWPcDmYwkpTM8HVz6ouq1mZa77wjVDGPHAIEP9XbXOeqj90IDS2fb7Ose8cKXpdA3rjqIx69OuDO6U2AjXq7DGfeD5YpC6GDTCH93gMZF6oBK0kNuxiYcbieME5+kaCpRIPQu1J6RzKeyMoJ6vqlHMaxhNOwnte62pszBU9Q38GJcSqA5ZYwlwk5ywnjTwB7VO9tqCfmv445KF3ZyIYGVfGv6nkcewrFRje3JP4glrxTWok6SnZ7WNKzD2qdbdMR6diyJd3RVjRz+UrSkfupYOz6D57BjxZRHmCk8KwqlAJDB0sn3Q97UiZ5e/vp48/G/MyUi3vZv8PkPC+uq7JtwBYu1LmQfDMAIdSTCDGnXfhojuq+b2ENE1oWyh3j2mCidGx1EeIfy7QzXU1s7DEIH6vMBdPqHfFpzVLTRZSkxJC7GMOjdPomqaHs4OcRRktCa/INCPlzuQNG1B5wD9QLySmITFOKIicQGkPpVuaXWd2zam42I7iQA2inzx/sJZKwMi1cveP41+8u5tEpFy/TvjgH3pyJ7znLEWQ+jsrfLKJsWzOx/vYRuaSKkaXO9b3+jhD+E91nik8gTIt1a6O8L3XVaxZZWMK37xcTwrIeWM/nu81/ kHV0zjW+ 2mG47TzfB9eRWRQ99k1NpH5tGMM4jxvh3+oYUfwpa+RKCsKvtR90fnFn3yA+qB1Vq5+9Qf1UzHEh0Wvg5z5gxTqY38dYvGzMxrTzRQ4dqCSkU7o2UQnNKgv6vhSLi1vJrYbUe9f5FmU4MSwVKQxZzmy5IdQJmv/zoVKkWHmLD7Ok6iUckb39wFrShJ0R6laAFBqbt1b1PYSRMG0nqO29n13wPQC1hBr3flTXhCk/x/CpZgiGlGJ67XaXo5Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: While platform firmware takes some responsibility for mapping the RAM capacity of CXL devices present at boot, the OS is responsible for mapping the remainder and hot-added devices. Platform firmware is also responsible for identifying the platform general purpose memory pool, typically DDR attached DRAM, and arranging for the remainder to be 'Soft Reserved'. That reservation allows the CXL subsystem to route the memory to core-mm via memory-hotplug (dax_kmem), or leave it for dedicated access (device-dax). The new 'struct cxl_dax_region' object allows for a CXL memory resource (region) to be published, but also allow for udev and module policy to act on that event. It also prevents cxl_core.ko from having a module loading dependency on any drivers/dax/ modules. Signed-off-by: Dan Williams --- MAINTAINERS | 1 drivers/cxl/acpi.c | 3 + drivers/cxl/core/core.h | 3 + drivers/cxl/core/port.c | 2 + drivers/cxl/core/region.c | 108 ++++++++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxl.h | 12 +++++ drivers/dax/Kconfig | 13 +++++ drivers/dax/Makefile | 2 + drivers/dax/cxl.c | 53 ++++++++++++++++++++++ drivers/dax/hmem/hmem.c | 14 ++++++ 10 files changed, 208 insertions(+), 3 deletions(-) create mode 100644 drivers/dax/cxl.c diff --git a/MAINTAINERS b/MAINTAINERS index 7f86d02cb427..73a9f3401e0e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6035,6 +6035,7 @@ M: Dan Williams M: Vishal Verma M: Dave Jiang L: nvdimm@lists.linux.dev +L: linux-cxl@vger.kernel.org S: Supported F: drivers/dax/ diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index ad0849af42d7..8ebb9a74790d 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -731,7 +731,8 @@ static void __exit cxl_acpi_exit(void) cxl_bus_drain(); } -module_init(cxl_acpi_init); +/* load before dax_hmem sees 'Soft Reserved' CXL ranges */ +subsys_initcall(cxl_acpi_init); module_exit(cxl_acpi_exit); MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS(CXL); diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 479f01da6d35..cde475e13216 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -15,12 +15,14 @@ extern struct device_attribute dev_attr_create_ram_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; +extern const struct device_type cxl_dax_region_type; extern const struct device_type cxl_region_type; void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled); #define CXL_REGION_ATTR(x) (&dev_attr_##x.attr) #define CXL_REGION_TYPE(x) (&cxl_region_type) #define SET_CXL_REGION_ATTR(x) (&dev_attr_##x.attr), #define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type) +#define CXL_DAX_REGION_TYPE(x) (&cxl_dax_region_type) int cxl_region_init(void); void cxl_region_exit(void); #else @@ -38,6 +40,7 @@ static inline void cxl_region_exit(void) #define CXL_REGION_TYPE(x) NULL #define SET_CXL_REGION_ATTR(x) #define CXL_PMEM_REGION_TYPE(x) NULL +#define CXL_DAX_REGION_TYPE(x) NULL #endif struct cxl_send_command; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 8d0895cbae93..0faeb1ffc212 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -46,6 +46,8 @@ static int cxl_device_id(struct device *dev) return CXL_DEVICE_NVDIMM; if (dev->type == CXL_PMEM_REGION_TYPE()) return CXL_DEVICE_PMEM_REGION; + if (dev->type == CXL_DAX_REGION_TYPE()) + return CXL_DEVICE_DAX_REGION; if (is_cxl_port(dev)) { if (is_cxl_root(to_cxl_port(dev))) return CXL_DEVICE_ROOT; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 6fe8c70790df..86fa54015a99 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2261,6 +2261,75 @@ static struct cxl_pmem_region *cxl_pmem_region_alloc(struct cxl_region *cxlr) return cxlr_pmem; } +static void cxl_dax_region_release(struct device *dev) +{ + struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); + + kfree(cxlr_dax); +} + +static const struct attribute_group *cxl_dax_region_attribute_groups[] = { + &cxl_base_attribute_group, + NULL, +}; + +const struct device_type cxl_dax_region_type = { + .name = "cxl_dax_region", + .release = cxl_dax_region_release, + .groups = cxl_dax_region_attribute_groups, +}; + +static bool is_cxl_dax_region(struct device *dev) +{ + return dev->type == &cxl_dax_region_type; +} + +struct cxl_dax_region *to_cxl_dax_region(struct device *dev) +{ + if (dev_WARN_ONCE(dev, !is_cxl_dax_region(dev), + "not a cxl_dax_region device\n")) + return NULL; + return container_of(dev, struct cxl_dax_region, dev); +} +EXPORT_SYMBOL_NS_GPL(to_cxl_dax_region, CXL); + +static struct lock_class_key cxl_dax_region_key; + +static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr) +{ + struct cxl_region_params *p = &cxlr->params; + struct cxl_dax_region *cxlr_dax; + struct device *dev; + + down_read(&cxl_region_rwsem); + if (p->state != CXL_CONFIG_COMMIT) { + cxlr_dax = ERR_PTR(-ENXIO); + goto out; + } + + cxlr_dax = kzalloc(sizeof(*cxlr_dax), GFP_KERNEL); + if (!cxlr_dax) { + cxlr_dax = ERR_PTR(-ENOMEM); + goto out; + } + + cxlr_dax->hpa_range.start = p->res->start; + cxlr_dax->hpa_range.end = p->res->end; + + dev = &cxlr_dax->dev; + cxlr_dax->cxlr = cxlr; + device_initialize(dev); + lockdep_set_class(&dev->mutex, &cxl_dax_region_key); + device_set_pm_not_required(dev); + dev->parent = &cxlr->dev; + dev->bus = &cxl_bus_type; + dev->type = &cxl_dax_region_type; +out: + up_read(&cxl_region_rwsem); + + return cxlr_dax; +} + static void cxlr_pmem_unregister(void *_cxlr_pmem) { struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem; @@ -2345,6 +2414,42 @@ static int devm_cxl_add_pmem_region(struct cxl_region *cxlr) return rc; } +static void cxlr_dax_unregister(void *_cxlr_dax) +{ + struct cxl_dax_region *cxlr_dax = _cxlr_dax; + + device_unregister(&cxlr_dax->dev); +} + +static int devm_cxl_add_dax_region(struct cxl_region *cxlr) +{ + struct cxl_dax_region *cxlr_dax; + struct device *dev; + int rc; + + cxlr_dax = cxl_dax_region_alloc(cxlr); + if (IS_ERR(cxlr_dax)) + return PTR_ERR(cxlr_dax); + + dev = &cxlr_dax->dev; + rc = dev_set_name(dev, "dax_region%d", cxlr->id); + if (rc) + goto err; + + rc = device_add(dev); + if (rc) + goto err; + + dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent), + dev_name(dev)); + + return devm_add_action_or_reset(&cxlr->dev, cxlr_dax_unregister, + cxlr_dax); +err: + put_device(dev); + return rc; +} + static int match_decoder_by_range(struct device *dev, void *data) { struct range *r1, *r2 = data; @@ -2606,8 +2711,7 @@ static int cxl_region_probe(struct device *dev) p->res->start, p->res->end, cxlr, is_system_ram) > 0) return 0; - dev_dbg(dev, "TODO: hookup devdax\n"); - return 0; + return devm_cxl_add_dax_region(cxlr); default: dev_dbg(&cxlr->dev, "unsupported region mode: %d\n", cxlr->mode); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 4c6ee6c96f23..336b65d333a2 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -505,6 +505,12 @@ struct cxl_pmem_region { struct cxl_pmem_region_mapping mapping[]; }; +struct cxl_dax_region { + struct device dev; + struct cxl_region *cxlr; + struct range hpa_range; +}; + /** * struct cxl_port - logical collection of upstream port devices and * downstream port devices to construct a CXL memory @@ -699,6 +705,7 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); #define CXL_DEVICE_MEMORY_EXPANDER 5 #define CXL_DEVICE_REGION 6 #define CXL_DEVICE_PMEM_REGION 7 +#define CXL_DEVICE_DAX_REGION 8 #define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") #define CXL_MODALIAS_FMT "cxl:t%d" @@ -715,6 +722,7 @@ struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct device *dev); #ifdef CONFIG_CXL_REGION bool is_cxl_pmem_region(struct device *dev); struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); +struct cxl_dax_region *to_cxl_dax_region(struct device *dev); void cxl_add_to_region(struct cxl_endpoint_decoder *cxled); #else static inline bool is_cxl_pmem_region(struct device *dev) @@ -725,6 +733,10 @@ static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) { return NULL; } +static inline struct cxl_dax_region *to_cxl_dax_region(struct device *dev) +{ + return NULL; +} static inline void cxl_add_to_region(struct cxl_endpoint_decoder *cxled) { } diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index 1163eb62e5f6..bd06e16c7ac8 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -45,6 +45,19 @@ config DEV_DAX_HMEM Say M if unsure. +config DEV_DAX_CXL + tristate "CXL DAX: direct access to CXL RAM regions" + depends on CXL_REGION && DEV_DAX + default CXL_REGION && DEV_DAX + help + CXL RAM regions are either mapped by platform-firmware + and published in the initial system-memory map as "System RAM", mapped + by platform-firmware as "Soft Reserved", or dynamically provisioned + after boot by the CXL driver. In the latter two cases a device-dax + instance is created to access that unmapped-by-default address range. + Per usual it can remain as dedicated access via a device interface, or + converted to "System RAM" via the dax_kmem facility. + config DEV_DAX_HMEM_DEVICES depends on DEV_DAX_HMEM && DAX def_bool y diff --git a/drivers/dax/Makefile b/drivers/dax/Makefile index 90a56ca3b345..5ed5c39857c8 100644 --- a/drivers/dax/Makefile +++ b/drivers/dax/Makefile @@ -3,10 +3,12 @@ obj-$(CONFIG_DAX) += dax.o obj-$(CONFIG_DEV_DAX) += device_dax.o obj-$(CONFIG_DEV_DAX_KMEM) += kmem.o obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o +obj-$(CONFIG_DEV_DAX_CXL) += dax_cxl.o dax-y := super.o dax-y += bus.o device_dax-y := device.o dax_pmem-y := pmem.o +dax_cxl-y := cxl.o obj-y += hmem/ diff --git a/drivers/dax/cxl.c b/drivers/dax/cxl.c new file mode 100644 index 000000000000..ccdf8de85bd5 --- /dev/null +++ b/drivers/dax/cxl.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Intel Corporation. All rights reserved. */ +#include +#include + +#include "../cxl/cxl.h" +#include "bus.h" + +static int cxl_dax_region_probe(struct device *dev) +{ + struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); + int nid = phys_to_target_node(cxlr_dax->hpa_range.start); + struct cxl_region *cxlr = cxlr_dax->cxlr; + struct dax_region *dax_region; + struct dev_dax_data data; + struct dev_dax *dev_dax; + + if (nid == NUMA_NO_NODE) + nid = memory_add_physaddr_to_nid(cxlr_dax->hpa_range.start); + + dax_region = alloc_dax_region(dev, cxlr->id, &cxlr_dax->hpa_range, nid, + PMD_SIZE, IORESOURCE_DAX_KMEM); + if (!dax_region) + return -ENOMEM; + + data = (struct dev_dax_data) { + .dax_region = dax_region, + .id = -1, + .size = range_len(&cxlr_dax->hpa_range), + }; + dev_dax = devm_create_dev_dax(&data); + if (IS_ERR(dev_dax)) + return PTR_ERR(dev_dax); + + /* child dev_dax instances now own the lifetime of the dax_region */ + dax_region_put(dax_region); + return 0; +} + +static struct cxl_driver cxl_dax_region_driver = { + .name = "cxl_dax_region", + .probe = cxl_dax_region_probe, + .id = CXL_DEVICE_DAX_REGION, + .drv = { + .suppress_bind_attrs = true, + }, +}; + +module_cxl_driver(cxl_dax_region_driver); +MODULE_ALIAS_CXL(CXL_DEVICE_DAX_REGION); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Intel Corporation"); +MODULE_IMPORT_NS(CXL); diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 5ec08f9f8a57..e5fe8b39fb94 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -72,6 +72,13 @@ static int hmem_register_device(struct device *host, int target_nid, long id; int rc; + if (IS_ENABLED(CONFIG_CXL_REGION) && + region_intersects(res->start, resource_size(res), IORESOURCE_MEM, + IORES_DESC_CXL) != REGION_DISJOINT) { + dev_dbg(host, "deferring range to CXL: %pr\n", res); + return 0; + } + rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, IORES_DESC_SOFT_RESERVED); if (rc != REGION_INTERSECTS) @@ -157,6 +164,13 @@ static __exit void dax_hmem_exit(void) module_init(dax_hmem_init); module_exit(dax_hmem_exit); +/* Allow for CXL to define its own dax regions */ +#if IS_ENABLED(CONFIG_CXL_REGION) +#if IS_MODULE(CONFIG_CXL_ACPI) +MODULE_SOFTDEP("pre: cxl_acpi"); +#endif +#endif + MODULE_ALIAS("platform:hmem*"); MODULE_ALIAS("platform:hmem_platform*"); MODULE_LICENSE("GPL v2");