From patchwork Mon Feb 6 01:02:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BF3CC636CD for ; Mon, 6 Feb 2023 01:03:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2D8A96B007B; Sun, 5 Feb 2023 20:03:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 289196B007D; Sun, 5 Feb 2023 20:03:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 151996B007E; Sun, 5 Feb 2023 20:03:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 066376B007B for ; Sun, 5 Feb 2023 20:03:00 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D3D5E1406A1 for ; Mon, 6 Feb 2023 01:02:59 +0000 (UTC) X-FDA: 80435067678.06.64B4D0D Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf24.hostedemail.com (Postfix) with ESMTP id A7C1B180016 for ; Mon, 6 Feb 2023 01:02:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dL0oz9mO; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645378; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wEOiU6QpO3ymx0xLdLYjdGFirFQKBSl8KjCe46IoVio=; b=aKT26rCUYMHbOWl+aEr+p5E02FrC3FOrDarxe3rZOye5+u7F1C0xCjuwuKWWDT7clA+fTQ lHG6JKaDNB3QiGEbmJh3AxxNI1EhJ1OklDlt9dMxVLNR1DW6HpBqfZCjYk3YBi7W/T9ju2 pd8XmdlutRBGdBDjldo0ijrjS8mY0Hs= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dL0oz9mO; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645378; a=rsa-sha256; cv=none; b=dudUvqZwpNabL20wANTnS9PT6xFMfBqusbQMIXWlIehGJPDkTvQS+sy8Bt/mv853UFsQC7 4EweFM+NphXBaH7Dh8v9i8uhwPm9yV/qKWFuPS7OWhrJqcmr9hWkQtyNm1rrb2uaE2/o8M U0OHfHvxTnu4o/f2pSvKQ9qXijqBWiI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645377; x=1707181377; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2NFt5YIQN4Pv/UJUgAAJbE/vCys4h+IOB5VcXpizhjY=; b=dL0oz9mOolNG2TuoGvxn4nrTmr9p9Q4GzxTt7u9Vs/Ne4ZpTmlBMojLr JZnB6iS3wbppxMKVcJIIENUdQ9hLZiG5wJ25P2qpjTXvfdhSbv9TJLzsh AeHfczri/tiaxuaxrFzo4Uzqx5tWOScpxW/TK7Mgb8mR22LZ24Jh23Hzv oXr6KQpGa4AyWmadad0Htu63v3XB5lX5btTEDAR/CYsJTc/dD5YzxJkK8 Spln6wehYlxKj1At75mrrNR9BFkVxfhMq8Ounrmj+83HPFuWTScHMq5pe DMoXKZzCWo+u/rmGI+qU9CiCJ8Ayrc5ifVjRnxsIGSanhOe3A7pvTj2V8 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="331243808" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="331243808" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:57 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="643855748" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="643855748" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:02:57 -0800 Subject: [PATCH 05/18] cxl/region: Add volatile region creation support From: Dan Williams To: linux-cxl@vger.kernel.org Cc: dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:02:56 -0800 Message-ID: <167564537678.847146.4066579806086171091.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A7C1B180016 X-Stat-Signature: agxx5ks9katwwz5jh99oxu5jd1eg1jo3 X-Rspam-User: X-HE-Tag: 1675645377-867982 X-HE-Meta: U2FsdGVkX18nyy6ICw0qRNfN3XcJgfgClnVkyQmiluTfbs49cjW6GcPgRvIhrFRIBF6XQLI0nextwoZHa+R7XxW+3rIE0IGxaLLQc0cmqtiWjwL+BdL09Bp96mSqh2BG+6PK8qJAvSJwOihJlZBEHfSJFitPWil5UGbyQopWoEUehBscrhUvWK2JNwMXLKJjZOSgcahW0zbAFYxy6R6KMnG8RNE/D8v/IrBsLWtnFDGmTixksK8U2qjdiawl9XoySL8B7JyYui8aCxC+ysokZrm7nJ/2PHbitjt2RGbVQ8Gm22E0JaR29ykCES06zyA4+4KcmAW37bbQW1tmka83WvUtAnHADZxhYu5KTZNpO8XEVyocGBsRCoVkodbgv9U/Fuast0WBKZiibu3EpGabVSxLo0IrkzN2lH04Pqe+O2vKbbJOErT/09xU62pjEaFEDPEPqweif4ovMbH0jbhLqmI3usAD2f/665xQpFAFvoRoIFT7qs214bgiTn2UCotffx3gQtkXf0WkN8Tone0vsESpBaRUFyJTySBT/XLmFVctuQS5PvalzjbTViNgIngfV2M/imy10TjAga5A6FYKF7NZfHZ1UOAwWXwn/SN1fT52DzrndY8vZszxEm/ed4aVQORh0tF5l5iIw+R+NU2MSLa+eF1BlyfTXr1Zu17xFbgfOJ7+jPPkYGYtKHscSuIYrHxzJWtSgynURvAOiJCY8VWPtCTCpIFDibp3/JQUzHDtI9FFaeKo94eZsaQPk2d9D8K9tP26SF09+R9sdTk5mx24S7wch+vSdk5mnkzoo0Sex91uY47Rf7RXNQyQMmpAblgjvwjQuteiF4GnBNKEnJn6dh6pQqNoBEiCQ/0gSQr0BZ9TdzLI9Qvxieq9yms0Z46NYLjnkHo+BVavzLJA8BgHa1tUwVeX2dSn5lXFZa9/eAXqxX4191T+qfyDvljuWC8UAo2QbE8DIMU7YJL bYdqn4f8 lhQCditBjf9yrKNAS4UnnoIUTu8v9sjKOIU70CBTXV14AFV2q2Mtzo1gJ/c6i8XjFJUEPDGAiCQdywFpO3sKH9lW8Mcx4B/xM673sgYd4wyRaSIWM0N11f4q9Tb1fVRs6pLC86ijBif4CN1mf2OxrYC45FGTRQmlpo0w8NA1nTyadiQoHs2fpAQ0TYd0bDnu6XE0Wg6lbGCujPWtWSbrtUR7lz8KMScih1TlpHskPqxj5D7MSH5Di178Ldh64TWKQ/w9CyoLT9Zc1+/eRXjH3/s/2PA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Expand the region creation infrastructure to enable 'ram' (volatile-memory) regions. The internals of create_pmem_region_store() and create_pmem_region_show() are factored out into helpers __create_region() and __create_region_show() for the 'ram' case to reuse. Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- Documentation/ABI/testing/sysfs-bus-cxl | 22 +++++----- drivers/cxl/core/core.h | 1 drivers/cxl/core/port.c | 14 ++++++ drivers/cxl/core/region.c | 71 +++++++++++++++++++++++++------ 4 files changed, 82 insertions(+), 26 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 4c4e1cbb1169..3acf2f17a73f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -285,20 +285,20 @@ Description: interleave_granularity). -What: /sys/bus/cxl/devices/decoderX.Y/create_pmem_region -Date: May, 2022 -KernelVersion: v6.0 +What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region +Date: May, 2022, January, 2023 +KernelVersion: v6.0 (pmem), v6.3 (ram) Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process - of defining a new persistent memory region (interleave-set) - within the decode range bounded by root decoder 'decoderX.Y'. - The value written must match the current value returned from - reading this attribute. An atomic compare exchange operation is - done on write to assign the requested id to a region and - allocate the region-id for the next creation attempt. EBUSY is - returned if the region name written does not match the current - cached value. + of defining a new persistent, or volatile memory region + (interleave-set) within the decode range bounded by root decoder + 'decoderX.Y'. The value written must match the current value + returned from reading this attribute. An atomic compare exchange + operation is done on write to assign the requested id to a + region and allocate the region-id for the next creation attempt. + EBUSY is returned if the region name written does not match the + current cached value. What: /sys/bus/cxl/devices/decoderX.Y/delete_region diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 8c04672dca56..5eb873da5a30 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -11,6 +11,7 @@ extern struct attribute_group cxl_base_attribute_group; #ifdef CONFIG_CXL_REGION extern struct device_attribute dev_attr_create_pmem_region; +extern struct device_attribute dev_attr_create_ram_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 8566451cb22f..47e450c3a5a9 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -294,6 +294,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, SET_CXL_REGION_ATTR(create_pmem_region) + SET_CXL_REGION_ATTR(create_ram_region) SET_CXL_REGION_ATTR(delete_region) NULL, }; @@ -305,6 +306,13 @@ static bool can_create_pmem(struct cxl_root_decoder *cxlrd) return (cxlrd->cxlsd.cxld.flags & flags) == flags; } +static bool can_create_ram(struct cxl_root_decoder *cxlrd) +{ + unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_RAM; + + return (cxlrd->cxlsd.cxld.flags & flags) == flags; +} + static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute *a, int n) { struct device *dev = kobj_to_dev(kobj); @@ -313,7 +321,11 @@ static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute * if (a == CXL_REGION_ATTR(create_pmem_region) && !can_create_pmem(cxlrd)) return 0; - if (a == CXL_REGION_ATTR(delete_region) && !can_create_pmem(cxlrd)) + if (a == CXL_REGION_ATTR(create_ram_region) && !can_create_ram(cxlrd)) + return 0; + + if (a == CXL_REGION_ATTR(delete_region) && + !(can_create_pmem(cxlrd) || can_create_ram(cxlrd))) return 0; return a->mode; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 53d6dbe4de6d..8dea49c021b8 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1685,6 +1685,15 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, struct device *dev; int rc; + switch (mode) { + case CXL_DECODER_RAM: + case CXL_DECODER_PMEM: + break; + default: + dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); + return ERR_PTR(-EINVAL); + } + cxlr = cxl_region_alloc(cxlrd, id); if (IS_ERR(cxlr)) return cxlr; @@ -1713,12 +1722,38 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(rc); } +static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf) +{ + return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +} + static ssize_t create_pmem_region_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + return __create_region_show(to_cxl_root_decoder(dev), buf); +} - return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +static ssize_t create_ram_region_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return __create_region_show(to_cxl_root_decoder(dev), buf); +} + +static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, + enum cxl_decoder_mode mode, int id) +{ + int rc; + + rc = memregion_alloc(GFP_KERNEL); + if (rc < 0) + return ERR_PTR(rc); + + if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { + memregion_free(rc); + return ERR_PTR(-EBUSY); + } + + return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); } static ssize_t create_pmem_region_store(struct device *dev, @@ -1727,29 +1762,37 @@ static ssize_t create_pmem_region_store(struct device *dev, { struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); struct cxl_region *cxlr; - int id, rc; + int rc, id; rc = sscanf(buf, "region%d\n", &id); if (rc != 1) return -EINVAL; - rc = memregion_alloc(GFP_KERNEL); - if (rc < 0) - return rc; + cxlr = __create_region(cxlrd, CXL_DECODER_PMEM, id); + if (IS_ERR(cxlr)) + return PTR_ERR(cxlr); + return len; +} +DEVICE_ATTR_RW(create_pmem_region); - if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { - memregion_free(rc); - return -EBUSY; - } +static ssize_t create_ram_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + struct cxl_region *cxlr; + int rc, id; - cxlr = devm_cxl_add_region(cxlrd, id, CXL_DECODER_PMEM, - CXL_DECODER_EXPANDER); + rc = sscanf(buf, "region%d\n", &id); + if (rc != 1) + return -EINVAL; + + cxlr = __create_region(cxlrd, CXL_DECODER_RAM, id); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); - return len; } -DEVICE_ATTR_RW(create_pmem_region); +DEVICE_ATTR_RW(create_ram_region); static ssize_t region_show(struct device *dev, struct device_attribute *attr, char *buf)