From patchwork Fri Feb 10 09:05:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9328C636CD for ; Fri, 10 Feb 2023 09:05:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 727D66B007B; Fri, 10 Feb 2023 04:05:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D6EF6B0089; Fri, 10 Feb 2023 04:05:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59EBC6B008C; Fri, 10 Feb 2023 04:05:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3796E6B007B for ; Fri, 10 Feb 2023 04:05:32 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D18941414AB for ; Fri, 10 Feb 2023 09:05:31 +0000 (UTC) X-FDA: 80450798862.09.91E51B6 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf16.hostedemail.com (Postfix) with ESMTP id 98AEA180018 for ; Fri, 10 Feb 2023 09:05:29 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=UWFGI2G9; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019930; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qIOTL5xh8Pqs43N+5OJhNWaRvw62EgV5/mOUXhULrnU=; b=X1t2OWlVntiGzHivjZ6fsqQxe5V0UER2hjyspS4ixhRUnOGKZuYSPgdIxtj+V2uulurcp2 ZmsYGg8w6THibNyp7BNiKbKOmZPxozQvM6vrVb9omK+rRN+mWb2NhUP7hjBXyrKGs23P0R 4pB5E75bvnNTjyMS8HfmACu11Yq5PGs= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=UWFGI2G9; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019930; a=rsa-sha256; cv=none; b=mu5VvsT/vDjTZggTQVsKuJC9x2KvKqYuOEHgdRMO10QSlK7C6kSvQX0gfvkETcNUtCMxHv fKf7EdcDeCH2H9k3PozLiRIbP3dV8am3u5uRk98hdvlB0OlLLkPn5m1/5R0R21fKJBUMy6 yiMy3Hwf8z1jSbthg3ka085ZJPMykUs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019929; x=1707555929; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0DbfECaRxiOQUgGeyWMl1ZfzEksrLbe++d0Gmjbv5io=; b=UWFGI2G9E/Z82lZhr7Rdg0UrxhlNk2JoBVMi1vRsNnewdRK/BOxv5zHW DuN1IqbcSl39gOrxmZpgHr4ByB8wKeYnMU3L9fM/oduvdUTpkV4CCE6pR HYo2r7+RFx9jER1h2ZE+TzFRBre9bAkPCVp9vtjby8t4z9pKIogAEECWm Qyyy1OB2CRYfLbppTO5/u19bojnddC6XuyNrioqpRly9p4ricwGhaUQu1 zfS4B4/cFxVpJ4rC/d+F7M64NTQ5JHfBbfsQoMbcthXAUVx98RHewVcE+ ql8SmJvE6LW8OKvgSxH7mEWBD54rIdkFgA77+/FQOPqccdbvU9nx5hOa9 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="314018691" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="314018691" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:28 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="736669726" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="736669726" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:28 -0800 Subject: [PATCH v2 01/20] cxl/memdev: Fix endpoint port removal From: Dan Williams To: linux-cxl@vger.kernel.org Cc: vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:27 -0800 Message-ID: <167601992789.1924368.8083994227892600608.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Stat-Signature: kw85xagaqftjuyytc6xeeqahr6irwpb4 X-Rspam-User: X-Rspamd-Queue-Id: 98AEA180018 X-Rspamd-Server: rspam06 X-HE-Tag: 1676019929-818589 X-HE-Meta: U2FsdGVkX1+c9UfYjzHTNVHMGN1qrET8U3CWK/lhZGyobqMsIrM30fS6aTp7BE9hddtkwUw4Q7C9mKzoCohN9eDspBifbeISlI+y333C0UAvhC8VVCtrUWguPAfHVDlL1gG56SKCAoIszCXICdAASG/JUgzkChNVVwqEE2fwI9ysa6zDiIPYqjIyrcGPxvSayESjyY3lbsLwkRLmtGGTnpF/Itk9FxzhcRkYQNHi8dOXNgF/Zs6/kl28d7g/iljfO9lkD+yk/Qc/38myFb/7PrDSRCy3w9CU6htBh2NBtylvqAN19sG5B6jR94FGt+ZBVl4ky31xzTslJygclKoQtqPRHiysUbylutkwvVF+9kCqcIOHb0bpTucDN3TKuZrhGzDtQDKgJioUdUwA22xH/20/AOeJU8i0ROVbQbcsLFzrmFZYxxajXie7XRfb6ez1OyeA+dxrKINK644qx5SzTaM/4v+EK8056AWyEOeLF+tWc3jAT8I0+WlLFqtHUwTKu1lQy0563GCSDd7zNI3dKZTBTSNueiKIWT8ZwYXHO4epkqVtJPuhW+v1y86JQy43WcqtazgZYLGYRlUkHUX+wWZrj4J4fYq/g5H2Z7dbmFHTDlDGQ1sWa6CMVL0AfMtTWc/llLPkqezLrrctdW9aurmv/dc3jSABBEjUUFW3nWvxuB8j6+gG7ghkxBZyfeD/cmOv1J/ASguDIj55czNROO+UUoPVr3GWTn6OKH8L42IwjWBqDj2ESVnG3xKzFKmh7vb26gsb/JV9/0TvVanSTpXV1boBkp3yBAR82+gSdFtBorSLOt+Agc8xxm45Wwy+W7gqeyaNJRwrYl5xP/IM86Ct/w2gvGH4qaYKaO7b1aXWNRdjxcWmurgEbSUr3AhEG2wYlzKVM/J/wlC+mufdCWQy1HZ9IymAimKX4flKqGkTKUuF3fEi3cT9bJEIcTXIBMxkHrKNZpZc54cnY02 xr3Bctg1 ilNwPeBZMj73FrOwbb78/UIEIManLwN5eVnMfBjiZvRCGEBsa2FEzLS+dWwvxGAr5W5qOVWV6WHPqlIAXhHmh5rv11q+PsoZck1gqn3VQ7Ab1HhG60vkU8qC5Kq0wpG/T+EDL+t9je3G1ciVciiHt4GC5dXD25I1ZNRGAswLjmsFvsHCAYLEaq22hTi611A4egSECYQ9eYA/sr/b8jNxlpGr4544zndKxpg5tgoOFdtUg5HD0yUZ8pC8saQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Testing of ram region support [1], stimulates a long standing bug in cxl_detach_ep() where some cxl_ep_remove() cleanup is skipped due to inability to walk ports after dports have been unregistered. That results in a failure to re-register a memdev after the port is re-enabled leading to a crash like the following: cxl_port_setup_targets: cxl region4: cxl_host_bridge.0:port4 iw: 1 ig: 256 general protection fault, ... [..] RIP: 0010:cxl_region_setup_targets+0x897/0x9e0 [cxl_core] dev_name at include/linux/device.h:700 (inlined by) cxl_port_setup_targets at drivers/cxl/core/region.c:1155 (inlined by) cxl_region_setup_targets at drivers/cxl/core/region.c:1249 [..] Call Trace: attach_target+0x39a/0x760 [cxl_core] ? __mutex_unlock_slowpath+0x3a/0x290 cxl_add_to_region+0xb8/0x340 [cxl_core] ? lockdep_hardirqs_on+0x7d/0x100 discover_region+0x4b/0x80 [cxl_port] ? __pfx_discover_region+0x10/0x10 [cxl_port] device_for_each_child+0x58/0x90 cxl_port_probe+0x10e/0x130 [cxl_port] cxl_bus_probe+0x17/0x50 [cxl_core] Change the port ancestry walk to be by depth rather than by dport. This ensures that even if a port has unregistered its dports a deferred memdev cleanup will still be able to cleanup the memdev's interest in that port. The parent_port->dev.driver check is only needed for determining if the bottom up removal beat the top-down removal, but cxl_ep_remove() can always proceed. Fixes: 2703c16c75ae ("cxl/core/port: Add switch port enumeration") Link: http://lore.kernel.org/r/167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com [1] Signed-off-by: Dan Williams Reviewed-by: Vishal Verma --- drivers/cxl/core/memdev.c | 1 + drivers/cxl/core/port.c | 58 +++++++++++++++++++++++++-------------------- drivers/cxl/cxlmem.h | 2 ++ 3 files changed, 35 insertions(+), 26 deletions(-) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index a74a93310d26..3a8bc2b06047 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -246,6 +246,7 @@ static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds, if (rc < 0) goto err; cxlmd->id = rc; + cxlmd->depth = -1; dev = &cxlmd->dev; device_initialize(dev); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 410c036c09fa..317bcf4dbd9d 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1207,6 +1207,7 @@ int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint) get_device(&endpoint->dev); dev_set_drvdata(dev, endpoint); + cxlmd->depth = endpoint->depth; return devm_add_action_or_reset(dev, delete_endpoint, cxlmd); } EXPORT_SYMBOL_NS_GPL(cxl_endpoint_autoremove, CXL); @@ -1241,50 +1242,55 @@ static void reap_dports(struct cxl_port *port) } } +struct detach_ctx { + struct cxl_memdev *cxlmd; + int depth; +}; + +static int port_has_memdev(struct device *dev, const void *data) +{ + const struct detach_ctx *ctx = data; + struct cxl_port *port; + + if (!is_cxl_port(dev)) + return 0; + + port = to_cxl_port(dev); + if (port->depth != ctx->depth) + return 0; + + return !!cxl_ep_load(port, ctx->cxlmd); +} + static void cxl_detach_ep(void *data) { struct cxl_memdev *cxlmd = data; - struct device *iter; - for (iter = &cxlmd->dev; iter; iter = grandparent(iter)) { - struct device *dport_dev = grandparent(iter); + for (int i = cxlmd->depth - 1; i >= 1; i--) { struct cxl_port *port, *parent_port; + struct detach_ctx ctx = { + .cxlmd = cxlmd, + .depth = i, + }; + struct device *dev; struct cxl_ep *ep; bool died = false; - if (!dport_dev) - break; - - port = find_cxl_port(dport_dev, NULL); - if (!port) - continue; - - if (is_cxl_root(port)) { - put_device(&port->dev); + dev = bus_find_device(&cxl_bus_type, NULL, &ctx, + port_has_memdev); + if (!dev) continue; - } + port = to_cxl_port(dev); parent_port = to_cxl_port(port->dev.parent); device_lock(&parent_port->dev); - if (!parent_port->dev.driver) { - /* - * The bottom-up race to delete the port lost to a - * top-down port disable, give up here, because the - * parent_port ->remove() will have cleaned up all - * descendants. - */ - device_unlock(&parent_port->dev); - put_device(&port->dev); - continue; - } - device_lock(&port->dev); ep = cxl_ep_load(port, cxlmd); dev_dbg(&cxlmd->dev, "disconnect %s from %s\n", ep ? dev_name(ep->ep) : "", dev_name(&port->dev)); cxl_ep_remove(port, ep); if (ep && !port->dead && xa_empty(&port->endpoints) && - !is_cxl_root(parent_port)) { + !is_cxl_root(parent_port) && parent_port->dev.driver) { /* * This was the last ep attached to a dynamically * enumerated port. Block new cxl_add_ep() and garbage diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index ab138004f644..c9da3c699a21 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -38,6 +38,7 @@ * @cxl_nvb: coordinate removal of @cxl_nvd if present * @cxl_nvd: optional bridge to an nvdimm if the device supports pmem * @id: id number of this memdev instance. + * @depth: endpoint port depth */ struct cxl_memdev { struct device dev; @@ -47,6 +48,7 @@ struct cxl_memdev { struct cxl_nvdimm_bridge *cxl_nvb; struct cxl_nvdimm *cxl_nvd; int id; + int depth; }; static inline struct cxl_memdev *to_cxl_memdev(struct device *dev) From patchwork Fri Feb 10 09:05:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9249BC05027 for ; Fri, 10 Feb 2023 09:05:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1333A6B0117; Fri, 10 Feb 2023 04:05:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E37C6B0118; Fri, 10 Feb 2023 04:05:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EEDB06B0119; Fri, 10 Feb 2023 04:05:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C4E7E6B0117 for ; Fri, 10 Feb 2023 04:05:39 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 75908A13DC for ; Fri, 10 Feb 2023 09:05:39 +0000 (UTC) X-FDA: 80450799198.11.4020481 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by imf29.hostedemail.com (Postfix) with ESMTP id 4FB41120020 for ; Fri, 10 Feb 2023 09:05:36 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GOPIvwsO; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf29.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019936; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ajMtGjhx8RF0hshevNRrA730dykzDVSqBo8rswDcsN8=; b=GHut1qnbZ+dK0RTaBMdv/Xkg82RVBPC7h8xDr0gdp6qwTRN9y+ybxdVGWmHy5i9m69Nf8z 1N61RFG5wayHBtG739zQqSy04YX6ZB+0hzhGMTc7r7EWj13LhTuRfQc+97GbmZInKdplbq Mf9p/VR8zvxfrShrVAvFkE+Q/UNbYOM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GOPIvwsO; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf29.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019936; a=rsa-sha256; cv=none; b=H/aJt4znGkgcIKiDtV00qvqeFkFxXl0Bbb3gecUQ27ZqemT7plPBPmo9ixH5gAqzLHiIR8 XikbpGXcQWZT6Y4AZlJol58xUDr0+CF4oog1K86WhdZ7uDowppQ9Kc9QOmBAcBOqiQwgk4 g1F29wULh2AFzf5FXxgiQAd/VvbJHSE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019936; x=1707555936; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M9WD4tNW7gQKGRRkfXq/VM0O9cOu0NWS0bJh4WHOXA8=; b=GOPIvwsOWIyQbxQKHPG8X4Sl4OuDJEVT9nKpdrgdV6hRQG+xVb3I6zo+ 1gNRHj5To8abn73UDi7Pst+y1cUlu7w+PlpEmXVSESTjfFdJ6pRaGkD/g lzQgKh6IHylZw4P+e4YjVkpzz6FkjI4NbZI6/EVJVoZyye8ujmip4/S1R lkwu5VLmLEsVubdF8mtknZl4YE0jFo7UjAI0Ez78fMwQ58toio68EMsvw eASksuxtOsIpAPSabwTIJzqlD3zMARWr9wA4tUEyHPytYe437/dGv0AuE 9oGZz9ttoXKy/rqyhswXCAggzgOO0j9pLwD40zsCFzl+gv/idtWIEcjlf Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="331677023" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="331677023" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:34 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="698364539" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="698364539" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:33 -0800 Subject: [PATCH v2 02/20] cxl/Documentation: Update references to attributes added in v6.0 From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Dave Jiang , Gregory Price , Ira Weiny , Davidlohr Bueso , Jonathan Cameron , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:33 -0800 Message-ID: <167601993360.1924368.14122892663883462813.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4FB41120020 X-Stat-Signature: 4afuszsw11ou7n4j8j5ysxrmt5ouqm4d X-HE-Tag: 1676019936-508632 X-HE-Meta: U2FsdGVkX19eSVx6JU4cnBNthEK/PoOOqIPQRs1HvCdsmmmoaNb+pr3MKkvwiK96vk86T9zDOcNx12hj1DoAzUjtnq8yJt0VbKOdu7qHZ8ZII1Go0aGY27fFLHMPk4X41jpqpBHHmKJFqWeZFVG4IkfGWYYRlF43Xq9EkBgoymWxMgaSUSxjMT1GJEPOrG+Fl61zrD7kQ97dw8Ttqp/F9Q38yvpa8Lqt1/IvnWmcPnXSMkQ5zGtcq/ZPQVdvm68hekbKwCvMBDaQKzX64D7UZ5KusHqggCm7Jr+MWqcW8ARfVXQTe/a6fdYM+sQMKnDbkpbqScUT4c4mPDWlM79CNEi9S0uQU/6o4t0nJqjIESNtdsuyKPVLnU40lJsRk+YlQAjhr9JoY267mT74iVDBSOIkvYy0Nz/u85JT5b5zprsy6WO6VsgaOqaO58svWShQEPFt8wgqmzA/gPhUqxiamEuZdsa/4NPUDhrFXmuujV3ZThkoHhdvTzRnFHKfer+8Cjmlab+9rLvZ7twU2a9zJ1nR2+vBf4fZS6uVxqbGHe9+pXNT9XDO/Feb5M2KUCHxi74KVtnXZrIesSvKUf+Yjx86PRfPYarSXk4JX3PBUs9Vh3p7aH6fYXpbwwIdnXmeZxjhMDjJgwgqU4NmfxNaIcf5SL0JU7UMMS9TkhC3gZT1k5vzGoj+9typNrNl65Mmvt7wJlebBD0Cxi7HxELdV5GnCOoBwDFLTQWjD/cuMcMZnlMjOklp7WPgHtelhjigTCKDMawgbaJsqA/CHNF4my1+ScF+lDGqiZuLATeNIRVgn42lwjGWfrubb9vsoPjUZLKczNDjdq/4XPm/4eq/p9ruvx1cVQbbDJ9xp6EQbr5UP5okfesaBFHL3XK+gGE3Pl/XZjCPnkhnR5T6NLBEJXvfq2iCZIhpqQVf1UfFm7lurjBVcVG6x03GExdSIpyjPO62Z+NIIhGaBTwn6Qb H4lHxy3d g52Fo/MBhIlCGuj+nrgVloXw3hZ4DHGDOkxSJOjEc9VP7NJ4S5mpEVgOVOezrH+6t/KGvs3iDgcW4h7cDbe1dl0g+6lAjfoMcEvEECvmy+QVhengCPpn48STgGGLb3HF1ZgBZoigjp32uNq8Gu2pLQkpuQHVbvN9SpGRf9A/aXaQPtGWnFbi7zj/bi3YVLnEJLWE3Gg5x5t48LrpVyAtwR2cVNVTU2pDo3eVs1m0yNSvhWj069O4/uRyl2ZsUI88Tf4/nbZfEHfmsLqJhiL7f5bQ7rYtefHI8JUXSnelgk6IjBFEBf9AbKtwfoX0+diIMfcLcJFRtJKA4KCF9Ou3X1K1+p7kki2m2FsyjshB99mhgZsnP3Hu9U20OCbSdsXHA4exjAs2HJRtLOFUg2jrxp/pZLOS05jNW5N+s9HB4VLK2CQVKrkpVccNpMA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Prior to Linus deciding that the kernel that following v5.19 would be v6.0, the CXL ABI documentation already referenced v5.20. In preparation for updating these entries update the kernel version to v6.0. Reviewed-by: Vishal Verma Reviewed-by: Dave Jiang Reviewed-by: Gregory Price Reviewed-by: Ira Weiny Reviewed-by: Davidlohr Bueso Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564535494.847146.12120939572640882946.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- Documentation/ABI/testing/sysfs-bus-cxl | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 329a7e46c805..5be032313e29 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -198,7 +198,7 @@ Description: What: /sys/bus/cxl/devices/endpointX/CDAT Date: July, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) If this sysfs entry is not present no DOE mailbox was @@ -209,7 +209,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/mode Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it @@ -229,7 +229,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/dpa_resource Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) When a CXL decoder is of devtype "cxl_decoder_endpoint", @@ -240,7 +240,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/dpa_size Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it @@ -260,7 +260,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/interleave_ways Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) The number of targets across which this decoder's host @@ -275,7 +275,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/interleave_granularity Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) The number of consecutive bytes of host physical address @@ -287,7 +287,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/create_pmem_region Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process @@ -303,7 +303,7 @@ Description: What: /sys/bus/cxl/devices/decoderX.Y/delete_region Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (WO) Write a string in the form 'regionZ' to delete that region, @@ -312,7 +312,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/uuid Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write a unique identifier for the region. This field must @@ -322,7 +322,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/interleave_granularity Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Set the number of consecutive bytes each device in the @@ -333,7 +333,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/interleave_ways Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Configures the number of devices participating in the @@ -343,7 +343,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/size Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) System physical address space to be consumed by the region. @@ -360,7 +360,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/resource Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RO) A region is a contiguous partition of a CXL root decoder @@ -372,7 +372,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/target[0..N] Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write an endpoint decoder object name to 'targetX' where X @@ -391,7 +391,7 @@ Description: What: /sys/bus/cxl/devices/regionZ/commit Date: May, 2022 -KernelVersion: v5.20 +KernelVersion: v6.0 Contact: linux-cxl@vger.kernel.org Description: (RW) Write a boolean 'true' string value to this attribute to From patchwork Fri Feb 10 09:05:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BCF6C636CD for ; Fri, 10 Feb 2023 09:05:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E04116B0118; Fri, 10 Feb 2023 04:05:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB59E6B011A; Fri, 10 Feb 2023 04:05:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7C426B011B; Fri, 10 Feb 2023 04:05:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9F5DA6B0118 for ; Fri, 10 Feb 2023 04:05:45 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 54D48A1363 for ; Fri, 10 Feb 2023 09:05:45 +0000 (UTC) X-FDA: 80450799450.29.1F22449 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf15.hostedemail.com (Postfix) with ESMTP id 11C36A0016 for ; Fri, 10 Feb 2023 09:05:42 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GthjjzCw; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019943; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zMQUl4J6X7ULaIrRgxt3l1Taiz3cx+Xhc0qmS3HTx30=; b=iSNJh3GKpLGuW173gQ6Z0CJD5gqiH7KjWbwcxYP7haDq5kR1LDqY05mmOCuCudu90F1z3v oizfE8oeb6OoTyoCJcbGZmtJXzcMR1aUGFw6q48IP22h3XwdDzMrccbTD69QOhYQGGJYu0 InayJeOdGF2TCJD+HU/4TKFvNFy/9Vk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GthjjzCw; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019943; a=rsa-sha256; cv=none; b=Om6oylTqFPpxk8i2ao+8i9AKU1QrO/PTJcWcT1h4k5ePYZA62jyDw4GKQZARXwuQ2wS4IB OpIdICFluW0TEm6/V4s8spfXcet8PcE5B3YGM9x+7AQDrIiaanCe39f7dT2h3WA708G1Rt nX3J0EcnKJWKnSgS9E/YrhjXMm2f4s4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019943; x=1707555943; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3cQC8uwKOt3b1h6vNcSkmE8AzC7Sg9UhcvwbHDHzsBA=; b=GthjjzCwOi7uG3kETx0bPYCBAtGeC5W3+34do27cxP9cTAKM45YtHTht /C4OircWRoJnikFyxQRUeqv0vXUdWpQlUPBK/S5QwUdjZRBTj5j1ysPQr E9BpSSsFVWXy+C9nrmlQwkHD/xIwmJsRHmnOO+l8GYPwiJFi68kK7JA8E MW8AavKI4k54QXKUBqqCkbD3uDS/nDNvMFP++Qk+mfd4ouFTQ8aRNV/NA 8KNdwh9REjB2fxj+SBAX3wUpaWjOORpfm/xfOxbxJeqTD/+7Qu1KLSRgJ 2/cFRI5t1pYJnC3wOV9tRJSPwlv9QE+Q3oXnoDel6CEocnQ0fYj/3ollR w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416599983" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="416599983" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:40 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="669930135" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="669930135" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:39 -0800 Subject: [PATCH v2 03/20] cxl/region: Add a mode attribute for regions From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Dave Jiang , Gregory Price , Ira Weiny , Jonathan Cameron , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:39 -0800 Message-ID: <167601993930.1924368.4305018565539515665.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 11C36A0016 X-Stat-Signature: dwnb4fmes9wyxzfisa6xo89h7yxt4qir X-Rspam-User: X-HE-Tag: 1676019942-138262 X-HE-Meta: U2FsdGVkX19oWH8yyOc7eaCuSuVLH50mIR5dtM/oUj8lRo6qemXgn8E02zx0JG647MFUMRD8Nxv++O+I+GeYAPMtCXqpcta72j4+O3EZzGhzWMG5KLm+ddNkn+6cKYXvLw/q6cOh8tFVNbnBpqkwK0kBh+QsxQURq9f0mlPuQCt4k8T+U93MF4xPbk0E6EOEwPqw7UXgImrmup8LOmwrJLrYh0KDOIL5HKfNhFLgS/qM7CttwLWyb5RALfOHpNtK15ICrAntcpBMWGtmO7TcLCwWtlmmre8OCfAc0mpdzulKWDJDJ6C8GWex30ISk9cm1sHwrM5GDQl/hFkQF4b/OWKW7kbcQ9evWNiE3SbI09LDYjY+PuLb86gOZUvVL/uERdZeC/uUsRa69Rv1O5g0Q2qNI/vhGtFQvy58bOOEgu9hd2ox5lR9t311AL4M85cfbWA/27kdfM2TMny5UxE8eHAbiMU5h/vW0LuJHK6247IXe1vnrW4zHRMBSc5VQdaVFRxC83KEYZCDdO1temnmATf7ainY0Cpnf+RMn4gwbHFrcChBtTljg6g5VTWL1PnxN6fDMpCYL86qC6/I48/GJ8YzTTwy+S90pqIW0gBT2BTbHbTZ869QNV1rfqF4lm9w69rnAmZ+K/SyrEkjLs4Mfh3qYloTkVA8OzW1h42JOVNnEIP6st6T/rstYK4HuMDhemrtaLWyA15GxeK6TxkY6GTEgvM+wcYTvI02XSnq61NRT3IygA6PHpCkI8K4XbmKhGZ3RwsKSCzF+DqCxeGosFsazu/zGw3lYJEtQlN65t8syXanY41X4ZfXJBiZKJjSGN5Fy+MBl8KWn+SB2mUmO8AaGJ8TBOyLkU6t6odq3fey9d1GEH91WZniPujcTKNCWdeTgV8y8Z/Dj10XY1JQvU43+O6Sr8o37Uc47vaolyMVjjXzpRVmjJmTSE6w/rM3WyRkxiP2Z4hgqjgypdz WH+D6kma plQ+7I9aPFKi/ghKnJTC2/jgJlU/MGCbjKVG0beCmHr6K/sEX82KQg8HfzCCcqG00jzXbhOm//8m9TifndBUJ1B3AkqtcTmvWHOVfuzHgYvklG8y5KBf3qlqMA8fDANS9AalMcd7XtsiCG0ZziDB2dB/kxaUDTCt8PV6P35xuzOrzVu9qbny+pk4/IpfRGyfX1wsmyj1klZDMcKLlgo+zOie9OkbH+kMJ13EHQLWuAbomXhUrlsmOjaufEe+1ZGREwfQ2l1WiR2pDc2qsTog98dRc9g4UW5eVnfr3ut9+ysZ9d7yT5G6vhtHwd7JbONuk5uyZrX4TQJEI0/E+tk73IkPR0qIx5OS20KtgJokK275y1iRxnCz+JYuoYg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for a new region type, "ram" regions, add a mode attribute to clarify the mode of the decoders that can be added to a region. Share the internals of mode_show() (for decoders) with the region case. Reviewed-by: Vishal Verma Reviewed-by: Dave Jiang Reviewed-by: Gregory Price Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564536041.847146.11330354943211409793.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- Documentation/ABI/testing/sysfs-bus-cxl | 11 +++++++++++ drivers/cxl/core/port.c | 12 +----------- drivers/cxl/core/region.c | 10 ++++++++++ drivers/cxl/cxl.h | 14 ++++++++++++++ 4 files changed, 36 insertions(+), 11 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 5be032313e29..058b0c45001f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -358,6 +358,17 @@ Description: results in the same address being allocated. +What: /sys/bus/cxl/devices/regionZ/mode +Date: January, 2023 +KernelVersion: v6.3 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) The mode of a region is established at region creation time + and dictates the mode of the endpoint decoder that comprise the + region. For more details on the possible modes see + /sys/bus/cxl/devices/decoderX.Y/mode + + What: /sys/bus/cxl/devices/regionZ/resource Date: May, 2022 KernelVersion: v6.0 diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 317bcf4dbd9d..1e541956f605 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -180,17 +180,7 @@ static ssize_t mode_show(struct device *dev, struct device_attribute *attr, { struct cxl_endpoint_decoder *cxled = to_cxl_endpoint_decoder(dev); - switch (cxled->mode) { - case CXL_DECODER_RAM: - return sysfs_emit(buf, "ram\n"); - case CXL_DECODER_PMEM: - return sysfs_emit(buf, "pmem\n"); - case CXL_DECODER_NONE: - return sysfs_emit(buf, "none\n"); - case CXL_DECODER_MIXED: - default: - return sysfs_emit(buf, "mixed\n"); - } + return sysfs_emit(buf, "%s\n", cxl_decoder_mode_name(cxled->mode)); } static ssize_t mode_store(struct device *dev, struct device_attribute *attr, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 60828d01972a..17d2d0c12725 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -458,6 +458,15 @@ static ssize_t resource_show(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RO(resource); +static ssize_t mode_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + + return sysfs_emit(buf, "%s\n", cxl_decoder_mode_name(cxlr->mode)); +} +static DEVICE_ATTR_RO(mode); + static int alloc_hpa(struct cxl_region *cxlr, resource_size_t size) { struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); @@ -585,6 +594,7 @@ static struct attribute *cxl_region_attrs[] = { &dev_attr_interleave_granularity.attr, &dev_attr_resource.attr, &dev_attr_size.attr, + &dev_attr_mode.attr, NULL, }; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index aa3af3bb73b2..ca76879af1de 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -320,6 +320,20 @@ enum cxl_decoder_mode { CXL_DECODER_DEAD, }; +static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) +{ + static const char * const names[] = { + [CXL_DECODER_NONE] = "none", + [CXL_DECODER_RAM] = "ram", + [CXL_DECODER_PMEM] = "pmem", + [CXL_DECODER_MIXED] = "mixed", + }; + + if (mode >= CXL_DECODER_NONE && mode <= CXL_DECODER_MIXED) + return names[mode]; + return "mixed"; +} + /** * struct cxl_endpoint_decoder - Endpoint / SPA to DPA decoder * @cxld: base cxl_decoder_object From patchwork Fri Feb 10 09:05:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FAE2C05027 for ; Fri, 10 Feb 2023 09:05:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB7246B0112; Fri, 10 Feb 2023 04:05:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D66CE6B011A; Fri, 10 Feb 2023 04:05:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2F286B011C; Fri, 10 Feb 2023 04:05:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9C7E66B0112 for ; Fri, 10 Feb 2023 04:05:51 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4A48916150C for ; Fri, 10 Feb 2023 09:05:51 +0000 (UTC) X-FDA: 80450799702.10.FBE7C41 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by imf16.hostedemail.com (Postfix) with ESMTP id EB3CA180018 for ; Fri, 10 Feb 2023 09:05:48 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aiEEDLlS; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0su2l4GKACBH+3TpTGM2GIGa+tzyjdoHTwfxlcXxQzk=; b=U8TGKj74/r7UJQMdfekE13t3iH48IXispEZNKAyd76tc6kBNnXYlS8rMkiCQ44wXFg5wID W/Qj2CNoK+Zu4lRSCRO9TJ0ThwadWZ6hBMM2I8iXWhcWY5Sl9WFyjQROyIGud7MMAxPTsG SPAODX/Wsy6G/DGGxXF/kg+wLxXBSNY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aiEEDLlS; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf16.hostedemail.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019949; a=rsa-sha256; cv=none; b=QOiyLRxiX5g677O0Zsu/QAEaOc29cgKiw7/xFShgW/NTGDH0WXrieJjuVK3yDcXLtpLbqz XcCh5GOHA5zQrOrbe2TsRdu/N/r3m8dWWl6b7Jp5Ij/wa4/a5gVC03wijCy35SX8D0WouM y/g8iCE1FshtYWU2qTLJCUAOAMDOCCc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019949; x=1707555949; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8rEeQZ+8vrd28/mihuN4IYSQYwRcfCWFNTCZHITf2LM=; b=aiEEDLlSU9gpv+tfe7MUjllV3cHt0ej4WUjpqZWWReRVqT7spfJAU53r 6A1htXMWmN1/QgjtRENRa7m5Kd/C4MlvMxq2vPG/Z42r2FusqFZOtFsFS bewJFDoMQCZ8DaSDSSPiQX/XTbRVevSENtWUmQXTzjhxZ/p80pBT/in7w inF6wK9L/8K0u3cI+1YFG9+0RCerCHoWy/DkARjmtqCcEF3YjR1PAJGZa BEPCQSL/wpih7xLl0tSIFa+H/IAdVAoRp2SHdCm54akbsqTZiKX98qqoW e1XCWytGoQNLVPC0E/4O+VvTRPb1kunmlW+FGBytXGwUHEfs+Vt/1F958 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="331677061" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="331677061" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:46 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="698364566" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="698364566" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:45 -0800 Subject: [PATCH v2 04/20] cxl/region: Support empty uuids for non-pmem regions From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:45 -0800 Message-ID: <167601994558.1924368.12612811533724694444.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: EB3CA180018 X-Stat-Signature: r61pasw8bsxxphpb4pkz5rbbr599mt7o X-HE-Tag: 1676019948-412488 X-HE-Meta: U2FsdGVkX19qckoILHIqIifMT8m5JgpAFQjPNruuUIxgwvrddriO3HwAW6NF9T05J5lcafeBrPPM7/JI7bi5TbeewTGVNrucySOAAeQOTH5tyY9yqjdp945iGgDzvo9bdNGCNJbmDWM3C5n1ioRnrMLtFpGKrmOW2b1dQt4mR8g0Z1Z+xS6Chzjy3WsXQ1TL0MX6lTeAzgg7BUbKfdatuS51zAG/nuiE7MU941n4s5Fz9Y//9sEtjlhNQR92/Eba1qKGM0gitQwtUxhIcjwrjs3yF07JIzMEmALy4L85PL5PuLv4zkh9p5Sf/+jqgbDYVD68uDZZRgSpEKx6FCArInrzxaXSnfBpZkTfgl6LfYdHfRJ1S/RTcVmEzoTmf88taLbstVXsTOwImN/pHrmgPp3fQaGYKt6d3L6pjNSjMD38PfpYxbg9V+icGOrdKd+nvUm9HoDnntbMJvm3L8WdzfxelCkePZD1bBtHDN1LWKRhEGhJRTlNSxJAhQp3Emdi0sefyOp2KvQ/ZxvteekwmusEcSutat6Z/Ml+JEAftZyC+hEWi0qSeTP/rXPyIY9BsufqIqkHdPsmR59rcR1YSiZcDGv8n+TydGIpesCPcwNoJdc6mEm84Fmq0kB83O6+zdSc1NkLvJd7jzRGI/imGTGbXPLCj28/Rgbx+AfiUYGAllQVf/f3pOmv1Gv8bWoi9iWxxiVs+TP1STpdDcXEVSxxZJm7k9oiOvok8Q1pQvMjK1qJJFkaurooenFuxWsXrB3n4Jxyi6GeUdUmJirWFKGvSgx9XX3sm4NSmaJDTWCqILfEd007S4weglEbfy/PFEsTgUl2OB84Xsym67+a/zbAX8Dm696aMnhGI0OSHaFup207/8XkPmuF4tuCBFUP+ShDKUp5tkcEcfiEtQE6vEMSztFu+iTX5TAUqc7IeewqcVRZFE+wfJpHJzZMWsAdALPJYFH/rDVw4YVz5MF YNViQDAA ufQqdc/I8gynVckxbYaRAhdz2wh/6pZPwexr7WT2SLideC4cI6SmrvGHol/v3EWKc2SAVU5rxSSPb/gitkE+/WRWJ1oWSADCxVOzM1UlEqe+SKbusPK/teF7WTEUWDPnPXfawRjxRBC4e/VmaFX0RjQPqgZX0qedwlAl7/5F6Pjr3/c4u87TW9eqv5xjum4O70aSx1+EsS/DwfJ7B9HW550ewBjAK3fm7NsunbJa0HjWRJchXG30rZPteXf2R5J6nK5IdJApSVa6QTdesWbCmwCPKfp/bHDzWQvpPXOrgfKbiQqc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Shipping versions of the cxl-cli utility expect all regions to have a 'uuid' attribute. In preparation for 'ram' regions, update the 'uuid' attribute to return an empty string which satisfies the current expectations of 'cxl list -R'. Otherwise, 'cxl list -R' fails in the presence of regions with the 'uuid' attribute missing. Force the attribute to be read-only as there is no facility or expectation for a 'ram' region to recall its uuid from one boot to the next. Reviewed-by: Vishal Verma Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564536587.847146.12703125206459604597.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny --- Documentation/ABI/testing/sysfs-bus-cxl | 3 ++- drivers/cxl/core/region.c | 11 +++++++++-- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 058b0c45001f..4c4e1cbb1169 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -317,7 +317,8 @@ Contact: linux-cxl@vger.kernel.org Description: (RW) Write a unique identifier for the region. This field must be set for persistent regions and it must not conflict with the - UUID of another region. + UUID of another region. For volatile ram regions this + attribute is a read-only empty string. What: /sys/bus/cxl/devices/regionZ/interleave_granularity diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 17d2d0c12725..0fc80478ff6b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -45,7 +45,10 @@ static ssize_t uuid_show(struct device *dev, struct device_attribute *attr, rc = down_read_interruptible(&cxl_region_rwsem); if (rc) return rc; - rc = sysfs_emit(buf, "%pUb\n", &p->uuid); + if (cxlr->mode != CXL_DECODER_PMEM) + rc = sysfs_emit(buf, "\n"); + else + rc = sysfs_emit(buf, "%pUb\n", &p->uuid); up_read(&cxl_region_rwsem); return rc; @@ -300,8 +303,12 @@ static umode_t cxl_region_visible(struct kobject *kobj, struct attribute *a, struct device *dev = kobj_to_dev(kobj); struct cxl_region *cxlr = to_cxl_region(dev); + /* + * Support tooling that expects to find a 'uuid' attribute for all + * regions regardless of mode. + */ if (a == &dev_attr_uuid.attr && cxlr->mode != CXL_DECODER_PMEM) - return 0; + return 0444; return a->mode; } From patchwork Fri Feb 10 09:05:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEA2BC05027 for ; Fri, 10 Feb 2023 09:05:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85EA86B011A; Fri, 10 Feb 2023 04:05:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 80EE46B011D; Fri, 10 Feb 2023 04:05:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AFD66B011E; Fri, 10 Feb 2023 04:05:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4A1516B011A for ; Fri, 10 Feb 2023 04:05:57 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 00CBA1C6D31 for ; Fri, 10 Feb 2023 09:05:56 +0000 (UTC) X-FDA: 80450799954.23.F39D24F Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf19.hostedemail.com (Postfix) with ESMTP id E2EA21A0005 for ; Fri, 10 Feb 2023 09:05:54 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JzWgV6G0; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf19.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019955; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sXevTYWZEFDu/kuHSXYci1sYVSQvNQ6E8gmnztP0Loo=; b=bYEpfHnz3OEenYn8UzfNtMS2oIOiHlwizFQ02OUaifVPKIn0K3YCXQfkuOd+ZpMtrJ2nZD hJhQPzFq1og9iKKxd9zQPNir0+WZqm7dnL9E5sD1nzwMWniGWCZI6/L9tR6u3DSwLavGML GTPgBPDyhyEF9ES387oUSLhHrmmM4xE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JzWgV6G0; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf19.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019955; a=rsa-sha256; cv=none; b=o1Kdln9wdVuETKO/Ciz+V4qrLKKFpLs5MFxRLZeet8sKI4cZ7F67YYe4NRrDvs0gpVDYUB Q1xRXlm0uFi1BJ3HG+SPlGatLflAQYLMe4hfduP+lliU9pB6+oc0qjbD0+64mjgm6RzzCF s8nyf3X4vCnJc85tZxLuuRtSFGJCGwU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019955; x=1707555955; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pOZoUMpBUKRLc8QIK289XEjoJsII4YkEt62Y5Arb8us=; b=JzWgV6G0ISbYIOXom1ktU4OHQDxZeIOX2xVXRHG3i9wchIBfVBdm+D7i x/JuyofdkJuhov9BQSr+5Mq+aFGUKDUx9iUYws35xEZyjC72nUiQfJZqn LUbUrjL1dw9yLfjwMm4ywPdfJvcCYUQOBv+qR4zsjkczxk0h7OmK7aZs9 rBjBKn5gDxzdCqOZ0UpcE94nqjvTrkvVEGxtVzzgl0uE3W4R12i9feAbw wVqYBPwlC5JITuGp0QdnV4VkV3WDKxaoqPtflGiwYAT2RmonWVMSTv9NM s6nRyUmFv+d3VvlaqXmw/tUjejbfcLpc8eHH72SLeagMEco9BcSqSMBUw w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416600013" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="416600013" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:53 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="669930208" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="669930208" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:51 -0800 Subject: [PATCH v2 05/20] cxl/region: Validate region mode vs decoder mode From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Gregory Price , Dave Jiang , Ira Weiny , Jonathan Cameron , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:51 -0800 Message-ID: <167601995111.1924368.7459128614177994602.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: E2EA21A0005 X-Stat-Signature: r64h7yatmdzooafbpj13nh1uhortiz6w X-HE-Tag: 1676019954-205034 X-HE-Meta: U2FsdGVkX199g8/AyKhDv4A2X3WX1Q/5i/CUA0V7mrtSzEYSkPdu6csE9B6GUAA7vlLCrRiZWXwXf++RKVBwFgFhcxevXdXHDzQ+qB8c5PW/GZCIHPzL5CzYaKF1D1xQLoCXKfSk9eYyQ/gVC+iz5GtC6y5ftipv4PmsLG3S4LYmuEZpTPIzV7USbX+nF+3ZPHxa2qGts1n/KK0iYYw5Z74Khge9kcYgpiOJfx02rPfGyuKHaBtbSqyOSaJkI5PWpor69FyUdX21c+lF4KtZMvbi/lOiAkZZopGw5BQTCEpKF2Vjtj0AFsnHypdpPETC6Mq8ySOTmFl/b87j2KJIluI4B/YN7vUJ16GPkgouN5twCKQKNcWNl9Js1jMUsOQJO6VK5kn4HVptprMfFnfmforYgPvPQ/q1dkF4RJU2Ru7nszUUPL29zWwUiUIaY/szxbowl0PuHB0x5MJXW6jOGbwWcaF6Shkvw79FLN3EuSL9Qo0eIktSIYuntdBZ2bvMQr606OIC3LsnKzhdLwosGU/ShtH9oM++HELFuQN6UYd7Xd2fKgSPDt65NacV8734QWW1PfPx9WZVxVgtefcZSMJ6Ln5tOYg3Ci46XTwE7DiaXnnWpKaPXvL0OcSQ1qoPDwvcqdqZbLZPFnZjQgQ8c0Lw+E/fvo4ByVi31Z+AoLHHx1pIU4C++hC9OpVU2UwGlXbD5DmQ3abCnUSr9E9RGU4lqGzLCiyGNmvvsexnpvW3a1Y4DSG6UV7ajg8aIudp1Aq5dEPLFraj2Lj+GfqjVMOk5RqfqvgG88eu/oym0GiRRl0RCGdNqukdmywu0P+lXW+WUj1UmjN/WYySUCjOrSR5804fOUJRJwQbrRB1PZ+xsDIO0f9KQ9JhvJpi2aMM+JBxXNshObBPo86H8QXjBzQwWvKc1XejKiFAKHgg8zc174cD34p2tJ1xm6CMU5bgAl9n/sVReTdpuU29Hae gnrpjKuS AUlEzxy+hGJr3fZB+wiAmFOhVJ00BzKmZFEtmu3djKdCyUtDPkhE2/CnlyxMfkKbek8cm+VvQuTbNe+CbcxmlPXfVMNOQKNOGqSX7LBpE0Gucw/MErSFJqxElqq3eEeWomlYpkbO4SBslimQ3CzIMxkmU9sGijQocV93s83R6084vf7bkddkPr2rbTvG5/bYKOujjPZNSqv35hnc+9c11iEwyMyMRUdTunxKhJEUCCV/qTLX441tB9KQ/7+4X6R9ARlWsuLDhW1ysiP9f6pdzfYmM1emFtLMeKd4H/x9hn+gGJJU6+F9yOWatnSyLTH8CUauhbOEMTEPMP/4ixwX6XyGeTYlZz9IyHxU6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for a new region mode, do not, for example, allow 'ram' decoders to be assigned to 'pmem' regions and vice versa. Reviewed-by: Vishal Verma Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564537131.847146.9020072654741860107.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- drivers/cxl/core/region.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 0fc80478ff6b..285835145e9b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1221,6 +1221,12 @@ static int cxl_region_attach(struct cxl_region *cxlr, struct cxl_dport *dport; int i, rc = -ENXIO; + if (cxled->mode != cxlr->mode) { + dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", + dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); + return -EINVAL; + } + if (cxled->mode == CXL_DECODER_DEAD) { dev_dbg(&cxlr->dev, "%s dead\n", dev_name(&cxled->cxld.dev)); return -ENODEV; From patchwork Fri Feb 10 09:05:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80882C636CD for ; Fri, 10 Feb 2023 09:06:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 132986B011D; Fri, 10 Feb 2023 04:06:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E2996B011F; Fri, 10 Feb 2023 04:06:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EECC16B0120; Fri, 10 Feb 2023 04:06:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CA4BC6B011D for ; Fri, 10 Feb 2023 04:06:15 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8170D16150C for ; Fri, 10 Feb 2023 09:06:15 +0000 (UTC) X-FDA: 80450800710.11.AFA1890 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf15.hostedemail.com (Postfix) with ESMTP id BC95EA000F for ; Fri, 10 Feb 2023 09:06:12 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fvyPPhsC; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019973; a=rsa-sha256; cv=none; b=JPPbJ7FGWrq9LMy65xiBMMAEUDXr87bo+fTTwpDaaVdSm+C/Mc8xkY3sMzAKWJ6k2044CX xm8gQA80Erzv45HWrOZUrG1QnNz29DuwG+5kpMr8K4+kkpZi8fEOBJ5xN5zlkHsIyT8+06 9YI1QymGw0pyhgdHmQUHhEEQWmMbdxs= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fvyPPhsC; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019973; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wwuxx7bh+7ePNt2vu7+dCEZZpWGglxPpYkNjrsP1BZ4=; b=np1+1vMt061XJRocbJ/eJKN3W2q/gMtbV+8UKOb1T5bCHrUrVW83gC+/f84PeXRFVJQIw6 fCQxEIcYgFtI/rfbV8hQK48UQ1TOeDpir1FSMOivBduWpBOy49qz49KLH3aVwpkGF/+j2f V3uml3a0JAnP1xoXQtJ/N+yRmIAb4sU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019973; x=1707555973; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g9WOYiZgLx/JR1gosHNg5WOS/EaNuj1iypU5OOVGJsQ=; b=fvyPPhsCH3Kd81pwOUvnvW1vUwNtZT5ANa+lZNe/rpykXGlPTBnouuNi YhqepTOj5cxWEI8hw6Mh/SHAN0t2aH7M+MIVhWolSYwipKkj03jWvMvc5 5gChsO2BL91D+X/gm5V3QgNw7hWF8M8IsN4jiIDg1VaKgGjnJhFEfJGmL 6i8YwVCCARWDPIPILcwxf7rIMryZ+Abub+C8j9e/d9YV90EyN/gRXQQt6 2rYQcyaMZDQFA0m1MoUWkXiCocP9AzRRZnwNj5q9tCRRSyAcnjNaC1GSd Izpe/Vw+0vvhu/W0SZG4MhBE6Id3UcAA4j9Zp3PRudRRjm6IZeD9CWGcM A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416600026" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="416600026" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:58 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="669930220" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="669930220" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:05:58 -0800 Subject: [PATCH v2 06/20] cxl/region: Add volatile region creation support From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Gregory Price , Dave Jiang , Ira Weiny , Jonathan Cameron , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:05:57 -0800 Message-ID: <167601995775.1924368.352616146815830591.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: BC95EA000F X-Rspamd-Server: rspam01 X-Stat-Signature: c8gmzquen6qmwnac9xmtuz3fu6cyurhm X-HE-Tag: 1676019972-590851 X-HE-Meta: U2FsdGVkX187xrGWYPF6pef5vTHO9HQj5CB3aYD0W4Tio1l0ie9k7IuFRQJuaLBSxWuZv80ReGy51D8/mqmaY1831C/Dev2aR9EsYNqaqhTtBoKYV0ntCMW6kM26hOKNNpLAYHnIIxBvXSLG0VW6M+wlLxNqXG0yYsd+oQgcxWUNcTum5iuADGj1iRN6qrQTuLC/tBHu9+nxwIeNS97bM1tPxbPXQLcuMT09UsQeHhfr+2gN53AQfTvaEhYUcI6UNIEN21/tGhx6y+SAtCSjMGgBDCtoVb296U3xae17QF0+cC3j6Ws6JoP3M8tgANUd1bu3+A6opmuniibfigN+s4W9Ft93EDJiy6EXwQ2Sk0hBN3yduZcTkPJfTD8o+dp6bV+gy4o8QV4yt3LgyzRFQJHeTzcmyoL91wMaKxIXBzD8iuCI5yPapKci5EB3aqaneb51a9UA4PqX5PVwaKzK1nvx1n5QWEIVKhV/z0jDORlCEHLimJFVTh/d4thDJC2T82Dto/nqN7a43ue63hHFzS15EZ3zCfzOLOtlblB+oHDMnwXfqvHYCHFBxsD1jrB7FhpZL+ed7po1Mu9RxCKlL2U5tD+MwmYOgEEigOQzC8Y8EoxlTHqEVseh/FaV4oYbSjQtJpl4+7kkRGACd7rJIwssIbtXvVtwisLtueXAPSob2WwM6eJS6XH7PnfviOfgg9s+/5GbnkPmquiPycsG5jg2vd5tADiBxkUoiBcLWjEqxZaeSKz+qmtZ4OMy6mhri7XRfxIvBeGDbVCC5hlXUEI1chGRyxRy84WlxbYLujc5Hx/vd0epw3eIF/28PQ7uKbVRvULJtTeKR6VR5Da5E/F9J2ouT0X2ltRR+2/GIW2V/xyr7jHsoFRHLB5BSvXkHZOVgd+movhlflOO3d3Q6q/IWQGH8sdLs7FLltwhEmkABagYMcBIcpx4W4WXR5ucM/oSHem6gi+tHG3m6WH GfAobVnD 2AdrtkDV+Aajjyw8XhmBa4nYHVzyK+sBaWO+/6pMj0sD8hUXuBWAOJp6ghKjtHLN9Jq5/88yNiBrJfMlsnTA8kNuIOZhitKrxc5H+kpHehY47l2IBJwWax/x0XmAfZ+DX4dEZw+1KcJjFmJCZboQ6FLhOhCiLNWO5qecfVX5/lZ8tAenacck9F0BiCGv7osv66I0C6R5Oyz6COqTP0T3hTTtr8YTxOrddsj1KRJCw5B0W0zGCDttGz3Mjslj81c7FV9D4KMCv7Qp29/OiQlzQmu+MfV3lNu0nzJWB0RiCiM94QeZyig+GIpANl+/bVOOgkvZPa68+1L++cXEqTw037jNBm0ci2VkGQNxvs9kFNSJhucELf4aYIuRxqthKsn4xTAOD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Expand the region creation infrastructure to enable 'ram' (volatile-memory) regions. The internals of create_pmem_region_store() and create_pmem_region_show() are factored out into helpers __create_region() and __create_region_show() for the 'ram' case to reuse. Reviewed-by: Vishal Verma Reviewed-by: Gregory Price Reviewed-by: Dave Jiang Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564537678.847146.4066579806086171091.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- Documentation/ABI/testing/sysfs-bus-cxl | 22 +++++----- drivers/cxl/core/core.h | 1 drivers/cxl/core/port.c | 14 ++++++ drivers/cxl/core/region.c | 71 +++++++++++++++++++++++++------ 4 files changed, 83 insertions(+), 25 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 4c4e1cbb1169..3acf2f17a73f 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -285,20 +285,20 @@ Description: interleave_granularity). -What: /sys/bus/cxl/devices/decoderX.Y/create_pmem_region -Date: May, 2022 -KernelVersion: v6.0 +What: /sys/bus/cxl/devices/decoderX.Y/create_{pmem,ram}_region +Date: May, 2022, January, 2023 +KernelVersion: v6.0 (pmem), v6.3 (ram) Contact: linux-cxl@vger.kernel.org Description: (RW) Write a string in the form 'regionZ' to start the process - of defining a new persistent memory region (interleave-set) - within the decode range bounded by root decoder 'decoderX.Y'. - The value written must match the current value returned from - reading this attribute. An atomic compare exchange operation is - done on write to assign the requested id to a region and - allocate the region-id for the next creation attempt. EBUSY is - returned if the region name written does not match the current - cached value. + of defining a new persistent, or volatile memory region + (interleave-set) within the decode range bounded by root decoder + 'decoderX.Y'. The value written must match the current value + returned from reading this attribute. An atomic compare exchange + operation is done on write to assign the requested id to a + region and allocate the region-id for the next creation attempt. + EBUSY is returned if the region name written does not match the + current cached value. What: /sys/bus/cxl/devices/decoderX.Y/delete_region diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 8c04672dca56..5eb873da5a30 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -11,6 +11,7 @@ extern struct attribute_group cxl_base_attribute_group; #ifdef CONFIG_CXL_REGION extern struct device_attribute dev_attr_create_pmem_region; +extern struct device_attribute dev_attr_create_ram_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 1e541956f605..9e5df64ea6b5 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -294,6 +294,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, SET_CXL_REGION_ATTR(create_pmem_region) + SET_CXL_REGION_ATTR(create_ram_region) SET_CXL_REGION_ATTR(delete_region) NULL, }; @@ -305,6 +306,13 @@ static bool can_create_pmem(struct cxl_root_decoder *cxlrd) return (cxlrd->cxlsd.cxld.flags & flags) == flags; } +static bool can_create_ram(struct cxl_root_decoder *cxlrd) +{ + unsigned long flags = CXL_DECODER_F_TYPE3 | CXL_DECODER_F_RAM; + + return (cxlrd->cxlsd.cxld.flags & flags) == flags; +} + static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute *a, int n) { struct device *dev = kobj_to_dev(kobj); @@ -313,7 +321,11 @@ static umode_t cxl_root_decoder_visible(struct kobject *kobj, struct attribute * if (a == CXL_REGION_ATTR(create_pmem_region) && !can_create_pmem(cxlrd)) return 0; - if (a == CXL_REGION_ATTR(delete_region) && !can_create_pmem(cxlrd)) + if (a == CXL_REGION_ATTR(create_ram_region) && !can_create_ram(cxlrd)) + return 0; + + if (a == CXL_REGION_ATTR(delete_region) && + !(can_create_pmem(cxlrd) || can_create_ram(cxlrd))) return 0; return a->mode; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 285835145e9b..e440db8611a4 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1689,6 +1689,15 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, struct device *dev; int rc; + switch (mode) { + case CXL_DECODER_RAM: + case CXL_DECODER_PMEM: + break; + default: + dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); + return ERR_PTR(-EINVAL); + } + cxlr = cxl_region_alloc(cxlrd, id); if (IS_ERR(cxlr)) return cxlr; @@ -1717,12 +1726,38 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(rc); } +static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf) +{ + return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +} + static ssize_t create_pmem_region_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + return __create_region_show(to_cxl_root_decoder(dev), buf); +} - return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); +static ssize_t create_ram_region_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + return __create_region_show(to_cxl_root_decoder(dev), buf); +} + +static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, + enum cxl_decoder_mode mode, int id) +{ + int rc; + + rc = memregion_alloc(GFP_KERNEL); + if (rc < 0) + return ERR_PTR(rc); + + if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { + memregion_free(rc); + return ERR_PTR(-EBUSY); + } + + return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_EXPANDER); } static ssize_t create_pmem_region_store(struct device *dev, @@ -1731,29 +1766,39 @@ static ssize_t create_pmem_region_store(struct device *dev, { struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); struct cxl_region *cxlr; - int id, rc; + int rc, id; rc = sscanf(buf, "region%d\n", &id); if (rc != 1) return -EINVAL; - rc = memregion_alloc(GFP_KERNEL); - if (rc < 0) - return rc; + cxlr = __create_region(cxlrd, CXL_DECODER_PMEM, id); + if (IS_ERR(cxlr)) + return PTR_ERR(cxlr); - if (atomic_cmpxchg(&cxlrd->region_id, id, rc) != id) { - memregion_free(rc); - return -EBUSY; - } + return len; +} +DEVICE_ATTR_RW(create_pmem_region); + +static ssize_t create_ram_region_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev); + struct cxl_region *cxlr; + int rc, id; - cxlr = devm_cxl_add_region(cxlrd, id, CXL_DECODER_PMEM, - CXL_DECODER_EXPANDER); + rc = sscanf(buf, "region%d\n", &id); + if (rc != 1) + return -EINVAL; + + cxlr = __create_region(cxlrd, CXL_DECODER_RAM, id); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); return len; } -DEVICE_ATTR_RW(create_pmem_region); +DEVICE_ATTR_RW(create_ram_region); static ssize_t region_show(struct device *dev, struct device_attribute *attr, char *buf) From patchwork Fri Feb 10 09:06:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8E91C05027 for ; Fri, 10 Feb 2023 09:06:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66BF66B011F; Fri, 10 Feb 2023 04:06:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C7776B0121; Fri, 10 Feb 2023 04:06:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 465BB6B0122; Fri, 10 Feb 2023 04:06:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1998F6B011F for ; Fri, 10 Feb 2023 04:06:17 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B6BD8C1488 for ; Fri, 10 Feb 2023 09:06:16 +0000 (UTC) X-FDA: 80450800752.01.CD7511E Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf19.hostedemail.com (Postfix) with ESMTP id 93B5E1A001A for ; Fri, 10 Feb 2023 09:06:14 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FoSVZzCp; spf=pass (imf19.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019974; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oA+zrlhsxTicB4JREMkAp7hINgQp3GNpARD4FxAUpgs=; b=Xir3Eg2AGrX+As2LHOacQ4Yy8jLK4YM6heJhhwiTvZDAmekWBuwrNkklK5UFHKMOgH/sdF DrtSdYiNsDFZKxQPJCZqQBMhvemM9pddU4tHtLfq82+gygyiAlm4VvOeYm3Vw9j7cGt7Zk nI+2yq7JeMtRsiiD0p8+x+jSx7+hReo= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=FoSVZzCp; spf=pass (imf19.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019974; a=rsa-sha256; cv=none; b=WwbUuyfW28t1m1Nqp+4+FtdZ6hkKydM23jUETRx/QNU+IItQh8kvHxrBNg0qQc5j7p8nIA 0vDOxlIUnzuuKvDPEGnzMfnI/YyWs4Oq8NGHfS8D7RNuzvv/SoNw1bETQ744whxB2geMQk hWDYMtjOOLKjtNN6AMlekkbRbPViBSo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019974; x=1707555974; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=s+JhIpX+g9sjvv5ZiADOIwQseU67ppQ0dtunIobOaPU=; b=FoSVZzCpEuK93BjViEPsgOsWDT/V0KFsVcwyCVR6CBKAMDaTxb+C/h7a gZ0S78N3FUi59hasDyfFVACMrsfnkwDsh73jTTmKZRgmD9Dkjx4OHExop gSR57dlpmyHSy2BOcUz/NGxK7emo4RrAKaUNttBkIv+XJ2dxs78lWMmHG f05pycl7xkdf4l5ldFoxBq9TKIy5NZVffpioCmLywXS2U7C4J66+OCVUD 7eW0paqHmG2DX76EckYUVhfICaB8+wDNcAY/Ac8AKmx5ND3U8MOxB10BS 8v40f6XJ2NiL5f3PkuHxIkVHO6k521ZY4OG/GNbouBHuxWUVbhAnp4AIp w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416600043" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="416600043" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:04 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="669930234" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="669930234" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:04 -0800 Subject: [PATCH v2 07/20] cxl/region: Refactor attach_target() for autodiscovery From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Dave Jiang , Ira Weiny , Jonathan Cameron , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:04 -0800 Message-ID: <167601996393.1924368.2202255054618600069.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: co1hoe7kmb3wfkqqzez1gj1h3tttbtib X-Rspamd-Queue-Id: 93B5E1A001A X-HE-Tag: 1676019974-90986 X-HE-Meta: U2FsdGVkX19Ud1HMEmMweEKU+G9VLOUYgT6zJGjzci0cbvfhJjpqJ1H9/2bMhUw4hb+Y8NfmdCDZMxGt8bwvczbJMzaET9sDziHeZ/B+ZZt3bm2hKD8g41KZ/b3wrPcRDzmvryTsqMAE3U7dyTG8CNY1NBsuo5un9DqQ5VvpuB7tz7tfwMAnaD1ss2UiDulGDzWv1pStAsQOC1jSmvgDETo61ITjYOHkLzQQaawCjynnCWqTfPPxf6NrEPuAS7pRZBvVOloCTV5/p/iYnI4etnJ0S6tjxWNs1pYLozSrgYDu28k2i/t3aIaAlHutfXT2a+AB8iM2cHBaXh6cM9NeA29e2f/ac99Gvtj9+NwV2VrO+WrIjXlo9XcoADrhByKUWLM03X9E7j7COx/bdXKjaxouRJNkCs1oEdAt/7774kOcFTWUakz7x+ONtPv9ERbZ53CcuZOjbRdgYRJTFyMbm7JDHBbIfkPLDaYoqr7i/sBG1ym7Puwym7KazPSgXLttDVEhSvY3M7WSdpUlUHSvStvwBL7Hz5B/aNFxfa7tYrLuCX5Z/1/DHOp9m0yf1EUXPIfHiHxB3xxe8qq64t21i2EiSRfdwewDdOD+B/1w1RQr6aDH5j+JR0w9MBF8A+D35lqG0G0vB9mnXs0Jl8Y0mMmRtg3BY563hOMy2j6ONSaHM3fPT8c19BhpXGrVSPop0xGKqY2lNu7/XgrG7QXeBf82FCNPBivWxrrXf4sTjYTphyq1NwslJlIOyRhJPOi0owylQSAeFngvyTPUGUUTYqioCE3z/xJgqLb7IZb1aSCfXEMXxjMkkY0+PIzrPUwCPhXi1L2NV+tA2fiW6uk3USagUewK77QAyYDGCINvEoBxHEph7Y/tHLm68gS29bwDJxAgMJUCm/H1Jr8ZFkJOkogdL12R/4Gg2Dt9Pd2T+Aa7RA9azZJesYGAfRl8sm2395JxMGBBHtzNN7h2JsZ UQIRV23V UisfxvkipxswJ3e4diJPeZyBaR9+vaR6V0/AJIYTgHRjrzWx2NjajSK0qJdFy8TQp1cxjJ6DBw3YFM8Rfun3VgxRKn0mEuT71vkahvyBr6ZrbbUAuL6CrSiiUS5jxPOMoIHxLgnWRWV6lE139nvlgSgx1O60H16fqtj5/iTcZnjoxqnBbp56eRy/BaUQRnYxdHtCiHHOFryTLRpSjT/OOVodtIRmGYHVJCo89cgqMVAmN5EW8fLp5LX8zK1XnjSKNHOvSqDCbEgDyOFbivjD+I7NllFdTraP+hlAIic6g1UCioNLckGZXNKCiLjCnlgzmvZ2e17ht49Bc+rM/6Yp7C3V+PA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Region autodiscovery is the process of kernel creating 'struct cxl_region' object to represent active CXL memory ranges it finds already active in hardware when the driver loads. Typically this happens when platform firmware establishes CXL memory regions and then publishes them in the memory map. However, this can also happen in the case of kexec-reboot after the kernel has created regions. In the autodiscovery case the region creation process starts with a known endpoint decoder. Refactor attach_target() into a helper that is suitable to be called from either sysfs, for runtime region creation, or from cxl_port_probe() after it has enumerated all endpoint decoders. The cxl_port_probe() context is an async device-core probing context, so it is not appropriate to allow SIGTERM to interrupt the assembly process. Refactor attach_target() to take @cxled and @state as arguments where @state indicates whether waiting from the region rwsem is interruptible or not. No behavior change is intended. Reviewed-by: Vishal Verma Reviewed-by: Dave Jiang Reviewed-by: Ira Weiny Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564538227.847146.16305045998592488364.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- drivers/cxl/core/region.c | 47 +++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 19 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e440db8611a4..040bbd39c81d 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1422,31 +1422,25 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled) up_write(&cxl_region_rwsem); } -static int attach_target(struct cxl_region *cxlr, const char *decoder, int pos) +static int attach_target(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, int pos, + unsigned int state) { - struct device *dev; - int rc; - - dev = bus_find_device_by_name(&cxl_bus_type, NULL, decoder); - if (!dev) - return -ENODEV; - - if (!is_endpoint_decoder(dev)) { - put_device(dev); - return -EINVAL; - } + int rc = 0; - rc = down_write_killable(&cxl_region_rwsem); + if (state == TASK_INTERRUPTIBLE) + rc = down_write_killable(&cxl_region_rwsem); + else + down_write(&cxl_region_rwsem); if (rc) - goto out; + return rc; + down_read(&cxl_dpa_rwsem); - rc = cxl_region_attach(cxlr, to_cxl_endpoint_decoder(dev), pos); + rc = cxl_region_attach(cxlr, cxled, pos); if (rc == 0) set_bit(CXL_REGION_F_INCOHERENT, &cxlr->flags); up_read(&cxl_dpa_rwsem); up_write(&cxl_region_rwsem); -out: - put_device(dev); return rc; } @@ -1484,8 +1478,23 @@ static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos, if (sysfs_streq(buf, "\n")) rc = detach_target(cxlr, pos); - else - rc = attach_target(cxlr, buf, pos); + else { + struct device *dev; + + dev = bus_find_device_by_name(&cxl_bus_type, NULL, buf); + if (!dev) + return -ENODEV; + + if (!is_endpoint_decoder(dev)) { + rc = -EINVAL; + goto out; + } + + rc = attach_target(cxlr, to_cxl_endpoint_decoder(dev), pos, + TASK_INTERRUPTIBLE); +out: + put_device(dev); + } if (rc < 0) return rc; From patchwork Fri Feb 10 09:06:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07899C64EC4 for ; Fri, 10 Feb 2023 09:06:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8AD816B0121; Fri, 10 Feb 2023 04:06:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E8876B0123; Fri, 10 Feb 2023 04:06:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68A0A6B0124; Fri, 10 Feb 2023 04:06:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2ABFE6B0121 for ; Fri, 10 Feb 2023 04:06:18 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DFBE0C1355 for ; Fri, 10 Feb 2023 09:06:17 +0000 (UTC) X-FDA: 80450800794.22.87AA8F2 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf15.hostedemail.com (Postfix) with ESMTP id C4FA3A000F for ; Fri, 10 Feb 2023 09:06:15 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JSHxYX70; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019976; a=rsa-sha256; cv=none; b=3DWY6dSnti4KsRY39A7WfuJlduwEZb/0Ge7ovDc9TM9PV6saGGstWcBECM4KAYDvn20UVY obbVuKrKB92+1cvWZMg+gm1zE9unjK06EPXkvuPbVAbjuHy2b2I7ZY30CERDvkFSI4Nw2D Atb1ZMPfUyV9K24TWJvZ5YnSFTdLqBM= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JSHxYX70; spf=pass (imf15.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019976; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mfUEpWzaCwLjRg/Zjh5TXxp63MmUw9zTaXLKAaLX4Xg=; b=CXFAAxQv4kWgbY4dm2oqCMZdfxn/Y3OQiaGM8iQ8pczBbAgdKVezesu21saV80DwTtgBeB zVNPLXCPZ+iZ2m+eGQY53+Mn1qdvRykrGR/TmCIyo0PGRxP6btrS6IHrUoYiGm4EzGEl9w rIiMtJFp1CvKa5hb4JiNsd6794vYXhI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019975; x=1707555975; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fhaP5oPHi1onz90GhrHdXiTbxa3g3s/6YpVTWY4pO0U=; b=JSHxYX701Clq6zXbr/KYF3yvV9vve2AyF2a6BgVEk7+OkN08K/bRgRhW XBfbQNhg/zp37M6Tn409rIWiPwpoI9dlVkIht5iTBbws5MdL5HZBDBrtX d+he9BFR+5kMxxFKVD1chUYv5VNESK+WN1ybfTm1p1sKZlBV4g8CXlyWg lmbhnIrVIR3cgGcM6597h9CDlpP2rhWoxgRoipt7Cs/29HPc+tmb/mo42 QAVQswxM2bkVgYK9zvSOocOt39wCtZlHZYHMKZGMxUZ7LLxi0H4R0WS9O 2z5GIKpR+rl1gYlpFyhB4w9eQDmPuU7iZ1yVvZmyihQhdFt5db66f+tcs g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416600066" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="416600066" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:10 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="669930260" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="669930260" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:10 -0800 Subject: [PATCH v2 08/20] cxl/region: Cleanup target list on attach error From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:09 -0800 Message-ID: <167601996980.1924368.390423634911157277.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C4FA3A000F X-Rspamd-Server: rspam01 X-Stat-Signature: h6zyx4gbkiro3h5tm9tm67b5irziaf11 X-HE-Tag: 1676019975-141546 X-HE-Meta: U2FsdGVkX1+zBwP0JGlKKM6c6DrtEU8pP1/IFqwSYpMm6x7wE22n4jvmFoU9r81sJD1d8qBPK0Oq75T3FFUkcL4Scm89qK9KOTYxlbrOHN1KOC+4YS3eSoTiAd6VwUxf/F/zEdpXXaX8iOH8ogZl/yC/EJNZkljT0VQK/hbe9RnaCRPSx4wNn9NXp9WIQ4Z5yg7GO74KdD25aTFVHNNDweDLxJO6T/vUz+gQDzfBxFOSxoxvvetFctOkdPQktnagAPM62oKxePKr7JKBFcF1mCkFRXrLbP+5nYj5ZgXu3S+Oroglwk9HgBkXfIC405WWJP8zwEtuq61dPZ+JTTe3SuUaAbd5wUQE0CCVnUHaoFr2YZOEC+0QFycaAVEbkj+MosDNUSPfRDZTOddsmS0ncTSoja304YSVtrxWjYgZ5ma2W6Kb4mCfJ49dmRZuCII7HM21ohZZUVsDLshaIEdGh4AGLJwfmW8asp75PMPTmhKudFPc3VzU1P/hOcZIaC5+Z9t2kJ7Eo07Z25yefwApKB83Mp1TuoJOZbrVRLzVB0ZaVuCIddficDerIGnpU1oT+KkEBJKEVznUfhaXjwnKE0aaGm5/TExyTo9Ul876QCozLPZqIQnHIaLIkHNpDVpTxqoKrx/mKWYgItR1Vc8fUDD09E5mWwz0yBK2I32yFtzjrCSPBadg+FeT1PxSgKrqlu1m2trj5RPynCr9wQkUOQQTVWV0iBXOTZzQhsVVP6q5kFutu6ab8P787U1W83DirQN0lV8Q4DqhOMH1ICjt7/MPlbmG+e0V1Q5eLCTsfqtvixw7HvMkVEvNTZAcyTwP5jws29+Hm6yfoKhuQmZOAQQN96KTejv2bu9bpifG7SoB32NNdWwVzg+06hUD6bophouuezPRR+9fcxu+OmX9Wa9BFvfA729xiLujnTpIrRpJQlUqAkitCoCsdwhfCQP8ii2fqJQBg5ZIcQ3Nkne ZSoio89i YurORs4qZtkHnIkBfklmfRlhLfzJuXVa3ccM4IPYr2rKS0ehILma6ScxzLwb8kQQUP7FmbcmcUv3fHaDWvnCTe/FJ39t0S16IpGJm+/zC3BKMSpOxp3mq7A3sIQVUecgtYZadzO7nmZ8Gf2lQybTMt63XzVrZcp1MZAmqV8va9iUEkO8ReI4iAvsm7E86irVaprsEi8v+0yWZQ5afYbgqIFpwuckqS6vgQmdxaOTFE72+n8j5Ri/WgvprEtmWghGQ/xY32I2IM+X82lo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000449, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Jonathan noticed that the target list setup is not unwound completely upon error. Undo all the setup in the 'err_decrement:' exit path. Fixes: 27b3f8d13830 ("cxl/region: Program target lists") Reported-by: Jonathan Cameron Link: http://lore.kernel.org/r/20230208123031.00006990@Huawei.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Vishal Verma Reviewed-by: Ira Weiny --- drivers/cxl/core/region.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 040bbd39c81d..ae7d3adcd41a 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1347,6 +1347,8 @@ static int cxl_region_attach(struct cxl_region *cxlr, err_decrement: p->nr_targets--; + cxled->pos = -1; + p->targets[pos] = NULL; err: for (iter = ep_port; !is_cxl_root(iter); iter = to_cxl_port(iter->dev.parent)) From patchwork Fri Feb 10 09:06:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AD42C64EC5 for ; Fri, 10 Feb 2023 09:06:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0BEFD6B0123; Fri, 10 Feb 2023 04:06:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F3AE66B0125; Fri, 10 Feb 2023 04:06:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8F2E280003; Fri, 10 Feb 2023 04:06:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A99036B0123 for ; Fri, 10 Feb 2023 04:06:20 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 55E5D1C6EB4 for ; Fri, 10 Feb 2023 09:06:20 +0000 (UTC) X-FDA: 80450800920.03.4866412 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf24.hostedemail.com (Postfix) with ESMTP id 15530180002 for ; Fri, 10 Feb 2023 09:06:17 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HjqGL8Fn; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019978; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6+B/NurnkO1gFFMola1RLz46jU3golVCaNgR+4ekC0k=; b=p04zucvyNsmmqHYUB03cZeBMWbBn911Ni26YOS3XAfEp9NVxYNjW8mibaTd8yM9bRMleft pftdq8hVw3Ztmg6b/Nl4m14Iwr7RetJPRDQ2Vy2gvAndapUQIdZomxjiobtzZm80eTDVrB eC4z7hv1kWfnt5HCig2NDFKV6/4hPPA= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HjqGL8Fn; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019978; a=rsa-sha256; cv=none; b=4qz7F22Qv9aduVNb9dw5ygRdMy7UzVvqRFi/IMJZKL/W8Q+eU2iv1PBrboOTgbgLBWKDCc 6a9jPhQ5uadzrJOOHCPSOFTUaVyfF2kuEJVTW5E9AZOsUv0qQahhKDJvPl53Oe7kOoe7Yf TH/yq3D8pyKBdEGT601ABuPS6VgLOpw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019978; x=1707555978; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tyAA/0gIK0IYL4MQ6rj9f+aiydACvqCY4T+pqHWSbEY=; b=HjqGL8Fntiusjv/cwnqTzx5oj3CTYgFEGwVUscIcJbKEPz/riqlJwBGO BV97dvJxVm0jI7h26kXQ8jp+E3Zk1Mbh6J8gJh4fekHk820hJ1lqS0+OK GS9BScKmlQ4LEyFeTAWcfcadvFBm4jFmyNdmZz7di4MsIC2ilyoqhXfGe TwdcJNlAkjCWO/Sz6VpODCoKCIG1Q9W9XGaLBcZFHT0eQBe/mXk5CCsRo 2mrQur1B6z8Y5emOZVzHMw4X/NvPmBZp8DeTCozW7QX51HZpTLnYFcLzm wxJn4cBoN2gZII9MX6DVqQdgZeJXRfLMdZzTScFmFC2uTqoJ3FCgczrSk g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329003042" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="329003042" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:16 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="913463794" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="913463794" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:16 -0800 Subject: [PATCH v2 09/20] cxl/region: Move region-position validation to a helper From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:15 -0800 Message-ID: <167601997584.1924368.4615769326126138969.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: 15530180002 X-Stat-Signature: 9oe4g9jpdnhmquomnrydeqzwrxesewfn X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1676019977-748429 X-HE-Meta: U2FsdGVkX1/ONRB/nlV6zvwq3ZcCRI4rwTfrhltU9xXuzaUIr5lPGcfljp6YX1yvJ8kuZ028xzh/Y76k6oWM35jQG+huymBRzo+KWD0Nxv5ekFNP2x6Es+3sIB47B2qGVnRI+vJcpX3RANl3cFBXKwoKFvQ5gYY+K8WshoVUunJYVns6v0Yyfdgx0PS24+qIm3i6IhrupzRVlF2Y8PJRaXRtdiNAwkZ9ZZp5mDI+H5nty1bfsLywsV7C7QAOOlf+lWRcOrJPa2VVpmdslU3/C3jlhlTRM1wS8fPlTokLa/Q66c6L7kgMHCjgjTXUPZYtVhTE8a0JsD4G5gkjCC1L6tE4O4IVHLXUJaA4Ek3uXYgpFeCRbKahmLWYlu7otQHT6PuWT67ShuYmlC2c+GlHGPBoAPwEHvklW/RfDqGIXhsHi7H9e7xtHd1lp0aoXGBAK2j5FhL8reROBksBvKNDZG7KMBe4+/I7Ia70yH+YtwOHG8L4rwkjeRquvRdGkBYA1g3ReDGlgqW19NhWU/JslJ1lSdjtsR5RtLoDVJBYlwXCrwksJT2/prjtEnhW9+1iUfqxaZIsdMwJUEcnDntWzsxzYNGdDrWdMfT7A8PSCRLOAFH2aeDI1TltaFw2Bw61L4rJtJ7emSI7YlWiV4Cr/Utx1dnpyoWHeFmqOvasOKNuXcvQ8nPN5KXast0ToiCzfJIdDgwOW8YLY/PBBO2qXAxSGZbxD0DOeujYRlgR/DdV06+1fCrKQQbIc7SPONCXQFdUE9/3vcmswOhmA5zs5+GS2BioKoyMNsb78XnJFBvbGh6I81nt7xV3gCZuDOoRa3PNs1LgVyTeN+tMgSsz6JCXXtr54uarrHRzBwIWeLPpLMXPd+v9OUJ6r+TaIhfIAOh7wE2TIBoLaJHmIVZlnPOv7yeGCymg0ZntXzTiqEG8i7+3/wuXi+kcA3jOPS5FWdYqptJs/IqavxICOC6 CqDQG3aQ oU259KgCisW2ShSYO9o7h10MbFHl9b8/UUYhxZJUvzIj8vZkwILkNNHyfCZnjyF78fCrxJrPVcuyJ9YsH0emmMbJK8LzTIcP8Ki8gRNn7SEpNSPX/0Iadws7nm60+bO3ARwDWWs6HBj8x2R0OFuJ3vj/eeC1MJKddZf4HHO9FmItUl2uGO0CjXj7pwTz3gWgpTDxtDIsZRWY/slc6P8vkF77yt7UC7qgDGwNXhu4AfNm8WX+oO/BXC5twqPa8EldqJVO+6RUPj8yaXyI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for region autodiscovery, that needs all devices discovered before their relative position in the region can be determined, consolidate all position dependent validation in a helper. Recall that in the on-demand region creation flow the end-user picks the position of a given endpoint decoder in a region. In the autodiscovery case the position of an endpoint decoder can only be determined after all other endpoint decoders that claim to decode the region's address range have been enumerated and attached. So, in the autodiscovery case endpoint decoders may be attached before their relative position is known. Once all decoders arrive, then positions can be determined and validated with cxl_region_validate_position() the same as user initiated on-demand creation. Reviewed-by: Vishal Verma Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564538779.847146.8356062886811511706.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- drivers/cxl/core/region.c | 119 +++++++++++++++++++++++++++++---------------- 1 file changed, 76 insertions(+), 43 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index ae7d3adcd41a..691605f1e120 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1211,35 +1211,13 @@ static int cxl_region_setup_targets(struct cxl_region *cxlr) return 0; } -static int cxl_region_attach(struct cxl_region *cxlr, - struct cxl_endpoint_decoder *cxled, int pos) +static int cxl_region_validate_position(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, + int pos) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); - struct cxl_port *ep_port, *root_port, *iter; struct cxl_region_params *p = &cxlr->params; - struct cxl_dport *dport; - int i, rc = -ENXIO; - - if (cxled->mode != cxlr->mode) { - dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", - dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); - return -EINVAL; - } - - if (cxled->mode == CXL_DECODER_DEAD) { - dev_dbg(&cxlr->dev, "%s dead\n", dev_name(&cxled->cxld.dev)); - return -ENODEV; - } - - /* all full of members, or interleave config not established? */ - if (p->state > CXL_CONFIG_INTERLEAVE_ACTIVE) { - dev_dbg(&cxlr->dev, "region already active\n"); - return -EBUSY; - } else if (p->state < CXL_CONFIG_INTERLEAVE_ACTIVE) { - dev_dbg(&cxlr->dev, "interleave config missing\n"); - return -ENXIO; - } + int i; if (pos < 0 || pos >= p->interleave_ways) { dev_dbg(&cxlr->dev, "position %d out of range %d\n", pos, @@ -1278,6 +1256,71 @@ static int cxl_region_attach(struct cxl_region *cxlr, } } + return 0; +} + +static int cxl_region_attach_position(struct cxl_region *cxlr, + struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder *cxled, + const struct cxl_dport *dport, int pos) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_port *iter; + int rc; + + if (cxlrd->calc_hb(cxlrd, pos) != dport) { + dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + dev_name(&cxlrd->cxlsd.cxld.dev)); + return -ENXIO; + } + + for (iter = cxled_to_port(cxled); !is_cxl_root(iter); + iter = to_cxl_port(iter->dev.parent)) { + rc = cxl_port_attach_region(iter, cxlr, cxled, pos); + if (rc) + goto err; + } + + return 0; + +err: + for (iter = cxled_to_port(cxled); !is_cxl_root(iter); + iter = to_cxl_port(iter->dev.parent)) + cxl_port_detach_region(iter, cxlr, cxled); + return rc; +} + +static int cxl_region_attach(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, int pos) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_region_params *p = &cxlr->params; + struct cxl_port *ep_port, *root_port; + struct cxl_dport *dport; + int rc = -ENXIO; + + if (cxled->mode != cxlr->mode) { + dev_dbg(&cxlr->dev, "%s region mode: %d mismatch: %d\n", + dev_name(&cxled->cxld.dev), cxlr->mode, cxled->mode); + return -EINVAL; + } + + if (cxled->mode == CXL_DECODER_DEAD) { + dev_dbg(&cxlr->dev, "%s dead\n", dev_name(&cxled->cxld.dev)); + return -ENODEV; + } + + /* all full of members, or interleave config not established? */ + if (p->state > CXL_CONFIG_INTERLEAVE_ACTIVE) { + dev_dbg(&cxlr->dev, "region already active\n"); + return -EBUSY; + } else if (p->state < CXL_CONFIG_INTERLEAVE_ACTIVE) { + dev_dbg(&cxlr->dev, "interleave config missing\n"); + return -ENXIO; + } + ep_port = cxled_to_port(cxled); root_port = cxlrd_to_port(cxlrd); dport = cxl_find_dport_by_dev(root_port, ep_port->host_bridge); @@ -1288,13 +1331,6 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -ENXIO; } - if (cxlrd->calc_hb(cxlrd, pos) != dport) { - dev_dbg(&cxlr->dev, "%s:%s invalid target position for %s\n", - dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), - dev_name(&cxlrd->cxlsd.cxld.dev)); - return -ENXIO; - } - if (cxled->cxld.target_type != cxlr->type) { dev_dbg(&cxlr->dev, "%s:%s type mismatch: %d vs %d\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), @@ -1318,12 +1354,13 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -EINVAL; } - for (iter = ep_port; !is_cxl_root(iter); - iter = to_cxl_port(iter->dev.parent)) { - rc = cxl_port_attach_region(iter, cxlr, cxled, pos); - if (rc) - goto err; - } + rc = cxl_region_validate_position(cxlr, cxled, pos); + if (rc) + return rc; + + rc = cxl_region_attach_position(cxlr, cxlrd, cxled, dport, pos); + if (rc) + return rc; p->targets[pos] = cxled; cxled->pos = pos; @@ -1349,10 +1386,6 @@ static int cxl_region_attach(struct cxl_region *cxlr, p->nr_targets--; cxled->pos = -1; p->targets[pos] = NULL; -err: - for (iter = ep_port; !is_cxl_root(iter); - iter = to_cxl_port(iter->dev.parent)) - cxl_port_detach_region(iter, cxlr, cxled); return rc; } From patchwork Fri Feb 10 09:06:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 532F5C636CD for ; Fri, 10 Feb 2023 09:06:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E78D66B0125; Fri, 10 Feb 2023 04:06:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E031C6B0127; Fri, 10 Feb 2023 04:06:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC9AE6B0128; Fri, 10 Feb 2023 04:06:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9C8606B0125 for ; Fri, 10 Feb 2023 04:06:25 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4D48941526 for ; Fri, 10 Feb 2023 09:06:25 +0000 (UTC) X-FDA: 80450801130.20.3F1320A Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf24.hostedemail.com (Postfix) with ESMTP id 14EFD180002 for ; Fri, 10 Feb 2023 09:06:22 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="DOw/R0nU"; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019983; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=R9IdUqZqUxZIFEXVIPczGCz0YkXfxkHqRn8UN4DeZdo=; b=2Lea1+hr12enuoEnBgjoQJ5O/YHwBrg+cxmI7LSeoHf7g2zRLHewMNEtNI/Br1MTNiIUUo Nq6wt6Kc8bEeUR9Hpc/bdNIrZL1RtAqgbi9q2GiG4AtlIBYifWEGDEO/xXt/FC9XJhmyiI qFwsTpUqz+hphHKqc2xYOG+NfKmUYs0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="DOw/R0nU"; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019983; a=rsa-sha256; cv=none; b=nJYmLCUAwClGxHWJrN3DPaZF/xMOuu6CE6grIlF04GUR3hAyr9/4tHo2znN+5n1M89pelu B0ILOPqWiwl0e8zmHtw1eewuJ44U3J4y+1E8msOu48N/cisUk2zkdo0EmPVuQ2BTARr+7o gGCR1k3OxEbODlgB1KUeQQdKjoiptu0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019983; x=1707555983; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8q/1cNxswZmepEnA6xfwY56HjCTVePeFSsIba+FKR6Y=; b=DOw/R0nUF9DorjxeXGLtVECIWcqagkUPm/VDTElnT0j///4Vh2r+3vRd wINVuTpqXuKiqd6fRHkvjCn2SokTUcTO2Sy6MfNVy2u4gtaBD1i8mEXfi tCcI/HI1PalcIy5Hg8Ck1ofUFSABTchhU2Upi9gdPvq9U1gDu6nBpIdRN 4omGyeSScUtw511ZtgmatCzf+jI0DRt6Jyc6hcDITuD+4UyKa95XXJhKo qqGT9/J+s4q+Ak0JIuqtX0u6MHwLX2IQcgUVxcQv0vVxGr8gSQFSnGSIP 8ure2TMjxiJcD4R0471t2yDS/Pm9MswXmWIZgmd5b5pIR/z2XCVSCK3xa A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329003052" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="329003052" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:22 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="913463813" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="913463813" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:22 -0800 Subject: [PATCH v2 10/20] kernel/range: Uplevel the cxl subsystem's range_contains() helper From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Kees Cook , Vishal Verma , Jonathan Cameron , Dave Jiang , Gregory Price , Ira Weiny , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:21 -0800 Message-ID: <167601998163.1924368.6067392174077323935.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: 14EFD180002 X-Stat-Signature: fssqjshhcdpkedenf9ns6kkg5e46eb3z X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1676019982-371350 X-HE-Meta: U2FsdGVkX1+Mt50cV/clTN99ifXYAGpoO8m8AIumsnoFNwxCPChQL5qVzKOZJWwci7wd373/y89pFNkgo19Pm8RX6IUg5GYb+oXk53QCADopv3ADjiML+g3SDN3xSKmAfNR5d9ClIabdZAIGEkSimzANRvoohZF+kXgSTfRfWsXTFrsBuuGxZ3O+w5qq3FDSj8JID3KCqq0n4LqF7Y1JRb/UElWns8RZvzzaUqnDR6ZQMm/dieEU3WtTgtM37OuU8Nn290abxIhLGXqhTALwCpOII+R9r9jjEqPCIFWm06AqsYI3PZ42iD13LHbHoqA5NtUfQWneXvyvxAcGAgFq/e7rMeKzshQIzb8C912T3R+QDBvWhPebj2GKXZzWtQZ58br1p6+mL+hp6dRX2UclSIN16aGHeFZ222Zi0uutHj/Ep30dJr9Xl6oVGn79LbYFEMkj97NBb+3xeoa2h6bRRFRf0jGhIjB8E+2Kgh+w/04KwXwJXQyeY/DP/icvRAGjOOWiprO8XDJ32YTbTCVysNpkUlP83BGugc5f+N01MVsnrrWXzehdsvxS3uFQjN3QrhMiIWCxJEfLvQ+M7ibJSO7tMyzD05bzl77RZ/3LUJ7CpEe6uy8dnyzT2ZJf+l/cgo6nmDEcztwCXL6zgPEjrD1zsHIi3EbIWIV3j/sBMLsvQGO3DJ12hPOj+NcAzB3HtGsflws3K09C1XDuTVSNVyBBvtVRI3ull7FOTO7rj132EDaIrBRb09ELWdVREf3kZSZOAnz6Y42I0uxY7/IiYnafKyaf5vk1sdGDj8f+vHUr8u6kD0s4H0n/yuIssDvwVPFt5q1z94ETuGiVbjS7Nj9MkDoTh0kaYk+/lX1hH1A/3J220q5Wt3cGSNGOfqOWh+6aqUpcvpUj+K8yIatWk167qDEp5i5DYJgcwfcVw7YY8FyA7X2+kQhO/eyiDNHA5xpJwVR/mIkP7eJ890V h5qxEZus hXWaHqZV7i/MVdvS7pivjJ6/uFewrLaxnrVH+me8/J0GiZaVWO3NpOh3T/K8cC7HYdSLpWVwQOw92GGge06+HbLSjuQIQxY3Cs3MAWEO5fsrUc4M9IYkBhDjs/4/Q55WWfCHL0B5tDYBWXBhQZ2Ui5/Z6gfuXuk0invSVQbaeXAzM7/oXrbVptDY89dEaHbnZNgbEPBgFT8Ks6sN/GSrhQMaVhbUw+Nk1LA8jmADj1KKfGwCuaKKHmFZ+AfJK2SMP7VI5PDm8NKwP0LxIRDTzmlICeYng2zC11j+8L1AjzMki9CzjvMFNiN0r9ecN3kDk0W8X8M058omPJxPIauaqmsq649FoBFVYHkOH+bs+XXLVg5sRMVX7VhzN4lt4JDcJOC0oJ5IY57ovigm0+S/bij2UBO3DeDsrjOlOrnbQFC0SRxiWpRCGkJ+wlUuoAhdhI6NqSn4uXJTTiwU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In support of the CXL subsystem's use of 'struct range' to track decode address ranges, add a common range_contains() implementation with identical semantics as resource_contains(); The existing 'range_contains()' in lib/stackinit_kunit.c is namespaced with a 'stackinit_' prefix. Cc: Kees Cook Reviewed-by: Vishal Verma Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Gregory Price Reviewed-by: Ira Weiny Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564539327.847146.788601375229324484.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- drivers/cxl/core/pci.c | 5 ----- include/linux/range.h | 5 +++++ lib/stackinit_kunit.c | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 1d1492440287..9ed2120dbf8a 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -214,11 +214,6 @@ static int devm_cxl_enable_mem(struct device *host, struct cxl_dev_state *cxlds) return devm_add_action_or_reset(host, clear_mem_enable, cxlds); } -static bool range_contains(struct range *r1, struct range *r2) -{ - return r1->start <= r2->start && r1->end >= r2->end; -} - /* require dvsec ranges to be covered by a locked platform window */ static int dvsec_range_allowed(struct device *dev, void *arg) { diff --git a/include/linux/range.h b/include/linux/range.h index 274681cc3154..7efb6a9b069b 100644 --- a/include/linux/range.h +++ b/include/linux/range.h @@ -13,6 +13,11 @@ static inline u64 range_len(const struct range *range) return range->end - range->start + 1; } +static inline bool range_contains(struct range *r1, struct range *r2) +{ + return r1->start <= r2->start && r1->end >= r2->end; +} + int add_range(struct range *range, int az, int nr_range, u64 start, u64 end); diff --git a/lib/stackinit_kunit.c b/lib/stackinit_kunit.c index 4591d6cf5e01..05947a2feb93 100644 --- a/lib/stackinit_kunit.c +++ b/lib/stackinit_kunit.c @@ -31,8 +31,8 @@ static volatile u8 forced_mask = 0xff; static void *fill_start, *target_start; static size_t fill_size, target_size; -static bool range_contains(char *haystack_start, size_t haystack_size, - char *needle_start, size_t needle_size) +static bool stackinit_range_contains(char *haystack_start, size_t haystack_size, + char *needle_start, size_t needle_size) { if (needle_start >= haystack_start && needle_start + needle_size <= haystack_start + haystack_size) @@ -175,7 +175,7 @@ static noinline void test_ ## name (struct kunit *test) \ \ /* Validate that compiler lined up fill and target. */ \ KUNIT_ASSERT_TRUE_MSG(test, \ - range_contains(fill_start, fill_size, \ + stackinit_range_contains(fill_start, fill_size, \ target_start, target_size), \ "stack fill missed target!? " \ "(fill %zu wide, target offset by %d)\n", \ From patchwork Fri Feb 10 09:06:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44E82C636D3 for ; Fri, 10 Feb 2023 09:06:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D41096B0127; Fri, 10 Feb 2023 04:06:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CCA076B0129; Fri, 10 Feb 2023 04:06:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B6AA56B012A; Fri, 10 Feb 2023 04:06:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 86AA16B0127 for ; Fri, 10 Feb 2023 04:06:31 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 49EA21614E8 for ; Fri, 10 Feb 2023 09:06:31 +0000 (UTC) X-FDA: 80450801382.13.00AB54D Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf24.hostedemail.com (Postfix) with ESMTP id 3B18F180015 for ; Fri, 10 Feb 2023 09:06:28 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ROTz6GR1; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019989; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AtuA4o2L5tjCl0Jt1xo6GMSwWiqm5f8u48cxLegVdnE=; b=8B8kNO4rmbK2AT9GygollS6aax3t5/h2RqFhZI0ehQGVKNFhnGrSiiWNfp3w9zlB9r9qSk ayFEafU3mCz0Tp8mtoSzWan7/BlbZtJGWziFaWzDThepTNZC6k9sV8RHywpeNQiYxbOFVg 4SmOasShtsIbJoahC7ixGxwoJfTLTwE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ROTz6GR1; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019989; a=rsa-sha256; cv=none; b=3BK5fIfpd7h2GrrzvnfjWchI5WDN9Rlzx8fH3OTg7xeO+LU3E93HfUR6uZYqcnX+i5mzJT 9deaYh/0tz76Phg83gQ64aKh7NIhcFd+1D2ndXbGZzqlOFTjEdi38tpKRHofmFpJx7cAcd cbMHBObZFPBLUexjSkXTiN2jWGapvgA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019989; x=1707555989; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QvyAdNBc9ZYsmtnrtNpMiAkkV624GUaybC07KzHXR5s=; b=ROTz6GR1lLgZ/UGNGI1F5F3KZIcEtNPAQipPFHlhx+l2ym3IZUOe/rka v21sG4kvO5KJ6PDn0qhC+au9KGGeqStppK03cg+DOkqlo7OTzC23Vd/4l e/58E5xg52nG0nCAQsivYn7qf4PEvVKpknIvheTQxL+zBJgNxcY4BzCNt KcaQLayHLJ2Ae0C6nFsesny21iqnTPUSD6urzLmSNvDFI1EZk72+GXZYC HhKB9Osht0Bnt5LepzMoIky4nYHZxlAd2S0LgM/ZQz9FsAhAn+iuPN6wh C/fZ+1BzpAFQVsG1ZllOdhtiKb+Vd8fsH4hyC4h7oJI0q85wGy3/BvJ6/ w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329003075" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="329003075" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:28 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="913463849" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="913463849" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:28 -0800 Subject: [PATCH v2 11/20] cxl/region: Enable CONFIG_CXL_REGION to be toggled From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Vishal Verma , Jonathan Cameron , Dave Jiang , Gregory Price , Fan Ni , dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:27 -0800 Message-ID: <167601998765.1924368.258370414771847699.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: 3B18F180015 X-Stat-Signature: 1k3bdrodp8oxikbokcujtxtnttoqmb1a X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1676019988-337910 X-HE-Meta: U2FsdGVkX1+YUFeKb6gu1ujBsq5yIXfbGVUYWqsdWxbPyErpwtGHy1TEmJPONfqhYFEPr9HhTj/xBnMQiVEcLxb1YdWYFw5+JK9M6NQo5RPXXD/3+tZBBivOaSxez9qu3qJn09MJCs4C0jE+yyRRTlUjadoYUs/BhtPje9BHW4xBdgiy1zn51SeJyvHtTc9P0hzSPGdvQbPyAkxeOrnxl8F2QHXyAetitihrsaKUB6+654hEC2sZhUP9K2wLgwYRwL+kAqUDTjrxEE7z3tQfRHoFQOSxnpRBzhNyfmDBDAe0ArLX6ZrHA6aRQ++2EOYQrbzlejQzi0ZoFr4TOEFcJOwjFpv1+rfzykoRvjJFMbIb+gnkPVMPdvH917RLUU+689+NVtEpAYBx6HFLnW6Zd1PBx9hRAgPcwIAsMMJS/yyy3H/RtFrjeZnWQc01jzhdEFPbFg5J7ZXKYwZBlojGqGLbP/5k8GkqTwVIp+NFkH4YWqrjjOM/UKl20UpR2ZBM3FTU0s7Z5A2N7/V2BD5ETcesns84VDGwdCFPUzIwRUDuHq1jQiCeKJouBJObCgRbpcAwgD37bXPO/yBr6JoMDZMFn573Kek+I9FCwOlFme23MNjFFDh5w4kaVJg5fqqQgwdQKxPaNyW5UKm+vzF1tiUY0Ru92CwI+vHNdYr5pS7bzvR+L1t0wPVEWF33CUjtSfmB96VnxF/6tuJBT9fMCXnPtqeqH5v8+ra8yzR6JjlhZ3AoPDQkizc/iCM1VzKbGIXuWw49hE/VUnXd4GJQ/MPKAtBdpY3AyclhTYIsuwIE+RtbJMlUuLhmQHPLr2YgnHTK+wEP9KcwmNN1lDlQoZ8alPP2mCiB2Fd427aZCBrHvODw0eqysG1dPK/ZsXo4ndTipww3zZ4QEVcRjaSm6AiAjyIq0izkkFnZiUNtNDUSXQKnDko9jkPLLJrfGJtvaVZpOB8gmUy1zYLKtc7 QhkD3i/b W4++A6OelE5zwE57VKLtcvTgSaFe6PbRYhoD7yucQ74RyGh6K085tl5FfzCQTav8Qb/vky5wjNRVV/7Pkqgvg1IuvvjKsZ2lq4IeQcNBS4Ig0LKgRltP9siUhrc6u80ERyoQgADX3YBxulmPPnWgtNguKCVU00wq+kN/nYrCQT9ouCXknIuANCpDLlWVn7F/6PMbOOU3BKnLm0zm3M6EqOwJWO0Xx+Lj5YXgtXnhi7r4SbajPLNGzSR6Pbo81CpkJ+mZ2Gty505zkK0z2eR5O617iV5rIF6yVZtvDPT5WZgaOwWq0ZH0pv7reDo3C0FRHaJezh/vPU1hTUI+DNm0dZoAL9tX9jI9AF5UnsMVwptCoWkIT0/aW14K20Flnf/pYvTZ+Gq0DVpubBJc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add help text and a label so the CXL_REGION config option can be toggled. This is mainly to enable compile testing without region support. Reviewed-by: Vishal Verma Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Gregory Price Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564539875.847146.16213498614174558767.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams --- drivers/cxl/Kconfig | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 0ac53c422c31..163c094e67ae 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -104,12 +104,22 @@ config CXL_SUSPEND depends on SUSPEND && CXL_MEM config CXL_REGION - bool + bool "CXL: Region Support" default CXL_BUS # For MAX_PHYSMEM_BITS depends on SPARSEMEM select MEMREGION select GET_FREE_REGION + help + Enable the CXL core to enumerate and provision CXL regions. A CXL + region is defined by one or more CXL expanders that decode a given + system-physical address range. For CXL regions established by + platform-firmware this option enables memory error handling to + identify the devices participating in a given interleaved memory + range. Otherwise, platform-firmware managed CXL is enabled by being + placed in the system address map and does not need a driver. + + If unsure say 'y' config CXL_REGION_INVALIDATION_TEST bool "CXL: Region Cache Management Bypass (TEST)" From patchwork Fri Feb 10 09:06:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 324D0C6379F for ; Fri, 10 Feb 2023 09:06:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE1D6280004; Fri, 10 Feb 2023 04:06:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B69F7280003; Fri, 10 Feb 2023 04:06:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E3086B012C; Fri, 10 Feb 2023 04:06:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6FA076B0129 for ; Fri, 10 Feb 2023 04:06:37 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0A22A140E8C for ; Fri, 10 Feb 2023 09:06:37 +0000 (UTC) X-FDA: 80450801634.02.97FFA9C Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf24.hostedemail.com (Postfix) with ESMTP id EBE0618001C for ; Fri, 10 Feb 2023 09:06:34 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DKFgmhDB; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676019995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LZhzV0L5VQceoAk8Yw7eRbPmAiaBjiH42qaw9YHJfuE=; b=1tpFYoNOknCaTplELn9RUDBy60E02WbVBGW9CeKDfpjBGflP8V6rOyT7I3gIuJEhgCjR6B KmQzuyw/VHPk/qq01d4eUyTbOiWGg0dJwdun3FI1HdGkUGuXxB88nGYX4L7/HJtoxofad3 bt8HdU5+6vl9iIXcoqYi/7PS28S+U1Y= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DKFgmhDB; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676019995; a=rsa-sha256; cv=none; b=Wngf4AakTRdhN1wT/do+JR98shI5Lc+yoUplGJUWaETZosien5hQ+i1BqX9FZxgHkrqANe /2y2IxnLuEcy6KizWfts3uuch5Vh+8wSwYNFqNL3FMj+16g13nbZiIxrF7akI7N/P0uBP6 JHKRULgrRb/vYFqBtYTLvyLLXseWHp4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676019995; x=1707555995; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Gr+5ktwk+FAUTV+9ED5wsHL+1owhmcENlu36isQTlcE=; b=DKFgmhDByf00cvxpChvoeCLf27/lmyUwLqTbfbZ2+qebY/snLqfoNCqm DyapwJ7Yq9zAWyCZFA8vQUaVH1GGtWhsQ8GqMHcGe9b071/avLXd1b4Fz yVKQg0uHchCsOX9suSd6gz4izQZQx0n4OB4HruKOMxxE7P4FjJHq9o3/r 56Y2aON2tddl+CRnRwRRSsJ33LRTCxdl43zEnqIeSKuw/J+qnerpktgW7 S/wNGxH4FpPzBwyjWAg/T3URsxHj00+JXZlnwpUztKdSuByaR8lJhKHa+ Xi0mR6FJvhQeDsmy7nVxgb9InCXpBVTrggqsE+nf/V7rIJtihbTPas4UJ w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329003098" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="329003098" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:34 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="913463888" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="913463888" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:34 -0800 Subject: [PATCH v2 12/20] cxl/port: Split endpoint and switch port probe From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:33 -0800 Message-ID: <167601999378.1924368.15071142145866277623.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: EBE0618001C X-Stat-Signature: hnzxj6wtm1ycofaf84y4af8fwwm4huqt X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1676019994-217366 X-HE-Meta: U2FsdGVkX195pxpqVrofbiSGaVS/OP4uxaj9EcWQky9A1NufOVfGvwD+pWNeXlkVs0eGjg66EIHFuZAwmkLP2NQBvmPFAiZrrjK0pqEPDQf5G0aXOxQ0p/Hs/Ts401vJeroaNqW2iaZc+fmZeblhAF6Nas+oNiB7RPqPoL2mzpxIRIuaZ+ApwDosbz7zltHdfui8D8PpL9M0tfyrf+dqsM64pSWypqmkYN8goOib63qyAcPhAaPoxFSde3+aO5pEInLpccqRxYcScrT7k7/Nqu+rxWkfz0hLJskIRdyAwqwD7nKR7MJsSTUQNlwOPISll4KskuUsc1LjoK+yWwDziXQrBH30LOSzcWRbwHKoAXJQC5KeqjQZA0eg44l10NcvCfB8OmYYLAHIo/0v8aTWj0mDzixbpB661vKTSiQPfIx5fyPCWEP4+Y3ENJoiLD8QLIcNhP7CwgQ2mM/v1vIFxuN0K3FzzlDAl2k8CIEj1RNKe6ESYzYfjtLd9D0sYm80I8iRHVGcQ8dJFTKPkobbplzWujOUuh+QbbwTRTUBynHA1CjSj0bg2JS7MxadIn5F2vYAYgYcYZg5bVXeUFxnxumy7gRyrmsrNhYM4pE5uEea1dI6ye3rt1M7z+MBgn2iJaKZuyM6Z3rJmNLSH+GPVsCbw6D1lycIYM0G8mHDW9GH9q32H7+0tvF01WD7p5IxkXkZHuj9NNdoBaOI4cC/3VV9jh1sgVhGa5lbiOLktTbcxuB+RKYct8tqLVX6wqnpw5r/fv84h2rSw4fqTevafQu60rgVCdMnD+WcezfifSkgBDRyFo3hslF6p4W0IFE67TYqqZWhZJaI74kk0qARgnyNgFLXUlDviNpDVu1Cd4De79XyN7bYZ0urw7D8WDtEbcLffkQcSVv7494gaqbFXCWMtdtunDt9pIi/ozfO8DC0pC4juh8NskG8h5iWP/VNOv73AqY1I1UJK6xzvy6 ZdywzhnM zf7gbCgzXZI3mwMAXNDYcqDi9bals0+ENY8PKglZS6c+VDwDfXK1Yrh5D2kdF1lhM1kgSd9tZAARhxDxGDrbRnIQtuOYA1FR/V4q0PklczYgL8dftn14FJOfWiepvpVYjvmzsuL1nyeQ5DDZSTIUmSYG5tDyNMyujej1gxqOxC7LT+vJgDt+y7Qn6wLsms5bj/l+GsVAvzCVkPE1sdGmepn4ky4gBl/QvHuz+w9JzQ2DGcIsRMwz5XZNY+0odJ2rQonFetZHu7T5utvI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Jonathan points out that the shared code between the switch and endpoint case is small. Before adding another is_cxl_endpoint() conditional, just split the two cases. Rather than duplicate the "Couldn't enumerate decoders" error message take the opportunity to improve the error messages in devm_cxl_enumerate_decoders(). Reported-by: Jonathan Cameron Link: http://lore.kernel.org/r/20230208170724.000067ec@Huawei.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Vishal Verma --- drivers/cxl/core/hdm.c | 11 ++++++-- drivers/cxl/port.c | 69 +++++++++++++++++++++++++++--------------------- 2 files changed, 47 insertions(+), 33 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index dcc16d7cb8f3..a0891c3464f1 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -826,7 +826,8 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) cxled = cxl_endpoint_decoder_alloc(port); if (IS_ERR(cxled)) { dev_warn(&port->dev, - "Failed to allocate the decoder\n"); + "Failed to allocate decoder%d.%d\n", + port->id, i); return PTR_ERR(cxled); } cxld = &cxled->cxld; @@ -836,7 +837,8 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) cxlsd = cxl_switch_decoder_alloc(port, target_count); if (IS_ERR(cxlsd)) { dev_warn(&port->dev, - "Failed to allocate the decoder\n"); + "Failed to allocate decoder%d.%d\n", + port->id, i); return PTR_ERR(cxlsd); } cxld = &cxlsd->cxld; @@ -844,13 +846,16 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) rc = init_hdm_decoder(port, cxld, target_map, hdm, i, &dpa_base); if (rc) { + dev_warn(&port->dev, + "Failed to initialize decoder%d.%d\n", + port->id, i); put_device(&cxld->dev); return rc; } rc = add_hdm_decoder(port, cxld, target_map); if (rc) { dev_warn(&port->dev, - "Failed to add decoder to port\n"); + "Failed to add decoder%d.%d\n", port->id, i); return rc; } } diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 5453771bf330..a8d46a67b45e 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -30,55 +30,64 @@ static void schedule_detach(void *cxlmd) schedule_cxl_memdev_detach(cxlmd); } -static int cxl_port_probe(struct device *dev) +static int cxl_switch_port_probe(struct cxl_port *port) { - struct cxl_port *port = to_cxl_port(dev); struct cxl_hdm *cxlhdm; int rc; + rc = devm_cxl_port_enumerate_dports(port); + if (rc < 0) + return rc; - if (!is_cxl_endpoint(port)) { - rc = devm_cxl_port_enumerate_dports(port); - if (rc < 0) - return rc; - if (rc == 1) - return devm_cxl_add_passthrough_decoder(port); - } + if (rc == 1) + return devm_cxl_add_passthrough_decoder(port); cxlhdm = devm_cxl_setup_hdm(port); if (IS_ERR(cxlhdm)) return PTR_ERR(cxlhdm); - if (is_cxl_endpoint(port)) { - struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport); - struct cxl_dev_state *cxlds = cxlmd->cxlds; + return devm_cxl_enumerate_decoders(cxlhdm); +} - /* Cache the data early to ensure is_visible() works */ - read_cdat_data(port); +static int cxl_endpoint_port_probe(struct cxl_port *port) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_hdm *cxlhdm; + int rc; + + cxlhdm = devm_cxl_setup_hdm(port); + if (IS_ERR(cxlhdm)) + return PTR_ERR(cxlhdm); - get_device(&cxlmd->dev); - rc = devm_add_action_or_reset(dev, schedule_detach, cxlmd); - if (rc) - return rc; + /* Cache the data early to ensure is_visible() works */ + read_cdat_data(port); - rc = cxl_hdm_decode_init(cxlds, cxlhdm); - if (rc) - return rc; + get_device(&cxlmd->dev); + rc = devm_add_action_or_reset(&port->dev, schedule_detach, cxlmd); + if (rc) + return rc; - rc = cxl_await_media_ready(cxlds); - if (rc) { - dev_err(dev, "Media not active (%d)\n", rc); - return rc; - } - } + rc = cxl_hdm_decode_init(cxlds, cxlhdm); + if (rc) + return rc; - rc = devm_cxl_enumerate_decoders(cxlhdm); + rc = cxl_await_media_ready(cxlds); if (rc) { - dev_err(dev, "Couldn't enumerate decoders (%d)\n", rc); + dev_err(&port->dev, "Media not active (%d)\n", rc); return rc; } - return 0; + return devm_cxl_enumerate_decoders(cxlhdm); +} + +static int cxl_port_probe(struct device *dev) +{ + struct cxl_port *port = to_cxl_port(dev); + + if (is_cxl_endpoint(port)) + return cxl_endpoint_port_probe(port); + return cxl_switch_port_probe(port); } static ssize_t CDAT_read(struct file *filp, struct kobject *kobj, From patchwork Fri Feb 10 09:06:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56AE8C6379F for ; Fri, 10 Feb 2023 09:06:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2B616B012B; Fri, 10 Feb 2023 04:06:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DB3A76B012C; Fri, 10 Feb 2023 04:06:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C08846B012D; Fri, 10 Feb 2023 04:06:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 88A626B012B for ; Fri, 10 Feb 2023 04:06:43 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 37DBA12156C for ; Fri, 10 Feb 2023 09:06:43 +0000 (UTC) X-FDA: 80450801886.20.9D3CCB9 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf24.hostedemail.com (Postfix) with ESMTP id DFFD6180002 for ; Fri, 10 Feb 2023 09:06:40 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hift6rgH; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020001; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e89mlhu2ySq7nFrCLKxjgxfNJQsoVq2Ek6iOiT85WTw=; b=nrKOyobkLNANyLIZnR29aL0pcjukOn8v9CdHMbTJw10ySE2rm4/NwH3nPKvX5lcGGK8SHK 8yiZg8kAa/Z4Aad5m/vze9IgqPa6xgClv07BDDuEOkJ8g73hWAx7YOgkNGXch5WOxlL/VE XfNnb3tx0HAvEOtK+XpJW94xf2RaMYE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hift6rgH; spf=pass (imf24.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020001; a=rsa-sha256; cv=none; b=w6sPRRZhegKwZhDfAAqSS96v8f37yZozhr70mtg3PzMD3XpnQtRAFhQ6GgdQdJZOIW0Ry0 rIjlqHbV88eevet6dRfwlaEsYeXnUkGAQooMWqRIe6bzIL+QLbWJ+xPfk6i1MUow5mq7jl Z21Txmzzeo4wOYPyItxPKSFXIjMksq8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020001; x=1707556001; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xg87SwclHTz/KBBQ2o6kd8aRFgOzWf7Clc07w0ozOXQ=; b=hift6rgH9cja3HEa8kJ/aNmVzcDAYV0IqWYQuEVbH33jngSoBD9eM4LF DIgY8A2nb69hYS4fTqI+UkBU65gjA1arx41WzES/eTRO1fJz6wPzGHKG4 nvWUAhI5M9gxJN+qqBwZUOs53F6Gbt2wkckCCfpDio87Lyrd6ZqGowKje chlUaHcoJcHpCsI2dLuSUnAbkyq0M0uhhFG7ZWB3vHp5tpTh92L4ULZnH ubU+u20cOUVF4PtvdTqVmoGuhEYMuYaICuSqOPx4Zdw427HjOotxkQbpI kuHvnTDw1d33r/MxiOC6pCZmRSO+nQhyI5ttJpr1gXgAXbkxdCCZzR+9O A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="329003114" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="329003114" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:40 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="913463913" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="913463913" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:40 -0800 Subject: [PATCH v2 13/20] cxl/region: Add region autodiscovery From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:39 -0800 Message-ID: <167601999958.1924368.9366954455835735048.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspamd-Queue-Id: DFFD6180002 X-Stat-Signature: j4eztxiwziqxgg8ygjdpkkn8ytdjskuk X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1676020000-130817 X-HE-Meta: U2FsdGVkX1/OPlMX3yCWWRxyu450iLK0ptMGQyuI/LU8AwiJm5H6GVHyD+o0cbL1pJL59bxprhEoj9GkI6wHMs92WlGbY2uxChwTL3ALxzS/ksAWXy6utBmGKho8zw8J8I6TZ8XZBGie+FRN8OJgd4k3YlK66PrH2GNYZ0CSZ4IzQ5tjBHpaxmcQnj958RSMYY1Y48LyfBfnH4tyKF6VzE/uEde+hc2j3YNpp+NOzbDWjjnP9gpcjMsIZXiLtVZRBw42fJ/cdYlaicwMD4mEWNUCrjkVLFao3pRWXIpBQZOwSW/3/zuXCpvFhJoU/DsJZjJPg2yxLLTFoNu+0eJIdWVN/R5QpFjUo4a54Jkq6swQbCrDq4XJy8s1s2wwh1HpeSgCner0xMQaM8rtHPcIZDLWPf4MUQkxzsvc+5FBfT1xlCIJCVMPCNNBOqcH9kbAKFzuv4mMuBpRp7bWSWL8UIhRPij96kU7gsHwUBGlzKK3zBBYuqEmBvNBz8mMSiTu6MC0nwMeXUonCd/KcwUU9X4iwStFNerKwGG7ZvTdalpGCJdoX3s30Ab0S2z5m3EEij8hHtoRcHT1rpbYALHFYEznRZxaqe/NfDTelqlyoupYoAKPsAx0M9VGpdcBeHKaOn5oLVl7+VuBy+WbdthIVo65uB+/9mtdQqOZ7Kv3jAoqN315iBfMvVNFkFG8+SmV7gmq++Ca4JoUhzm8lPzMzgn/zH/b5ekMGInsjyw5FrND4OdWSdrAWzQhJ/d8zHrKHpynKLNBZVInrv/doxU3SbQQVq5cQu6rtG1Rp2ZWmkpDglg2PPaptgjpkLVgJG6y+dZ7WPpQ2JY/8zey3hW4y3LKRC6MLYTPl+9DacMJgOiwTbWCuxvP8PXGMsFix63zaG2uj6LIzJctuW/o2PHykPTccEeUJN0ghbp05iIVGoICeL31AZk5JYThW7FZ4/rNXgHFiX8MPwqvDiHxW/6 szIAyvYK 9Jyg2XXE8dK04z9IQP/AS0UDCOYUUvIUdQoqJP7s74Mhh6W/NA4ns6EUqAdH4ehBe9E/WP98UNM/I8CFZVzqQGer5n8uDFnYq8L6+ufzdlJ+fYM+mG5Kyep1Wj0fFWqAz02L5w8z6S60Dxguy9vV87BVd6SMnkBDlG3FjKjHaDN+DiRgb2aPHk5j71tH7eXkmv+GCCTCAg0pQ8Kt3ZrcyLdj9k2w3H8oTPo4/qM/fk0JDcogakSJLOY2iDmaUQ/lfkgHr4kow7Els1+g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Region autodiscovery is an asynchronous state machine advanced by cxl_port_probe(). After the decoders on an endpoint port are enumerated they are scanned for actively enabled instances. Each active decoder is flagged for auto-assembly CXL_DECODER_F_AUTO and attached to a region. If a region does not already exist for the address range setting of the decoder one is created. That creation process may race with other decoders of the same region being discovered since cxl_port_probe() is asynchronous. A new 'struct cxl_root_decoder' lock, @range_lock, is introduced to mitigate that race. Once all decoders have arrived, "p->nr_targets == p->interleave_ways", they are sorted by their relative decode position. The sort algorithm involves finding the point in the cxl_port topology where one leg of the decode leads to deviceA and the other deviceB. At that point in the topology the target order in the 'struct cxl_switch_decoder' indicates the relative position of those endpoint decoders in the region. >From that point the region goes through the same setup and validation steps as user-created regions, but instead of programming the decoders it validates that driver would have written the same values to the decoders as were already present. Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564540972.847146.17096178433176097831.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Vishal Verma --- drivers/cxl/core/hdm.c | 11 + drivers/cxl/core/port.c | 2 drivers/cxl/core/region.c | 497 ++++++++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxl.h | 29 +++ drivers/cxl/port.c | 48 ++++ 5 files changed, 576 insertions(+), 11 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index a0891c3464f1..8c29026a4b9d 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -676,6 +676,14 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) port->commit_end--; cxld->flags &= ~CXL_DECODER_F_ENABLE; + /* Userspace is now responsible for reconfiguring this decoder */ + if (is_endpoint_decoder(&cxld->dev)) { + struct cxl_endpoint_decoder *cxled; + + cxled = to_cxl_endpoint_decoder(&cxld->dev); + cxled->state = CXL_DECODER_STATE_MANUAL; + } + return 0; } @@ -783,6 +791,9 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, return rc; } *dpa_base += dpa_size + skip; + + cxled->state = CXL_DECODER_STATE_AUTO; + return 0; } diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 9e5df64ea6b5..59620528571a 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -446,6 +446,7 @@ bool is_endpoint_decoder(struct device *dev) { return dev->type == &cxl_decoder_endpoint_type; } +EXPORT_SYMBOL_NS_GPL(is_endpoint_decoder, CXL); bool is_root_decoder(struct device *dev) { @@ -1628,6 +1629,7 @@ struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, } cxlrd->calc_hb = calc_hb; + mutex_init(&cxlrd->range_lock); cxld = &cxlsd->cxld; cxld->dev.type = &cxl_decoder_root_type; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 691605f1e120..3f6453da2c51 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -524,7 +525,12 @@ static void cxl_region_iomem_release(struct cxl_region *cxlr) if (device_is_registered(&cxlr->dev)) lockdep_assert_held_write(&cxl_region_rwsem); if (p->res) { - remove_resource(p->res); + /* + * Autodiscovered regions may not have been able to insert their + * resource. + */ + if (p->res->parent) + remove_resource(p->res); kfree(p->res); p->res = NULL; } @@ -1105,12 +1111,35 @@ static int cxl_port_setup_targets(struct cxl_port *port, return rc; } - cxld->interleave_ways = iw; - cxld->interleave_granularity = ig; - cxld->hpa_range = (struct range) { - .start = p->res->start, - .end = p->res->end, - }; + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { + if (cxld->interleave_ways != iw || + cxld->interleave_granularity != ig || + cxld->hpa_range.start != p->res->start || + cxld->hpa_range.end != p->res->end || + ((cxld->flags & CXL_DECODER_F_ENABLE) == 0)) { + dev_err(&cxlr->dev, + "%s:%s %s expected iw: %d ig: %d %pr\n", + dev_name(port->uport), dev_name(&port->dev), + __func__, iw, ig, p->res); + dev_err(&cxlr->dev, + "%s:%s %s got iw: %d ig: %d state: %s %#llx:%#llx\n", + dev_name(port->uport), dev_name(&port->dev), + __func__, cxld->interleave_ways, + cxld->interleave_granularity, + (cxld->flags & CXL_DECODER_F_ENABLE) ? + "enabled" : + "disabled", + cxld->hpa_range.start, cxld->hpa_range.end); + return -ENXIO; + } + } else { + cxld->interleave_ways = iw; + cxld->interleave_granularity = ig; + cxld->hpa_range = (struct range) { + .start = p->res->start, + .end = p->res->end, + }; + } dev_dbg(&cxlr->dev, "%s:%s iw: %d ig: %d\n", dev_name(port->uport), dev_name(&port->dev), iw, ig); add_target: @@ -1121,7 +1150,17 @@ static int cxl_port_setup_targets(struct cxl_port *port, dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), pos); return -ENXIO; } - cxlsd->target[cxl_rr->nr_targets_set] = ep->dport; + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { + if (cxlsd->target[cxl_rr->nr_targets_set] != ep->dport) { + dev_dbg(&cxlr->dev, "%s:%s: %s expected %s at %d\n", + dev_name(port->uport), dev_name(&port->dev), + dev_name(&cxlsd->cxld.dev), + dev_name(ep->dport->dport), + cxl_rr->nr_targets_set); + return -ENXIO; + } + } else + cxlsd->target[cxl_rr->nr_targets_set] = ep->dport; inc = 1; out_target_set: cxl_rr->nr_targets_set += inc; @@ -1163,6 +1202,13 @@ static void cxl_region_teardown_targets(struct cxl_region *cxlr) struct cxl_ep *ep; int i; + /* + * In the auto-discovery case skip automatic teardown since the + * address space is already active + */ + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) + return; + for (i = 0; i < p->nr_targets; i++) { cxled = p->targets[i]; cxlmd = cxled_to_memdev(cxled); @@ -1195,8 +1241,8 @@ static int cxl_region_setup_targets(struct cxl_region *cxlr) iter = to_cxl_port(iter->dev.parent); /* - * Descend the topology tree programming targets while - * looking for conflicts. + * Descend the topology tree programming / validating + * targets while looking for conflicts. */ for (ep = cxl_ep_load(iter, cxlmd); iter; iter = ep->next, ep = cxl_ep_load(iter, cxlmd)) { @@ -1291,6 +1337,185 @@ static int cxl_region_attach_position(struct cxl_region *cxlr, return rc; } +static int cxl_region_attach_auto(struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled, int pos) +{ + struct cxl_region_params *p = &cxlr->params; + + if (cxled->state != CXL_DECODER_STATE_AUTO) { + dev_err(&cxlr->dev, + "%s: unable to add decoder to autodetected region\n", + dev_name(&cxled->cxld.dev)); + return -EINVAL; + } + + if (pos >= 0) { + dev_dbg(&cxlr->dev, "%s: expected auto position, not %d\n", + dev_name(&cxled->cxld.dev), pos); + return -EINVAL; + } + + if (p->nr_targets >= p->interleave_ways) { + dev_err(&cxlr->dev, "%s: no more target slots available\n", + dev_name(&cxled->cxld.dev)); + return -ENXIO; + } + + /* + * Temporarily record the endpoint decoder into the target array. Yes, + * this means that userspace can view devices in the wrong position + * before the region activates, and must be careful to understand when + * it might be racing region autodiscovery. + */ + pos = p->nr_targets; + p->targets[pos] = cxled; + cxled->pos = pos; + p->nr_targets++; + + return 0; +} + +static struct cxl_port *next_port(struct cxl_port *port) +{ + if (!port->parent_dport) + return NULL; + return port->parent_dport->port; +} + +static int decoder_match_range(struct device *dev, void *data) +{ + struct cxl_endpoint_decoder *cxled = data; + struct cxl_switch_decoder *cxlsd; + + if (!is_switch_decoder(dev)) + return 0; + + cxlsd = to_cxl_switch_decoder(dev); + return range_contains(&cxlsd->cxld.hpa_range, &cxled->cxld.hpa_range); +} + +static void find_positions(const struct cxl_switch_decoder *cxlsd, + const struct cxl_port *iter_a, + const struct cxl_port *iter_b, int *a_pos, + int *b_pos) +{ + int i; + + for (i = 0, *a_pos = -1, *b_pos = -1; i < cxlsd->nr_targets; i++) { + if (cxlsd->target[i] == iter_a->parent_dport) + *a_pos = i; + else if (cxlsd->target[i] == iter_b->parent_dport) + *b_pos = i; + if (*a_pos >= 0 && *b_pos >= 0) + break; + } +} + +static int cmp_decode_pos(const void *a, const void *b) +{ + struct cxl_endpoint_decoder *cxled_a = *(typeof(cxled_a) *)a; + struct cxl_endpoint_decoder *cxled_b = *(typeof(cxled_b) *)b; + struct cxl_memdev *cxlmd_a = cxled_to_memdev(cxled_a); + struct cxl_memdev *cxlmd_b = cxled_to_memdev(cxled_b); + struct cxl_port *port_a = cxled_to_port(cxled_a); + struct cxl_port *port_b = cxled_to_port(cxled_b); + struct cxl_port *iter_a, *iter_b, *port = NULL; + struct cxl_switch_decoder *cxlsd; + struct device *dev; + int a_pos, b_pos; + unsigned int seq; + + /* Exit early if any prior sorting failed */ + if (cxled_a->pos < 0 || cxled_b->pos < 0) + return 0; + + /* + * Walk up the hierarchy to find a shared port, find the decoder that + * maps the range, compare the relative position of those dport + * mappings. + */ + for (iter_a = port_a; iter_a; iter_a = next_port(iter_a)) { + struct cxl_port *next_a, *next_b; + + next_a = next_port(iter_a); + if (!next_a) + break; + + for (iter_b = port_b; iter_b; iter_b = next_port(iter_b)) { + next_b = next_port(iter_b); + if (next_a != next_b) + continue; + port = next_a; + break; + } + + if (port) + break; + } + + if (!port) { + dev_err(cxlmd_a->dev.parent, + "failed to find shared port with %s\n", + dev_name(cxlmd_b->dev.parent)); + goto err; + } + + dev = device_find_child(&port->dev, cxled_a, decoder_match_range); + if (!dev) { + struct range *range = &cxled_a->cxld.hpa_range; + + dev_err(port->uport, + "failed to find decoder that maps %#llx-%#llx\n", + range->start, range->end); + goto err; + } + + cxlsd = to_cxl_switch_decoder(dev); + do { + seq = read_seqbegin(&cxlsd->target_lock); + find_positions(cxlsd, iter_a, iter_b, &a_pos, &b_pos); + } while (read_seqretry(&cxlsd->target_lock, seq)); + + put_device(dev); + + if (a_pos < 0 || b_pos < 0) { + dev_err(port->uport, + "failed to find shared decoder for %s and %s\n", + dev_name(cxlmd_a->dev.parent), + dev_name(cxlmd_b->dev.parent)); + goto err; + } + + dev_dbg(port->uport, "%s comes %s %s\n", dev_name(cxlmd_a->dev.parent), + a_pos - b_pos < 0 ? "before" : "after", + dev_name(cxlmd_b->dev.parent)); + + return a_pos - b_pos; +err: + cxled_a->pos = -1; + return 0; +} + +static int cxl_region_sort_targets(struct cxl_region *cxlr) +{ + struct cxl_region_params *p = &cxlr->params; + int i, rc = 0; + + sort(p->targets, p->nr_targets, sizeof(p->targets[0]), cmp_decode_pos, + NULL); + + for (i = 0; i < p->nr_targets; i++) { + struct cxl_endpoint_decoder *cxled = p->targets[i]; + + if (cxled->pos < 0) + rc = -ENXIO; + cxled->pos = i; + } + + dev_dbg(&cxlr->dev, "region sort %s\n", rc ? "failed" : "successful"); + return rc; +} + static int cxl_region_attach(struct cxl_region *cxlr, struct cxl_endpoint_decoder *cxled, int pos) { @@ -1354,6 +1579,50 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -EINVAL; } + if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { + int i; + + rc = cxl_region_attach_auto(cxlr, cxled, pos); + if (rc) + return rc; + + /* await more targets to arrive... */ + if (p->nr_targets < p->interleave_ways) + return 0; + + /* + * All targets are here, which implies all PCI enumeration that + * affects this region has been completed. Walk the topology to + * sort the devices into their relative region decode position. + */ + rc = cxl_region_sort_targets(cxlr); + if (rc) + return rc; + + for (i = 0; i < p->nr_targets; i++) { + cxled = p->targets[i]; + ep_port = cxled_to_port(cxled); + dport = cxl_find_dport_by_dev(root_port, + ep_port->host_bridge); + rc = cxl_region_attach_position(cxlr, cxlrd, cxled, + dport, i); + if (rc) + return rc; + } + + rc = cxl_region_setup_targets(cxlr); + if (rc) + return rc; + + /* + * If target setup succeeds in the autodiscovery case + * then the region is already committed. + */ + p->state = CXL_CONFIG_COMMIT; + + return 0; + } + rc = cxl_region_validate_position(cxlr, cxled, pos); if (rc) return rc; @@ -2087,6 +2356,193 @@ static int devm_cxl_add_pmem_region(struct cxl_region *cxlr) return rc; } +static int match_decoder_by_range(struct device *dev, void *data) +{ + struct range *r1, *r2 = data; + struct cxl_root_decoder *cxlrd; + + if (!is_root_decoder(dev)) + return 0; + + cxlrd = to_cxl_root_decoder(dev); + r1 = &cxlrd->cxlsd.cxld.hpa_range; + return range_contains(r1, r2); +} + +static int match_region_by_range(struct device *dev, void *data) +{ + struct cxl_region_params *p; + struct cxl_region *cxlr; + struct range *r = data; + int rc = 0; + + if (!is_cxl_region(dev)) + return 0; + + cxlr = to_cxl_region(dev); + p = &cxlr->params; + + down_read(&cxl_region_rwsem); + if (p->res && p->res->start == r->start && p->res->end == r->end) + rc = 1; + up_read(&cxl_region_rwsem); + + return rc; +} + +/* Establish an empty region covering the given HPA range */ +static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder *cxled) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_port *port = cxlrd_to_port(cxlrd); + struct range *hpa = &cxled->cxld.hpa_range; + struct cxl_region_params *p; + struct cxl_region *cxlr; + struct resource *res; + int rc; + + do { + cxlr = __create_region(cxlrd, cxled->mode, + atomic_read(&cxlrd->region_id)); + } while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY); + + if (IS_ERR(cxlr)) { + dev_err(cxlmd->dev.parent, + "%s:%s: %s failed assign region: %ld\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + __func__, PTR_ERR(cxlr)); + return cxlr; + } + + down_write(&cxl_region_rwsem); + p = &cxlr->params; + if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) { + dev_err(cxlmd->dev.parent, + "%s:%s: %s autodiscovery interrupted\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + __func__); + rc = -EBUSY; + goto err; + } + + set_bit(CXL_REGION_F_AUTO, &cxlr->flags); + + res = kmalloc(sizeof(*res), GFP_KERNEL); + if (!res) { + rc = -ENOMEM; + goto err; + } + + *res = DEFINE_RES_MEM_NAMED(hpa->start, range_len(hpa), + dev_name(&cxlr->dev)); + rc = insert_resource(cxlrd->res, res); + if (rc) { + /* + * Platform-firmware may not have split resources like "System + * RAM" on CXL window boundaries see cxl_region_iomem_release() + */ + dev_warn(cxlmd->dev.parent, + "%s:%s: %s %s cannot insert resource\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), + __func__, dev_name(&cxlr->dev)); + } + + p->res = res; + p->interleave_ways = cxled->cxld.interleave_ways; + p->interleave_granularity = cxled->cxld.interleave_granularity; + p->state = CXL_CONFIG_INTERLEAVE_ACTIVE; + + rc = sysfs_update_group(&cxlr->dev.kobj, get_cxl_region_target_group()); + if (rc) + goto err; + + dev_dbg(cxlmd->dev.parent, "%s:%s: %s %s res: %pr iw: %d ig: %d\n", + dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), __func__, + dev_name(&cxlr->dev), p->res, p->interleave_ways, + p->interleave_granularity); + + /* ...to match put_device() in cxl_add_to_region() */ + get_device(&cxlr->dev); + up_write(&cxl_region_rwsem); + + return cxlr; + +err: + up_write(&cxl_region_rwsem); + devm_release_action(port->uport, unregister_region, cxlr); + return ERR_PTR(rc); +} + +int cxl_add_to_region(struct cxl_port *root, struct cxl_endpoint_decoder *cxled) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct range *hpa = &cxled->cxld.hpa_range; + struct cxl_decoder *cxld = &cxled->cxld; + struct cxl_root_decoder *cxlrd; + struct cxl_region_params *p; + struct cxl_region *cxlr; + bool attach = false; + struct device *dev; + int rc; + + dev = device_find_child(&root->dev, &cxld->hpa_range, + match_decoder_by_range); + if (!dev) { + dev_err(cxlmd->dev.parent, + "%s:%s no CXL window for range %#llx:%#llx\n", + dev_name(&cxlmd->dev), dev_name(&cxld->dev), + cxld->hpa_range.start, cxld->hpa_range.end); + return -ENXIO; + } + + cxlrd = to_cxl_root_decoder(dev); + + /* + * Ensure that if multiple threads race to construct_region() for @hpa + * one does the construction and the others add to that. + */ + mutex_lock(&cxlrd->range_lock); + dev = device_find_child(&cxlrd->cxlsd.cxld.dev, hpa, + match_region_by_range); + if (!dev) + cxlr = construct_region(cxlrd, cxled); + else + cxlr = to_cxl_region(dev); + mutex_unlock(&cxlrd->range_lock); + + if (IS_ERR(cxlr)) { + rc = PTR_ERR(cxlr); + goto out; + } + + attach_target(cxlr, cxled, -1, TASK_UNINTERRUPTIBLE); + + down_read(&cxl_region_rwsem); + p = &cxlr->params; + attach = p->state == CXL_CONFIG_COMMIT; + up_read(&cxl_region_rwsem); + + if (attach) { + int rc = device_attach(&cxlr->dev); + + /* + * If device_attach() fails the range may still be active via + * the platform-firmware memory map, otherwise the driver for + * regions is local to this file, so driver matching can't fail. + */ + if (rc < 0) + dev_err(&cxlr->dev, "failed to enable, range: %pr\n", + p->res); + } + + put_device(&cxlr->dev); +out: + put_device(&cxlrd->cxlsd.cxld.dev); + return rc; +} +EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, CXL); + static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) { if (!test_bit(CXL_REGION_F_INCOHERENT, &cxlr->flags)) @@ -2111,6 +2567,15 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) return 0; } +static int is_system_ram(struct resource *res, void *arg) +{ + struct cxl_region *cxlr = arg; + struct cxl_region_params *p = &cxlr->params; + + dev_dbg(&cxlr->dev, "%pr has System RAM: %pr\n", p->res, res); + return 1; +} + static int cxl_region_probe(struct device *dev) { struct cxl_region *cxlr = to_cxl_region(dev); @@ -2144,6 +2609,18 @@ static int cxl_region_probe(struct device *dev) switch (cxlr->mode) { case CXL_DECODER_PMEM: return devm_cxl_add_pmem_region(cxlr); + case CXL_DECODER_RAM: + /* + * The region can not be manged by CXL if any portion of + * it is already online as 'System RAM' + */ + if (walk_iomem_res_desc(IORES_DESC_NONE, + IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY, + p->res->start, p->res->end, cxlr, + is_system_ram) > 0) + return 0; + dev_dbg(dev, "TODO: hookup devdax\n"); + return 0; default: dev_dbg(&cxlr->dev, "unsupported region mode: %d\n", cxlr->mode); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ca76879af1de..c8ee4bb8cce6 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -261,6 +261,8 @@ resource_size_t cxl_rcrb_to_component(struct device *dev, * cxl_decoder flags that define the type of memory / devices this * decoder supports as well as configuration lock status See "CXL 2.0 * 8.2.5.12.7 CXL HDM Decoder 0 Control Register" for details. + * Additionally indicate whether decoder settings were autodetected, + * user customized. */ #define CXL_DECODER_F_RAM BIT(0) #define CXL_DECODER_F_PMEM BIT(1) @@ -334,12 +336,22 @@ static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) return "mixed"; } +/* + * Track whether this decoder is reserved for region autodiscovery, or + * free for userspace provisioning. + */ +enum cxl_decoder_state { + CXL_DECODER_STATE_MANUAL, + CXL_DECODER_STATE_AUTO, +}; + /** * struct cxl_endpoint_decoder - Endpoint / SPA to DPA decoder * @cxld: base cxl_decoder_object * @dpa_res: actively claimed DPA span of this decoder * @skip: offset into @dpa_res where @cxld.hpa_range maps * @mode: which memory type / access-mode-partition this decoder targets + * @state: autodiscovery state * @pos: interleave position in @cxld.region */ struct cxl_endpoint_decoder { @@ -347,6 +359,7 @@ struct cxl_endpoint_decoder { struct resource *dpa_res; resource_size_t skip; enum cxl_decoder_mode mode; + enum cxl_decoder_state state; int pos; }; @@ -380,6 +393,7 @@ typedef struct cxl_dport *(*cxl_calc_hb_fn)(struct cxl_root_decoder *cxlrd, * @region_id: region id for next region provisioning event * @calc_hb: which host bridge covers the n'th position by granularity * @platform_data: platform specific configuration data + * @range_lock: sync region autodiscovery by address range * @cxlsd: base cxl switch decoder */ struct cxl_root_decoder { @@ -387,6 +401,7 @@ struct cxl_root_decoder { atomic_t region_id; cxl_calc_hb_fn calc_hb; void *platform_data; + struct mutex range_lock; struct cxl_switch_decoder cxlsd; }; @@ -436,6 +451,13 @@ struct cxl_region_params { */ #define CXL_REGION_F_INCOHERENT 0 +/* + * Indicate whether this region has been assembled by autodetection or + * userspace assembly. Prevent endpoint decoders outside of automatic + * detection from being added to the region. + */ +#define CXL_REGION_F_AUTO 1 + /** * struct cxl_region - CXL region * @dev: This region's device @@ -699,6 +721,8 @@ struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct device *dev); #ifdef CONFIG_CXL_REGION bool is_cxl_pmem_region(struct device *dev); struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); +int cxl_add_to_region(struct cxl_port *root, + struct cxl_endpoint_decoder *cxled); #else static inline bool is_cxl_pmem_region(struct device *dev) { @@ -708,6 +732,11 @@ static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) { return NULL; } +static inline int cxl_add_to_region(struct cxl_port *root, + struct cxl_endpoint_decoder *cxled) +{ + return 0; +} #endif /* diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index a8d46a67b45e..d88518836c2d 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -30,6 +30,34 @@ static void schedule_detach(void *cxlmd) schedule_cxl_memdev_detach(cxlmd); } +static int discover_region(struct device *dev, void *root) +{ + struct cxl_endpoint_decoder *cxled; + int rc; + + if (!is_endpoint_decoder(dev)) + return 0; + + cxled = to_cxl_endpoint_decoder(dev); + if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) + return 0; + + if (cxled->state != CXL_DECODER_STATE_AUTO) + return 0; + + /* + * Region enumeration is opportunistic, if this add-event fails, + * continue to the next endpoint decoder. + */ + rc = cxl_add_to_region(root, cxled); + if (rc) + dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", + cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); + + return 0; +} + + static int cxl_switch_port_probe(struct cxl_port *port) { struct cxl_hdm *cxlhdm; @@ -54,6 +82,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport); struct cxl_dev_state *cxlds = cxlmd->cxlds; struct cxl_hdm *cxlhdm; + struct cxl_port *root; int rc; cxlhdm = devm_cxl_setup_hdm(port); @@ -78,7 +107,24 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) return rc; } - return devm_cxl_enumerate_decoders(cxlhdm); + rc = devm_cxl_enumerate_decoders(cxlhdm); + if (rc) + return rc; + + /* + * This can't fail in practice as CXL root exit unregisters all + * descendant ports and that in turn synchronizes with cxl_port_probe() + */ + root = find_cxl_root(&cxlmd->dev); + + /* + * Now that all endpoint decoders are successfully enumerated, try to + * assemble regions from committed decoders + */ + device_for_each_child(&port->dev, root, discover_region); + put_device(&root->dev); + + return 0; } static int cxl_port_probe(struct device *dev) From patchwork Fri Feb 10 09:06:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95DE6C636D3 for ; Fri, 10 Feb 2023 09:06:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26A466B012C; Fri, 10 Feb 2023 04:06:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F3D56B012E; Fri, 10 Feb 2023 04:06:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 094726B012F; Fri, 10 Feb 2023 04:06:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D08556B012C for ; Fri, 10 Feb 2023 04:06:51 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7274F1A1470 for ; Fri, 10 Feb 2023 09:06:51 +0000 (UTC) X-FDA: 80450802222.19.A2AD457 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf20.hostedemail.com (Postfix) with ESMTP id 007181C001D for ; Fri, 10 Feb 2023 09:06:47 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="BMAOWS/+"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020008; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tV88dzAlUKNImheD2bKLCBJdt5Wk/uU4dheckYl9950=; b=5nSl5oa3ceuWZbjFTfvX0wvHW9LgWjtfmLAcRzdeat7W7tbuFnl6+LJLWG0E7XwYDrSGhV IUs8Km7zt2RGYrf3nT2cgWnSGQSqSK4lJUCVmFBjeVA7eVZtl1XlHuK9UgwHz4niS5bzcI VPM7J1s5THRH/rrvxUz0XpUi2KYJUcY= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="BMAOWS/+"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020008; a=rsa-sha256; cv=none; b=CGDIDXf0SRviSi445dSim3VoeqhW6+Ly4EqRpfT6/L7al6HrwE1WI17BFp5YGtvp7VI5mQ JlFjv6qIBwbYV7SHxlPaJk3c897zo7hfVzHagJ8nbg2QL8LqpELIF4nGgCgLM49NVqYJF8 yFrXfJdli+7iV01Jvgd+pEwOi6N/KLY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020008; x=1707556008; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rZ3O3B8mdY6zXcZ47a8wBl453HTwxqIFLhvZISKfgJs=; b=BMAOWS/+7t5qZ2jhDuC187+/L9QQ8Npu+zD1rceC3NeE+XK74648E7gS d/EkhR0bxvJFKSnLaLk8zbQ4Oz3jWA1wv21BUnABLmpDfM09gZ2qyrqfv vpWZGSgXCNb+4BAur7eF+TgLzVy+CZu4bva25zl0wxOicFHK0lDn9oBIO auUZIN/o4S7OKhVrrTjoJSN390Kv7/7BaA5BbfklBYTX5yiANjdU5GPxW i5QvNFQuzcnFS0Y8vkwEojjCqVOcfdQFchQr9eGB2JR3nvb0cWdaqK6Ms EbVibYOENYbiWGRSBqgHERNtJfXo7bAJXKQl+Ba7mw/qJ1T8OsWySUr/Q w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="328062568" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="328062568" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:46 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="700392672" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="700392672" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:45 -0800 Subject: [PATCH v2 14/20] tools/testing/cxl: Define a fixed volatile configuration to parse From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:45 -0800 Message-ID: <167602000547.1924368.11613151863880268868.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 007181C001D X-Stat-Signature: dna3fwaz4wyfko9zygowpkothr3r91zc X-HE-Tag: 1676020007-165788 X-HE-Meta: U2FsdGVkX1/X5u7nZM67Y1g290qV6zYCRCwNuGSvfZDCPaIUEzXerr2w461s4uBMUka9qSta3vcRNPX6hC2agHOUUh/FbxxEKOUW2hSdMQD+dVFi/Y1KiHE6MgpB3NLYKm4gu99cHEHK6rC2lar+L3w/cVOSlUcqC4PWZt8bbRFMJUp/7Rjp93bXZWtL9ZaY8OXuWMj4FPT2bR2fZUt9T3j9YxzKNFwe9kCZdw1smpnQKcP6JccJJjUS5wyjZa4MK2GkS3Wru2Fgn28QvuPBjoBrQqjLmxmeZGVgIGXFzf598R3haqmgXjSsDwoEuHu8Sue9L1VJwUya4TF+ROD4UnRhdHcOAUf5tJrrzHgmtHB1QOohyNexTIjKkiPW5aJR0JWLGMD3KHpdvJBjn9A7ecD75R6+OnWjEw6O0PYUE9XbHo3XhTdOQsv/V+4siR5H3O6uHlDW95YH4FKj+IRk8yA6l06USfXYTu//BsMGDwSeZVXTPKhyixJrzQAGcB6d7b36Ltgq8N4bpjxCRBRvgyNlkS55W+3JX7bk7V4HFtBZyXkyWbefxU9DKSRWVbpFfcZVu6DT59iKdlFhX70wkO1zWVc5TqSC/vUCICOLg7uW8J6JIMy+Vee+/d06Gl4d3bMzabp/LESJXDbWhNCRKj4+yL8nMwH47y8lTNdta1i74guxHyZeW98JeS9O3ncYMyclvFydrt1fGLk25vR9Libkf7aNzcwB2w+mrTl1ZNcMC+gt7Om19oNUwKGBeDYuspKNw5YZqZ1g6wJcXUG8eEU/DEv0S8gKi5W2W1C/Dg+NxClwauGcscj0M2DsG1Hzn9uUw9OyjZ9w/pGRQiZctFvy9OqJZY8pXVdc5L7vwY0J/ihbJFN7OmZlgCU9Z+CyMvWjmejZKmvKSA3axi7w/QeEJAf3y8OaAPVLt9pBjFmjPU5jPizfjIll5Kzn4Sb/3ZfNT8DJGR4YZfaSWF/ E7dV/yUl lWVEhW/G9Ksgf+aR/67v629dr4EDUU+McThuZ4hsVQvR2aZhRsgHOeiBdQI7N7wooDIggY236uDtlGr1SljpHWnIL8mZ7m6Ad4ap31Rnk7/7d9s1OIpGS1vRLXOnSrhXLeoZBTcQ0wItx4UEZvo0nP5qfNtu2lEP2wHMltINtYA806nGQQz+jySgFDGYd6LaIE5lYgnPJH48coru1MoThfzZncu4+M4D/pquEsKN3AlTkMP/vDbXHezqIEKmhjRK+Hb/DMw1CJKc2UME= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Take two endpoints attached to the first switch on the first host-bridge in the cxl_test topology and define a pre-initialized region. This is a x2 interleave underneath a x1 CXL Window. $ modprobe cxl_test $ # cxl list -Ru { "region":"region3", "resource":"0xf010000000", "size":"512.00 MiB (536.87 MB)", "interleave_ways":2, "interleave_granularity":4096, "decode_state":"commit" } Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564541523.847146.12199636368812381475.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/cxl/core/core.h | 3 - drivers/cxl/core/hdm.c | 3 + drivers/cxl/core/port.c | 2 + drivers/cxl/cxl.h | 2 + drivers/cxl/cxlmem.h | 3 + tools/testing/cxl/test/cxl.c | 147 +++++++++++++++++++++++++++++++++++++++--- 6 files changed, 146 insertions(+), 14 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 5eb873da5a30..479f01da6d35 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -57,9 +57,6 @@ resource_size_t cxl_dpa_size(struct cxl_endpoint_decoder *cxled); resource_size_t cxl_dpa_resource_start(struct cxl_endpoint_decoder *cxled); extern struct rw_semaphore cxl_dpa_rwsem; -bool is_switch_decoder(struct device *dev); -struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev); - int cxl_memdev_init(void); void cxl_memdev_exit(void); void cxl_mbox_init(void); diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 8c29026a4b9d..80eccae6ba9e 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -279,7 +279,7 @@ static int __cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, return 0; } -static int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, +int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, resource_size_t base, resource_size_t len, resource_size_t skipped) { @@ -295,6 +295,7 @@ static int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, return devm_add_action_or_reset(&port->dev, cxl_dpa_release, cxled); } +EXPORT_SYMBOL_NS_GPL(devm_cxl_dpa_reserve, CXL); resource_size_t cxl_dpa_size(struct cxl_endpoint_decoder *cxled) { diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 59620528571a..b45d2796ef35 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -458,6 +458,7 @@ bool is_switch_decoder(struct device *dev) { return is_root_decoder(dev) || dev->type == &cxl_decoder_switch_type; } +EXPORT_SYMBOL_NS_GPL(is_switch_decoder, CXL); struct cxl_decoder *to_cxl_decoder(struct device *dev) { @@ -485,6 +486,7 @@ struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev) return NULL; return container_of(dev, struct cxl_switch_decoder, cxld.dev); } +EXPORT_SYMBOL_NS_GPL(to_cxl_switch_decoder, CXL); static void cxl_ep_release(struct cxl_ep *ep) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index c8ee4bb8cce6..2ac344235235 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -653,8 +653,10 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct cxl_decoder *to_cxl_decoder(struct device *dev); struct cxl_root_decoder *to_cxl_root_decoder(struct device *dev); +struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev); struct cxl_endpoint_decoder *to_cxl_endpoint_decoder(struct device *dev); bool is_root_decoder(struct device *dev); +bool is_switch_decoder(struct device *dev); bool is_endpoint_decoder(struct device *dev); struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, unsigned int nr_targets, diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index c9da3c699a21..bf7d4c5c8612 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -81,6 +81,9 @@ static inline bool is_cxl_endpoint(struct cxl_port *port) } struct cxl_memdev *devm_cxl_add_memdev(struct cxl_dev_state *cxlds); +int devm_cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, + resource_size_t base, resource_size_t len, + resource_size_t skipped); static inline struct cxl_ep *cxl_ep_load(struct cxl_port *port, struct cxl_memdev *cxlmd) diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 920bd969c554..5342f69d70d2 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -703,6 +703,142 @@ static int mock_decoder_reset(struct cxl_decoder *cxld) return 0; } +static void default_mock_decoder(struct cxl_decoder *cxld) +{ + cxld->hpa_range = (struct range){ + .start = 0, + .end = -1, + }; + + cxld->interleave_ways = 1; + cxld->interleave_granularity = 256; + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->commit = mock_decoder_commit; + cxld->reset = mock_decoder_reset; +} + +static int first_decoder(struct device *dev, void *data) +{ + struct cxl_decoder *cxld; + + if (!is_switch_decoder(dev)) + return 0; + cxld = to_cxl_decoder(dev); + if (cxld->id == 0) + return 1; + return 0; +} + +static void mock_init_hdm_decoder(struct cxl_decoder *cxld) +{ + struct acpi_cedt_cfmws *window = mock_cfmws[0]; + struct platform_device *pdev = NULL; + struct cxl_endpoint_decoder *cxled; + struct cxl_switch_decoder *cxlsd; + struct cxl_port *port, *iter; + const int size = SZ_512M; + struct cxl_memdev *cxlmd; + struct cxl_dport *dport; + struct device *dev; + bool hb0 = false; + u64 base; + int i; + + if (is_endpoint_decoder(&cxld->dev)) { + cxled = to_cxl_endpoint_decoder(&cxld->dev); + cxlmd = cxled_to_memdev(cxled); + WARN_ON(!dev_is_platform(cxlmd->dev.parent)); + pdev = to_platform_device(cxlmd->dev.parent); + + /* check is endpoint is attach to host-bridge0 */ + port = cxled_to_port(cxled); + do { + if (port->uport == &cxl_host_bridge[0]->dev) { + hb0 = true; + break; + } + if (is_cxl_port(port->dev.parent)) + port = to_cxl_port(port->dev.parent); + else + port = NULL; + } while (port); + port = cxled_to_port(cxled); + } + + /* + * The first decoder on the first 2 devices on the first switch + * attached to host-bridge0 mock a fake / static RAM region. All + * other decoders are default disabled. Given the round robin + * assignment those devices are named cxl_mem.0, and cxl_mem.4. + * + * See 'cxl list -BMPu -m cxl_mem.0,cxl_mem.4' + */ + if (!hb0 || pdev->id % 4 || pdev->id > 4 || cxld->id > 0) { + default_mock_decoder(cxld); + return; + } + + base = window->base_hpa; + cxld->hpa_range = (struct range) { + .start = base, + .end = base + size - 1, + }; + + cxld->interleave_ways = 2; + eig_to_granularity(window->granularity, &cxld->interleave_granularity); + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->flags = CXL_DECODER_F_ENABLE; + cxled->state = CXL_DECODER_STATE_AUTO; + port->commit_end = cxld->id; + devm_cxl_dpa_reserve(cxled, 0, size / cxld->interleave_ways, 0); + cxld->commit = mock_decoder_commit; + cxld->reset = mock_decoder_reset; + + /* + * Now that endpoint decoder is set up, walk up the hierarchy + * and setup the switch and root port decoders targeting @cxlmd. + */ + iter = port; + for (i = 0; i < 2; i++) { + dport = iter->parent_dport; + iter = dport->port; + dev = device_find_child(&iter->dev, NULL, first_decoder); + /* + * Ancestor ports are guaranteed to be enumerated before + * @port, and all ports have at least one decoder. + */ + if (WARN_ON(!dev)) + continue; + cxlsd = to_cxl_switch_decoder(dev); + if (i == 0) { + /* put cxl_mem.4 second in the decode order */ + if (pdev->id == 4) + cxlsd->target[1] = dport; + else + cxlsd->target[0] = dport; + } else + cxlsd->target[0] = dport; + cxld = &cxlsd->cxld; + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->flags = CXL_DECODER_F_ENABLE; + iter->commit_end = 0; + /* + * Switch targets 2 endpoints, while host bridge targets + * one root port + */ + if (i == 0) + cxld->interleave_ways = 2; + else + cxld->interleave_ways = 1; + cxld->interleave_granularity = 256; + cxld->hpa_range = (struct range) { + .start = base, + .end = base + size - 1, + }; + put_device(dev); + } +} + static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) { struct cxl_port *port = cxlhdm->port; @@ -748,16 +884,7 @@ static int mock_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) cxld = &cxled->cxld; } - cxld->hpa_range = (struct range) { - .start = 0, - .end = -1, - }; - - cxld->interleave_ways = min_not_zero(target_count, 1); - cxld->interleave_granularity = SZ_4K; - cxld->target_type = CXL_DECODER_EXPANDER; - cxld->commit = mock_decoder_commit; - cxld->reset = mock_decoder_reset; + mock_init_hdm_decoder(cxld); if (target_count) { rc = device_for_each_child(port->uport, &ctx, From patchwork Fri Feb 10 09:06:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DA03C64EC4 for ; Fri, 10 Feb 2023 09:06:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCAFF6B012E; Fri, 10 Feb 2023 04:06:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C54226B0130; Fri, 10 Feb 2023 04:06:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF4DD6B0131; Fri, 10 Feb 2023 04:06:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8811F6B012E for ; Fri, 10 Feb 2023 04:06:55 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 381DC1A05F9 for ; Fri, 10 Feb 2023 09:06:55 +0000 (UTC) X-FDA: 80450802390.24.164A785 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf20.hostedemail.com (Postfix) with ESMTP id 2BDEC1C000D for ; Fri, 10 Feb 2023 09:06:52 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=L2BSlSYB; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020013; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xORu5vY/uLlTocbR/Ujmdh107QrHuVBUFZOWrFptnF8=; b=KHdcXUtY6gfgzD+9A888draaMpS78SMqgewc7ZW1UFJtr1e9dhY2lx/wAW669nScT9P7P0 L6vuwWaQ8cXY93HI5598lUpYz2fyg1+Fwia/hq98VfT4/wiRv1o3qqosy7KVilk4ARrBNj Jm9kKVj+T2ftdQFRGIkyn2t5z4ez5mY= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=L2BSlSYB; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020013; a=rsa-sha256; cv=none; b=HCTfUFNJvOKihH9xIhpUBXMKUO8WnAccWWfWkDPIkbOsaIQUBRJ8YumZeMIux5umWiSUS/ kIpgcJ+fQ/SsfRIBIDXDjDIm46te49/8FL5PT7/SviTDDsKgi01b6w3fGDiyDn1bAH7qGY 1RTCLQH2o7xdD7Hbnl8AoRD/sNxl78k= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020013; x=1707556013; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Kiq8SiWxreJ/WplvYfP6rIo46j43k+e9EnBH+6Fzm+Q=; b=L2BSlSYB+kESKX/5othgWdaeXwqbvFYTDA3nSKCx425ekiJubiWR2IS/ UZF/hzfKT54qhEIDUUFCERkJaHY6Itk9JfI9Qh4jn/Ao6Sp42jJJ8y8Ui FMYF75+WyVUJX1/wXug49r1z0+TNRUjACY9tRj6lzYODUiVm2F07l9Smv GfbenDy/5GVOnZuWTKSi06Mh4l30rPZhxC9XHQrK51qv4dZEs2FSZeAd8 OEr3n/eBgixzS9ocRtJXOvBuNxo34iuNQcTpg7hhuYQKu4qteYSG8k0Es j5zfXWN727pNhPyv7HN1xuuZFFRE7aDUp2JJAtNU7nVWwrZU2lipMsy3g w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="328062585" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="328062585" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:52 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="700392732" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="700392732" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:51 -0800 Subject: [PATCH v2 15/20] dax/hmem: Move HMAT and Soft reservation probe initcall level From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:51 -0800 Message-ID: <167602001107.1924368.11562316181038595611.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2BDEC1C000D X-Stat-Signature: b7qp38krgab4fe4xsi3rqfc3opoguo9h X-HE-Tag: 1676020012-626218 X-HE-Meta: U2FsdGVkX1+/w9Y5G9yN8GAuNyt2hOepCLVq9rDKWqJ22mvNDrnUK53e5k0JWo/FXRSUjW7Z46R+rGZAg84Uzm39TuQkcn6zYUkilOStpdqvzB3QEey8pM1iupR/Y/W5fU2+d0UDBunR4+JFJRRm1zHIMhcqPWKArRhyuRjPX5QRNsoiIj7ADOMqEbfivGqPQDbr0jdGBzA6zr6nRtaVrJ12uRX2zKdLa1jbwyHbEigUpZAMo+1/9QAK2grSU9e/faKzTNvVhbTBQlFqrJWZ9eNqZonyNupwh0Ot30RVKcUA973zW+Da59plcGQaep8xW/bQtXJpX/Qg0GBfWcePqXBpSW6LtZWXBPQZ4GmHd2vk44SloGTO8LtIcwre6MsmwModJmVGDAgUOEdTMhpQmj7yPV5lo5nbjT2NrlzPCuSzt35U0AD+EGatiWth7uX5+Vd4R2NZulHqqpk6OrxTXlxjFAdwtcXmWH7VbO0oEIPnqJc/ma3DhxMuXYec8r4n9uOKTxxfZgOPOhIEgzaTEfs7+nWo1SeH3GIU1A6K/QNrJIWyAfYqBvauky3NRcj+c/O1L+GJ7332QWfC/dtwcW/RtPXyQrPh6Gde+5cVdwQ7Uu9ws8H2qIX5xt1xIRZwbCKeM6BJZS4jxqsdgUKx3T9ziYLYmMXoDNsUfVoSWEq0KroBdlLudr02NJZKeyJ2qGfyLKmjiWlaGp5KO8cLgK3i/IPjrE44xiUfuGbSDgXV7mATgAjk6xDvyNm2BLO3o7/xqfrgB40OlqMHBT+OUC1oPtxBpXa0EPaqFkjlEdmEPmPA0xTZOSNwcBrutGAa9EYnGSF2ySDqDwDaX8aidUaIlIeC30I/DvVdTadtjoRxQBFX6ZxLYZpsB3QG+39YuFruS4ao1PnvgYl2mje65+1d3uMDT2JPSH0JqNqDBIZ4I3UT+NRPPSiF1IKQDErHZBPnM8k4ScAHO4piBd/ UjxRq9k6 sBHMX52dbnBLlZKmzOexUm8YAORPDbZDJIIHwv7mcdCIwk8oKGde+01HwuUjAvSugiv1cA10drZ3WgaW6TEw4Itpxow3ES67R3rIYowrzGwMUkl9L1CIImjPSh62O9sZhfJu8lLJpWZC9ntZbeQQxJQuQey02SkFD0KGn4TBjqcN9W/4DvCxUGHSjDQ7U5gNTyPLeSNvzWvOwmPTBM4CTOfep9NyMOOGOsVlAkrQTNDHd23XZ3uF5R6l3nyK34NJAYUydYjItZl6cdRI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for moving more filtering of "hmem" ranges into the dax_hmem.ko module, update the initcall levels. HMAT range registration moves to subsys_initcall() to be done before Soft Reservation probing, and Soft Reservation probing is moved to device_initcall() to be done before dax_hmem.ko initialization if it is built-in. Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564542109.847146.10113972881782419363.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Dave Jiang Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/acpi/numa/hmat.c | 2 +- drivers/dax/hmem/Makefile | 3 ++- drivers/dax/hmem/device.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index 605a0c7053be..ff24282301ab 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -869,4 +869,4 @@ static __init int hmat_init(void) acpi_put_table(tbl); return 0; } -device_initcall(hmat_init); +subsys_initcall(hmat_init); diff --git a/drivers/dax/hmem/Makefile b/drivers/dax/hmem/Makefile index 57377b4c3d47..d4c4cd6bccd7 100644 --- a/drivers/dax/hmem/Makefile +++ b/drivers/dax/hmem/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_DEV_DAX_HMEM) += dax_hmem.o +# device_hmem.o deliberately precedes dax_hmem.o for initcall ordering obj-$(CONFIG_DEV_DAX_HMEM_DEVICES) += device_hmem.o +obj-$(CONFIG_DEV_DAX_HMEM) += dax_hmem.o device_hmem-y := device.o dax_hmem-y := hmem.o diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index 903325aac991..20749c7fab81 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -104,4 +104,4 @@ static __init int hmem_init(void) * As this is a fallback for address ranges unclaimed by the ACPI HMAT * parsing it must be at an initcall level greater than hmat_init(). */ -late_initcall(hmem_init); +device_initcall(hmem_init); From patchwork Fri Feb 10 09:06:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D80E3C636CD for ; Fri, 10 Feb 2023 09:07:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 529996B0130; Fri, 10 Feb 2023 04:07:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B2E96B0132; Fri, 10 Feb 2023 04:07:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3538C6B0133; Fri, 10 Feb 2023 04:07:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0A35B6B0130 for ; Fri, 10 Feb 2023 04:07:00 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B5A43A105B for ; Fri, 10 Feb 2023 09:06:59 +0000 (UTC) X-FDA: 80450802558.22.5B22C3B Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf20.hostedemail.com (Postfix) with ESMTP id B22FA1C0005 for ; Fri, 10 Feb 2023 09:06:57 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bjL7kD6w; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020017; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=00hrzvXj48UxL/UzFy6wcAqvbe+Blz0CTrlShVrrkFc=; b=WxvJCuD/hh8cpnyhOp0+1tlBoGt10zupuJJadFj7q2q5Dht+nDHtLDtFfYa0ZBJe0CgFpJ VEdXpsuwuvCBxGvDHJs/AUzrXDzHXI6r7Gwohm//DyjGokjOqlC7uiM4tMj41fpX7v9ryD tI63yZOkfmnja6vh76V71FlQtFcTeUw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=bjL7kD6w; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020017; a=rsa-sha256; cv=none; b=uhl89WlYB11arTnPrX9SjCE+qCo4lUyY1mbaeF2UWdG8qgzXYLhkHE798pXJidmDIGJvMu 53BCfej8+gc/BDo7T0px7PjC/EpWS/sj8EylbZXdr1KTpg7ovYr8F5035buMDMtFPjDFSl lhFFw6z5tmiDE3UgbifAPwlSOxlIKSM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020017; x=1707556017; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M0vAv81xsakge/rDyjzGZncp54zlO+3hec2ycZsFjWQ=; b=bjL7kD6w8GGUjIZSWicLIalQNsWqsbaRBxPspFexmkz7KKdBp1Hd420K B5c8CTo8I9JLmq/77DxGVViIQs5ZbyzC9unAxn2JPogwRqYwdGVobxa0h pxAMOHCRl5siYqZPcKztORh+KeNbTC4p5aY1h5neHghUjMTfpv3xW0FDS do3QUscHdewNrczWsaO1YlC4fXzeUwUg6MiC2cQn4VbkpaDw3xiolf/M7 JhbbhGXErkf0Q0OrI28cu/C97tN9+IfnpoPxzAK4KtQKsDYBnjhm59rNR LXZsOzgJ8Xp+ciWSSM77MeDCk7xrJ6TsVNlmJUoZZ5+lbeEMdSUo0RNhb w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="328062606" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="328062606" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:57 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="700392810" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="700392810" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:06:56 -0800 Subject: [PATCH v2 16/20] dax/hmem: Drop unnecessary dax_hmem_remove() From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , Gregory Price , Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:06:56 -0800 Message-ID: <167602001664.1924368.9102029637928071240.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B22FA1C0005 X-Stat-Signature: rbc8s5su8hdciuhdpaqpnaqtnunwon6b X-HE-Tag: 1676020017-611910 X-HE-Meta: U2FsdGVkX1/Pd/0MHY9yajRiK7VP1LZ7fJwAn4N67GxzGxoBJoeN60thz8yljfwO+KnCYSFuZGM0HepQQHx/QrU+AAk5Rz50wc3JMq9Dg6WcbYjdxWzNmiAxLDGqd/mv/gwRwUOYzBRcfAcSkLGLVEZFqLwESOKMz5qBJclM5xR6LeIsKonBiDuplxxENpUdk9PWAstWoBKQbTTkLB4WAWPG0n/e+Rf4X1Hh95V0jbckq4bQTVzFPrSq9rtvUwznEIRGijqZeskuJJ6JF71fsMLdTYR9tv+o1E60oRBEJkLRq2t+o8e/W7ETbIEuBSceWFqjekNfP/3kqkb4lh/GHHsQu0vxLnftJcb11asoRTZ1fMX5w2YpiX6vMC0EMi/RjII7SoffI2Ve/7slniK6lRWDkT1SCbAcpX8lfSdck8/r//3+QcJm3A0tpDLfRacuR9fwHZLsumUjq5MamSsNrG0WFzn1jP579kWh3whu04hNoUy8eA1RaIxAnrlSzBOIKaqWFdDrB//UNZDzOQc5o+NSZm9mhUtH/tBrDbYeWWt7lFOTrK8llWpCXks2B8vLLtWW8c3nL1Zkued0NVyqMhujDMpZNzWksODPQO1ri2Y+BrEZcHytxh7SSKaORwPR9WtKGBJTPbN7Zzsrhsrok2bDJ2ZBGI+kfIh0ksjGI9jOjUrI8UYl4NQwRXfnoQIgYsv4qa0T0sFyD6t0j/HAFwVDGDjQkX0GhW9Z+Dk7IhqkKj/w4/WdB+elejgHPm9kaiSMxFiKHZ2aITncScCIsKrKGEdaZUDBAwz6OdGK/OBXosdN9GHSXWFTDGTiqymbmQgjg2w5Xi+4+z5/pWe+a9U/s7iO6m/RICqPsqnWFqunNW/s0x/xh87Y1ijB3O280Ng92AHG6NIMTyTJDdyO2mNcRdb/wpPLNTJgVOVxWQ+5uNMNpmdolshs+8SYZBwBh40PGrjTevtfRGyUaAI vNxs9Hni EHf4H0fh7TdV8/ZesS6g5DninJJ7K1l7rPrf8BHdZkr8TkyvFlLS3UgLwEEVSycFoHK7Dsnz2M8ihyBBSwKVJzoJbuy3NbBAoHCKjv/9YCcXLU0sO4mvnlHUK58An5fLN1lBRdipofoGUYzWOF2LtUtVcHLJbrmxmCA5Q87hFd6NWw2AiuP66I7PkVt6/TGacMbAcH9iw0TxAgYRaeWG5WDK5glwEh0n0ReH05H0p5bws2bUIMLwtd39ljYpisJoX6LL10lupYtS+emxKsrLHbrVQlj6XF0HeDzn+99bAbLKLOzkPH5aDKbM3bDniezhn4R8WbmC+8pbb7XGQQlsn9y6RJWZ7CZs0wIoB/NR+GDrspOUqXPWytoXRGAacX2sDZMJTAElxP7PeFig= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Empty driver remove callbacks can just be elided. Reviewed-by: Jonathan Cameron Reviewed-by: Gregory Price Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564542679.847146.17174404738816053065.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/dax/hmem/hmem.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 1bf040dbc834..c7351e0dc8ff 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -44,15 +44,8 @@ static int dax_hmem_probe(struct platform_device *pdev) return 0; } -static int dax_hmem_remove(struct platform_device *pdev) -{ - /* devm handles teardown */ - return 0; -} - static struct platform_driver dax_hmem_driver = { .probe = dax_hmem_probe, - .remove = dax_hmem_remove, .driver = { .name = "hmem", }, From patchwork Fri Feb 10 09:07:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C1DBC05027 for ; Fri, 10 Feb 2023 09:07:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E4F26B0132; Fri, 10 Feb 2023 04:07:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 071246B0134; Fri, 10 Feb 2023 04:07:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4FCD6B0135; Fri, 10 Feb 2023 04:07:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B99646B0132 for ; Fri, 10 Feb 2023 04:07:05 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 796E0413F0 for ; Fri, 10 Feb 2023 09:07:05 +0000 (UTC) X-FDA: 80450802810.14.5633DFF Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf20.hostedemail.com (Postfix) with ESMTP id 54C921C0005 for ; Fri, 10 Feb 2023 09:07:03 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JpzUwbbI; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020023; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fWZIGDEV/0CAjDraCP4eeSVsFl8hwSA/coPiht4uvQM=; b=m2NM5pf4Bl3NkVnV5LQz2OBitycTrIuWlthimHDl3i682jqwykKd22jqHLiqSBLGltpQLG iINWAZiFC0cyUWTFB6fpPUQJlEse+wz13aCOB5kmGZKneRfjHdcvX2bbiuiQuhwVsELwi1 3/vXDJ/bb1CaZ23XTfVtqCmEHTT7RK8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JpzUwbbI; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020023; a=rsa-sha256; cv=none; b=Rx6yIGgL+YF8JkO0oszEps52NJ4+s9SexwtC6Hlb/MMCdufZi8hzaK43I/EC0E6eWtGQaA sj4gxy7OGMEt+v5+kCgBFcB8S7SplGUCh659syE0UC2ie7SlIboMIJZ1BTQe57tSXx4W5U T+ji85yq2CY8L1sNXMnBzFERDjUwrr0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020023; x=1707556023; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fsFyJVK/yMU8PeF2io/JqvsaT5S0XIoYNB1iom5H8o0=; b=JpzUwbbIq7mBgQ76cYlGttvVrllYFRZIMo2utqs504MfwtOzmIKGk11s 9zvC9MDsNi3qJaHVmgZMRyLPlUnDrQhJ03Kv127dIlpjhFW65/EAeem3L 9FLvsfClekL6jF6VFRsRJdaPjAmXbnuNjgQQX2NnNAjjDXkaXuwv5MQ1X nsp+nMGUelTNnT4+autqwJzpZRMsq7PHYjD8kQsCf7kX+77vvJJIMm5Rv CE2voMLjtpZny5aR37vgxgQWqMEJu5H4ejPdjBE2G6B41jWxKKS9s2cKI 9dsrk7Z/dXQvQBLj+cMXJpBiAgojgurO9G2qYBVaA09CAPY757JJ7wXRi w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="328062622" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="328062622" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:02 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="700392901" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="700392901" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:02 -0800 Subject: [PATCH v2 17/20] dax/hmem: Convey the dax range via memregion_info() From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Jonathan Cameron , Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:07:02 -0800 Message-ID: <167602002217.1924368.7036275892522551624.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 54C921C0005 X-Stat-Signature: ibe8jsaoxwr1xuxhiwznmszw3jb9rgh6 X-HE-Tag: 1676020023-984168 X-HE-Meta: U2FsdGVkX1+gCADtonhlkCAvjnWAWy+vPGuwpjxz0Zehm5XN83nHMHHs2oCUoEM5b9vSQlfVx+pVOy5z9LkpLE/eSKRC2w/4SptP1e+wEfhyMQ+RwpvwHREBfhMSmbX4ubLX3je4i57tCb7X3tJ1N+/WSsmUAqem9dXV7A8MTCy7UAZ+PBAPIl4BkRglSEl+r0SFAJd8wO3u2fdVW1Ry4TKAnaDsVdmS9HOJYcajiNFsuspE1gvPybR6nH8ihuC6nkAyYWThFLHX0uxNO8d+9x9CWcWiMd+dhkTHy+rnkfC5ri2zhHLifhCqSWZPDqf8qpqFy1Unx2QaQFyoNdd8Xk8x6y07xqEM8In2C+69JdXMgiMacQAOEPlj+l1VAwlJC+QhZFOgIUBliQC+EoHPfTY1gd6QzNBPRQyC5wrqLXYCIhYniEIHWiLG3vs2/dlTJhtJO0Zs9d0R1bJOvAu+dRYEV/rxoyWYQiVNcCIPLl6gTHOYYUZr39wC2Mv9TlmcsARZxdJv/PPLX5Afy75h+PKQUksNhTBFN6U228I7+ClO1Whd1NkDuzfoQotB7uDhMyl9fStS/CEse+x7yk/Vtd6V8hKpx4i8YHhyEotocggH0Vka9iPHFGt12NDAU3Z7TXDBAb03cUp9lxhvPXxId6W4mHYMyPr0UkrefHxJBtBwUuhTK+qWF6xeIHP8ZNP0mIei+JSCu84l6Lx1K1ao9cyN6tnjiwEGBrLBQYqm0fLz80y7rPdtICDel64LgslaAX5Fu4269asvLWUnQ8nGM5cf+3IIEEibayvXKeGRJ0r8vlIfW6ghUYekrAHeGsVlKwj9p139Iyqix34S+ogY+Nt7i4CgCEGYU2y2jLYqnUJ4bK10H7Qj+5Im51kX5//5OMIJdurKY84jESxH4bEinP/RpyqcdPw097xkWJLbSAhF5bJjvL/wxPm35p4gR4upqKXE6xQGF3dM8Yexf6o HfGcpdiz PZrtWA7VeME4jmEQM890ZinTSgGuABOoJO+Iog5hvghvWiKi3ZWbHfI9OsMEZFyfX/CpJCWPsNVUVuJsvn9nJAkvV5S/c8i3I8LcymSeIHVj6B9q2v9ZSksPqKANQiz6n7z7FFzYwfp8JeyfllpcCeLjeaTPTrvbuFEDHb9+uJ4Kia2u9yA1jFfAT7/lfMf3O9O3GV2QjbWsQ81IQYSnKk3n6sZSzPdnO3M1xirhEZ87sfN0ziqC+lcR744qXeXgQ1INR0P6jaqJ6IMwZUH6EIIc0jKEqm/ce15peu7p63v3OHvpCDWDe/zjORzs71A/sescYIYshJKJRQCEArh+6fS9qlw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for hmem platform devices to be unregistered, stop using platform_device_add_resources() to convey the address range. The platform_device_add_resources() API causes an existing "Soft Reserved" iomem resource to be re-parented under an inserted platform device resource. When that platform device is deleted it removes the platform device resource and all children. Instead, it is sufficient to convey just the address range and let request_mem_region() insert resources to indicate the devices active in the range. This allows the "Soft Reserved" resource to be re-enumerated upon the next probe event. Reviewed-by: Jonathan Cameron Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564543303.847146.11045895213318648441.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/dax/hmem/device.c | 37 ++++++++++++++----------------------- drivers/dax/hmem/hmem.c | 14 +++----------- include/linux/memregion.h | 2 ++ 3 files changed, 19 insertions(+), 34 deletions(-) diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index 20749c7fab81..b1b339bccfe5 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -15,15 +15,8 @@ static struct resource hmem_active = { .flags = IORESOURCE_MEM, }; -void hmem_register_device(int target_nid, struct resource *r) +void hmem_register_device(int target_nid, struct resource *res) { - /* define a clean / non-busy resource for the platform device */ - struct resource res = { - .start = r->start, - .end = r->end, - .flags = IORESOURCE_MEM, - .desc = IORES_DESC_SOFT_RESERVED, - }; struct platform_device *pdev; struct memregion_info info; int rc, id; @@ -31,55 +24,53 @@ void hmem_register_device(int target_nid, struct resource *r) if (nohmem) return; - rc = region_intersects(res.start, resource_size(&res), IORESOURCE_MEM, - IORES_DESC_SOFT_RESERVED); + rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, + IORES_DESC_SOFT_RESERVED); if (rc != REGION_INTERSECTS) return; id = memregion_alloc(GFP_KERNEL); if (id < 0) { - pr_err("memregion allocation failure for %pr\n", &res); + pr_err("memregion allocation failure for %pr\n", res); return; } pdev = platform_device_alloc("hmem", id); if (!pdev) { - pr_err("hmem device allocation failure for %pr\n", &res); + pr_err("hmem device allocation failure for %pr\n", res); goto out_pdev; } - if (!__request_region(&hmem_active, res.start, resource_size(&res), + if (!__request_region(&hmem_active, res->start, resource_size(res), dev_name(&pdev->dev), 0)) { - dev_dbg(&pdev->dev, "hmem range %pr already active\n", &res); + dev_dbg(&pdev->dev, "hmem range %pr already active\n", res); goto out_active; } pdev->dev.numa_node = numa_map_to_online_node(target_nid); info = (struct memregion_info) { .target_node = target_nid, + .range = { + .start = res->start, + .end = res->end, + }, }; rc = platform_device_add_data(pdev, &info, sizeof(info)); if (rc < 0) { - pr_err("hmem memregion_info allocation failure for %pr\n", &res); - goto out_resource; - } - - rc = platform_device_add_resources(pdev, &res, 1); - if (rc < 0) { - pr_err("hmem resource allocation failure for %pr\n", &res); + pr_err("hmem memregion_info allocation failure for %pr\n", res); goto out_resource; } rc = platform_device_add(pdev); if (rc < 0) { - dev_err(&pdev->dev, "device add failed for %pr\n", &res); + dev_err(&pdev->dev, "device add failed for %pr\n", res); goto out_resource; } return; out_resource: - __release_region(&hmem_active, res.start, resource_size(&res)); + __release_region(&hmem_active, res->start, resource_size(res)); out_active: platform_device_put(pdev); out_pdev: diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index c7351e0dc8ff..5025a8c9850b 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -15,25 +15,17 @@ static int dax_hmem_probe(struct platform_device *pdev) struct memregion_info *mri; struct dev_dax_data data; struct dev_dax *dev_dax; - struct resource *res; - struct range range; - - res = platform_get_resource(pdev, IORESOURCE_MEM, 0); - if (!res) - return -ENOMEM; mri = dev->platform_data; - range.start = res->start; - range.end = res->end; - dax_region = alloc_dax_region(dev, pdev->id, &range, mri->target_node, - PMD_SIZE, 0); + dax_region = alloc_dax_region(dev, pdev->id, &mri->range, + mri->target_node, PMD_SIZE, 0); if (!dax_region) return -ENOMEM; data = (struct dev_dax_data) { .dax_region = dax_region, .id = -1, - .size = region_idle ? 0 : resource_size(res), + .size = region_idle ? 0 : range_len(&mri->range), }; dev_dax = devm_create_dev_dax(&data); if (IS_ERR(dev_dax)) diff --git a/include/linux/memregion.h b/include/linux/memregion.h index bf83363807ac..c01321467789 100644 --- a/include/linux/memregion.h +++ b/include/linux/memregion.h @@ -3,10 +3,12 @@ #define _MEMREGION_H_ #include #include +#include #include struct memregion_info { int target_node; + struct range range; }; #ifdef CONFIG_MEMREGION From patchwork Fri Feb 10 09:07:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77B3DC636CD for ; Fri, 10 Feb 2023 09:07:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 132756B0134; Fri, 10 Feb 2023 04:07:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BB7A6B0136; Fri, 10 Feb 2023 04:07:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7863280003; Fri, 10 Feb 2023 04:07:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id BC5C26B0134 for ; Fri, 10 Feb 2023 04:07:11 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 679561C5238 for ; Fri, 10 Feb 2023 09:07:11 +0000 (UTC) X-FDA: 80450803062.05.95B4D33 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf20.hostedemail.com (Postfix) with ESMTP id 341FD1C0013 for ; Fri, 10 Feb 2023 09:07:09 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GZOR9hvZ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020029; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9rVPB7BJWp1I9rOjRhMwtbu2duvG7e6LSS/dfBPdUcU=; b=SitQPvA67DjO4QHhNJWr6x4Z2lDZodXAZVelXIyzzyLy5AJYe10oBEZmOCN3CtV+tGGD75 Y6hbQ4s/Bb0nN015TLEhoxlNGLY14l3VJQMMmMNuyQaVZa7BWw4EpEN7/L+SjxjKinOXeG Jg06sSWtGXqFoJr5ewr6TF5LcVDkCtM= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GZOR9hvZ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020029; a=rsa-sha256; cv=none; b=J68tAhTcA1Gg8JCl34MiA4ynYUmwzKNyHZDAIfY/PEtE7qtLm3WBf9a0tbyjHXyuzZEAcZ UO5WAfzKraJFg+Fm+IqmcbgFXkxgRfweuOOMcuvuyTU9jYjFyoJ1zQ7wbp91nAGqhrVz24 pcOgjDcj87SB3VMI53b0Zs3OkQ28FhY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020029; x=1707556029; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yz6quCcaJ6esd5lp1w4m7/rotepEpujCREL7aGBC9MI=; b=GZOR9hvZFKZabo8OAq4ZJ8BIIGatBl0sesQrSTqJiQKOyy5or0CncQhc BAld/n12a3gRTUjV0FzqupHUCgdmkvxyF9FwBaCL3Eunw/7UUKtftcCk5 888XHGMhkhx/zfDky+xpsmhLE1Qv9PimraHILoQf6/Q+3U7tKN8WvItUu TpjRm+wP/YGGkiwu4fdqj45DSMREfPBzkN1vRezZbJjJ/asPasipxwpTM 18wBMemjLew+wM7NJJd9GsGJWDPDnMussUj1IdD75ww1Nhv65XUVxkIi8 2z9p7JKNjV8U8cjtamQiatFdjNTXwFy5Q0hNJgRT60KwXZ/pZI7T+kCbj w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="328062645" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="328062645" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:08 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="700392965" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="700392965" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:08 -0800 Subject: [PATCH v2 18/20] dax/hmem: Move hmem device registration to dax_hmem.ko From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:07:07 -0800 Message-ID: <167602002771.1924368.5653558226424530127.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 341FD1C0013 X-Stat-Signature: z6ekkhnm49ny3trta3sr5nge66cu9hgz X-HE-Tag: 1676020029-27735 X-HE-Meta: U2FsdGVkX1/uJXxHSKn6ugASY8C5Hwn/Wu6D2pahNS8e0XgCrBSkQ8/3uXEykjD2mUzIieuepwdN0Isq3Cf5I8F9YC+ClNH9SiQFyMXVY0HaiT8w2Mrd5z9M8K27hTH58NgDb+D7h5hupYYK3GwVurLGbVcdmeXApYOxYT9cATspzA/mj4v9986+biz0+mQAIbLFcMy39RVKxdDrvd/S+2epkYxyQSKSE47dtma6z83TmoTAyZ45KH/8Bq01BM6m4Zfh8g7MZPKn8R76UKbKsHVxZE4tPw5ICwMmI6Y1VKxelWctF/lyx1he1BYP0ivb7Cj6d8p4I1cDyWfJJUr2aE7cY0BNbsKahvW3At+s0Oy0NCYe3YPTheaIHVgkT2rCYVxiGReP3ALrkeWKE06MNw5MYVxilgzkUy9dr3aTNqCPbLCVGM3/ynKXG7NtZ8Sd+I/WoMJ2KIyYCSCQU8B+/Tjt1vksc6T36QhgLnRrg7tOXBhg4OOPOSf0NKA2fHrLd5OtE+EW7kCTpA+93oJYxGQ7LlXyvneFMVK1wNBaGkwlBOgWGYYwubS5Gxap/R3+tyXrM4bYKS7DS+Ek7Iu2cnFMjgZZm2w5AnVo53GZJTjmZbi2RJBeBYehQbhRf03is+WxyUyFHg/IDz4dNN3xJwu+X6bOv0kgRvMDJZdvVbjhxkHxVucGxCvQ032Q62kZ0OQZsiAQLqGNs9mKomc4YJWXp3HyAw1aEDMNgb/GMjocHJJAJz0lmvwtRVhEfdaRTTzSbvUU66uojsZvDRnogeTWC0LjVIDqHSzBKM/g9CnZnJvh3eMO/IecT5CYzzmY1iKEqiETmLeNE3b6kPo0AsSZWXN0BHlfWOWgGiMwJv3cMBzMpLZeEQS4IpDOp4Bpr3rqSgAXTgB+KI+IfmWGmarpMVN+rfILxBRo9rcbMwBy5jEAwtivzMUw12qRX4xlvsKN7QE8L801gROQQrE 0PK+et2h bUWl3ed6ufySk5zxthFaDe8RRJtRvg9ZSn861yywl5oKyMZ4Wge9IM5RJWZZfnTUAXYKYznnR2+fYhEFn9irx3rAAiE5ymMIPvm/2rnIDxFu3Y0vDcZCZ5R6cTmIw5nf+cwkdFgHn1hDpjLvL3rEn1JVkDwjMFVgwaUL8yhUf19nYeo2VAaPnv/l+UrPEkKU3fhWOPKDvUoAH9ydA7iD6t3dCK8lugTqXj5P4QhMgKs4Rftu2lDQpuvndtqGk03ZQAami0vgXkPvBi/Z5dFRKIynzQuHtrHmqK6lhIkI0lXc2tx1hMUPBFa7Ba0+oOdWit253 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for the CXL region driver to take over the responsibility of registering device-dax instances for CXL regions, move the registration of "hmem" devices to dax_hmem.ko. Previously the builtin component of this enabling (drivers/dax/hmem/device.o) would register platform devices for each address range and trigger the dax_hmem.ko module to load and attach device-dax instances to those devices. Now, the ranges are collected from the HMAT and EFI memory map walking, but the device creation is deferred. A new "hmem_platform" device is created which triggers dax_hmem.ko to load and register the platform devices. Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564543923.847146.9030380223622044744.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/acpi/numa/hmat.c | 2 - drivers/dax/Kconfig | 2 - drivers/dax/hmem/device.c | 91 +++++++++++++++++++-------------------- drivers/dax/hmem/hmem.c | 105 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/dax.h | 7 ++- 5 files changed, 155 insertions(+), 52 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index ff24282301ab..bba268ecd802 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -718,7 +718,7 @@ static void hmat_register_target_devices(struct memory_target *target) for (res = target->memregions.child; res; res = res->sibling) { int target_nid = pxm_to_node(target->memory_pxm); - hmem_register_device(target_nid, res); + hmem_register_resource(target_nid, res); } } diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index 5fdf269a822e..d13c889c2a64 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -46,7 +46,7 @@ config DEV_DAX_HMEM Say M if unsure. config DEV_DAX_HMEM_DEVICES - depends on DEV_DAX_HMEM && DAX=y + depends on DEV_DAX_HMEM && DAX def_bool y config DEV_DAX_KMEM diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index b1b339bccfe5..f9e1a76a04a9 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -8,6 +8,8 @@ static bool nohmem; module_param_named(disable, nohmem, bool, 0444); +static bool platform_initialized; +static DEFINE_MUTEX(hmem_resource_lock); static struct resource hmem_active = { .name = "HMEM devices", .start = 0, @@ -15,71 +17,66 @@ static struct resource hmem_active = { .flags = IORESOURCE_MEM, }; -void hmem_register_device(int target_nid, struct resource *res) +int walk_hmem_resources(struct device *host, walk_hmem_fn fn) +{ + struct resource *res; + int rc = 0; + + mutex_lock(&hmem_resource_lock); + for (res = hmem_active.child; res; res = res->sibling) { + rc = fn(host, (int) res->desc, res); + if (rc) + break; + } + mutex_unlock(&hmem_resource_lock); + return rc; +} +EXPORT_SYMBOL_GPL(walk_hmem_resources); + +static void __hmem_register_resource(int target_nid, struct resource *res) { struct platform_device *pdev; - struct memregion_info info; - int rc, id; + struct resource *new; + int rc; - if (nohmem) + new = __request_region(&hmem_active, res->start, resource_size(res), "", + 0); + if (!new) { + pr_debug("hmem range %pr already active\n", res); return; + } - rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, - IORES_DESC_SOFT_RESERVED); - if (rc != REGION_INTERSECTS) - return; + new->desc = target_nid; - id = memregion_alloc(GFP_KERNEL); - if (id < 0) { - pr_err("memregion allocation failure for %pr\n", res); + if (platform_initialized) return; - } - pdev = platform_device_alloc("hmem", id); + pdev = platform_device_alloc("hmem_platform", 0); if (!pdev) { - pr_err("hmem device allocation failure for %pr\n", res); - goto out_pdev; - } - - if (!__request_region(&hmem_active, res->start, resource_size(res), - dev_name(&pdev->dev), 0)) { - dev_dbg(&pdev->dev, "hmem range %pr already active\n", res); - goto out_active; - } - - pdev->dev.numa_node = numa_map_to_online_node(target_nid); - info = (struct memregion_info) { - .target_node = target_nid, - .range = { - .start = res->start, - .end = res->end, - }, - }; - rc = platform_device_add_data(pdev, &info, sizeof(info)); - if (rc < 0) { - pr_err("hmem memregion_info allocation failure for %pr\n", res); - goto out_resource; + pr_err_once("failed to register device-dax hmem_platform device\n"); + return; } rc = platform_device_add(pdev); - if (rc < 0) { - dev_err(&pdev->dev, "device add failed for %pr\n", res); - goto out_resource; - } + if (rc) + platform_device_put(pdev); + else + platform_initialized = true; +} - return; +void hmem_register_resource(int target_nid, struct resource *res) +{ + if (nohmem) + return; -out_resource: - __release_region(&hmem_active, res->start, resource_size(res)); -out_active: - platform_device_put(pdev); -out_pdev: - memregion_free(id); + mutex_lock(&hmem_resource_lock); + __hmem_register_resource(target_nid, res); + mutex_unlock(&hmem_resource_lock); } static __init int hmem_register_one(struct resource *res, void *data) { - hmem_register_device(phys_to_target_node(res->start), res); + hmem_register_resource(phys_to_target_node(res->start), res); return 0; } diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 5025a8c9850b..e7bdff3132fa 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -3,6 +3,7 @@ #include #include #include +#include #include "../bus.h" static bool region_idle; @@ -43,8 +44,110 @@ static struct platform_driver dax_hmem_driver = { }, }; -module_platform_driver(dax_hmem_driver); +static void release_memregion(void *data) +{ + memregion_free((long) data); +} + +static void release_hmem(void *pdev) +{ + platform_device_unregister(pdev); +} + +static int hmem_register_device(struct device *host, int target_nid, + const struct resource *res) +{ + struct platform_device *pdev; + struct memregion_info info; + long id; + int rc; + + rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, + IORES_DESC_SOFT_RESERVED); + if (rc != REGION_INTERSECTS) + return 0; + + id = memregion_alloc(GFP_KERNEL); + if (id < 0) { + dev_err(host, "memregion allocation failure for %pr\n", res); + return -ENOMEM; + } + rc = devm_add_action_or_reset(host, release_memregion, (void *) id); + if (rc) + return rc; + + pdev = platform_device_alloc("hmem", id); + if (!pdev) { + dev_err(host, "device allocation failure for %pr\n", res); + return -ENOMEM; + } + + pdev->dev.numa_node = numa_map_to_online_node(target_nid); + info = (struct memregion_info) { + .target_node = target_nid, + .range = { + .start = res->start, + .end = res->end, + }, + }; + rc = platform_device_add_data(pdev, &info, sizeof(info)); + if (rc < 0) { + dev_err(host, "memregion_info allocation failure for %pr\n", + res); + goto out_put; + } + + rc = platform_device_add(pdev); + if (rc < 0) { + dev_err(host, "%s add failed for %pr\n", dev_name(&pdev->dev), + res); + goto out_put; + } + + return devm_add_action_or_reset(host, release_hmem, pdev); + +out_put: + platform_device_put(pdev); + return rc; +} + +static int dax_hmem_platform_probe(struct platform_device *pdev) +{ + return walk_hmem_resources(&pdev->dev, hmem_register_device); +} + +static struct platform_driver dax_hmem_platform_driver = { + .probe = dax_hmem_platform_probe, + .driver = { + .name = "hmem_platform", + }, +}; + +static __init int dax_hmem_init(void) +{ + int rc; + + rc = platform_driver_register(&dax_hmem_platform_driver); + if (rc) + return rc; + + rc = platform_driver_register(&dax_hmem_driver); + if (rc) + platform_driver_unregister(&dax_hmem_platform_driver); + + return rc; +} + +static __exit void dax_hmem_exit(void) +{ + platform_driver_unregister(&dax_hmem_driver); + platform_driver_unregister(&dax_hmem_platform_driver); +} + +module_init(dax_hmem_init); +module_exit(dax_hmem_exit); MODULE_ALIAS("platform:hmem*"); +MODULE_ALIAS("platform:hmem_platform*"); MODULE_LICENSE("GPL v2"); MODULE_AUTHOR("Intel Corporation"); diff --git a/include/linux/dax.h b/include/linux/dax.h index 2b5ecb591059..bf6258472e49 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -262,11 +262,14 @@ static inline bool dax_mapping(struct address_space *mapping) } #ifdef CONFIG_DEV_DAX_HMEM_DEVICES -void hmem_register_device(int target_nid, struct resource *r); +void hmem_register_resource(int target_nid, struct resource *r); #else -static inline void hmem_register_device(int target_nid, struct resource *r) +static inline void hmem_register_resource(int target_nid, struct resource *r) { } #endif +typedef int (*walk_hmem_fn)(struct device *dev, int target_nid, + const struct resource *res); +int walk_hmem_resources(struct device *dev, walk_hmem_fn fn); #endif From patchwork Fri Feb 10 09:07:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BEBAC636CD for ; Fri, 10 Feb 2023 09:07:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF6026B0136; Fri, 10 Feb 2023 04:07:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A7EBD280003; Fri, 10 Feb 2023 04:07:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D1B16B0139; Fri, 10 Feb 2023 04:07:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 56B156B0136 for ; Fri, 10 Feb 2023 04:07:18 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 18C20AB64F for ; Fri, 10 Feb 2023 09:07:18 +0000 (UTC) X-FDA: 80450803356.28.E4572BA Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf23.hostedemail.com (Postfix) with ESMTP id E9387140021 for ; Fri, 10 Feb 2023 09:07:15 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PEwG5yaK; spf=pass (imf23.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020036; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H+O0M4zatzFyKiYeQ21q8omDpceQ1caFZFs+o8Udjok=; b=DTfmXpA2tWqoccBm187KANCacF8Gj+ps4KG1BRWK70h4X19zGFgtYENjYEGdkraKGIiysK oENkTVMvdxq/J8iDjcWcpHJmkupuTJBBcRHplj03QdiSpZ58+jL3alTnVruQx28zh3/Aac FwoHeMxB59pOf1pKj+ivbpjqKvO1NIY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PEwG5yaK; spf=pass (imf23.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020036; a=rsa-sha256; cv=none; b=pydszlPp45jPVh7Cnqh3AktXRme27UM0wj3kVsFd8HxkC4mtouP7B7VA4AAngFCl2HH4X0 Ef/S+mAz2sJfCe2VR61cxyYEy9OTbsoE2f/MfG9lCy9GTpYLDzc5lAEc3xKdoNT/+//Mja TBj5ycclPBY+q7F01sICYtazvVvLqe4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020036; x=1707556036; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9J3LVeqpx/pEaciMjT7PQYSzPAi14MY2qHE7XN4c/uM=; b=PEwG5yaKnMByGJLUczG3Gxuj1rtia57WjXngOe4L+xfLG0WWkT5OO8DS EHBUsSvIskSqoNXqNszak9uwViokRBh00DI9sdKrYFHbOTmBPmbHN1Gqa f3uK1WQ7Zz31gt/OR/D47hkAPfJkB+6kyz/y9344a8Ih7yHTBaILI7iih rzruPP7zlQBdiVLIHHJXVALGt6IMsQD6/9uDlZggXlTe8l4SdfO43LtnY jZwDXQK9i0PI6l63K78cUxGzWtLPoLFMN1p2lbWAuQ1Obn4U+2HYiyDRT l/cTmIAtmVMjs/ddFfaIoe6Fopv/dP7YG91LEyzUNBp0RJVFz9d5xgCBs g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="310738836" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="310738836" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:14 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="661341213" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="661341213" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:13 -0800 Subject: [PATCH v2 19/20] dax: Assign RAM regions to memory-hotplug by default From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Michal Hocko , David Hildenbrand , Dave Hansen , Gregory Price , Fan Ni , vishal.l.verma@intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:07:13 -0800 Message-ID: <167602003336.1924368.6809503401422267885.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Stat-Signature: sw8na5za5a9ks5mm1iifntojnrr5q5mz X-Rspam-User: X-Rspamd-Queue-Id: E9387140021 X-Rspamd-Server: rspam06 X-HE-Tag: 1676020035-707582 X-HE-Meta: U2FsdGVkX1/VVuY2DwRoVqcWPTAQr8QaKbuvFF7/uhp4dT4GotdngNNY9ykmXeVQa9bU4G/fAlpoV28ftA4MeDl8tkOLTPFp/eIfmAU9pjpyrLnjrcyrL/YVEbAshTprbAdTXSs5Tf8F6IaYHqb6S+73GlWiXEfzpb2Mh/IP8TM/CwO6RvPqO0ZzaZcOCLsE9F4rrotykkkVRzMbOptKlOfmH8h3rr19Ya2UEKuSWzCkDM34+KRKydQxaW32Xabu/X/0l5sw2+YNfGGsdfNVky0xk2Bm+vvL41acXXbA8zoPYFsywSvlUPVhbB+HK0ayUnjzS6tGbE58QDzLYbBQOA/4u8tAw/2APiBMHjeUA1J/zoCxbQCR1S+PBxIDaa/vXDSmCZAJ4X5Bv7OafJAA/rfSCURDCGDpSMlMv2I1sGBe/0YDTwFAjHwJtCFwpuaCpSiGdnhRdv6rmHvuvgpMiMKzVnmOZPFzO4+5ZyMULZKcamFpa6S+nqqkndDZP5FziSkv5b7a/Qt82uJiZd1IDtlvCTRNr+RLr0rfi3iRFh6WInFLn+8TeYbMsISgCK0eMPsOCxlrBht5SUB505NeNnE+BmyN55XzXmamYfWUITT9OJIPJBO8Cq1vjjFzoN68yCPU5obomAfuyvQBQe6wMBO8zUlUTSmhaZpGV80Nuu55kAW4gzEppoVdtGFhqPdN1zG2UwFlx3yB1tb5yE3y779LL89IPFHx6WikFwqz3CnFKEzfDkG9sb5f9lpWE9IUYcU+2f/+MDVZN1n5YdVqiNGDYB+ehkpHSG9qKognkvc6DZo9F8LMdvf+Ed7kqbFPMveoNEkZSGzAXYS24WRtXvnXinPjDrnVIwFNM3Ux/7s0DK4WuK/o0NPFv+iXyONCpLW1dDPnhbi+CMjpzBZdxUOoFAWT9CVXb2474qIAXf4kg/AIISlMRhPq/h7Xxp8EF9s16SU+0pBf59l8y6H mrgmZBtZ hkWBmRBNgw0SxReoPdI/lQyt6srWnzMK2Amach7L9XlQkm86grEB1XICl8mvZOnHqyhF2RZMqXUgmQsX9olrjLLs4TQsuk0kg5SgldS/E579Cty47PqHk9Zw4qNsADLCiDIRwvAkXvZY4na1G73XHeoTt9kEGkK0/ir2QGu+5CyYHF9yF2CwZIs65YhKdlivFMOLfeICLzvoG4N7WRRMv5zZGUcxPAXfgLC0dEw+qOgrlDwBwoMNDQuzAcODlONM3Vtaez6DjmfeIllvnScQm4wYQ5Y/H0j/VO9qCODnumIEQyEdSH6XHcXhTSVtn3t3KoDhkvd0zB6YJGb6QypS/AX1zx+K0izjO3py+JPcvRAFNv0lutk8omZDCWzzRQM2EEx+0mccXzv00IwSbW8cdjG9FMtOkI87CqeFI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The default mode for device-dax instances is backwards for RAM-regions as evidenced by the fact that it tends to catch end users by surprise. "Where is my memory?". Recall that platforms are increasingly shipping with performance-differentiated memory pools beyond typical DRAM and NUMA effects. This includes HBM (high-bandwidth-memory) and CXL (dynamic interleave, varied media types, and future fabric attached possibilities). For this reason the EFI_MEMORY_SP (EFI Special Purpose Memory => Linux 'Soft Reserved') attribute is expected to be applied to all memory-pools that are not the general purpose pool. This designation gives an Operating System a chance to defer usage of a memory pool until later in the boot process where its performance properties can be interrogated and administrator policy can be applied. 'Soft Reserved' memory can be anything from too limited and precious to be part of the general purpose pool (HBM), too slow to host hot kernel data structures (some PMEM media), or anything in between. However, in the absence of an explicit policy, the memory should at least be made usable by default. The current device-dax default hides all non-general-purpose memory behind a device interface. The expectation is that the distribution of users that want the memory online by default vs device-dedicated-access by default follows the Pareto principle. A small number of enlightened users may want to do userspace memory management through a device, but general users just want the kernel to make the memory available with an option to get more advanced later. Arrange for all device-dax instances not backed by PMEM to default to attaching to the dax_kmem driver. From there the baseline memory hotplug policy (CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE / memhp_default_state=) gates whether the memory comes online or stays offline. Where, if it stays offline, it can be reliably converted back to device-mode where it can be partitioned, or fronted by a userspace allocator. So, if someone wants device-dax instances for their 'Soft Reserved' memory: 1/ Build a kernel with CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=n or boot with memhp_default_state=offline, or roll the dice and hope that the kernel has not pinned a page in that memory before step 2. 2/ Write a udev rule to convert the target dax device(s) from 'system-ram' mode to 'devdax' mode: daxctl reconfigure-device $dax -m devdax -f Cc: Michal Hocko Cc: David Hildenbrand Cc: Dave Hansen Reviewed-by: Gregory Price Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564544513.847146.4645646177864365755.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Dave Jiang Reviewed-by: Vishal Verma --- drivers/dax/Kconfig | 2 +- drivers/dax/bus.c | 53 ++++++++++++++++++++--------------------------- drivers/dax/bus.h | 12 +++++++++-- drivers/dax/device.c | 3 +-- drivers/dax/hmem/hmem.c | 12 ++++++++++- drivers/dax/kmem.c | 1 + 6 files changed, 46 insertions(+), 37 deletions(-) diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index d13c889c2a64..1163eb62e5f6 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -50,7 +50,7 @@ config DEV_DAX_HMEM_DEVICES def_bool y config DEV_DAX_KMEM - tristate "KMEM DAX: volatile-use of persistent memory" + tristate "KMEM DAX: map dax-devices as System-RAM" default DEV_DAX depends on DEV_DAX depends on MEMORY_HOTPLUG # for add_memory() and friends diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 1dad813ee4a6..012d576004e9 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -56,6 +56,25 @@ static int dax_match_id(struct dax_device_driver *dax_drv, struct device *dev) return match; } +static int dax_match_type(struct dax_device_driver *dax_drv, struct device *dev) +{ + enum dax_driver_type type = DAXDRV_DEVICE_TYPE; + struct dev_dax *dev_dax = to_dev_dax(dev); + + if (dev_dax->region->res.flags & IORESOURCE_DAX_KMEM) + type = DAXDRV_KMEM_TYPE; + + if (dax_drv->type == type) + return 1; + + /* default to device mode if dax_kmem is disabled */ + if (dax_drv->type == DAXDRV_DEVICE_TYPE && + !IS_ENABLED(CONFIG_DEV_DAX_KMEM)) + return 1; + + return 0; +} + enum id_action { ID_REMOVE, ID_ADD, @@ -216,14 +235,9 @@ static int dax_bus_match(struct device *dev, struct device_driver *drv) { struct dax_device_driver *dax_drv = to_dax_drv(drv); - /* - * All but the 'device-dax' driver, which has 'match_always' - * set, requires an exact id match. - */ - if (dax_drv->match_always) + if (dax_match_id(dax_drv, dev)) return 1; - - return dax_match_id(dax_drv, dev); + return dax_match_type(dax_drv, dev); } /* @@ -1413,13 +1427,10 @@ struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data) } EXPORT_SYMBOL_GPL(devm_create_dev_dax); -static int match_always_count; - int __dax_driver_register(struct dax_device_driver *dax_drv, struct module *module, const char *mod_name) { struct device_driver *drv = &dax_drv->drv; - int rc = 0; /* * dax_bus_probe() calls dax_drv->probe() unconditionally. @@ -1434,26 +1445,7 @@ int __dax_driver_register(struct dax_device_driver *dax_drv, drv->mod_name = mod_name; drv->bus = &dax_bus_type; - /* there can only be one default driver */ - mutex_lock(&dax_bus_lock); - match_always_count += dax_drv->match_always; - if (match_always_count > 1) { - match_always_count--; - WARN_ON(1); - rc = -EINVAL; - } - mutex_unlock(&dax_bus_lock); - if (rc) - return rc; - - rc = driver_register(drv); - if (rc && dax_drv->match_always) { - mutex_lock(&dax_bus_lock); - match_always_count -= dax_drv->match_always; - mutex_unlock(&dax_bus_lock); - } - - return rc; + return driver_register(drv); } EXPORT_SYMBOL_GPL(__dax_driver_register); @@ -1463,7 +1455,6 @@ void dax_driver_unregister(struct dax_device_driver *dax_drv) struct dax_id *dax_id, *_id; mutex_lock(&dax_bus_lock); - match_always_count -= dax_drv->match_always; list_for_each_entry_safe(dax_id, _id, &dax_drv->ids, list) { list_del(&dax_id->list); kfree(dax_id); diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index fbb940293d6d..8cd79ab34292 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -11,7 +11,10 @@ struct dax_device; struct dax_region; void dax_region_put(struct dax_region *dax_region); -#define IORESOURCE_DAX_STATIC (1UL << 0) +/* dax bus specific ioresource flags */ +#define IORESOURCE_DAX_STATIC BIT(0) +#define IORESOURCE_DAX_KMEM BIT(1) + struct dax_region *alloc_dax_region(struct device *parent, int region_id, struct range *range, int target_node, unsigned int align, unsigned long flags); @@ -25,10 +28,15 @@ struct dev_dax_data { struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data); +enum dax_driver_type { + DAXDRV_KMEM_TYPE, + DAXDRV_DEVICE_TYPE, +}; + struct dax_device_driver { struct device_driver drv; struct list_head ids; - int match_always; + enum dax_driver_type type; int (*probe)(struct dev_dax *dev); void (*remove)(struct dev_dax *dev); }; diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 5494d745ced5..ecdff79e31f2 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -475,8 +475,7 @@ EXPORT_SYMBOL_GPL(dev_dax_probe); static struct dax_device_driver device_dax_driver = { .probe = dev_dax_probe, - /* all probe actions are unwound by devm, so .remove isn't necessary */ - .match_always = 1, + .type = DAXDRV_DEVICE_TYPE, }; static int __init dax_init(void) diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index e7bdff3132fa..5ec08f9f8a57 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -11,15 +11,25 @@ module_param_named(region_idle, region_idle, bool, 0644); static int dax_hmem_probe(struct platform_device *pdev) { + unsigned long flags = IORESOURCE_DAX_KMEM; struct device *dev = &pdev->dev; struct dax_region *dax_region; struct memregion_info *mri; struct dev_dax_data data; struct dev_dax *dev_dax; + /* + * @region_idle == true indicates that an administrative agent + * wants to manipulate the range partitioning before the devices + * are created, so do not send them to the dax_kmem driver by + * default. + */ + if (region_idle) + flags = 0; + mri = dev->platform_data; dax_region = alloc_dax_region(dev, pdev->id, &mri->range, - mri->target_node, PMD_SIZE, 0); + mri->target_node, PMD_SIZE, flags); if (!dax_region) return -ENOMEM; diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index 4852a2dbdb27..918d01d3fbaa 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -239,6 +239,7 @@ static void dev_dax_kmem_remove(struct dev_dax *dev_dax) static struct dax_device_driver device_dax_kmem_driver = { .probe = dev_dax_kmem_probe, .remove = dev_dax_kmem_remove, + .type = DAXDRV_KMEM_TYPE, }; static int __init dax_kmem_init(void) From patchwork Fri Feb 10 09:07:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13135567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A131C636D3 for ; Fri, 10 Feb 2023 09:07:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1315F6B0139; Fri, 10 Feb 2023 04:07:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BC846B013A; Fri, 10 Feb 2023 04:07:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E76A96B013B; Fri, 10 Feb 2023 04:07:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BAAD46B0139 for ; Fri, 10 Feb 2023 04:07:22 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 72723161510 for ; Fri, 10 Feb 2023 09:07:22 +0000 (UTC) X-FDA: 80450803524.09.29FC4DC Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf23.hostedemail.com (Postfix) with ESMTP id 3792114001D for ; Fri, 10 Feb 2023 09:07:20 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ny88ZzDw; spf=pass (imf23.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676020040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qdSWoiUrfD4Tq2O1D1E/x5/rH4GNmxT6oT39d3A3lXQ=; b=HgqK/IOI7sh4LiUn3W9KfTXiLn+nAT9tq3sg0GfATowN72rOm9YaCFSgeazRuCHpUdWuNX MI8GUNBXo3Uu9k5WI5G81MqWX6eipLbGL5ptDhUvjQIma4KAwhSkzWg1P38TprG3w7XPhF CxQQnpSQ2xmjMgYA5v1hBC7j0QZb8vA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ny88ZzDw; spf=pass (imf23.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676020040; a=rsa-sha256; cv=none; b=csuVqLMcl8lY6HmHuBdbrK5WsGgJeUV9TT4iNTbpO2qplJxbC+xn0grJFYWeyXE+jdfI8W lErxj35N8O4KJW4q+8x+cb2r0jqI8Wykew7sdH1LhVTp2Muy4pn6emmbLQsUktVvEdYN15 2frhmWeqGJCCa0QF3z0w8ZALR7S9xaM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676020040; x=1707556040; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hrm0j+K5sMknqLeRHZPi95fuEMBi83jRYl1v32nRVsM=; b=ny88ZzDwCPmzUosWzwOxB6RUzxgzTnchWCZFI+XHuVtK51P3hvWz1+Yt Ph5KRBX+hdeKiukdjBBFQ5KQZa8JJr+tFdxQAi0HFi9D9twbcnsVeAB8r nZplY75rffCDqT30HL6+bByGmO6UvtIWh/GztcRGTQBZRaWyHVZhxOMAw e18MH4ZNMrmo8+8GGaeS96o4V1w0bVAX14KGFPMcxpf8JWgLjRXw9w6PS 4j3UM9p9g07KZX2Oky5LPuMrF+6cjZAIh9so9GLsvxXe15t5UO5WZlIMo 2I3keN7HptSKcBaNBdmp9sc8NemIw1PnZm/c/mw3yT1auf4OnOR/kbOGk g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="310738863" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="310738863" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:19 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="661341226" X-IronPort-AV: E=Sophos;i="5.97,286,1669104000"; d="scan'208";a="661341226" Received: from hrchavan-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.46.42]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2023 01:07:19 -0800 Subject: [PATCH v2 20/20] cxl/dax: Create dax devices for CXL RAM regions From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Fan Ni , vishal.l.verma@intel.com, dave.hansen@linux.intel.com, linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Fri, 10 Feb 2023 01:07:19 -0800 Message-ID: <167602003896.1924368.10335442077318970468.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> References: <167601992097.1924368.18291887895351917895.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Stat-Signature: e8eodfrray6o6j9tmdwqrpwjpsb5b4zk X-Rspam-User: X-Rspamd-Queue-Id: 3792114001D X-Rspamd-Server: rspam06 X-HE-Tag: 1676020040-158286 X-HE-Meta: U2FsdGVkX19XyoOj7KjqyrA2hNPdWWBU80vANp5eB/9MFzk9H7LW7csRg/fLv/z5hfo1pZx6coo69KIEfHCSpKAPQbE3YJKXqYK10RiHIb/HsiWnWuSio9wQI0wfzoKx3YiqtHCwgp3stLjs+2abHE0NLm0/ZeEv6BJV3ZZwuZGgJNdiuh67MgEn7RowqXymVkXzRF5OMzlc7GiyPaJ8Xxv9lNCfeZwxby7Yfdnaurve9Gmd9DFeB7N8Hn/qh0kA2baTDnMElBDegQZMyWBFiE8sEDKPkKDLZVQyi3If7dB8mZx4O3ydLnJWvM2aR3m8U5xTwKsNtmgmrs1AC7RRTZ7iS24gF5Lt2FjH4YOQrzVCTm70sm2hzDxpyocdTyJPuVEAUoIF/WJMmavJclykcEvhsv/ecIv/wTcGQ0R27LGmKTWrQetk4TtgJJ1lbkTYrQzPVzqc0a8vGp3jhQBYN65YUY67uT/MdfSdUDoxd39oFln5RHqqJJiSPD8+2L6rgjZIdHOEfAusaDromr6diR60Zr/YI5tV8oKpzEMlWmHaXNcBB+XIWhI4g0j6ok61cEC9o5XxBZx8tetKoa8ttrPzSqCrmVqmeqVpNczfAVVnWwIDKM+0Xv4u9BNjklbgrA+Tl0fgbLRjdT9omvUN+/6Y4V81W69RPKPISbMRVbbRhHU2+2ggERGQadU61ncZWW2dTLIfweet6HQWmi13k1UJw5Qs4qU0MVB83xRt1jP8t1ShNWVHcnnFtdVMLK3ENgYs9wfGMmUxUYXkYTp1VBnBZi4gw76YO5M0NAj8yTr1CXBfK380KzPw69GNhpxmxf0Uu9+4x48ivJdyprbDpRhar0lCxa6WnpNcQc9HW2Y1+DXlRGoWEqFhYROlKv5ajDg4ILrMyQv+WdnZ8GAX7QY/hWkdSrkOEfeagaEKUqBBAombsfAVliUUXoMT0SSmZqP0mlMpzVzaoUZewck n+YbM5aR a3WI3S2oVhMHo/XqTFx/5k0xXFrfYwg7W/nAjXT65rLuvBjRj5NhRncrUHuhGXKVPBsz7wEM7cM6erb9hbpipUhKiYv8qofREY6soTys7YYHBkR0rbmR742T5nlMkAAZYvrvewczKMpzBQyWdmreMDr+O3zsFrArCd1/Bvco9vquIhShYHMXI2zDmt+94cwyCpx0N0ubgg0uT9qOAHGYx7o3z+r1LWxDwAeRaCMG1QrTCfD0XBUIIydAs4qxEvJjaBzRaLUf6QzMamjc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: While platform firmware takes some responsibility for mapping the RAM capacity of CXL devices present at boot, the OS is responsible for mapping the remainder and hot-added devices. Platform firmware is also responsible for identifying the platform general purpose memory pool, typically DDR attached DRAM, and arranging for the remainder to be 'Soft Reserved'. That reservation allows the CXL subsystem to route the memory to core-mm via memory-hotplug (dax_kmem), or leave it for dedicated access (device-dax). The new 'struct cxl_dax_region' object allows for a CXL memory resource (region) to be published, but also allow for udev and module policy to act on that event. It also prevents cxl_core.ko from having a module loading dependency on any drivers/dax/ modules. Tested-by: Fan Ni Link: https://lore.kernel.org/r/167564545116.847146.4741351262959589920.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang --- MAINTAINERS | 1 drivers/cxl/acpi.c | 3 + drivers/cxl/core/core.h | 3 + drivers/cxl/core/port.c | 4 +- drivers/cxl/core/region.c | 108 ++++++++++++++++++++++++++++++++++++++++++++- drivers/cxl/cxl.h | 12 +++++ drivers/dax/Kconfig | 13 +++++ drivers/dax/Makefile | 2 + drivers/dax/cxl.c | 53 ++++++++++++++++++++++ drivers/dax/hmem/hmem.c | 14 ++++++ 10 files changed, 209 insertions(+), 4 deletions(-) create mode 100644 drivers/dax/cxl.c diff --git a/MAINTAINERS b/MAINTAINERS index 7f86d02cb427..73a9f3401e0e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6035,6 +6035,7 @@ M: Dan Williams M: Vishal Verma M: Dave Jiang L: nvdimm@lists.linux.dev +L: linux-cxl@vger.kernel.org S: Supported F: drivers/dax/ diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index ad0849af42d7..8ebb9a74790d 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -731,7 +731,8 @@ static void __exit cxl_acpi_exit(void) cxl_bus_drain(); } -module_init(cxl_acpi_init); +/* load before dax_hmem sees 'Soft Reserved' CXL ranges */ +subsys_initcall(cxl_acpi_init); module_exit(cxl_acpi_exit); MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS(CXL); diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 479f01da6d35..cde475e13216 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -15,12 +15,14 @@ extern struct device_attribute dev_attr_create_ram_region; extern struct device_attribute dev_attr_delete_region; extern struct device_attribute dev_attr_region; extern const struct device_type cxl_pmem_region_type; +extern const struct device_type cxl_dax_region_type; extern const struct device_type cxl_region_type; void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled); #define CXL_REGION_ATTR(x) (&dev_attr_##x.attr) #define CXL_REGION_TYPE(x) (&cxl_region_type) #define SET_CXL_REGION_ATTR(x) (&dev_attr_##x.attr), #define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type) +#define CXL_DAX_REGION_TYPE(x) (&cxl_dax_region_type) int cxl_region_init(void); void cxl_region_exit(void); #else @@ -38,6 +40,7 @@ static inline void cxl_region_exit(void) #define CXL_REGION_TYPE(x) NULL #define SET_CXL_REGION_ATTR(x) #define CXL_PMEM_REGION_TYPE(x) NULL +#define CXL_DAX_REGION_TYPE(x) NULL #endif struct cxl_send_command; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index b45d2796ef35..0bb7a5ff724b 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -46,6 +46,8 @@ static int cxl_device_id(struct device *dev) return CXL_DEVICE_NVDIMM; if (dev->type == CXL_PMEM_REGION_TYPE()) return CXL_DEVICE_PMEM_REGION; + if (dev->type == CXL_DAX_REGION_TYPE()) + return CXL_DEVICE_DAX_REGION; if (is_cxl_port(dev)) { if (is_cxl_root(to_cxl_port(dev))) return CXL_DEVICE_ROOT; @@ -2015,6 +2017,6 @@ static void cxl_core_exit(void) debugfs_remove_recursive(cxl_debugfs); } -module_init(cxl_core_init); +subsys_initcall(cxl_core_init); module_exit(cxl_core_exit); MODULE_LICENSE("GPL v2"); diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 3f6453da2c51..91d334080cab 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2272,6 +2272,75 @@ static struct cxl_pmem_region *cxl_pmem_region_alloc(struct cxl_region *cxlr) return cxlr_pmem; } +static void cxl_dax_region_release(struct device *dev) +{ + struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); + + kfree(cxlr_dax); +} + +static const struct attribute_group *cxl_dax_region_attribute_groups[] = { + &cxl_base_attribute_group, + NULL, +}; + +const struct device_type cxl_dax_region_type = { + .name = "cxl_dax_region", + .release = cxl_dax_region_release, + .groups = cxl_dax_region_attribute_groups, +}; + +static bool is_cxl_dax_region(struct device *dev) +{ + return dev->type == &cxl_dax_region_type; +} + +struct cxl_dax_region *to_cxl_dax_region(struct device *dev) +{ + if (dev_WARN_ONCE(dev, !is_cxl_dax_region(dev), + "not a cxl_dax_region device\n")) + return NULL; + return container_of(dev, struct cxl_dax_region, dev); +} +EXPORT_SYMBOL_NS_GPL(to_cxl_dax_region, CXL); + +static struct lock_class_key cxl_dax_region_key; + +static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr) +{ + struct cxl_region_params *p = &cxlr->params; + struct cxl_dax_region *cxlr_dax; + struct device *dev; + + down_read(&cxl_region_rwsem); + if (p->state != CXL_CONFIG_COMMIT) { + cxlr_dax = ERR_PTR(-ENXIO); + goto out; + } + + cxlr_dax = kzalloc(sizeof(*cxlr_dax), GFP_KERNEL); + if (!cxlr_dax) { + cxlr_dax = ERR_PTR(-ENOMEM); + goto out; + } + + cxlr_dax->hpa_range.start = p->res->start; + cxlr_dax->hpa_range.end = p->res->end; + + dev = &cxlr_dax->dev; + cxlr_dax->cxlr = cxlr; + device_initialize(dev); + lockdep_set_class(&dev->mutex, &cxl_dax_region_key); + device_set_pm_not_required(dev); + dev->parent = &cxlr->dev; + dev->bus = &cxl_bus_type; + dev->type = &cxl_dax_region_type; +out: + up_read(&cxl_region_rwsem); + + return cxlr_dax; +} + static void cxlr_pmem_unregister(void *_cxlr_pmem) { struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem; @@ -2356,6 +2425,42 @@ static int devm_cxl_add_pmem_region(struct cxl_region *cxlr) return rc; } +static void cxlr_dax_unregister(void *_cxlr_dax) +{ + struct cxl_dax_region *cxlr_dax = _cxlr_dax; + + device_unregister(&cxlr_dax->dev); +} + +static int devm_cxl_add_dax_region(struct cxl_region *cxlr) +{ + struct cxl_dax_region *cxlr_dax; + struct device *dev; + int rc; + + cxlr_dax = cxl_dax_region_alloc(cxlr); + if (IS_ERR(cxlr_dax)) + return PTR_ERR(cxlr_dax); + + dev = &cxlr_dax->dev; + rc = dev_set_name(dev, "dax_region%d", cxlr->id); + if (rc) + goto err; + + rc = device_add(dev); + if (rc) + goto err; + + dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent), + dev_name(dev)); + + return devm_add_action_or_reset(&cxlr->dev, cxlr_dax_unregister, + cxlr_dax); +err: + put_device(dev); + return rc; +} + static int match_decoder_by_range(struct device *dev, void *data) { struct range *r1, *r2 = data; @@ -2619,8 +2724,7 @@ static int cxl_region_probe(struct device *dev) p->res->start, p->res->end, cxlr, is_system_ram) > 0) return 0; - dev_dbg(dev, "TODO: hookup devdax\n"); - return 0; + return devm_cxl_add_dax_region(cxlr); default: dev_dbg(&cxlr->dev, "unsupported region mode: %d\n", cxlr->mode); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 2ac344235235..b1395c46baec 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -513,6 +513,12 @@ struct cxl_pmem_region { struct cxl_pmem_region_mapping mapping[]; }; +struct cxl_dax_region { + struct device dev; + struct cxl_region *cxlr; + struct range hpa_range; +}; + /** * struct cxl_port - logical collection of upstream port devices and * downstream port devices to construct a CXL memory @@ -707,6 +713,7 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); #define CXL_DEVICE_MEMORY_EXPANDER 5 #define CXL_DEVICE_REGION 6 #define CXL_DEVICE_PMEM_REGION 7 +#define CXL_DEVICE_DAX_REGION 8 #define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") #define CXL_MODALIAS_FMT "cxl:t%d" @@ -725,6 +732,7 @@ bool is_cxl_pmem_region(struct device *dev); struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); int cxl_add_to_region(struct cxl_port *root, struct cxl_endpoint_decoder *cxled); +struct cxl_dax_region *to_cxl_dax_region(struct device *dev); #else static inline bool is_cxl_pmem_region(struct device *dev) { @@ -739,6 +747,10 @@ static inline int cxl_add_to_region(struct cxl_port *root, { return 0; } +static inline struct cxl_dax_region *to_cxl_dax_region(struct device *dev) +{ + return NULL; +} #endif /* diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index 1163eb62e5f6..bd06e16c7ac8 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -45,6 +45,19 @@ config DEV_DAX_HMEM Say M if unsure. +config DEV_DAX_CXL + tristate "CXL DAX: direct access to CXL RAM regions" + depends on CXL_REGION && DEV_DAX + default CXL_REGION && DEV_DAX + help + CXL RAM regions are either mapped by platform-firmware + and published in the initial system-memory map as "System RAM", mapped + by platform-firmware as "Soft Reserved", or dynamically provisioned + after boot by the CXL driver. In the latter two cases a device-dax + instance is created to access that unmapped-by-default address range. + Per usual it can remain as dedicated access via a device interface, or + converted to "System RAM" via the dax_kmem facility. + config DEV_DAX_HMEM_DEVICES depends on DEV_DAX_HMEM && DAX def_bool y diff --git a/drivers/dax/Makefile b/drivers/dax/Makefile index 90a56ca3b345..5ed5c39857c8 100644 --- a/drivers/dax/Makefile +++ b/drivers/dax/Makefile @@ -3,10 +3,12 @@ obj-$(CONFIG_DAX) += dax.o obj-$(CONFIG_DEV_DAX) += device_dax.o obj-$(CONFIG_DEV_DAX_KMEM) += kmem.o obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o +obj-$(CONFIG_DEV_DAX_CXL) += dax_cxl.o dax-y := super.o dax-y += bus.o device_dax-y := device.o dax_pmem-y := pmem.o +dax_cxl-y := cxl.o obj-y += hmem/ diff --git a/drivers/dax/cxl.c b/drivers/dax/cxl.c new file mode 100644 index 000000000000..ccdf8de85bd5 --- /dev/null +++ b/drivers/dax/cxl.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Intel Corporation. All rights reserved. */ +#include +#include + +#include "../cxl/cxl.h" +#include "bus.h" + +static int cxl_dax_region_probe(struct device *dev) +{ + struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); + int nid = phys_to_target_node(cxlr_dax->hpa_range.start); + struct cxl_region *cxlr = cxlr_dax->cxlr; + struct dax_region *dax_region; + struct dev_dax_data data; + struct dev_dax *dev_dax; + + if (nid == NUMA_NO_NODE) + nid = memory_add_physaddr_to_nid(cxlr_dax->hpa_range.start); + + dax_region = alloc_dax_region(dev, cxlr->id, &cxlr_dax->hpa_range, nid, + PMD_SIZE, IORESOURCE_DAX_KMEM); + if (!dax_region) + return -ENOMEM; + + data = (struct dev_dax_data) { + .dax_region = dax_region, + .id = -1, + .size = range_len(&cxlr_dax->hpa_range), + }; + dev_dax = devm_create_dev_dax(&data); + if (IS_ERR(dev_dax)) + return PTR_ERR(dev_dax); + + /* child dev_dax instances now own the lifetime of the dax_region */ + dax_region_put(dax_region); + return 0; +} + +static struct cxl_driver cxl_dax_region_driver = { + .name = "cxl_dax_region", + .probe = cxl_dax_region_probe, + .id = CXL_DEVICE_DAX_REGION, + .drv = { + .suppress_bind_attrs = true, + }, +}; + +module_cxl_driver(cxl_dax_region_driver); +MODULE_ALIAS_CXL(CXL_DEVICE_DAX_REGION); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Intel Corporation"); +MODULE_IMPORT_NS(CXL); diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 5ec08f9f8a57..e5fe8b39fb94 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -72,6 +72,13 @@ static int hmem_register_device(struct device *host, int target_nid, long id; int rc; + if (IS_ENABLED(CONFIG_CXL_REGION) && + region_intersects(res->start, resource_size(res), IORESOURCE_MEM, + IORES_DESC_CXL) != REGION_DISJOINT) { + dev_dbg(host, "deferring range to CXL: %pr\n", res); + return 0; + } + rc = region_intersects(res->start, resource_size(res), IORESOURCE_MEM, IORES_DESC_SOFT_RESERVED); if (rc != REGION_INTERSECTS) @@ -157,6 +164,13 @@ static __exit void dax_hmem_exit(void) module_init(dax_hmem_init); module_exit(dax_hmem_exit); +/* Allow for CXL to define its own dax regions */ +#if IS_ENABLED(CONFIG_CXL_REGION) +#if IS_MODULE(CONFIG_CXL_ACPI) +MODULE_SOFTDEP("pre: cxl_acpi"); +#endif +#endif + MODULE_ALIAS("platform:hmem*"); MODULE_ALIAS("platform:hmem_platform*"); MODULE_LICENSE("GPL v2");