From patchwork Fri Apr 12 07:07:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Daisuke Kobayashi (Fujitsu)" X-Patchwork-Id: 13627000 Received: from esa11.hc1455-7.c3s2.iphmx.com (esa11.hc1455-7.c3s2.iphmx.com [207.54.90.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B04FE524B8; Fri, 12 Apr 2024 07:04:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=207.54.90.137 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712905499; cv=none; b=tZWeBPo+oyYO2LWtc35bK40A3j/teMnATRTYxe4kJyL8gPlzXVR/5KJH/vuO6/tSxIKXRxIb86m1mhq/lLaygHIv7Rpl841V4Big0p25x8TKXjdIpoA0Sg8iLQg25BsSI/vv3dGc6V5eGQ1CXXrVW6gkd2S0D/Wth005DFjKzLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712905499; c=relaxed/simple; bh=bZoKYgzce3mt6j7p8NSqBxytIhrtz4Jf1jdPXHOwSho=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kMdgmlH/UaS2Zy+BINd07TwXFXtkYsBDCCGwEGASqmer24fUrJkHpGDY79BY9mD7Uq4fKQZPTn4vEZtKD6TtHl/xdgmQcF8MrSmSN2R0aaNoJEZVe7m0Ks6UC8obxXbxqRcBX7zLILDjc9ZZUirwFKyZu5kWyK+bXnaupw4Caqs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com; spf=pass smtp.mailfrom=fujitsu.com; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b=TW/GKxb7; arc=none smtp.client-ip=207.54.90.137 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b="TW/GKxb7" DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1712905497; x=1744441497; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bZoKYgzce3mt6j7p8NSqBxytIhrtz4Jf1jdPXHOwSho=; b=TW/GKxb78cK4sTIXqRzFIfnMy988CDMKX0NlposCoXSrXU7c4sAFccAJ xJtFz87SHtkIn7GkrKyOh8tb7aAeYgskURIstfUlyU3tA5SQhplkESIGA oDO5+iCn4p41UUqjU6X4fdT/0Xhp6E7YMi+zP4JrEPaXgJG4qk6kaqMJw /61Jt1JjuA76+EZF7Dl2Yo2t+X4J4WfpHXvDg8D25zI0is6Q+7QzA3eKE g3F+dDjFoWnpyvkJk6jfq6Hn/jUvQEf/bUVJBXX3CB0LretNGSp2fx43h OJgZtlDKSf/g1qyQJkiu4MSz5CGTHy60v6CSPbDs6F3mMNF6s4Se0lxcI A==; X-IronPort-AV: E=McAfee;i="6600,9927,11041"; a="134763264" X-IronPort-AV: E=Sophos;i="6.07,195,1708354800"; d="scan'208";a="134763264" Received: from unknown (HELO oym-r4.gw.nic.fujitsu.com) ([210.162.30.92]) by esa11.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Apr 2024 16:04:48 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id 3CDF7DCB6A; Fri, 12 Apr 2024 16:04:46 +0900 (JST) Received: from m3002.s.css.fujitsu.com (msm3.b.css.fujitsu.com [10.128.233.104]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 72254106CDA; Fri, 12 Apr 2024 16:04:45 +0900 (JST) Received: from cxl-test.. (unknown [10.118.236.45]) by m3002.s.css.fujitsu.com (Postfix) with ESMTP id 37EE9209C7C0; Fri, 12 Apr 2024 16:04:45 +0900 (JST) From: "Kobayashi,Daisuke" To: kobayashi.da-06@jp.fujitsu.com, linux-cxl@vger.kernel.org Cc: y-goto@fujitsu.com, linux-pci@vger.kernel.org, mj@ucw.cz, dan.j.williams@intel.com, "Kobayashi,Daisuke" Subject: [PATCH v5 1/2] cxl/core/regs: Add rcd_regs initialization at __rcrb_to_component() Date: Fri, 12 Apr 2024 16:07:14 +0900 Message-ID: <20240412070715.16160-2-kobayashi.da-06@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240412070715.16160-1-kobayashi.da-06@fujitsu.com> References: <20240412070715.16160-1-kobayashi.da-06@fujitsu.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 Add rcd_regs and its initialization at __rcrb_to_component() to cache the cxl1.1 device link status information. Reduce access to the memory map area where the RCRB is located by caching the cxl1.1 device link status information. Signed-off-by: "Kobayashi,Daisuke" --- drivers/cxl/core/regs.c | 16 ++++++++++++++++ drivers/cxl/cxl.h | 3 +++ 2 files changed, 19 insertions(+) diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 372786f80955..e0e96be0ca7d 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -514,6 +514,8 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *ri u32 bar0, bar1; u16 cmd; u32 id; + u16 offset; + u32 cap_hdr; if (which == CXL_RCRB_UPSTREAM) rcrb += SZ_4K; @@ -537,6 +539,20 @@ resource_size_t __rcrb_to_component(struct device *dev, struct cxl_rcrb_info *ri cmd = readw(addr + PCI_COMMAND); bar0 = readl(addr + PCI_BASE_ADDRESS_0); bar1 = readl(addr + PCI_BASE_ADDRESS_1); + offset = FIELD_GET(GENMASK(7, 0), readw(addr + PCI_CAPABILITY_LIST)); + cap_hdr = readl(addr + offset); + while ((cap_hdr & GENMASK(7, 0)) != PCI_CAP_ID_EXP) { + offset = (cap_hdr >> 8) & GENMASK(7, 0); + if (offset == 0) + break; + cap_hdr = readl(addr + offset); + } + if (offset) { + ri->rcd_lnkcap = readl(addr + offset + PCI_EXP_LNKCAP); + ri->rcd_lnkctrl = readl(addr + offset + PCI_EXP_LNKCTL); + ri->rcd_lnkstatus = readl(addr + offset + PCI_EXP_LNKSTA); + } + iounmap(addr); release_mem_region(rcrb, SZ_4K); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 003feebab79b..2dc827c301a1 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -647,6 +647,9 @@ cxl_find_dport_by_dev(struct cxl_port *port, const struct device *dport_dev) struct cxl_rcrb_info { resource_size_t base; u16 aer_cap; + u16 rcd_lnkctrl; + u16 rcd_lnkstatus; + u32 rcd_lnkcap; }; /** From patchwork Fri Apr 12 07:07:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Daisuke Kobayashi (Fujitsu)" X-Patchwork-Id: 13627013 Received: from esa6.hc1455-7.c3s2.iphmx.com (esa6.hc1455-7.c3s2.iphmx.com [68.232.139.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E5BDC2FD; Fri, 12 Apr 2024 07:06:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=68.232.139.139 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712905569; cv=none; b=AWTzZ+0nj4A5sIm0hY5xLN1vL6cmiZVkURAOnDZvKuwjLTB8xsRuoBM3udcYdbCsj5tyzh5JXnpiB3ZJCysxIli1EYvXM1A3RXpWwsywZ2n6i/ewmvkhcz0AGzVfG0dCK8GzNqwlMHh7Y+rdIm5192Eix/txABO9TSWF0myNpl0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712905569; c=relaxed/simple; bh=jZH5+5sXxf3MbvFul0+ZE3IuoRWAmGkRhN2oWgt6yQ0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cnf4Z4gT9wUqDxgrOQqHTlr77bm3BKF0S7/YgTqMUV0VWfC3beOdJ5qaf6ggGlzbnn+iet8KZXrlBM++asgN0HVpR0KR3cUUACBBC/EsWA1KIPbPO/AkOYPbjTLFA4qdWutc0bCohj/fSeqgm+fZvjUS3OjxMljHJ6bPV2/5VZ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com; spf=pass smtp.mailfrom=fujitsu.com; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b=Fj1iJnDR; arc=none smtp.client-ip=68.232.139.139 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b="Fj1iJnDR" DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1712905567; x=1744441567; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jZH5+5sXxf3MbvFul0+ZE3IuoRWAmGkRhN2oWgt6yQ0=; b=Fj1iJnDRzaLX6fBr0kh0EgH1zhTnqrnLKNcnPyQCY9Qa5jMhf0g7x8KV 5EQgr4lHrifcFY80AltiOX1ZOBHsVJruEKStyCNOcmsEwULD6LkbSH8uq 3dpTEBSSmrCjCt9xsDZP+HK+/4dcZ1fJQoCIyhTwr6RXwQG3QGNgFQ4O4 p8Yepelk9eLP6NxUJftRg3n3jiCqcKPmxzp8xTk8LTUaB5niG/JGazpdz FU4uYn0ltmGCWuwBUDhaUAjylYu8qrwuFwjWUDAYrNQD81N7j/oRP5Y+s c1xihYedoV2d2VDbZzqv+pMON/ZmJPtucP1VWhg6RvMXjwiUclh7zgYJQ g==; X-IronPort-AV: E=McAfee;i="6600,9927,11041"; a="157308498" X-IronPort-AV: E=Sophos;i="6.07,195,1708354800"; d="scan'208";a="157308498" Received: from unknown (HELO oym-r4.gw.nic.fujitsu.com) ([210.162.30.92]) by esa6.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Apr 2024 16:04:54 +0900 Received: from oym-m1.gw.nic.fujitsu.com (oym-nat-oym-m1.gw.nic.fujitsu.com [192.168.87.58]) by oym-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id 92F672F7FD; Fri, 12 Apr 2024 16:04:51 +0900 (JST) Received: from m3002.s.css.fujitsu.com (msm3.b.css.fujitsu.com [10.128.233.104]) by oym-m1.gw.nic.fujitsu.com (Postfix) with ESMTP id BFE51FDA74; Fri, 12 Apr 2024 16:04:50 +0900 (JST) Received: from cxl-test.. (unknown [10.118.236.45]) by m3002.s.css.fujitsu.com (Postfix) with ESMTP id 864AD209C7C0; Fri, 12 Apr 2024 16:04:50 +0900 (JST) From: "Kobayashi,Daisuke" To: kobayashi.da-06@jp.fujitsu.com, linux-cxl@vger.kernel.org Cc: y-goto@fujitsu.com, linux-pci@vger.kernel.org, mj@ucw.cz, dan.j.williams@intel.com, "Kobayashi,Daisuke" Subject: [PATCH v5 2/2] cxl/pci: Add sysfs attribute for CXL 1.1 device link status Date: Fri, 12 Apr 2024 16:07:15 +0900 Message-ID: <20240412070715.16160-3-kobayashi.da-06@fujitsu.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240412070715.16160-1-kobayashi.da-06@fujitsu.com> References: <20240412070715.16160-1-kobayashi.da-06@fujitsu.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 Add sysfs attribute for CXL 1.1 device link status to the cxl pci device. In CXL1.1, the link status of the device is included in the RCRB mapped to the memory mapped register area. Critically, that arrangement makes the link status and control registers invisible to existing PCI user tooling. Export those registers via sysfs with the expectation that PCI user tooling will alternatively look for these sysfs files when attempting to access to these CXL 1.1 endpoints registers. Signed-off-by: "Kobayashi,Daisuke" --- drivers/cxl/pci.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 2ff361e756d6..b2d8198ab532 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -786,6 +786,103 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, return 0; } +static ssize_t rcd_link_cap_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_port *port; + struct cxl_dport *dport; + struct cxl_dev_state *cxlds = dev_get_drvdata(dev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *endpoint_parent; + + port = cxl_mem_find_port(cxlmd, &dport); + if (!port) + return -EINVAL; + + endpoint_parent = port->uport_dev; + if (!endpoint_parent) + return -ENXIO; + + guard(device)(endpoint_parent); + if (!endpoint_parent->driver) + return -ENXIO; + + return sysfs_emit(buf, "%x\n", dport->rcrb.rcd_lnkcap); +} +static DEVICE_ATTR_RO(rcd_link_cap); + +static ssize_t rcd_link_ctrl_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_port *port; + struct cxl_dport *dport; + struct cxl_dev_state *cxlds = dev_get_drvdata(dev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *endpoint_parent; + + port = cxl_mem_find_port(cxlmd, &dport); + if (!port) + return -EINVAL; + + endpoint_parent = port->uport_dev; + if (!endpoint_parent) + return -ENXIO; + + guard(device)(endpoint_parent); + if (!endpoint_parent->driver) + return -ENXIO; + + return sysfs_emit(buf, "%x\n", dport->rcrb.rcd_lnkctrl); +} +static DEVICE_ATTR_RO(rcd_link_ctrl); + +static ssize_t rcd_link_status_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_port *port; + struct cxl_dport *dport; + struct cxl_dev_state *cxlds = dev_get_drvdata(dev); + struct cxl_memdev *cxlmd = cxlds->cxlmd; + struct device *endpoint_parent; + + port = cxl_mem_find_port(cxlmd, &dport); + if (!port) + return -EINVAL; + + endpoint_parent = port->uport_dev; + if (!endpoint_parent) + return -ENXIO; + + guard(device)(endpoint_parent); + if (!endpoint_parent->driver) + return -ENXIO; + + return sysfs_emit(buf, "%x\n", dport->rcrb.rcd_lnkstatus); +} +static DEVICE_ATTR_RO(rcd_link_status); + +static struct attribute *cxl_rcd_attrs[] = { + &dev_attr_rcd_link_cap.attr, + &dev_attr_rcd_link_ctrl.attr, + &dev_attr_rcd_link_status.attr, + NULL +}; + +static umode_t cxl_rcd_visible(struct kobject *kobj, + struct attribute *a, int n) +{ + struct device *dev = kobj_to_dev(kobj); + struct pci_dev *pdev = to_pci_dev(dev); + + if (is_cxl_restricted(pdev)) + return a->mode; + + return 0; +} + +static struct attribute_group cxl_rcd_group = { + .attrs = cxl_rcd_attrs, + .is_visible = cxl_rcd_visible, +}; +__ATTRIBUTE_GROUPS(cxl_rcd); + static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct pci_host_bridge *host_bridge = pci_find_host_bridge(pdev->bus); @@ -969,6 +1066,7 @@ static struct pci_driver cxl_pci_driver = { .id_table = cxl_mem_pci_tbl, .probe = cxl_pci_probe, .err_handler = &cxl_error_handlers, + .dev_groups = cxl_rcd_groups, .driver = { .probe_type = PROBE_PREFER_ASYNCHRONOUS, },