From patchwork Mon Sep 16 17:35:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13805711 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8CF7E18B1A for ; Mon, 16 Sep 2024 17:35:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508163; cv=none; b=Z/AS9057NdkfOA2KM7w9iTNkESjcAdSi5dYOm7krox5sL5fg11F/S/+wjJukfsugJm2UrC0wFLYzA2svfAsB8IMqjdJHHZHQ/tKoGk58ZUMwUZMLHHZg6B+J5nDy2Ww+0eknJDivzclETorO5LORm7HGB9c9Hp+6yc3DoyEKtHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508163; c=relaxed/simple; bh=IgCFi554874G/H00XXDx0h2QitS9Vv9eO5ErFyeI37U=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Jyzq7VHk2GmWY+AWW+NGv/4umDLtb/utanfBhQlMR5kvH9XeyaMgTOrxTw2tq6P7xa/V97WzbnpMlDplffRmbvN6G7EgYUWPhMxdf/dJHn0ZMUCTvSK5fBxvZCWPy454Nd2vxPcYBfpAmBdTxUkvmsAhnbb5ihoLUMzitu1OLi0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X6sWr2mfpz6K6gc; Tue, 17 Sep 2024 01:35:52 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 1BE831401F3; Tue, 17 Sep 2024 01:35:57 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 16 Sep 2024 19:35:54 +0200 From: Jonathan Cameron To: , Markus Armbruster , CC: , , , Dave Jiang , Huang Ying , Michael Roth , Subject: [PATCH 1/6] hw/pci-bridge/cxl_root_port: Provide x-speed and x-width properties. Date: Mon, 16 Sep 2024 18:35:13 +0100 Message-ID: <20240916173518.1843023-2-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> References: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) Approach copied from gen_pcie_root_port.c Previously the link defaulted to a maximum of 2.5GT/s and 1x. Enable setting it's maximum values. The actual value after 'training' will depend on the downstream device configuration. Signed-off-by: Jonathan Cameron Reviewed-by: Fan Ni --- hw/pci-bridge/cxl_root_port.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci-bridge/cxl_root_port.c b/hw/pci-bridge/cxl_root_port.c index 2dd10239bd..5e2156d7ba 100644 --- a/hw/pci-bridge/cxl_root_port.c +++ b/hw/pci-bridge/cxl_root_port.c @@ -24,6 +24,7 @@ #include "hw/pci/pcie_port.h" #include "hw/pci/msi.h" #include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" #include "hw/sysbus.h" #include "qapi/error.h" #include "hw/cxl/cxl.h" @@ -206,6 +207,10 @@ static Property gen_rp_props[] = { -1), DEFINE_PROP_SIZE("pref64-reserve", CXLRootPort, res_reserve.mem_pref_64, -1), + DEFINE_PROP_PCIE_LINK_SPEED("x-speed", PCIESlot, + speed, PCIE_LINK_SPEED_64), + DEFINE_PROP_PCIE_LINK_WIDTH("x-width", PCIESlot, + width, PCIE_LINK_WIDTH_32), DEFINE_PROP_END_OF_LIST() }; From patchwork Mon Sep 16 17:35:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13805712 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C40018B1A for ; Mon, 16 Sep 2024 17:36:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508195; cv=none; b=jOZbTO1ebWiixHrsP9e39MCbAGtTTD6Xz82aIyDpBybsfWFXSnpJcCl2tGQMvE6cA6H/hJqvtv7MNQqO0wPTBEM0Fbp73DVQ7PqPTZMyMvMt6jUDI7NQaNvAczsV64fXkpUW8ZwBDKblRA17S9pI6iY2KURy38ryjXJHmmdAQFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508195; c=relaxed/simple; bh=n0g697SEwFq/+vRqbHJCSaPjULhZU+WS7rFwKnvkORc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hHD+6nnJFtY3q+/Ulg2Y/2HffbyozeiT/obV7rWfJI9mb9aQ6UKn19XsHuCzY5FWI3zxmXqGuUQ6C+ZyzOBH9T8pllKhTV6BwHBdlLG2bbNfQd7LtOY+qyomrdmDmutnAkeMdp7A17mWS1T6baD7IZMVnZQXxrTQgVv01FUhnPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X6sRk6RTKz6K5pZ; Tue, 17 Sep 2024 01:32:18 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 16E9E14058E; Tue, 17 Sep 2024 01:36:31 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 16 Sep 2024 19:36:28 +0200 From: Jonathan Cameron To: , Markus Armbruster , CC: , , , Dave Jiang , Huang Ying , Michael Roth , Subject: [PATCH 2/6] hw/pci-bridge/cxl_upstream: Provide x-speed and x-width properties. Date: Mon, 16 Sep 2024 18:35:14 +0100 Message-ID: <20240916173518.1843023-3-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> References: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) Copied from gen_pcie_root_port.c Drop the previous code that ensured a valid value in s->width, s->speed as now a default is provided so this will always be set. Note this changes the default settings but it is unlikely to have a negative effect on software as will only affect ports with now downstream device. All other ports will use the settings from that device. Signed-off-by: Jonathan Cameron --- hw/pci-bridge/cxl_downstream.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/hw/pci-bridge/cxl_downstream.c b/hw/pci-bridge/cxl_downstream.c index 4b42984360..c347ac06f3 100644 --- a/hw/pci-bridge/cxl_downstream.c +++ b/hw/pci-bridge/cxl_downstream.c @@ -13,6 +13,8 @@ #include "hw/pci/msi.h" #include "hw/pci/pcie.h" #include "hw/pci/pcie_port.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" #include "hw/cxl/cxl.h" #include "qapi/error.h" @@ -210,24 +212,20 @@ static void cxl_dsp_exitfn(PCIDevice *d) pci_bridge_exitfn(d); } -static void cxl_dsp_instance_post_init(Object *obj) -{ - PCIESlot *s = PCIE_SLOT(obj); - - if (!s->speed) { - s->speed = QEMU_PCI_EXP_LNK_2_5GT; - } - - if (!s->width) { - s->width = QEMU_PCI_EXP_LNK_X1; - } -} +static Property cxl_dsp_props[] = { + DEFINE_PROP_PCIE_LINK_SPEED("x-speed", PCIESlot, + speed, PCIE_LINK_SPEED_64), + DEFINE_PROP_PCIE_LINK_WIDTH("x-width", PCIESlot, + width, PCIE_LINK_WIDTH_16), + DEFINE_PROP_END_OF_LIST() +}; static void cxl_dsp_class_init(ObjectClass *oc, void *data) { DeviceClass *dc = DEVICE_CLASS(oc); PCIDeviceClass *k = PCI_DEVICE_CLASS(oc); + device_class_set_props(dc, cxl_dsp_props); k->config_write = cxl_dsp_config_write; k->realize = cxl_dsp_realize; k->exit = cxl_dsp_exitfn; @@ -243,7 +241,6 @@ static const TypeInfo cxl_dsp_info = { .name = TYPE_CXL_DSP, .instance_size = sizeof(CXLDownstreamPort), .parent = TYPE_PCIE_SLOT, - .instance_post_init = cxl_dsp_instance_post_init, .class_init = cxl_dsp_class_init, .interfaces = (InterfaceInfo[]) { { INTERFACE_PCIE_DEVICE }, From patchwork Mon Sep 16 17:35:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13805713 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B786C2D032 for ; Mon, 16 Sep 2024 17:37:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508229; cv=none; b=BP8K6ubLG7IpwH6T+sptz6Vg/LcAPln7XxdEyUSTynDPWsrIWWWW5N+rLAuyczB8AG3q0U1wPBie4zBwrAC+K+gGcjzh1DTKHVPhMsekLF+r2LVIbHTNal0dApYER5In+sbrbVziHCASwKW7eFPdTILiFaMZxumQgRZWOBPisCc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508229; c=relaxed/simple; bh=UjL2gGQy97TSGA7dQmI2qPPndwta9qFFJdmfjc5/Jg8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YcJgOyw88Gqj1e5vJk7+3P5nOAoEJLQWCypWSez+qCjasFJ7AwKmWT7IOUazC4ydYjHFjhNcoSu3QLrmQqhKelN/vl0fXJ/73Vkkh2uhjgNBNlKDF/5M1ynWqwT4mzp7t5D6GwzLsC7PT2OQYPr4x8Dq3nDM2QP4sZJMw0KRPCQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X6sSP0849z6K9DM; Tue, 17 Sep 2024 01:32:53 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 283A5140F1C; Tue, 17 Sep 2024 01:37:05 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 16 Sep 2024 19:37:02 +0200 From: Jonathan Cameron To: , Markus Armbruster , CC: , , , Dave Jiang , Huang Ying , Michael Roth , Subject: [PATCH 3/6] hw/pcie: Factor out PCI Express link register filling common to EP. Date: Mon, 16 Sep 2024 18:35:15 +0100 Message-ID: <20240916173518.1843023-4-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> References: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) Whilst not all link related registers are common between RP / Switch DSP and EP / Switch USP many of them are. Factor that group out to save on duplication when adding EP / Swtich USP configurability. Signed-off-by: Jonathan Cameron --- hw/pci/pcie.c | 87 ++++++++++++++++++++++++++++----------------------- 1 file changed, 48 insertions(+), 39 deletions(-) diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index 4b2f0805c6..1ac6d89dcf 100644 --- a/hw/pci/pcie.c +++ b/hw/pci/pcie.c @@ -105,46 +105,18 @@ pcie_cap_v1_fill(PCIDevice *dev, uint8_t port, uint8_t type, uint8_t version) pci_set_word(cmask + PCI_EXP_LNKSTA, 0); } -static void pcie_cap_fill_slot_lnk(PCIDevice *dev) +/* Includes setting the target speed default */ +static void pcie_cap_fill_lnk(uint8_t *exp_cap, PCIExpLinkWidth width, + PCIExpLinkSpeed speed) { - PCIESlot *s = (PCIESlot *)object_dynamic_cast(OBJECT(dev), TYPE_PCIE_SLOT); - uint8_t *exp_cap = dev->config + dev->exp.exp_cap; - - /* Skip anything that isn't a PCIESlot */ - if (!s) { - return; - } - /* Clear and fill LNKCAP from what was configured above */ pci_long_test_and_clear_mask(exp_cap + PCI_EXP_LNKCAP, PCI_EXP_LNKCAP_MLW | PCI_EXP_LNKCAP_SLS); pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP, - QEMU_PCI_EXP_LNKCAP_MLW(s->width) | - QEMU_PCI_EXP_LNKCAP_MLS(s->speed)); - - /* - * Link bandwidth notification is required for all root ports and - * downstream ports supporting links wider than x1 or multiple link - * speeds. - */ - if (s->width > QEMU_PCI_EXP_LNK_X1 || - s->speed > QEMU_PCI_EXP_LNK_2_5GT) { - pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP, - PCI_EXP_LNKCAP_LBNC); - } - - if (s->speed > QEMU_PCI_EXP_LNK_2_5GT) { - /* - * Hot-plug capable downstream ports and downstream ports supporting - * link speeds greater than 5GT/s must hardwire PCI_EXP_LNKCAP_DLLLARC - * to 1b. PCI_EXP_LNKCAP_DLLLARC implies PCI_EXP_LNKSTA_DLLLA, which - * we also hardwire to 1b here. 2.5GT/s hot-plug slots should also - * technically implement this, but it's not done here for compatibility. - */ - pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP, - PCI_EXP_LNKCAP_DLLLARC); - /* the PCI_EXP_LNKSTA_DLLLA will be set in the hotplug function */ + QEMU_PCI_EXP_LNKCAP_MLW(width) | + QEMU_PCI_EXP_LNKCAP_MLS(speed)); + if (speed > QEMU_PCI_EXP_LNK_2_5GT) { /* * Target Link Speed defaults to the highest link speed supported by * the component. 2.5GT/s devices are permitted to hardwire to zero. @@ -152,7 +124,7 @@ static void pcie_cap_fill_slot_lnk(PCIDevice *dev) pci_word_test_and_clear_mask(exp_cap + PCI_EXP_LNKCTL2, PCI_EXP_LNKCTL2_TLS); pci_word_test_and_set_mask(exp_cap + PCI_EXP_LNKCTL2, - QEMU_PCI_EXP_LNKCAP_MLS(s->speed) & + QEMU_PCI_EXP_LNKCAP_MLS(speed) & PCI_EXP_LNKCTL2_TLS); } @@ -161,27 +133,64 @@ static void pcie_cap_fill_slot_lnk(PCIDevice *dev) * actually a reference to the highest bit supported in this register. * We assume the device supports all link speeds. */ - if (s->speed > QEMU_PCI_EXP_LNK_5GT) { + if (speed > QEMU_PCI_EXP_LNK_5GT) { pci_long_test_and_clear_mask(exp_cap + PCI_EXP_LNKCAP2, ~0U); pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP2, PCI_EXP_LNKCAP2_SLS_2_5GB | PCI_EXP_LNKCAP2_SLS_5_0GB | PCI_EXP_LNKCAP2_SLS_8_0GB); - if (s->speed > QEMU_PCI_EXP_LNK_8GT) { + if (speed > QEMU_PCI_EXP_LNK_8GT) { pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP2, PCI_EXP_LNKCAP2_SLS_16_0GB); } - if (s->speed > QEMU_PCI_EXP_LNK_16GT) { + if (speed > QEMU_PCI_EXP_LNK_16GT) { pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP2, PCI_EXP_LNKCAP2_SLS_32_0GB); } - if (s->speed > QEMU_PCI_EXP_LNK_32GT) { + if (speed > QEMU_PCI_EXP_LNK_32GT) { pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP2, PCI_EXP_LNKCAP2_SLS_64_0GB); } } } +static void pcie_cap_fill_slot_lnk(PCIDevice *dev) +{ + PCIESlot *s = (PCIESlot *)object_dynamic_cast(OBJECT(dev), TYPE_PCIE_SLOT); + uint8_t *exp_cap = dev->config + dev->exp.exp_cap; + + /* Skip anything that isn't a PCIESlot */ + if (!s) { + return; + } + + /* + * Link bandwidth notification is required for all root ports and + * downstream ports supporting links wider than x1 or multiple link + * speeds. + */ + if (s->width > QEMU_PCI_EXP_LNK_X1 || + s->speed > QEMU_PCI_EXP_LNK_2_5GT) { + pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP, + PCI_EXP_LNKCAP_LBNC); + } + + if (s->speed > QEMU_PCI_EXP_LNK_2_5GT) { + /* + * Hot-plug capable downstream ports and downstream ports supporting + * link speeds greater than 5GT/s must hardwire PCI_EXP_LNKCAP_DLLLARC + * to 1b. PCI_EXP_LNKCAP_DLLLARC implies PCI_EXP_LNKSTA_DLLLA, which + * we also hardwire to 1b here. 2.5GT/s hot-plug slots should also + * technically implement this, but it's not done here for compatibility. + */ + pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKCAP, + PCI_EXP_LNKCAP_DLLLARC); + /* the PCI_EXP_LNKSTA_DLLLA will be set in the hotplug function */ + } + + pcie_cap_fill_lnk(exp_cap, s->width, s->speed); +} + int pcie_cap_init(PCIDevice *dev, uint8_t offset, uint8_t type, uint8_t port, Error **errp) From patchwork Mon Sep 16 17:35:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13805714 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 190062D032 for ; Mon, 16 Sep 2024 17:37:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508263; cv=none; b=IwfPoK0MXWpIEHMOQdMZVdEh8EzUyYJ8iQSHdD+WY0p6+4HFWxSt0R1nvmWR4WHXLLnJx1lggfvgx9QiXBu2iwdLMhuK2oppTDXttbvgFdBnfT7D9edTs6MqMMa0bCmrNmqHIIgTxUeGv2Ot1g5dewqMn66u5UhS1hELlm7Al2g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508263; c=relaxed/simple; bh=ZsVx7Qo5HTEYcGw+/mev4T+2543t3qWs2AJLDsCiwzs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jVn8bca6O7sOD+imMO0MgasnJAndLdJx1BvlUaVWiG3LHdTZnPKetA+TQhsIIGHFY5liBYYqk1Rs49eeYRR4BQTagfKy3XDDlUgfR+X7trKh/FQ2FpVw5bnboVqs0EafaW4Z9U3vj+ZqGlAoGL0DfIy0ehD25eWArJ2VoaXXnOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X6sYp43Q8z6K6gN; Tue, 17 Sep 2024 01:37:34 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 4815F140AB8; Tue, 17 Sep 2024 01:37:39 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 16 Sep 2024 19:37:36 +0200 From: Jonathan Cameron To: , Markus Armbruster , CC: , , , Dave Jiang , Huang Ying , Michael Roth , Subject: [PATCH 4/6] hw/pcie: Provide a utility function for control of EP / SW USP link Date: Mon, 16 Sep 2024 18:35:16 +0100 Message-ID: <20240916173518.1843023-5-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> References: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) Whilst similar to existing PCIESlot link configuration a few registers need to be set differently so that the downstream device presents a 'configured' state that is then used to 'train' the upstream port on the link. Basically that means setting the status register to reflect it succeeding in training up to target settings. Signed-off-by: Jonathan Cameron --- include/hw/pci/pcie.h | 2 ++ hw/pci/pcie.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h index 5eddb90976..b8d59732bc 100644 --- a/include/hw/pci/pcie.h +++ b/include/hw/pci/pcie.h @@ -141,6 +141,8 @@ void pcie_acs_reset(PCIDevice *dev); void pcie_ari_init(PCIDevice *dev, uint16_t offset); void pcie_dev_ser_num_init(PCIDevice *dev, uint16_t offset, uint64_t ser_num); void pcie_ats_init(PCIDevice *dev, uint16_t offset, bool aligned); +void pcie_cap_fill_link_ep_usp(PCIDevice *dev, PCIExpLinkWidth width, + PCIExpLinkSpeed speed); void pcie_cap_slot_pre_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp); diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index 1ac6d89dcf..2738dbb28d 100644 --- a/hw/pci/pcie.c +++ b/hw/pci/pcie.c @@ -154,6 +154,24 @@ static void pcie_cap_fill_lnk(uint8_t *exp_cap, PCIExpLinkWidth width, } } +void pcie_cap_fill_link_ep_usp(PCIDevice *dev, PCIExpLinkWidth width, + PCIExpLinkSpeed speed) +{ + uint8_t *exp_cap = dev->config + dev->exp.exp_cap; + + /* + * For an end point or USP need to set the current status as well + * as the capabilities. + */ + pci_long_test_and_clear_mask(exp_cap + PCI_EXP_LNKSTA, + PCI_EXP_LNKSTA_CLS | PCI_EXP_LNKSTA_NLW); + pci_long_test_and_set_mask(exp_cap + PCI_EXP_LNKSTA, + QEMU_PCI_EXP_LNKSTA_NLW(width) | + QEMU_PCI_EXP_LNKSTA_CLS(speed)); + + pcie_cap_fill_lnk(exp_cap, width, speed); +} + static void pcie_cap_fill_slot_lnk(PCIDevice *dev) { PCIESlot *s = (PCIESlot *)object_dynamic_cast(OBJECT(dev), TYPE_PCIE_SLOT); From patchwork Mon Sep 16 17:35:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13805715 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4003115B115 for ; Mon, 16 Sep 2024 17:38:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508297; cv=none; b=jUuPdr9RBZl6L2nSMm1bfjQOS0A40nXriKcefs1nVm+U1xqMaNFI441hhujSx4aWztjYJs8ZHaCIMi0qkYyVOfHwanTgGc8QOtjL9/gvsWuHJtiA9Zs/SpvKMSQ2PIkIhVORBxrZUXeIrJgT7glFqK14tKHKyJlbVRsAHyTPh3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508297; c=relaxed/simple; bh=Vc0cuG0R+Bv0xjdfjMRrqLAlDPU+c0oVtu5oK0C6LVg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Seyb3Mh9NQl5pjEWYYZLfKihJjhOwh5ME8WuyPgmixcMCyTR8/7/Zl2yMsaTdcHLYEI1e//YL4VzSDgAcUk89MjBeXIPfcnb8cTL9ixLm0w22TlRrldcv6HHGJvsrpRqg8kRSIM+tUPRC64R1z37GtNU+Tjafs3dk2pHe+A/LHs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X6sZS2km3z6K6gh; Tue, 17 Sep 2024 01:38:08 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 1B294140B33; Tue, 17 Sep 2024 01:38:13 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 16 Sep 2024 19:38:10 +0200 From: Jonathan Cameron To: , Markus Armbruster , CC: , , , Dave Jiang , Huang Ying , Michael Roth , Subject: [PATCH 5/6] hw/mem/cxl-type3: Add properties to control link speed and width Date: Mon, 16 Sep 2024 18:35:17 +0100 Message-ID: <20240916173518.1843023-6-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> References: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) To establish performance characteristics of a CXL device when used via a particular CXL topology (root ports, switches, end points) it is necessary to set the appropriate link speed and width in the PCI Express capability structure. Provide x-speed and x-link properties for this. Signed-off-by: Jonathan Cameron --- include/hw/cxl/cxl_device.h | 4 ++++ hw/mem/cxl_type3.c | 6 ++++++ 2 files changed, 10 insertions(+) diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index fdd0f4e62b..e14e56ae4b 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -549,6 +549,10 @@ struct CXLType3Dev { CXLCCI vdm_fm_owned_ld_mctp_cci; CXLCCI ld0_cci; + /* PCIe link characteristics */ + PCIExpLinkSpeed speed; + PCIExpLinkWidth width; + /* DOE */ DOECap doe_cdat; diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 235ac40aeb..44d491d8f6 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -17,6 +17,7 @@ #include "hw/mem/pc-dimm.h" #include "hw/pci/pci.h" #include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" #include "qapi/error.h" #include "qemu/log.h" #include "qemu/module.h" @@ -1200,6 +1201,7 @@ static void ct3d_reset(DeviceState *dev) uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers; uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask; + pcie_cap_fill_link_ep_usp(PCI_DEVICE(dev), ct3d->width, ct3d->speed); cxl_component_register_init_common(reg_state, write_msk, CXL2_TYPE3_DEVICE); cxl_device_register_init_t3(ct3d); @@ -1229,6 +1231,10 @@ static Property ct3_props[] = { DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0), DEFINE_PROP_LINK("volatile-dc-memdev", CXLType3Dev, dc.host_dc, TYPE_MEMORY_BACKEND, HostMemoryBackend *), + DEFINE_PROP_PCIE_LINK_SPEED("x-speed", CXLType3Dev, + speed, PCIE_LINK_SPEED_32), + DEFINE_PROP_PCIE_LINK_WIDTH("x-width", CXLType3Dev, + width, PCIE_LINK_WIDTH_16), DEFINE_PROP_END_OF_LIST(), }; From patchwork Mon Sep 16 17:35:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13805716 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A61B2D032 for ; Mon, 16 Sep 2024 17:38:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508331; cv=none; b=g3iujngy9fOXAHF2q+oQkMq1siBumDSaaDOuGwuriwl3IKjX+IaDgewv5Xz9vzYWj+cKRZXVr3QEpMk+DGl33/Qh8ajkajY6Jrm6u1a6k4P7OVfUh2XKkAllkcOsZ8DhogcaC2CKv2IIyEOV6DUJQKnVBGq7GmHjvwi6UC5loqc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726508331; c=relaxed/simple; bh=wtCu5Nw+JuwiMxxvpBpZ3ib3qR2UBzUnvURcWm1prcg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=X1KWiarF22Vn0TapM3Bgx88CsJ/gQpLV0QBS6Lzc2OW7CWkqwh2VSb5px8h5mkTAzenExnTAiTkIMfl83uirVctFW/Vu0+yDFm3PY1tQn6uXJXGsQG6mYklXsw8HAaBwlJwwMerWHZj5avvWNSw15o4T5VxAo+lkWv1gLXb2qYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4X6sVM0t5Hz6K9FB; Tue, 17 Sep 2024 01:34:35 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 4D72C140F1C; Tue, 17 Sep 2024 01:38:47 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 16 Sep 2024 19:38:44 +0200 From: Jonathan Cameron To: , Markus Armbruster , CC: , , , Dave Jiang , Huang Ying , Michael Roth , Subject: [PATCH 6/6] hw/pci-bridge/cxl-upstream: Add properties to control link speed and width Date: Mon, 16 Sep 2024 18:35:18 +0100 Message-ID: <20240916173518.1843023-7-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> References: <20240916173518.1843023-1-Jonathan.Cameron@huawei.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) To establish performance characteristics of a CXL device when used via a particular CXL topology (root ports, switches, end points) it is necessary to set the appropriate link speed and width in the PCI Express capability structure. Provide x-speed and x-link properties for this. Signed-off-by: Jonathan Cameron --- include/hw/pci-bridge/cxl_upstream_port.h | 4 ++++ hw/pci-bridge/cxl_upstream.c | 6 ++++++ 2 files changed, 10 insertions(+) diff --git a/include/hw/pci-bridge/cxl_upstream_port.h b/include/hw/pci-bridge/cxl_upstream_port.h index 12635139f6..f208397ffe 100644 --- a/include/hw/pci-bridge/cxl_upstream_port.h +++ b/include/hw/pci-bridge/cxl_upstream_port.h @@ -12,6 +12,10 @@ typedef struct CXLUpstreamPort { /*< public >*/ CXLComponentState cxl_cstate; CXLCCI swcci; + + PCIExpLinkSpeed speed; + PCIExpLinkWidth width; + DOECap doe_cdat; uint64_t sn; } CXLUpstreamPort; diff --git a/hw/pci-bridge/cxl_upstream.c b/hw/pci-bridge/cxl_upstream.c index a5a39cc524..55f8b0053f 100644 --- a/hw/pci-bridge/cxl_upstream.c +++ b/hw/pci-bridge/cxl_upstream.c @@ -11,6 +11,7 @@ #include "qemu/osdep.h" #include "qemu/log.h" #include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" #include "hw/pci/msi.h" #include "hw/pci/pcie.h" #include "hw/pci/pcie_port.h" @@ -100,6 +101,7 @@ static void cxl_usp_reset(DeviceState *qdev) pci_bridge_reset(qdev); pcie_cap_deverr_reset(d); + pcie_cap_fill_link_ep_usp(d, usp->width, usp->speed); latch_registers(usp); } @@ -363,6 +365,10 @@ static void cxl_usp_exitfn(PCIDevice *d) static Property cxl_upstream_props[] = { DEFINE_PROP_UINT64("sn", CXLUpstreamPort, sn, UI64_NULL), DEFINE_PROP_STRING("cdat", CXLUpstreamPort, cxl_cstate.cdat.filename), + DEFINE_PROP_PCIE_LINK_SPEED("x-speed", CXLUpstreamPort, + speed, PCIE_LINK_SPEED_32), + DEFINE_PROP_PCIE_LINK_WIDTH("x-width", CXLUpstreamPort, + width, PCIE_LINK_WIDTH_16), DEFINE_PROP_END_OF_LIST() };