From patchwork Wed Apr 3 21:39:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 13616738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18112CD1288 for ; Wed, 3 Apr 2024 21:39:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kYsvu2as0VM++lCngjxFfEug7xFobHzDD/WywUQ7Qbo=; b=O/mksi3JwXDlc8W/tsCJXnMPAe V+YqYVzJQsAzLKNCoNOM+hW8Gvrgj695BX+9Hi5m0FJoGwGPdCvW/59EVennNeW8XBjcnqomjVUTA fB/T5DbqS3SfCzzK6XGNnyQX6xEY5VHuTgWCQOjV8obmPQOf5F7OPW5cpyoTxXxHDfh6zNKzvceDp J7zG2eTSchZyLNxTj4pl2ycHU1g9V3ZyTLJvo+lj29L444+LvTpNR5HCSia4mSbWwKEEhz4ltS2lB mCZG0oecmPv3hv2EQU6n+sVNV6UPJ5XHv9xEvLcJ0cCitMrd6bqu7EYEck/B391ZlHwMhUugAmMvy 4RHNwqaQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs8KJ-00000000HS6-1ZVl; Wed, 03 Apr 2024 21:39:23 +0000 Received: from mail-vk1-xa2e.google.com ([2607:f8b0:4864:20::a2e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs8KA-00000000HM3-3Kkb for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2024 21:39:16 +0000 Received: by mail-vk1-xa2e.google.com with SMTP id 71dfb90a1353d-4d43c633fecso128490e0c.3 for ; Wed, 03 Apr 2024 14:39:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1712180352; x=1712785152; darn=lists.infradead.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=7VBCEyaaU+8nci3Nv2Q43lb0TGQpEICGlo/quHlWch4=; b=FPpPcaNL9DpgWD6KxoqTSW4dLoKCRBHxImomS8jqIbY09iuwQJ5432LFOBRV1HD45a YgGh9MtrAcN3z2JGMoLY7AOG1Y97cpyNdF6w9idY6JuHYIG4Xyvc0Bnthj6SdLbmrmK9 N/hpUE5z3ltwK+oW7quzQQ4xwVsEbww6YtBmo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712180352; x=1712785152; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7VBCEyaaU+8nci3Nv2Q43lb0TGQpEICGlo/quHlWch4=; b=tFX+JyXld0wnNaMbRJhtchsMJXvCS0vEzthBSYanseTH7EnN9Nt/ibqj6/9GALGKlP lp4jD632hl9CxmEWefAoBwq9UIZ6iYkGSRAB18zd+FaJucQ0jvmD6oQ6wdSC4dfWxgDk qKg9FaABERrf5lc57LlZZz75o0RrlQo6GWF+xkqdzH700gjhxlPSTQKlLxOkT6Hp24Ha SaK8bo665y1sc9FP9BHpw7yIQodftgVpbUf5ti3i9zzdaHBCAKsxbcCO2TtmKgpCzjTw 0QVbZuKtIZ64C3AZrNESrRMRuSdSthplcjvqhIennkRyx23WtKALgj/zUVQdvZEufwMU vtQw== X-Forwarded-Encrypted: i=1; AJvYcCV35U/ldG3xXOijTRD7yjMuO1yipOQQMeWvy0jSYOkYBHjeNfbqXpWZaaSfOt3bYKuO4q02aR5XbBVDLidtl85zdiU0YISvXKImSPz1fHmUalk2dm8= X-Gm-Message-State: AOJu0Yx9Zz8BxKMYLxPoxSZgFJAZgf5A7Ln14Dcth8s42qlI6Xv3L0tm nWrsIUhNZGJyxgqQD5Rz1UHkTC+29VEdTAYZN+JMPL6ePD+hK9iXOxzdlq+TCQ== X-Google-Smtp-Source: AGHT+IErr7Hms47uq2WTpFuYryFwuFomKLlaAAOJj253kxVtX/49vIxPIfd0y1MFwlJjtp1XR8ctqw== X-Received: by 2002:a05:6122:a24:b0:4ca:615e:1b61 with SMTP id 36-20020a0561220a2400b004ca615e1b61mr651230vkn.10.1712180352188; Wed, 03 Apr 2024 14:39:12 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id pi10-20020a0562144a8a00b0069903cddc96sm1750739qvb.18.2024.04.03.14.39.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 14:39:11 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , Phil Elwell , bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Jim Quinlan , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v9 3/4] PCI: brcmstb: Set downstream maximum {no-}snoop LTR values Date: Wed, 3 Apr 2024 17:39:00 -0400 Message-Id: <20240403213902.26391-4-james.quinlan@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240403213902.26391-1-james.quinlan@broadcom.com> References: <20240403213902.26391-1-james.quinlan@broadcom.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240403_143914_944219_BA471E59 X-CRM114-Status: GOOD ( 21.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Most of our systems do not have FW or ACPI, so it is up to the RC driver to set the maximum LTR {no-}snoop latency values of downstream devices. We set them to a value that is slightly smaller than the value of our internal bus timeout register. Signed-off-by: Jim Quinlan --- drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++++++++- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index e3480ca4cd57..3d08b92d5bb8 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -182,6 +182,20 @@ #define DATA_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_DATA]) #define PCIE_RGR1_SW_INIT_1(pcie) (pcie->reg_offsets[RGR1_SW_INIT_1]) +/* + * What we call "LTR_FMT" is the 16 bit latency field format: + * [15:15] Requirement bit + * [12:10] Latency scale + * [09:00] Latency value + */ +#define LTR_FMT_TO_NS(p) (FIELD_GET(GENMASK(15, 15), (p)) \ + * ((unsigned long long)FIELD_GET(GENMASK(9, 0), (p)) \ + << (FIELD_GET(GENMASK(12, 10), (p)) * 5))) +#define BRCM_LTR_MAX_SCALE 4 /* Scale==4 => Each unit is 1,048,576ns */ +#define BRCM_LTR_MAX_VALUE 9 /* Using the above scale, roughly 9.4 ms */ +#define BRCM_LTR_MAX_NS ((unsigned long long)(BRCM_LTR_MAX_VALUE \ + << (5 * BRCM_LTR_MAX_SCALE))) + /* Rescal registers */ #define PCIE_DVT_PMU_PCIE_PHY_CTRL 0xc700 #define PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS 0x3 @@ -679,6 +693,49 @@ static void brcm_extend_internal_bus_timeout(struct brcm_pcie *pcie, u32 nsec) writel(216 * timeout_us, pcie->base + REG_OFFSET); } +/* Sets downstream device latency tolerance registers to max we can handle */ +static int brcm_set_dev_ltr_max(struct pci_dev *dev, void *data) +{ + u16 ltr_cap_offset = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + u16 ltr_fmt_cur, ltr_fmt = *(u16 *)data; + unsigned long long cur_nsec; + static const u16 reg_offsets[2] = { + PCI_LTR_MAX_SNOOP_LAT, + PCI_LTR_MAX_NOSNOOP_LAT, + }; + unsigned int i; + + if (!ltr_cap_offset || !dev->ltr_path) + return 0; + + /* + * FW may have already written a value so we want to respect that + * value if it is lower than ltr_fmt. Update the current value if + * it is 0 or if the new value is less than the current. + */ + for (i = 0; i < ARRAY_SIZE(reg_offsets); i++) { + pci_read_config_word(dev, ltr_cap_offset + reg_offsets[i], + <r_fmt_cur); + cur_nsec = LTR_FMT_TO_NS(ltr_fmt_cur); + if (cur_nsec == 0 || cur_nsec > BRCM_LTR_MAX_NS) + pci_write_config_word(dev, ltr_cap_offset + + reg_offsets[i], ltr_fmt); + } + + return 0; +} + +void brcm_set_downstream_devs_ltr_max(struct brcm_pcie *pcie) +{ + struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie); + u16 ltr_fmt = FIELD_PREP(GENMASK(9, 0), BRCM_LTR_MAX_VALUE) + | FIELD_PREP(GENMASK(12, 10), BRCM_LTR_MAX_SCALE) + | GENMASK(15, 15); + + if (bridge->native_ltr) + pci_walk_bus(bridge->bus, brcm_set_dev_ltr_max, <r_fmt); +} + /* The controller is capable of serving in both RC and EP roles */ static bool brcm_pcie_rc_mode(struct brcm_pcie *pcie) { @@ -1074,8 +1131,12 @@ static int brcm_pcie_start_link(struct brcm_pcie *pcie) return -ENODEV; } - /* Extend internal bus timeout to 8ms or so */ - brcm_extend_internal_bus_timeout(pcie, SZ_8M); + /* + * Extend internal bus timeout to 8-10ms, specifically to a value + * that is slightly larger than what we are using for the max + * {no-}snoop latency we will set in downstream devices. + */ + brcm_extend_internal_bus_timeout(pcie, BRCM_LTR_MAX_NS + 1000); if (pcie->gen) brcm_pcie_set_gen(pcie, pcie->gen); @@ -1616,6 +1677,9 @@ static int brcm_pcie_probe(struct platform_device *pdev) return ret; } + if (IS_ENABLED(CONFIG_PCIEASPM)) + brcm_set_downstream_devs_ltr_max(pcie); + return 0; fail: