From patchwork Tue Jan 3 06:34:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rajat Jain X-Patchwork-Id: 9494435 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 20E0F60414 for ; Tue, 3 Jan 2017 06:35:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12354266F3 for ; Tue, 3 Jan 2017 06:35:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 035A2269DA; Tue, 3 Jan 2017 06:35:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3EDC266F3 for ; Tue, 3 Jan 2017 06:35:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934550AbdACGew (ORCPT ); Tue, 3 Jan 2017 01:34:52 -0500 Received: from mail-pg0-f44.google.com ([74.125.83.44]:35436 "EHLO mail-pg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934536AbdACGek (ORCPT ); Tue, 3 Jan 2017 01:34:40 -0500 Received: by mail-pg0-f44.google.com with SMTP id i5so146617018pgh.2 for ; Mon, 02 Jan 2017 22:34:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eXEtLylqUgXHXxTSJNOAM8Dd1W9JCnyu99cfivi7HyM=; b=Drk0FlyRghpAuXUI8NJf/PZsLDobLXAsYPUnv98QeytfWbnAuilwIHmMKGdkMfQDW/ zpyG/hZ9bytLFWFXw97IndaX7bUCb11a3uctNARBTIzkw3u+BCGXJf3g9hmRFY1QCW3b OaREwAfVfNcDCNNqlurk/VT04leNcVPM9GEm8HYyiSDpHIWomPtIA2hhjWnCvq751LUC K2vHNiaUEY5AlTdRiO7xYbFndc5cSl7PEe5nXZdAWwA5+Jki4j7Bp+mFDe8YitWiUxvo kMIQHHle71AOvHqAUUjIY/Z3sk9gCdRLBAFyI3adDsWIb/dSPvmrEK+YKTsm3LFwwyZK Bitw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eXEtLylqUgXHXxTSJNOAM8Dd1W9JCnyu99cfivi7HyM=; b=SoBzA/ke3D0UMSes6BA/hIMAM7bpx0LVORdzWe5W7esVvD0nHJXk0CIPYlZasNZ5IS UWQPx4gGxd+k9+aR3JQBKVVgc+Dvke5lZTqLKlllEObVcYfb3pIEyRZP4wmSXTL36g/s iu36Rv1RoqOo2TuKsaNQiX/3HuEkTuxRceVXwj79K/ksOF7V6IMaUbBoTqIxHpOjoBvT LkT5rGozNDSM/8WfG+GadChCl+4DqWKjxn8S5SoByVLthffQ/j18JV3vxif9IBbRebf5 epySfOfJLH2bFWEgy8AKwDhGp6CM2k/2ciGpqy3o4i43ip6nmWxaABXQ+fOLNIcHpc7y r7XQ== X-Gm-Message-State: AIkVDXJ6LjhenVetLO07c87MfzxXSVTc6EiZa9nE1dfspv5szjy8Iqi5AFTIIkPjeSirzoB9 X-Received: by 10.84.140.3 with SMTP id 3mr132134906pls.61.1483425279276; Mon, 02 Jan 2017 22:34:39 -0800 (PST) Received: from rajat.mtv.corp.google.com ([172.22.64.13]) by smtp.gmail.com with ESMTPSA id g28sm72936251pgn.13.2017.01.02.22.34.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 02 Jan 2017 22:34:38 -0800 (PST) From: Rajat Jain To: Bjorn Helgaas , Keith Busch , Andreas Ziegler , Jonathan Yong , Shawn Lin , David Daney , Julia Lawall , Ram Amrani , Doug Ledford , Wang Sheng-Hui , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Rajat Jain , Rajat Jain , Brian Norris Subject: [PATCH 5/6] PCI/ASPM: Actually configure the L1 substate settings to the device Date: Mon, 2 Jan 2017 22:34:14 -0800 Message-Id: <1483425255-101923-6-git-send-email-rajatja@google.com> X-Mailer: git-send-email 2.8.0.rc3.226.g39d4020 In-Reply-To: <1483425255-101923-1-git-send-email-rajatja@google.com> References: <1483425255-101923-1-git-send-email-rajatja@google.com> Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add code to actually configure the L1 substate settigns on the upstream and downstream device, while taking care of the rules dictated by the PCIe spec. Signed-off-by: Rajat Jain --- drivers/pci/pcie/aspm.c | 106 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 104 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index a70afdf..6735f38 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -588,6 +588,92 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist) } } +static inline void pci_clear_and_set_dword(struct pci_dev *pdev, int pos, + u32 clear, u32 set) +{ + u32 val; + + pci_read_config_dword(pdev, pos, &val); + val &= ~clear; + val |= set; + pci_write_config_dword(pdev, pos, val); +} + +/* Configure the ASPM L1 substates */ +static void pcie_config_aspm_l1ss(struct pcie_link_state *link, u32 state) +{ + u32 val, enable_req; + struct pci_dev *child = link->downstream, *parent = link->pdev; + u32 up_cap_ptr = link->l1ss.up_cap_ptr; + u32 dw_cap_ptr = link->l1ss.dw_cap_ptr; + + enable_req = (link->aspm_enabled ^ state) & state; + + /* + * Here are the rules specified in the PCIe spec for enabling L1SS: + * - When enabling L1.x, enable bit at parent first, then at child + * - When disabling L1.x, disable bit at child first, then at parent + * - When enabling ASPM L1.x, need to disable L1 + * (at child followed by parent). + * - The ASPM/PCIPM L1.2 must be disabled while programming timin + * parameters + * + * To keep it simple, disable all L1SS bits first, and later enable + * what is needed. + */ + + /* Disable all L1 substates */ + pci_clear_and_set_dword(child, dw_cap_ptr + PCI_L1SS_CTL1, + PCI_L1SS_CTL1_L1SS_MASK, 0); + pci_clear_and_set_dword(parent, up_cap_ptr + PCI_L1SS_CTL1, + PCI_L1SS_CTL1_L1SS_MASK, 0); + /* + * If needed, disable L1, and it gets enabled later + * in pcie_config_aspm_link(). + */ + if (enable_req & (ASPM_STATE_L1_1 | ASPM_STATE_L1_2)) { + pcie_capability_clear_and_set_word(child, PCI_EXP_LNKCTL, + PCI_EXP_LNKCTL_ASPM_L1, 0); + pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL, + PCI_EXP_LNKCTL_ASPM_L1, 0); + } + + if (enable_req & ASPM_STATE_L1_2_MASK) { + + /* Program T_pwr_on in both ports */ + pci_write_config_dword(parent, up_cap_ptr + PCI_L1SS_CTL2, + link->l1ss.ctl2); + pci_write_config_dword(child, dw_cap_ptr + PCI_L1SS_CTL2, + link->l1ss.ctl2); + + /* Program T_cmn_mode in parent */ + pci_clear_and_set_dword(parent, up_cap_ptr + PCI_L1SS_CTL1, + 0xFF00, link->l1ss.ctl1); + + /* Program LTR L1.2 threshold in both ports */ + pci_clear_and_set_dword(parent, dw_cap_ptr + PCI_L1SS_CTL1, + 0xE3FF0000, link->l1ss.ctl1); + pci_clear_and_set_dword(child, dw_cap_ptr + PCI_L1SS_CTL1, + 0xE3FF0000, link->l1ss.ctl1); + } + + val = 0; + if (state & ASPM_STATE_L1_1) + val |= PCI_L1SS_CTL1_ASPM_L1_1; + if (state & ASPM_STATE_L1_2) + val |= PCI_L1SS_CTL1_ASPM_L1_2; + if (state & ASPM_STATE_L1_1_PCIPM) + val |= PCI_L1SS_CTL1_PCIPM_L1_1; + if (state & ASPM_STATE_L1_2_PCIPM) + val |= PCI_L1SS_CTL1_PCIPM_L1_2; + + /* Enable what we need to enable */ + pci_clear_and_set_dword(parent, up_cap_ptr + PCI_L1SS_CTL1, + PCI_L1SS_CAP_L1_PM_SS, val); + pci_clear_and_set_dword(child, dw_cap_ptr + PCI_L1SS_CTL1, + PCI_L1SS_CAP_L1_PM_SS, val); +} + static void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val) { pcie_capability_clear_and_set_word(pdev, PCI_EXP_LNKCTL, @@ -597,11 +683,23 @@ static void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val) static void pcie_config_aspm_link(struct pcie_link_state *link, u32 state) { u32 upstream = 0, dwstream = 0; - struct pci_dev *child, *parent = link->pdev; + struct pci_dev *child = link->downstream, *parent = link->pdev; struct pci_bus *linkbus = parent->subordinate; - /* Nothing to do if the link is already in the requested state */ + /* Enable only the states that were not explicitly disabled */ state &= (link->aspm_capable & ~link->aspm_disable); + + /* Can't enable any substates if L1 is not enabled */ + if (!(state & ASPM_STATE_L1)) + state &= ~ASPM_STATE_L1SS; + + /* Spec says both ports must be in D0 before enabling PCI PM substates*/ + if (parent->current_state != PCI_D0 || child->current_state != PCI_D0) { + state &= ~ASPM_STATE_L1_SS_PCIPM; + state |= (link->aspm_enabled & ASPM_STATE_L1_SS_PCIPM); + } + + /* Nothing to do if the link is already in the requested state */ if (link->aspm_enabled == state) return; /* Convert ASPM state to upstream/downstream ASPM register state */ @@ -613,6 +711,10 @@ static void pcie_config_aspm_link(struct pcie_link_state *link, u32 state) upstream |= PCI_EXP_LNKCTL_ASPM_L1; dwstream |= PCI_EXP_LNKCTL_ASPM_L1; } + + if (link->aspm_capable & ASPM_STATE_L1SS) + pcie_config_aspm_l1ss(link, state); + /* * Spec 2.0 suggests all functions should be configured the * same setting for ASPM. Enabling ASPM L1 should be done in