From patchwork Mon Nov 25 05:33:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 11259847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AFCB06C1 for ; Mon, 25 Nov 2019 05:34:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7E4A82082C for ; Mon, 25 Nov 2019 05:34:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=microsoft.com header.i=@microsoft.com header.b="Mmc2axwn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726029AbfKYFeN (ORCPT ); Mon, 25 Nov 2019 00:34:13 -0500 Received: from mail-eopbgr680109.outbound.protection.outlook.com ([40.107.68.109]:7331 "EHLO NAM04-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725882AbfKYFeN (ORCPT ); Mon, 25 Nov 2019 00:34:13 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I70rVeg89pQcqAmiHaDzjj3wprQUU3xUEnp79KIT7mBQuONcu+toK4JhHR1c1KjGD2KVokf+d9i7hVdkhgKv/X9owJgQ1LUoY/WTAF37X8U598Y/zxI61TIWM/l6MZ2p6vtqpDchKTT1C8c72GOihnMQos03GAyj+UETsRLssqq08vjpxQmhBzcxruNQQMPmbL3v2D1ZRlriWaqfJiwx3hR/7VXQi1IGCvPm0mJxYrru/MVM+rQ6w5+a0ggAhXRA0CKATnqPX9B/lGjn3llC2IWA0TqhmbKfc2tobi/3y19IaLxUPNCIzwhxgnnUWS7N/LIhV0pCC7PqrMylkNTtHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bC3B1muqgKYR4DdYCaiibwt3fQMT9P/hLKxSqPtMBOY=; b=DT3xJYaLk6gExpA4lpnGf5o5iQTjkURhtDg2M8BAaGaTCTlqKGseIsSqlVZvDGG/NDbo+snWO3RRb+JEHlHxTK5SJPhhBxBhzLM0jmrgzJkEC1XePH2NgGU4WLKDACDWPF6YAdsbeTCv6bQnLj7sPgRe6tdC7lzFrHivDYR/C6szbyv1lKRpcpIn6kxJJQR1iHMRjw+FIvTWS+iB4Woa9ZQ0kap6rgZNTG7LEqYpeZER5aMA+epU6vqzqHyaVRl8ZSZrra496AJyQTstCHGIKMUTjbg+vWwgo8f1RAhUcb4UKl3s/pJMJ7hvnY0Sj/M2WVab5LD76w7ZlVo9zwwtyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bC3B1muqgKYR4DdYCaiibwt3fQMT9P/hLKxSqPtMBOY=; b=Mmc2axwnGTNxC8vOfdmVEIHhwFX567QSV627EqP7/46nzjuat8pyg5p5UwcOOLP3Aa8E+IktAMRVc2XiGLperfCaSs6zYexr9zRT+PCSPBWGh39jMMOuL2LhO1iaPBDCV1XD1DNZlIzPrmyIxTTmzql7yP4virRt2DwatHH2G/Q= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; Received: from BYAPR21MB1141.namprd21.prod.outlook.com (20.179.57.138) by BYAPR21MB1189.namprd21.prod.outlook.com (20.179.57.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2516.1; Mon, 25 Nov 2019 05:34:08 +0000 Received: from BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89]) by BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89%3]) with mapi id 15.20.2495.014; Mon, 25 Nov 2019 05:34:08 +0000 From: Dexuan Cui To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, lorenzo.pieralisi@arm.com, bhelgaas@google.com, linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, mikelley@microsoft.com, Alexander.Levin@microsoft.com Cc: Dexuan Cui Subject: [PATCH v3 1/4] PCI: hv: Reorganize the code in preparation of hibernation Date: Sun, 24 Nov 2019 21:33:51 -0800 Message-Id: <1574660034-98780-2-git-send-email-decui@microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1574660034-98780-1-git-send-email-decui@microsoft.com> References: <1574660034-98780-1-git-send-email-decui@microsoft.com> Reply-To: decui@microsoft.com X-ClientProxiedBy: MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) To BYAPR21MB1141.namprd21.prod.outlook.com (2603:10b6:a03:108::10) MIME-Version: 1.0 Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (13.77.154.182) by MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=) via Frontend Transport; Mon, 25 Nov 2019 05:34:07 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [13.77.154.182] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: fba6987c-47e8-444e-b7c5-08d7716917e5 X-MS-TrafficTypeDiagnostic: BYAPR21MB1189:|BYAPR21MB1189: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3826; X-Forefront-PRVS: 0232B30BBC X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(376002)(366004)(136003)(346002)(189003)(199004)(6116002)(3846002)(4326008)(107886003)(6486002)(6436002)(10090500001)(3450700001)(2906002)(14444005)(50466002)(48376002)(25786009)(6512007)(76176011)(26005)(6506007)(386003)(51416003)(52116002)(66946007)(66556008)(66476007)(6636002)(4720700003)(86362001)(7736002)(186003)(2616005)(956004)(16526019)(36756003)(11346002)(446003)(66066001)(8936002)(8676002)(5660300002)(81156014)(47776003)(81166006)(1511001)(50226002)(10290500003)(478600001)(316002)(305945005)(6666004)(16586007)(43066004)(22452003)(921003)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:BYAPR21MB1189;H:BYAPR21MB1141.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; Received-SPF: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JaPC36N20U/JK1VMxBR0yMOSA4wSsIt2O627aawD+297a753FniAZgQZtcMf97iqcIQ0j0qqP34ienMVu9jWFhXroNJ9hspV87DAdW0w3Pn2TXzoj3b47pgx2jyg7dgq4cLA20OByzmoUpI88YuYB96omwOf+pCWl+eH7VQXy514NsIBevBMq1hUXCXiy6fWYb0SlHyw0L1uxGXkMnj7iwt0LEVd8Uos2mngGsQ7bEF6/m8PVU3g9m3g9bWkCdIC1ft4JblISRld9ng82e/coNf261eSGq+2jmJFZufNFzD340UFHcH4xoYCQOAoGs4FyaAc0gAQYQBMR+9lv9+Mj0VNnZrv9dChc5qREV6Vw520THSs4weifJ0Fl13rOfYH7k6AKuhTav7x/eW9doXHCmRwc1mGiN/FGIhbdmgnJSIu24MJV92OQ6y9xWu3Lah0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: fba6987c-47e8-444e-b7c5-08d7716917e5 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2019 05:34:08.5361 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zY31RFQzbXbpTsHOEuRE3921jG7oALHJpviMdnh4f0pYZcg1SEwUz79mipGodj6ZrVhhgHAso4VtkYDOdQ+Nnw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR21MB1189 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org There is no functional change. This is just preparatory to a later patch which adds the hibernation support for the pci-hyperv driver. Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley --- drivers/pci/controller/pci-hyperv.c | 43 +++++++++++++++++++---------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index f1f300218fab..65f18f840ce9 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -2379,7 +2379,9 @@ static void hv_pci_onchannelcallback(void *context) * failing if the host doesn't support the necessary protocol * level. */ -static int hv_pci_protocol_negotiation(struct hv_device *hdev) +static int hv_pci_protocol_negotiation(struct hv_device *hdev, + enum pci_protocol_version_t version[], + int num_version) { struct pci_version_request *version_req; struct hv_pci_compl comp_pkt; @@ -2403,8 +2405,8 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev) version_req = (struct pci_version_request *)&pkt->message; version_req->message_type.type = PCI_QUERY_PROTOCOL_VERSION; - for (i = 0; i < ARRAY_SIZE(pci_protocol_versions); i++) { - version_req->protocol_version = pci_protocol_versions[i]; + for (i = 0; i < num_version; i++) { + version_req->protocol_version = version[i]; ret = vmbus_sendpacket(hdev->channel, version_req, sizeof(struct pci_version_request), (unsigned long)pkt, VM_PKT_DATA_INBAND, @@ -2420,7 +2422,7 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev) } if (comp_pkt.completion_status >= 0) { - pci_protocol_version = pci_protocol_versions[i]; + pci_protocol_version = version[i]; dev_info(&hdev->device, "PCI VMBus probing: Using version %#x\n", pci_protocol_version); @@ -2930,7 +2932,8 @@ static int hv_pci_probe(struct hv_device *hdev, hv_set_drvdata(hdev, hbus); - ret = hv_pci_protocol_negotiation(hdev); + ret = hv_pci_protocol_negotiation(hdev, pci_protocol_versions, + ARRAY_SIZE(pci_protocol_versions)); if (ret) goto close; @@ -3011,7 +3014,7 @@ static int hv_pci_probe(struct hv_device *hdev, return ret; } -static void hv_pci_bus_exit(struct hv_device *hdev) +static int hv_pci_bus_exit(struct hv_device *hdev, bool hibernating) { struct hv_pcibus_device *hbus = hv_get_drvdata(hdev); struct { @@ -3027,16 +3030,20 @@ static void hv_pci_bus_exit(struct hv_device *hdev) * access the per-channel ringbuffer any longer. */ if (hdev->channel->rescind) - return; + return 0; - /* Delete any children which might still exist. */ - memset(&relations, 0, sizeof(relations)); - hv_pci_devices_present(hbus, &relations); + if (!hibernating) { + /* Delete any children which might still exist. */ + memset(&relations, 0, sizeof(relations)); + hv_pci_devices_present(hbus, &relations); + } ret = hv_send_resources_released(hdev); - if (ret) + if (ret) { dev_err(&hdev->device, "Couldn't send resources released packet(s)\n"); + return ret; + } memset(&pkt.teardown_packet, 0, sizeof(pkt.teardown_packet)); init_completion(&comp_pkt.host_event); @@ -3049,8 +3056,13 @@ static void hv_pci_bus_exit(struct hv_device *hdev) (unsigned long)&pkt.teardown_packet, VM_PKT_DATA_INBAND, VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED); - if (!ret) - wait_for_completion_timeout(&comp_pkt.host_event, 10 * HZ); + if (ret) + return ret; + + if (wait_for_completion_timeout(&comp_pkt.host_event, 10 * HZ) == 0) + return -ETIMEDOUT; + + return 0; } /** @@ -3062,6 +3074,7 @@ static void hv_pci_bus_exit(struct hv_device *hdev) static int hv_pci_remove(struct hv_device *hdev) { struct hv_pcibus_device *hbus; + int ret; hbus = hv_get_drvdata(hdev); if (hbus->state == hv_pcibus_installed) { @@ -3074,7 +3087,7 @@ static int hv_pci_remove(struct hv_device *hdev) hbus->state = hv_pcibus_removed; } - hv_pci_bus_exit(hdev); + ret = hv_pci_bus_exit(hdev, false); vmbus_close(hdev->channel); @@ -3091,7 +3104,7 @@ static int hv_pci_remove(struct hv_device *hdev) hv_put_dom_num(hbus->sysdata.domain); free_page((unsigned long)hbus); - return 0; + return ret; } static const struct hv_vmbus_device_id hv_pci_id_table[] = { From patchwork Mon Nov 25 05:33:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 11259853 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 183476C1 for ; Mon, 25 Nov 2019 05:38:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D0CFF20748 for ; Mon, 25 Nov 2019 05:38:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=microsoft.com header.i=@microsoft.com header.b="BRIfs3FN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725770AbfKYFiT (ORCPT ); Mon, 25 Nov 2019 00:38:19 -0500 Received: from mail-eopbgr680109.outbound.protection.outlook.com ([40.107.68.109]:7331 "EHLO NAM04-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725992AbfKYFiT (ORCPT ); Mon, 25 Nov 2019 00:38:19 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J727B7+1r45LJ+kzG/N1xFVquMlw5ZMIrK69clew/pFNhor4bopqDyknaNRDmGTQ/SuHs9wh2d+JxdJEhKjsNw95W2TQIYEX3fvTgxNawA6wbl2nHH1f7xUFkWKqDur3Bb4YrrgcXbNW697FSlmGSRJTKVHYaNLe37Trrf6ajbDFC3eTmDmnVHNu7m7pszwQk4gKN9s1QH6siSfdBezL+ygXdEdbuwe/s1rXXoDTeJVlZmwkX54Y0SwRSUOcppeuFCgYI4GY+J0ggiB+3epqFjAT9kThweSZX78SGqf7t8xifLaxhctYjYIv1ttY6loHn8c4cyDdTg0z/tLjFPZGVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R9vqP/9OMnBq4PWc3YRBg0/Eteu06Bqp8qfNu+jf7pQ=; b=QIDOBqtAfjvHswCBDwsrNBBH1S5Gb2iIoUkm9xOKScWPf1sEiLTogR4ZYmDoN2b0jAzFtLk2bwnGYdA5HlC7oNkcAAw+bj7zzJCWc8zEsjsFtUCIk66EUZX6DTld0/adb19j71WBOJkaddFsHe703sD+95GHsS8Jif/4EFZ3Zs9RmcWK3TGh/XmXlfX1YZYnyHdragHMUcR0Am81+IBIz2rZLFj/BA9c+RtbuIfP1+do4TBcYDC4KRRpNDW67ylLkFlQEEEuzP2kEO/okRs0EkEt56UCHES3TU9OyY77eniFIgo1rTurb6QI4P98SrnsR35iQAM2ALco3w/2L/UrnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R9vqP/9OMnBq4PWc3YRBg0/Eteu06Bqp8qfNu+jf7pQ=; b=BRIfs3FNe8kyV7SGQzwFviOIrV1HmnlTmP9iehp81RiLYvTyUqriGRE7LlUnGd0J+JsU9iWZcn+E6DsyVw4hm2I6+W2cq8bnka2S9gaYMot0bdN8H5IqOkF/N1jozrl2bILhiF60832fXbDHWJa74cIX4rcicV4A+A20R9BbmDM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; Received: from BYAPR21MB1141.namprd21.prod.outlook.com (20.179.57.138) by BYAPR21MB1189.namprd21.prod.outlook.com (20.179.57.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2516.1; Mon, 25 Nov 2019 05:34:09 +0000 Received: from BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89]) by BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89%3]) with mapi id 15.20.2495.014; Mon, 25 Nov 2019 05:34:09 +0000 From: Dexuan Cui To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, lorenzo.pieralisi@arm.com, bhelgaas@google.com, linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, mikelley@microsoft.com, Alexander.Levin@microsoft.com Cc: Dexuan Cui Subject: [PATCH v3 2/4] PCI: hv: Add the support of hibernation Date: Sun, 24 Nov 2019 21:33:52 -0800 Message-Id: <1574660034-98780-3-git-send-email-decui@microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1574660034-98780-1-git-send-email-decui@microsoft.com> References: <1574660034-98780-1-git-send-email-decui@microsoft.com> Reply-To: decui@microsoft.com X-ClientProxiedBy: MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) To BYAPR21MB1141.namprd21.prod.outlook.com (2603:10b6:a03:108::10) MIME-Version: 1.0 Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (13.77.154.182) by MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=) via Frontend Transport; Mon, 25 Nov 2019 05:34:08 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [13.77.154.182] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 0dd4cf78-adbb-4da6-5d8e-08d771691851 X-MS-TrafficTypeDiagnostic: BYAPR21MB1189:|BYAPR21MB1189: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-Forefront-PRVS: 0232B30BBC X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(376002)(366004)(136003)(346002)(189003)(199004)(6116002)(3846002)(4326008)(107886003)(6486002)(6436002)(10090500001)(3450700001)(2906002)(14444005)(50466002)(48376002)(25786009)(6512007)(76176011)(26005)(6506007)(386003)(51416003)(52116002)(66946007)(66556008)(66476007)(6636002)(4720700003)(86362001)(7736002)(186003)(2616005)(956004)(16526019)(36756003)(11346002)(446003)(66066001)(8936002)(8676002)(5660300002)(81156014)(47776003)(81166006)(1511001)(50226002)(10290500003)(478600001)(316002)(305945005)(6666004)(16586007)(43066004)(22452003)(921003)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:BYAPR21MB1189;H:BYAPR21MB1141.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; Received-SPF: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hDiYusnySB1OkwPh+HcX2gIchH0lcsCzal1X1AyIBfO3TXGK/RrPUO62CXdQlXNmZ9Xxquyf6a3UNVw2eNB0dXvUtjjqpRhJZb1jIkNnMrEKKHAokEhTMKa07cXar6xDufAhW6cT8GsPOpvqw/uB1HPRFz/+WKOAI8x8Gn4Yl9n/Xca9MSKHcq9PDDJc5mTPk6N9TH1KNUGtPgnR9VDXzanhK4cMk0vgwYE2DQcrIEjIK4AuUD6NriVvj29xz26aNHx/45LHYmb3LM0dBs7eJb5VtYoryeWeke2VKV75yIeeID4hF6ZImwWTpeZXOenXbakM1EdtUWounfx0bjTWAEkLsCKXj4Ab5NodHoW1aLiUKJghwWBipzwf/4QlpSsyFyvn9mztbqHy/AuLuWdLW6WDTiVuKB1B4ZfV6BJJtUmLjKhmyhvAj/EHnnlovbTS X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0dd4cf78-adbb-4da6-5d8e-08d771691851 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2019 05:34:09.1536 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 83UU6rd9ae9SazHyMLLWCk92KCw773TuOQAR77IAR8fBU3XXSHJkIhkFBLzG2m9rMK6SLTLYNW//HfEAsVeHRw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR21MB1189 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add suspend() and resume() functions so that Hyper-V virtual PCI devices are handled properly when the VM hibernates and resumes from hibernation. Note that the suspend() function must make sure there are no pending work items before calling vmbus_close(), since it runs in a process context as a callback in dpm_suspend(). When it starts to run, the channel callback hv_pci_onchannelcallback(), which runs in a tasklet context, can be still running concurrently and scheduling new work items onto hbus->wq in hv_pci_devices_present() and hv_pci_eject_device(), and the work item handlers can access the vmbus channel, which can be being closed by hv_pci_suspend(), e.g. the work item handler pci_devices_present_work() -> new_pcichild_device() writes to the vmbus channel. To eliminate the race, hv_pci_suspend() disables the channel callback tasklet, sets hbus->state to hv_pcibus_removing, and re-enables the tasklet. This way, when hv_pci_suspend() proceeds, it knows that no new work item can be scheduled, and then it flushes hbus->wq and safely closes the vmbus channel. Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley --- drivers/pci/controller/pci-hyperv.c | 125 +++++++++++++++++++++++++++- 1 file changed, 123 insertions(+), 2 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 65f18f840ce9..3c4996b073ca 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -455,6 +455,7 @@ enum hv_pcibus_state { hv_pcibus_init = 0, hv_pcibus_probed, hv_pcibus_installed, + hv_pcibus_removing, hv_pcibus_removed, hv_pcibus_maximum }; @@ -1681,6 +1682,23 @@ static void prepopulate_bars(struct hv_pcibus_device *hbus) spin_lock_irqsave(&hbus->device_list_lock, flags); + /* + * Clear the memory enable bit, in case it's already set. This occurs + * in the suspend path of hibernation, where the device is suspended, + * resumed and suspended again: see hibernation_snapshot() and + * hibernation_platform_enter(). + * + * If the memory enable bit is already set, Hyper-V sliently ignores + * the below BAR updates, and the related PCI device driver can not + * work, because reading from the device register(s) always returns + * 0xFFFFFFFF. + */ + list_for_each_entry(hpdev, &hbus->children, list_entry) { + _hv_pcifront_read_config(hpdev, PCI_COMMAND, 2, &command); + command &= ~PCI_COMMAND_MEMORY; + _hv_pcifront_write_config(hpdev, PCI_COMMAND, 2, command); + } + /* Pick addresses for the BARs. */ do { list_for_each_entry(hpdev, &hbus->children, list_entry) { @@ -2107,6 +2125,12 @@ static void hv_pci_devices_present(struct hv_pcibus_device *hbus, unsigned long flags; bool pending_dr; + if (hbus->state == hv_pcibus_removing) { + dev_info(&hbus->hdev->device, + "PCI VMBus BUS_RELATIONS: ignored\n"); + return; + } + dr_wrk = kzalloc(sizeof(*dr_wrk), GFP_NOWAIT); if (!dr_wrk) return; @@ -2223,11 +2247,19 @@ static void hv_eject_device_work(struct work_struct *work) */ static void hv_pci_eject_device(struct hv_pci_dev *hpdev) { + struct hv_pcibus_device *hbus = hpdev->hbus; + struct hv_device *hdev = hbus->hdev; + + if (hbus->state == hv_pcibus_removing) { + dev_info(&hdev->device, "PCI VMBus EJECT: ignored\n"); + return; + } + hpdev->state = hv_pcichild_ejecting; get_pcichild(hpdev); INIT_WORK(&hpdev->wrk, hv_eject_device_work); - get_hvpcibus(hpdev->hbus); - queue_work(hpdev->hbus->wq, &hpdev->wrk); + get_hvpcibus(hbus); + queue_work(hbus->wq, &hpdev->wrk); } /** @@ -3107,6 +3139,93 @@ static int hv_pci_remove(struct hv_device *hdev) return ret; } +static int hv_pci_suspend(struct hv_device *hdev) +{ + struct hv_pcibus_device *hbus = hv_get_drvdata(hdev); + enum hv_pcibus_state old_state; + int ret; + + /* + * hv_pci_suspend() must make sure there are no pending work items + * before calling vmbus_close(), since it runs in a process context + * as a callback in dpm_suspend(). When it starts to run, the channel + * callback hv_pci_onchannelcallback(), which runs in a tasklet + * context, can be still running concurrently and scheduling new work + * items onto hbus->wq in hv_pci_devices_present() and + * hv_pci_eject_device(), and the work item handlers can access the + * vmbus channel, which can be being closed by hv_pci_suspend(), e.g. + * the work item handler pci_devices_present_work() -> + * new_pcichild_device() writes to the vmbus channel. + * + * To eliminate the race, hv_pci_suspend() disables the channel + * callback tasklet, sets hbus->state to hv_pcibus_removing, and + * re-enables the tasklet. This way, when hv_pci_suspend() proceeds, + * it knows that no new work item can be scheduled, and then it flushes + * hbus->wq and safely closes the vmbus channel. + */ + tasklet_disable(&hdev->channel->callback_event); + + /* Change the hbus state to prevent new work items. */ + old_state = hbus->state; + if (hbus->state == hv_pcibus_installed) + hbus->state = hv_pcibus_removing; + + tasklet_enable(&hdev->channel->callback_event); + + if (old_state != hv_pcibus_installed) + return -EINVAL; + + flush_workqueue(hbus->wq); + + ret = hv_pci_bus_exit(hdev, true); + if (ret) + return ret; + + vmbus_close(hdev->channel); + + return 0; +} + +static int hv_pci_resume(struct hv_device *hdev) +{ + struct hv_pcibus_device *hbus = hv_get_drvdata(hdev); + enum pci_protocol_version_t version[1]; + int ret; + + hbus->state = hv_pcibus_init; + + ret = vmbus_open(hdev->channel, pci_ring_size, pci_ring_size, NULL, 0, + hv_pci_onchannelcallback, hbus); + if (ret) + return ret; + + /* Only use the version that was in use before hibernation. */ + version[0] = pci_protocol_version; + ret = hv_pci_protocol_negotiation(hdev, version, 1); + if (ret) + goto out; + + ret = hv_pci_query_relations(hdev); + if (ret) + goto out; + + ret = hv_pci_enter_d0(hdev); + if (ret) + goto out; + + ret = hv_send_resources_allocated(hdev); + if (ret) + goto out; + + prepopulate_bars(hbus); + + hbus->state = hv_pcibus_installed; + return 0; +out: + vmbus_close(hdev->channel); + return ret; +} + static const struct hv_vmbus_device_id hv_pci_id_table[] = { /* PCI Pass-through Class ID */ /* 44C4F61D-4444-4400-9D52-802E27EDE19F */ @@ -3121,6 +3240,8 @@ static struct hv_driver hv_pci_drv = { .id_table = hv_pci_id_table, .probe = hv_pci_probe, .remove = hv_pci_remove, + .suspend = hv_pci_suspend, + .resume = hv_pci_resume, }; static void __exit exit_hv_pci_drv(void) From patchwork Mon Nov 25 05:33:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 11259849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0ADA36C1 for ; Mon, 25 Nov 2019 05:34:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D2C7920674 for ; Mon, 25 Nov 2019 05:34:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=microsoft.com header.i=@microsoft.com header.b="CFtGevBL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725893AbfKYFeu (ORCPT ); Mon, 25 Nov 2019 00:34:50 -0500 Received: from mail-eopbgr680091.outbound.protection.outlook.com ([40.107.68.91]:28903 "EHLO NAM04-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725793AbfKYFeu (ORCPT ); Mon, 25 Nov 2019 00:34:50 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ETa1cVpVZC1TU7kNWt8mRKgv/pCsUGZwRcl+3FdvcaQAaby3gYjBQdDMraq4YsWP0cBQiCmO/rUIXHl/XPR3uhemeFx8xK8etELbiUCfu4gJYFVBTDkDep7okpLxeWxtElDXALidJEaJOAdOB0pjJNTWu1A3IchuMVF1ccbHJV1wD6faqQ61xnbUX0Cb3YZ0JXUmvnfyjWseE9gshDpzWGzidRJ2+q+yG/28Rrh7Vg8ZAgBvJjQuXbV9iza+koscLjFjRxzPjdPySIVnkb+MtjcEt1/SSgQr7rvc3XYo6HW/+6QNMmNCjqLzK8N122gXd69SYG5PZmsoMKI+RcJPkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ofaxl54vB3/BhWtrNaSdctzkbcMoXSCRWe1yDmN8/N0=; b=ciI7b63hPTGDifvm7MxrWkwZz6ADG4k4n9CcZ2NK3SOF0yye1B390A0v0FyK3hFTV1aOUmBHHcBDW+5AeSiMZM2RDItp2rtiXzQogiGgTx4uOXAYqQFkFcy1TN54KtmmMkVGPq3raGvAsFyIpTf0ZQMFguK0O2IXr7a2Xv8i+W2mnvxJTexXtIWXMBgIPInQ+AEjWDtSYjH5qno76KkeiDXWqQwLuYRotvHzB8gBp/FsNEz0yEJ5Wmgd7EChemt4aiAiZ1g5h8+juFCBknvqtnZK7Xugb3tLgw3aGAS3lfZWE+bQndE80uhMYEW8gcPdaNaiptV7mytArizpJBbPBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ofaxl54vB3/BhWtrNaSdctzkbcMoXSCRWe1yDmN8/N0=; b=CFtGevBLFc4SZH+vdF1lx0dV9SWR0BxGhG8HDCfJeBr8/Du7z6oV9bdYPwZt/avX+ui/qcCsf5zGoua5CSFNmK8HtDZ9NyTFu3XtB5VSVG4Uni0OeN61Qzrpip7JSc6t3U5+DvPVPjf7s9PkHLmkSDZf1u9xr2tyn8howSZksqs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; Received: from BYAPR21MB1141.namprd21.prod.outlook.com (20.179.57.138) by BYAPR21MB1189.namprd21.prod.outlook.com (20.179.57.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2516.1; Mon, 25 Nov 2019 05:34:09 +0000 Received: from BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89]) by BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89%3]) with mapi id 15.20.2495.014; Mon, 25 Nov 2019 05:34:09 +0000 From: Dexuan Cui To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, lorenzo.pieralisi@arm.com, bhelgaas@google.com, linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, mikelley@microsoft.com, Alexander.Levin@microsoft.com Cc: Dexuan Cui Subject: [PATCH v3 3/4] PCI: hv: Change pci_protocol_version to per-hbus Date: Sun, 24 Nov 2019 21:33:53 -0800 Message-Id: <1574660034-98780-4-git-send-email-decui@microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1574660034-98780-1-git-send-email-decui@microsoft.com> References: <1574660034-98780-1-git-send-email-decui@microsoft.com> Reply-To: decui@microsoft.com X-ClientProxiedBy: MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) To BYAPR21MB1141.namprd21.prod.outlook.com (2603:10b6:a03:108::10) MIME-Version: 1.0 Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (13.77.154.182) by MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=) via Frontend Transport; Mon, 25 Nov 2019 05:34:09 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [13.77.154.182] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: ed358042-5a07-4a38-dddc-08d7716918b1 X-MS-TrafficTypeDiagnostic: BYAPR21MB1189:|BYAPR21MB1189: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-Forefront-PRVS: 0232B30BBC X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(376002)(366004)(136003)(346002)(189003)(199004)(6116002)(3846002)(4326008)(107886003)(6486002)(6436002)(10090500001)(3450700001)(2906002)(14444005)(50466002)(48376002)(25786009)(6512007)(76176011)(26005)(6506007)(386003)(51416003)(52116002)(66946007)(66556008)(66476007)(6636002)(4720700003)(86362001)(7736002)(186003)(2616005)(956004)(16526019)(36756003)(11346002)(446003)(66066001)(8936002)(8676002)(5660300002)(81156014)(47776003)(81166006)(1511001)(50226002)(10290500003)(478600001)(316002)(305945005)(6666004)(16586007)(43066004)(22452003)(921003)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:BYAPR21MB1189;H:BYAPR21MB1141.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; Received-SPF: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HI2Z+xHsGezDNApzV9RgWplaJQprpIIQ5KSn3cmf3bcFDre7RI0MrSbZy+LsavP2mW4oafVBiriJdoHqLeyfvEWWe2ql5iucFze/CctRceyNJF9Iq/ECtA5dM/PPHP3IzaTMnouFlDaheEwmuC4dSeQnONarl5A75SntkG6hx5PZtxI6SuUUb+Y1UCaswcjDsDd8Gt/Q0xGlDMB8YoLd9PPusQBR52jKd+Y+6etwtth3Aapxedd5fK2zEI60oDmg5U4t551u4Z+Ml70b+WxeMZesCPjTbc+QaDMSdIokWRkK9zamD4vR2rscpNQCj/v5dNQAsaPIp7s2qwBoDU/stZYeRnIMe3IAPOcRM/XsOdkE8mkKxcJUh+apV0gQiOpMAfIrvuI6i2OxmCcMG+4fYZxMUxuxd19IrSq5sGQceS83QzMxwObLcnuxA7eNBZvR X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: ed358042-5a07-4a38-dddc-08d7716918b1 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2019 05:34:09.8090 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QHIfyWYOX3kYYRTvhk80m+44Z0joHNRmdfkZl+/w9/Uj0z6CJyvCW6vFujImdyki2UlgLNY88ZDw0z0AycamPA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR21MB1189 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org A VM can have multiple Hyper-V hbus. It's incorrect to set the global variable 'pci_protocol_version' when *every* hbus is initialized in hv_pci_protocol_negotiation(). This is not an issue in practice since every hbus should have the same value of hbus->protocol_version, but we should make the variable per-hbus, so in case we have busses with different protocol_versions, the driver can still work correctly. Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley --- drivers/pci/controller/pci-hyperv.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 3c4996b073ca..910fa016d095 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -76,11 +76,6 @@ static enum pci_protocol_version_t pci_protocol_versions[] = { PCI_PROTOCOL_VERSION_1_1, }; -/* - * Protocol version negotiated by hv_pci_protocol_negotiation(). - */ -static enum pci_protocol_version_t pci_protocol_version; - #define PCI_CONFIG_MMIO_LENGTH 0x2000 #define CFG_PAGE_OFFSET 0x1000 #define CFG_PAGE_SIZE (PCI_CONFIG_MMIO_LENGTH - CFG_PAGE_OFFSET) @@ -462,6 +457,8 @@ enum hv_pcibus_state { struct hv_pcibus_device { struct pci_sysdata sysdata; + /* Protocol version negotiated with the host */ + enum pci_protocol_version_t protocol_version; enum hv_pcibus_state state; refcount_t remove_lock; struct hv_device *hdev; @@ -1225,7 +1222,7 @@ static void hv_irq_unmask(struct irq_data *data) * negative effect (yet?). */ - if (pci_protocol_version >= PCI_PROTOCOL_VERSION_1_2) { + if (hbus->protocol_version >= PCI_PROTOCOL_VERSION_1_2) { /* * PCI_PROTOCOL_VERSION_1_2 supports the VP_SET version of the * HVCALL_RETARGET_INTERRUPT hypercall, which also coincides @@ -1395,7 +1392,7 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) ctxt.pci_pkt.completion_func = hv_pci_compose_compl; ctxt.pci_pkt.compl_ctxt = ∁ - switch (pci_protocol_version) { + switch (hbus->protocol_version) { case PCI_PROTOCOL_VERSION_1_1: size = hv_compose_msi_req_v1(&ctxt.int_pkts.v1, dest, @@ -2415,6 +2412,7 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev, enum pci_protocol_version_t version[], int num_version) { + struct hv_pcibus_device *hbus = hv_get_drvdata(hdev); struct pci_version_request *version_req; struct hv_pci_compl comp_pkt; struct pci_packet *pkt; @@ -2454,10 +2452,10 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev, } if (comp_pkt.completion_status >= 0) { - pci_protocol_version = version[i]; + hbus->protocol_version = version[i]; dev_info(&hdev->device, "PCI VMBus probing: Using version %#x\n", - pci_protocol_version); + hbus->protocol_version); goto exit; } @@ -2741,7 +2739,7 @@ static int hv_send_resources_allocated(struct hv_device *hdev) u32 wslot; int ret; - size_res = (pci_protocol_version < PCI_PROTOCOL_VERSION_1_2) + size_res = (hbus->protocol_version < PCI_PROTOCOL_VERSION_1_2) ? sizeof(*res_assigned) : sizeof(*res_assigned2); pkt = kmalloc(sizeof(*pkt) + size_res, GFP_KERNEL); @@ -2760,7 +2758,7 @@ static int hv_send_resources_allocated(struct hv_device *hdev) pkt->completion_func = hv_pci_generic_compl; pkt->compl_ctxt = &comp_pkt; - if (pci_protocol_version < PCI_PROTOCOL_VERSION_1_2) { + if (hbus->protocol_version < PCI_PROTOCOL_VERSION_1_2) { res_assigned = (struct pci_resources_assigned *)&pkt->message; res_assigned->message_type.type = @@ -3200,7 +3198,7 @@ static int hv_pci_resume(struct hv_device *hdev) return ret; /* Only use the version that was in use before hibernation. */ - version[0] = pci_protocol_version; + version[0] = hbus->protocol_version; ret = hv_pci_protocol_negotiation(hdev, version, 1); if (ret) goto out; From patchwork Mon Nov 25 05:33:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 11259851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 440856C1 for ; Mon, 25 Nov 2019 05:34:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1B94720674 for ; Mon, 25 Nov 2019 05:34:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=microsoft.com header.i=@microsoft.com header.b="dsqimAJc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727128AbfKYFev (ORCPT ); Mon, 25 Nov 2019 00:34:51 -0500 Received: from mail-eopbgr680091.outbound.protection.outlook.com ([40.107.68.91]:28903 "EHLO NAM04-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725882AbfKYFev (ORCPT ); Mon, 25 Nov 2019 00:34:51 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TUrcWutSwUTfSdcplrhCBefqR7ghxP1DajOcRMH0sLFZhGx8jtC5oRdmHaIjMrK5TWpqmrT1ITNfaaDMzQfGJQVd9NVcAPlmfKFOq/hmF+uEBZyc6GrbYvEqkgcrlZ/+ViApK0gKrtbfSdHsqLpn7/16FIapfC6Su+uE5zqHjfrEtD5GyLgtnZq3H7k3N9vnoOnbNrHPkAT6n1070mQH7ZquPXUpIMlfqBYH1JykHbVptBbnFoGGneU/K21W0jY2uG9xuiC1hgmD9UVC0ddaTOadQohQShgLHRnR53g6+Sat0z2eFVCLMedJHkOE7gRqKQsYAzePu0j0NFU1Egl57w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gEgzxQKdiYlMJ9oSRnvz5UH5uj0AWm6gNaaS12cBPMI=; b=KYarBpSPMspjj7bcO4r3rnUjtT5V3QQfGPVk9ZwUX0VJwyF3089y0djyv4EkcAAoKb411YwvLq0WmtVUomk9RRE7yEp4M0b8PPzbKedmhSZdjKPy8OJb3P+6TNAA4orte0fvdJZHd4cRxxJWTgG7AfzFWYTXGZFQ+JZsP+LdtvLMW8ZmKwnIabyIJsWL36F9IDkHHFZBRjWv4W0H/68V/5+KlNqspmpvtUDPwMUrn7g6vFZuM5bc7ZXK66TVB+NDMFg450a6gDj6e/dyRp6PiLzfftMfU+ZVlMK3RBv30el7eBOalK45E4lRQ5irq2nT8v+2d3OirZr270UGXwajcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gEgzxQKdiYlMJ9oSRnvz5UH5uj0AWm6gNaaS12cBPMI=; b=dsqimAJcL9YB0ro4KPpiBej3XUHestPSdIl0+m3i+w+OQ1EJOlxlrh0yCKf2CTpNN9SUdVY2L0XdIu8mFEKzatolpL3nM5qrZhEFEF2ObU7m5j+yLCOkc0UrIIkk1I05wuo6UrCmMkL1BvBbbfZ5GKSTKumDipV8FQJkDHXwUAQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=decui@microsoft.com; Received: from BYAPR21MB1141.namprd21.prod.outlook.com (20.179.57.138) by BYAPR21MB1189.namprd21.prod.outlook.com (20.179.57.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2516.1; Mon, 25 Nov 2019 05:34:10 +0000 Received: from BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89]) by BYAPR21MB1141.namprd21.prod.outlook.com ([fe80::5d0f:2e49:3464:7c89%3]) with mapi id 15.20.2495.014; Mon, 25 Nov 2019 05:34:10 +0000 From: Dexuan Cui To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, lorenzo.pieralisi@arm.com, bhelgaas@google.com, linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, mikelley@microsoft.com, Alexander.Levin@microsoft.com Cc: Dexuan Cui Subject: [PATCH v3 4/4] PCI: hv: Avoid a kmemleak false positive caused by the hbus buffer Date: Sun, 24 Nov 2019 21:33:54 -0800 Message-Id: <1574660034-98780-5-git-send-email-decui@microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1574660034-98780-1-git-send-email-decui@microsoft.com> References: <1574660034-98780-1-git-send-email-decui@microsoft.com> Reply-To: decui@microsoft.com X-ClientProxiedBy: MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) To BYAPR21MB1141.namprd21.prod.outlook.com (2603:10b6:a03:108::10) MIME-Version: 1.0 Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (13.77.154.182) by MWHPR20CA0011.namprd20.prod.outlook.com (2603:10b6:300:13d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=) via Frontend Transport; Mon, 25 Nov 2019 05:34:09 +0000 X-Mailer: git-send-email 1.8.3.1 X-Originating-IP: [13.77.154.182] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 8f33cce4-a402-4df3-52fe-08d771691918 X-MS-TrafficTypeDiagnostic: BYAPR21MB1189:|BYAPR21MB1189: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-Forefront-PRVS: 0232B30BBC X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(376002)(366004)(136003)(346002)(189003)(199004)(6116002)(3846002)(4326008)(107886003)(6486002)(6436002)(10090500001)(3450700001)(2906002)(14444005)(50466002)(48376002)(25786009)(6512007)(76176011)(26005)(6506007)(386003)(51416003)(52116002)(66946007)(66556008)(66476007)(6636002)(4720700003)(86362001)(7736002)(186003)(2616005)(956004)(16526019)(36756003)(11346002)(446003)(66066001)(8936002)(8676002)(5660300002)(81156014)(47776003)(81166006)(1511001)(50226002)(10290500003)(478600001)(316002)(305945005)(6666004)(16586007)(43066004)(22452003)(921003)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:BYAPR21MB1189;H:BYAPR21MB1141.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; Received-SPF: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rIk66HaTeCVEQuD1U+y2tg4s2XP1YMhUK0XBfpDsjDUAqhJlISKHQ10gNxE149aUiLc9mXFyBdSbnkzDlmbYlSby/Dh+SsVLNXLsgaMGYGr40tq3nuBhvjM+w6dYF+339dcDoBPa5cTXVFQVNIwBTkuPzjQe+51Ai7PnPfp0ZCiP9Ba+L6O0F6A2YRYu1MYtjO3GEiUWXH8WNDQkDaN0P8aZzAZi7xE//WCbBKOdYD9jYdOppbR1WgvqHFSHQyCQY7JXbKLwIbM8oq4BlZq+FRGwVUl3LRwofPvR6VXX4+MOVXMTOnYvONvd48EFa4g9r8cTmGdFreCs6NEnGzyWzUmxk/VcAHvzDkw7jZncMqyHyMO71nZWoRKj6VLoWNVw9zzxRwZ8quTD9YfLpWw6I8WQN2i03VLsr9omnq96XiZdPojdQipor1uPKkeRJGzw X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8f33cce4-a402-4df3-52fe-08d771691918 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2019 05:34:10.4915 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 52KrRUuUFFr0VFw66M/kF9uP28S3L4rs0tzQbXhjYBDieuN/xfZa1mo3ssUMBBap/BgTM3fclP0u9l3MbVRfaQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR21MB1189 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org With the recent 59bb47985c1d ("mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two)"), kzalloc() is able to allocate a 4KB buffer that is guaranteed to be 4KB-aligned. Here the size and alignment of hbus is important because hbus's field retarget_msi_interrupt_params must not cross a 4KB page boundary. Here we prefer kzalloc to get_zeroed_page(), because a buffer allocated by the latter is not tracked and scanned by kmemleak, and hence kmemleak reports the pointer contained in the hbus buffer (i.e. the hpdev struct, which is created in new_pcichild_device() and is tracked by hbus->children) as memory leak (false positive). If the kernel doesn't have 59bb47985c1d, get_zeroed_page() *must* be used to allocate the hbus buffer and we can avoid the kmemleak false positive by using kmemleak_alloc() and kmemleak_free() to ask kmemleak to track and scan the hbus buffer. Reported-by: Lili Deng Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley --- drivers/pci/controller/pci-hyperv.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 910fa016d095..be99862166d0 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -2902,9 +2902,27 @@ static int hv_pci_probe(struct hv_device *hdev, * hv_pcibus_device contains the hypercall arguments for retargeting in * hv_irq_unmask(). Those must not cross a page boundary. */ - BUILD_BUG_ON(sizeof(*hbus) > PAGE_SIZE); + BUILD_BUG_ON(sizeof(*hbus) > HV_HYP_PAGE_SIZE); - hbus = (struct hv_pcibus_device *)get_zeroed_page(GFP_KERNEL); + /* + * With the recent 59bb47985c1d ("mm, sl[aou]b: guarantee natural + * alignment for kmalloc(power-of-two)"), kzalloc() is able to allocate + * a 4KB buffer that is guaranteed to be 4KB-aligned. Here the size and + * alignment of hbus is important because hbus's field + * retarget_msi_interrupt_params must not cross a 4KB page boundary. + * + * Here we prefer kzalloc to get_zeroed_page(), because a buffer + * allocated by the latter is not tracked and scanned by kmemleak, and + * hence kmemleak reports the pointer contained in the hbus buffer + * (i.e. the hpdev struct, which is created in new_pcichild_device() and + * is tracked by hbus->children) as memory leak (false positive). + * + * If the kernel doesn't have 59bb47985c1d, get_zeroed_page() *must* be + * used to allocate the hbus buffer and we can avoid the kmemleak false + * positive by using kmemleak_alloc() and kmemleak_free() to ask + * kmemleak to track and scan the hbus buffer. + */ + hbus = (struct hv_pcibus_device *)kzalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL); if (!hbus) return -ENOMEM; hbus->state = hv_pcibus_init; @@ -3133,7 +3151,7 @@ static int hv_pci_remove(struct hv_device *hdev) hv_put_dom_num(hbus->sysdata.domain); - free_page((unsigned long)hbus); + kfree(hbus); return ret; }