From patchwork Tue Dec 3 03:46:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270271 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DDE1109A for ; Tue, 3 Dec 2019 03:48:37 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 56C262071F for ; Tue, 3 Dec 2019 03:48:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 56C262071F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B01FA1011366F; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2FE7F10113661 for ; Mon, 2 Dec 2019 19:51:52 -0800 (PST) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kkht001753 for ; Mon, 2 Dec 2019 22:48:29 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wn3pd3evp-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:29 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:16 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mG2Z63569958 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:16 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EADF852052; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 48B5852054; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 3C14AA03C6; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 01/27] memory_hotplug: Add a bounds check to __add_pages Date: Tue, 3 Dec 2019 14:46:29 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 19120303-0028-0000-0000-000003C3CC72 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0029-0000-0000-00002486E431 Message-Id: <20191203034655.51561-2-alastair@au1.ibm.com> X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 priorityscore=1501 mlxscore=0 adultscore=0 impostorscore=0 suspectscore=1 bulkscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: P4ZT3JN4DLLELHRU6732JWAZRWZZCXX6 X-Message-ID-Hash: P4ZT3JN4DLLELHRU6732JWAZRWZZCXX6 X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org, David Hildenbrand , Michal Hocko X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva On PowerPC, the address ranges allocated to OpenCAPI LPC memory are allocated from firmware. These address ranges may be higher than what older kernels permit, as we increased the maximum permissable address in commit 4ffe713b7587 ("powerpc/mm: Increase the max addressable memory to 2PB"). It is possible that the addressable range may change again in the future. In this scenario, we end up with a bogus section returned from __section_nr (see the discussion on the thread "mm: Trigger bug on if a section is not found in __section_nr"). Adding a check here means that we fail early and have an opportunity to handle the error gracefully, rather than rumbling on and potentially accessing an incorrect section. Further discussion is also on the thread ("powerpc: Perform a bounds check in arch_add_memory") http://lkml.kernel.org/r/20190827052047.31547-1-alastair@au1.ibm.com Signed-off-by: Alastair D'Silva Reviewed-by: David Hildenbrand Acked-by: Michal Hocko --- mm/memory_hotplug.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index f307bd82d750..b5be827ecda5 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -278,6 +278,23 @@ static int check_pfn_span(unsigned long pfn, unsigned long nr_pages, return 0; } +static int check_hotplug_memory_addressable(unsigned long pfn, + unsigned long nr_pages) +{ + const u64 max_addr = PFN_PHYS(pfn + nr_pages) - 1; + + if (max_addr >> MAX_PHYSMEM_BITS) { + const u64 max_allowed = (1ull << (MAX_PHYSMEM_BITS + 1)) - 1; + + WARN(1, + "Hotplugged memory exceeds maximum addressable address, range=%#llx-%#llx, maximum=%#llx\n", + PFN_PHYS(pfn), max_addr, max_allowed); + return -E2BIG; + } + + return 0; +} + /* * Reasonably generic function for adding memory. It is * expected that archs that support memory hotplug will @@ -291,6 +308,10 @@ int __ref __add_pages(int nid, unsigned long pfn, unsigned long nr_pages, unsigned long nr, start_sec, end_sec; struct vmem_altmap *altmap = restrictions->altmap; + err = check_hotplug_memory_addressable(pfn, nr_pages); + if (err) + return err; + if (altmap) { /* * Validate altmap is within bounds of the total request From patchwork Tue Dec 3 03:46:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17EB26C1 for ; Tue, 3 Dec 2019 03:48:30 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E654220848 for ; Tue, 3 Dec 2019 03:48:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E654220848 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 1ACB91011363F; Mon, 2 Dec 2019 19:51:52 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 702DB1011363B for ; Mon, 2 Dec 2019 19:51:50 -0800 (PST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kksL006780 for ; Mon, 2 Dec 2019 22:48:26 -0500 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6cy9sbj-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:26 -0500 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:23 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:16 -0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mFXj45088900 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:15 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AAA6142041; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 537F042042; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 44DE2A03C7; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 02/27] nvdimm: remove prototypes for nonexistent functions Date: Tue, 3 Dec 2019 14:46:30 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0020-0000-0000-00000392CC59 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0021-0000-0000-000021E9EAA8 Message-Id: <20191203034655.51561-3-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 priorityscore=1501 suspectscore=1 bulkscore=0 phishscore=0 mlxlogscore=709 adultscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: LXH3YFYF3R7CL2I7X6W2TWJDK47GQN22 X-Message-ID-Hash: LXH3YFYF3R7CL2I7X6W2TWJDK47GQN22 X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva These functions don't exist, so remove the prototypes for them. Signed-off-by: Alastair D'Silva Reviewed-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- drivers/nvdimm/nd-core.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/nvdimm/nd-core.h b/drivers/nvdimm/nd-core.h index 25fa121104d0..9f121a6aeb02 100644 --- a/drivers/nvdimm/nd-core.h +++ b/drivers/nvdimm/nd-core.h @@ -124,11 +124,7 @@ void nd_region_create_dax_seed(struct nd_region *nd_region); int nvdimm_bus_create_ndctl(struct nvdimm_bus *nvdimm_bus); void nvdimm_bus_destroy_ndctl(struct nvdimm_bus *nvdimm_bus); void nd_synchronize(void); -int nvdimm_bus_register_dimms(struct nvdimm_bus *nvdimm_bus); -int nvdimm_bus_register_regions(struct nvdimm_bus *nvdimm_bus); -int nvdimm_bus_init_interleave_sets(struct nvdimm_bus *nvdimm_bus); void __nd_device_register(struct device *dev); -int nd_match_dimm(struct device *dev, void *data); struct nd_label_id; char *nd_label_gen_id(struct nd_label_id *label_id, u8 *uuid, u32 flags); bool nd_is_uuid_unique(struct device *dev, u8 *uuid); From patchwork Tue Dec 3 03:46:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2AE8B6C1 for ; Tue, 3 Dec 2019 03:48:39 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 13E9D206DF for ; Tue, 3 Dec 2019 03:48:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 13E9D206DF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id D908510113677; Mon, 2 Dec 2019 19:51:55 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 22AEB10113660 for ; Mon, 2 Dec 2019 19:51:52 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kljd093288 for ; Mon, 2 Dec 2019 22:48:28 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 2wm6g99bmw-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:28 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:17 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mGhc35586160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:16 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EB13852057; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 4998B5205A; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 4D7DAA03CB; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 03/27] powerpc: Add OPAL calls for LPC memory alloc/release Date: Tue, 3 Dec 2019 14:46:31 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-4275-0000-0000-0000038A31DB X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-4276-0000-0000-0000389DCDD7 Message-Id: <20191203034655.51561-4-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1015 lowpriorityscore=0 bulkscore=0 phishscore=0 priorityscore=1501 impostorscore=0 suspectscore=1 spamscore=0 mlxscore=0 adultscore=0 mlxlogscore=818 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: QXYPLQSYQB65P4ECLNI7GWYAVX73CHWK X-Message-ID-Hash: QXYPLQSYQB65P4ECLNI7GWYAVX73CHWK X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva Add OPAL calls for LPC memory alloc/release Signed-off-by: Alastair D'Silva Acked-by: Andrew Donnellan Acked-by: Frederic Barrat --- arch/powerpc/include/asm/opal-api.h | 2 ++ arch/powerpc/include/asm/opal.h | 3 +++ arch/powerpc/platforms/powernv/opal-call.c | 2 ++ 3 files changed, 7 insertions(+) diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index 378e3997845a..2c88c02e69ed 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -208,6 +208,8 @@ #define OPAL_HANDLE_HMI2 166 #define OPAL_NX_COPROC_INIT 167 #define OPAL_XIVE_GET_VP_STATE 170 +#define OPAL_NPU_MEM_ALLOC 171 +#define OPAL_NPU_MEM_RELEASE 172 #define OPAL_MPIPL_UPDATE 173 #define OPAL_MPIPL_REGISTER_TAG 174 #define OPAL_MPIPL_QUERY_TAG 175 diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index a0cf8fba4d12..4db135fb54ab 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -39,6 +39,9 @@ int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t bdfn, uint64_t PE_handle); int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, long cap, uint64_t rate_phys, uint32_t size); +int64_t opal_npu_mem_alloc(uint64_t phb_id, uint32_t bdfn, + uint64_t size, uint64_t *bar); +int64_t opal_npu_mem_release(uint64_t phb_id, uint32_t bdfn); int64_t opal_console_write(int64_t term_number, __be64 *length, const uint8_t *buffer); diff --git a/arch/powerpc/platforms/powernv/opal-call.c b/arch/powerpc/platforms/powernv/opal-call.c index a2aa5e433ac8..27c4b93c774c 100644 --- a/arch/powerpc/platforms/powernv/opal-call.c +++ b/arch/powerpc/platforms/powernv/opal-call.c @@ -287,6 +287,8 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR); OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64); OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE); OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT); +OPAL_CALL(opal_npu_mem_alloc, OPAL_NPU_MEM_ALLOC); +OPAL_CALL(opal_npu_mem_release, OPAL_NPU_MEM_RELEASE); OPAL_CALL(opal_mpipl_update, OPAL_MPIPL_UPDATE); OPAL_CALL(opal_mpipl_register_tag, OPAL_MPIPL_REGISTER_TAG); OPAL_CALL(opal_mpipl_query_tag, OPAL_MPIPL_QUERY_TAG); From patchwork Tue Dec 3 03:46:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270259 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62B46159A for ; Tue, 3 Dec 2019 03:48:32 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3EB9F2186D for ; Tue, 3 Dec 2019 03:48:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3EB9F2186D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 5F14310113668; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id EB00910113637 for ; Mon, 2 Dec 2019 19:51:50 -0800 (PST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmbT083966 for ; Mon, 2 Dec 2019 22:48:27 -0500 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6xb0nxp-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:27 -0500 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:24 -0000 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:16 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33lZVF46661902 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:47:35 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9AD545204F; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 4911652059; Tue, 3 Dec 2019 03:48:15 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 60092A03CC; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 04/27] mm/memory_hotplug: Allow check_hotplug_memory_addressable to be called from drivers Date: Tue, 3 Dec 2019 14:46:32 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0016-0000-0000-000002D0158B X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0017-0000-0000-000033320CA4 Message-Id: <20191203034655.51561-5-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 suspectscore=3 lowpriorityscore=0 phishscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 adultscore=0 mlxlogscore=893 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: JHEVY7BJM5UXXKAZ2ZAHIMBKN3DAZ4XK X-Message-ID-Hash: JHEVY7BJM5UXXKAZ2ZAHIMBKN3DAZ4XK X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva When setting up OpenCAPI connected persistent memory, the range check may not be performed until quite late (or perhaps not at all, if the user does not establish a DAX device). This patch makes the range check callable so we can perform the check while probing the OpenCAPI SCM device. Signed-off-by: Alastair D'Silva --- include/linux/memory_hotplug.h | 5 +++++ mm/memory_hotplug.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index f46ea71b4ffd..3f3a010b60d2 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -339,6 +339,11 @@ static inline int remove_memory(int nid, u64 start, u64 size) static inline void __remove_memory(int nid, u64 start, u64 size) {} #endif /* CONFIG_MEMORY_HOTREMOVE */ +#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE +int check_hotplug_memory_addressable(unsigned long pfn, + unsigned long nr_pages); +#endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */ + extern void __ref free_area_init_core_hotplug(int nid); extern int __add_memory(int nid, u64 start, u64 size); extern int add_memory(int nid, u64 start, u64 size); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index b5be827ecda5..abd9624bd0c1 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -278,8 +278,8 @@ static int check_pfn_span(unsigned long pfn, unsigned long nr_pages, return 0; } -static int check_hotplug_memory_addressable(unsigned long pfn, - unsigned long nr_pages) +int check_hotplug_memory_addressable(unsigned long pfn, + unsigned long nr_pages) { const u64 max_addr = PFN_PHYS(pfn + nr_pages) - 1; From patchwork Tue Dec 3 03:46:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BAD606C1 for ; Tue, 3 Dec 2019 03:48:56 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A3BFA2158C for ; Tue, 3 Dec 2019 03:48:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3BFA2158C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9AF4D10097F03; Mon, 2 Dec 2019 19:52:01 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id F1C5B10113669 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmDT058038 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 2wm6g97ffq-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:26 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHfM41091192 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6BCB54C04A; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B896D4C040; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 732B3A03CD; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 05/27] powerpc: Map & release OpenCAPI LPC memory Date: Tue, 3 Dec 2019 14:46:33 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-4275-0000-0000-0000038A31DE X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-4276-0000-0000-0000389DCDD8 Message-Id: <20191203034655.51561-6-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 priorityscore=1501 suspectscore=3 clxscore=1015 mlxlogscore=522 impostorscore=0 lowpriorityscore=0 adultscore=0 malwarescore=0 spamscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: PLTNDJ5UDRFRW5P7NCUYNMHJTBN4QJFD X-Message-ID-Hash: PLTNDJ5UDRFRW5P7NCUYNMHJTBN4QJFD X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch adds platform support to map & release LPC memory. Signed-off-by: Alastair D'Silva Acked-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- arch/powerpc/include/asm/pnv-ocxl.h | 2 ++ arch/powerpc/platforms/powernv/ocxl.c | 42 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/arch/powerpc/include/asm/pnv-ocxl.h b/arch/powerpc/include/asm/pnv-ocxl.h index 7de82647e761..f8f8ffb48aa8 100644 --- a/arch/powerpc/include/asm/pnv-ocxl.h +++ b/arch/powerpc/include/asm/pnv-ocxl.h @@ -32,5 +32,7 @@ extern int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle) extern int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr); extern void pnv_ocxl_free_xive_irq(u32 irq); +extern u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size); +extern void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev); #endif /* _ASM_PNV_OCXL_H */ diff --git a/arch/powerpc/platforms/powernv/ocxl.c b/arch/powerpc/platforms/powernv/ocxl.c index 8c65aacda9c8..b56a48daf48c 100644 --- a/arch/powerpc/platforms/powernv/ocxl.c +++ b/arch/powerpc/platforms/powernv/ocxl.c @@ -475,6 +475,48 @@ void pnv_ocxl_spa_release(void *platform_data) } EXPORT_SYMBOL_GPL(pnv_ocxl_spa_release); +u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size) +{ + struct pci_controller *hose = pci_bus_to_host(pdev->bus); + struct pnv_phb *phb = hose->private_data; + u32 bdfn = pci_dev_id(pdev); + __be64 base_addr_be64; + u64 base_addr; + int rc; + + rc = opal_npu_mem_alloc(phb->opal_id, bdfn, size, &base_addr_be64); + if (rc) { + dev_warn(&pdev->dev, + "OPAL could not allocate LPC memory, rc=%d\n", rc); + return 0; + } + + base_addr = be64_to_cpu(base_addr_be64); + + rc = check_hotplug_memory_addressable(base_addr >> PAGE_SHIFT, + size >> PAGE_SHIFT); + if (rc) + return 0; + + return base_addr; +} +EXPORT_SYMBOL_GPL(pnv_ocxl_platform_lpc_setup); + +void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev) +{ + struct pci_controller *hose = pci_bus_to_host(pdev->bus); + struct pnv_phb *phb = hose->private_data; + u32 bdfn = pci_dev_id(pdev); + int rc; + + rc = opal_npu_mem_release(phb->opal_id, bdfn); + if (rc) + dev_warn(&pdev->dev, + "OPAL reported rc=%d when releasing LPC memory\n", rc); +} +EXPORT_SYMBOL_GPL(pnv_ocxl_platform_lpc_release); + + int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle) { struct spa_data *data = (struct spa_data *) platform_data; From patchwork Tue Dec 3 03:46:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87D24159A for ; Tue, 3 Dec 2019 03:48:49 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 709B72073B for ; Tue, 3 Dec 2019 03:48:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 709B72073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 0A66C10113693; Mon, 2 Dec 2019 19:51:59 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8018510113666 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmIg083990 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6xb0p0g-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:29 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHuF56492174 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 59F82AE04D; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B7FD0AE045; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 85EACA03CE; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 06/27] ocxl: Tally up the LPC memory on a link & allow it to be mapped Date: Tue, 3 Dec 2019 14:46:34 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0012-0000-0000-000003701286 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0013-0000-0000-000021ABCCBB Message-Id: <20191203034655.51561-7-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 suspectscore=3 lowpriorityscore=0 phishscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 adultscore=0 mlxlogscore=898 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: Y3WZQT6U773RSFH3LX2UU677SQBOKIZV X-Message-ID-Hash: Y3WZQT6U773RSFH3LX2UU677SQBOKIZV X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva Tally up the LPC memory on an OpenCAPI link & allow it to be mapped Signed-off-by: Alastair D'Silva --- drivers/misc/ocxl/core.c | 10 ++++++ drivers/misc/ocxl/link.c | 60 +++++++++++++++++++++++++++++++ drivers/misc/ocxl/ocxl_internal.h | 33 +++++++++++++++++ 3 files changed, 103 insertions(+) diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c index b7a09b21ab36..2531c6cf19a0 100644 --- a/drivers/misc/ocxl/core.c +++ b/drivers/misc/ocxl/core.c @@ -230,8 +230,18 @@ static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev) if (rc) goto err_free_pasid; + if (afu->config.lpc_mem_size || afu->config.special_purpose_mem_size) { + rc = ocxl_link_add_lpc_mem(afu->fn->link, afu->config.lpc_mem_offset, + afu->config.lpc_mem_size + + afu->config.special_purpose_mem_size); + if (rc) + goto err_free_mmio; + } + return 0; +err_free_mmio: + unmap_mmio_areas(afu); err_free_pasid: reclaim_afu_pasid(afu); err_free_actag: diff --git a/drivers/misc/ocxl/link.c b/drivers/misc/ocxl/link.c index 58d111afd9f6..d8503f0dc6ec 100644 --- a/drivers/misc/ocxl/link.c +++ b/drivers/misc/ocxl/link.c @@ -84,6 +84,11 @@ struct ocxl_link { int dev; atomic_t irq_available; struct spa *spa; + struct mutex lpc_mem_lock; + u64 lpc_mem_sz; /* Total amount of LPC memory presented on the link */ + u64 lpc_mem; + int lpc_consumers; + void *platform_data; }; static struct list_head links_list = LIST_HEAD_INIT(links_list); @@ -396,6 +401,8 @@ static int alloc_link(struct pci_dev *dev, int PE_mask, struct ocxl_link **out_l if (rc) goto err_spa; + mutex_init(&link->lpc_mem_lock); + /* platform specific hook */ rc = pnv_ocxl_spa_setup(dev, link->spa->spa_mem, PE_mask, &link->platform_data); @@ -711,3 +718,56 @@ void ocxl_link_free_irq(void *link_handle, int hw_irq) atomic_inc(&link->irq_available); } EXPORT_SYMBOL_GPL(ocxl_link_free_irq); + +int ocxl_link_add_lpc_mem(void *link_handle, u64 offset, u64 size) +{ + struct ocxl_link *link = (struct ocxl_link *) link_handle; + + // Check for overflow + if (offset > (offset + size)) + return -EINVAL; + + mutex_lock(&link->lpc_mem_lock); + link->lpc_mem_sz = max(link->lpc_mem_sz, offset + size); + + mutex_unlock(&link->lpc_mem_lock); + + return 0; +} + +u64 ocxl_link_lpc_map(void *link_handle, struct pci_dev *pdev) +{ + struct ocxl_link *link = (struct ocxl_link *) link_handle; + u64 lpc_mem; + + mutex_lock(&link->lpc_mem_lock); + if (link->lpc_mem) { + lpc_mem = link->lpc_mem; + + link->lpc_consumers++; + mutex_unlock(&link->lpc_mem_lock); + return lpc_mem; + } + + link->lpc_mem = pnv_ocxl_platform_lpc_setup(pdev, link->lpc_mem_sz); + if (link->lpc_mem) + link->lpc_consumers++; + lpc_mem = link->lpc_mem; + mutex_unlock(&link->lpc_mem_lock); + + return lpc_mem; +} + +void ocxl_link_lpc_release(void *link_handle, struct pci_dev *pdev) +{ + struct ocxl_link *link = (struct ocxl_link *) link_handle; + + mutex_lock(&link->lpc_mem_lock); + WARN_ON(--link->lpc_consumers < 0); + if (link->lpc_consumers == 0) { + pnv_ocxl_platform_lpc_release(pdev); + link->lpc_mem = 0; + } + + mutex_unlock(&link->lpc_mem_lock); +} diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h index 97415afd79f3..20b417e00949 100644 --- a/drivers/misc/ocxl/ocxl_internal.h +++ b/drivers/misc/ocxl/ocxl_internal.h @@ -141,4 +141,37 @@ int ocxl_irq_offset_to_id(struct ocxl_context *ctx, u64 offset); u64 ocxl_irq_id_to_offset(struct ocxl_context *ctx, int irq_id); void ocxl_afu_irq_free_all(struct ocxl_context *ctx); +/** + * ocxl_link_add_lpc_mem() - Increment the amount of memory required by an OpenCAPI link + * + * @link_handle: The OpenCAPI link handle + * @offset: The offset of the memory to add + * @size: The amount of memory to increment by + * + * Return 0 on success, negative on overflow + */ +int ocxl_link_add_lpc_mem(void *link_handle, u64 offset, u64 size); + +/** + * ocxl_link_lpc_map() - Map the LPC memory for an OpenCAPI device + * + * Since LPC memory belongs to a link, the whole LPC memory available + * on the link bust be mapped in order to make it accessible to a device. + * + * @link_handle: The OpenCAPI link handle + * @pdev: A device that is on the link + */ +u64 ocxl_link_lpc_map(void *link_handle, struct pci_dev *pdev); + +/** + * ocxl_link_lpc_release() - Release the LPC memory device for an OpenCAPI device + * + * Offlines LPC memory on an OpenCAPI link for a device. If this is the + * last device on the link to release the memory, unmap it from the link. + * + * @link_handle: The OpenCAPI link handle + * @pdev: A device that is on the link + */ +void ocxl_link_lpc_release(void *link_handle, struct pci_dev *pdev); + #endif /* _OCXL_INTERNAL_H_ */ From patchwork Tue Dec 3 03:46:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EF15159A for ; Tue, 3 Dec 2019 03:48:42 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC587206DF for ; Tue, 3 Dec 2019 03:48:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC587206DF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 439D610113671; Mon, 2 Dec 2019 19:51:57 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2943310113661 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33knjO021420 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wnehxj6t7-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHb553739674 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 671DBA4055; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C590CA4040; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 9DE35A03CF; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 07/27] ocxl: Add functions to map/unmap LPC memory Date: Tue, 3 Dec 2019 14:46:35 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0012-0000-0000-000003701287 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0013-0000-0000-000021ABCCBC Message-Id: <20191203034655.51561-8-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=1 impostorscore=0 mlxlogscore=844 priorityscore=1501 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: 52UB7B3ENOCM5DSXKTIFDTADSTC3FTEN X-Message-ID-Hash: 52UB7B3ENOCM5DSXKTIFDTADSTC3FTEN X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva Add functions to map/unmap LPC memory Signed-off-by: Alastair D'Silva --- drivers/misc/ocxl/config.c | 4 +++ drivers/misc/ocxl/core.c | 50 +++++++++++++++++++++++++++++++ drivers/misc/ocxl/ocxl_internal.h | 3 ++ include/misc/ocxl.h | 18 +++++++++++ 4 files changed, 75 insertions(+) diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c index c8e19bfb5ef9..fb0c3b6f8312 100644 --- a/drivers/misc/ocxl/config.c +++ b/drivers/misc/ocxl/config.c @@ -568,6 +568,10 @@ static int read_afu_lpc_memory_info(struct pci_dev *dev, afu->special_purpose_mem_size = total_mem_size - lpc_mem_size; } + + dev_info(&dev->dev, "Probed LPC memory of %#llx bytes and special purpose memory of %#llx bytes\n", + afu->lpc_mem_size, afu->special_purpose_mem_size); + return 0; } diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c index 2531c6cf19a0..98611faea219 100644 --- a/drivers/misc/ocxl/core.c +++ b/drivers/misc/ocxl/core.c @@ -210,6 +210,55 @@ static void unmap_mmio_areas(struct ocxl_afu *afu) release_fn_bar(afu->fn, afu->config.global_mmio_bar); } +int ocxl_afu_map_lpc_mem(struct ocxl_afu *afu) +{ + struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent); + + if ((afu->config.lpc_mem_size + afu->config.special_purpose_mem_size) == 0) + return 0; + + afu->lpc_base_addr = ocxl_link_lpc_map(afu->fn->link, dev); + if (afu->lpc_base_addr == 0) + return -EINVAL; + + if (afu->config.lpc_mem_size) { + afu->lpc_res.start = afu->lpc_base_addr + afu->config.lpc_mem_offset; + afu->lpc_res.end = afu->lpc_res.start + afu->config.lpc_mem_size - 1; + } + + if (afu->config.special_purpose_mem_size) { + afu->special_purpose_res.start = afu->lpc_base_addr + + afu->config.special_purpose_mem_offset; + afu->special_purpose_res.end = afu->special_purpose_res.start + + afu->config.special_purpose_mem_size - 1; + } + + return 0; +} +EXPORT_SYMBOL_GPL(ocxl_afu_map_lpc_mem); + +struct resource *ocxl_afu_lpc_mem(struct ocxl_afu *afu) +{ + return &afu->lpc_res; +} +EXPORT_SYMBOL_GPL(ocxl_afu_lpc_mem); + +static void unmap_lpc_mem(struct ocxl_afu *afu) +{ + struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent); + + if (afu->lpc_res.start || afu->special_purpose_res.start) { + void *link = afu->fn->link; + + ocxl_link_lpc_release(link, dev); + + afu->lpc_res.start = 0; + afu->lpc_res.end = 0; + afu->special_purpose_res.start = 0; + afu->special_purpose_res.end = 0; + } +} + static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev) { int rc; @@ -251,6 +300,7 @@ static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev) static void deconfigure_afu(struct ocxl_afu *afu) { + unmap_lpc_mem(afu); unmap_mmio_areas(afu); reclaim_afu_pasid(afu); reclaim_afu_actag(afu); diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h index 20b417e00949..9f4b47900e62 100644 --- a/drivers/misc/ocxl/ocxl_internal.h +++ b/drivers/misc/ocxl/ocxl_internal.h @@ -52,6 +52,9 @@ struct ocxl_afu { void __iomem *global_mmio_ptr; u64 pp_mmio_start; void *private; + u64 lpc_base_addr; /* Covers both LPC & special purpose memory */ + struct resource lpc_res; + struct resource special_purpose_res; }; enum ocxl_context_status { diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index 06dd5839e438..6f7c02f0d5e3 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -212,6 +212,24 @@ int ocxl_irq_set_handler(struct ocxl_context *ctx, int irq_id, // AFU Metadata +/** + * Map the LPC system & special purpose memory for an AFU + * + * Do not call this during device discovery, as there may me multiple + * devices on a link, and the memory is mapped for the whole link, not + * just one device. It should only be called after all devices have + * registered their memory on the link. + * + * afu: The AFU that has the LPC memory to map + */ +extern int ocxl_afu_map_lpc_mem(struct ocxl_afu *afu); + +/** + * Get the physical address range of LPC memory for an AFU + * afu: The AFU associated with the LPC memory + */ +extern struct resource *ocxl_afu_lpc_mem(struct ocxl_afu *afu); + /** * Get a pointer to the config for an AFU * From patchwork Tue Dec 3 03:46:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270265 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B13E2159A for ; Tue, 3 Dec 2019 03:48:34 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9A37820865 for ; Tue, 3 Dec 2019 03:48:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A37820865 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 8432710113664; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id ECE9F10113637 for ; Mon, 2 Dec 2019 19:51:51 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33knR7077047 for ; Mon, 2 Dec 2019 22:48:29 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wknuapy6t-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:28 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mH6I63111196 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 672B55204E; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id BC70F52050; Tue, 3 Dec 2019 03:48:16 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id AA740A03D2; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 08/27] ocxl: Save the device serial number in ocxl_fn Date: Tue, 3 Dec 2019 14:46:36 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0028-0000-0000-000003C3CC73 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0029-0000-0000-00002486E432 Message-Id: <20191203034655.51561-9-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 bulkscore=0 adultscore=0 mlxlogscore=484 phishscore=0 spamscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=1 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: 6KZM7B7XNDQMCKZTA5Z5HYMCOPZ4LPK2 X-Message-ID-Hash: 6KZM7B7XNDQMCKZTA5Z5HYMCOPZ4LPK2 X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch retrieves the serial number of the card and makes it available to consumers of the ocxl driver via the ocxl_fn struct. Signed-off-by: Alastair D'Silva Acked-by: Frederic Barrat Acked-by: Andrew Donnellan --- drivers/misc/ocxl/config.c | 46 ++++++++++++++++++++++++++++++++++++++ include/misc/ocxl.h | 1 + 2 files changed, 47 insertions(+) diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c index fb0c3b6f8312..a9203c309365 100644 --- a/drivers/misc/ocxl/config.c +++ b/drivers/misc/ocxl/config.c @@ -71,6 +71,51 @@ static int find_dvsec_afu_ctrl(struct pci_dev *dev, u8 afu_idx) return 0; } +/** + * Find a related PCI device (function 0) + * @device: PCI device to match + * + * Returns a pointer to the related device, or null if not found + */ +static struct pci_dev *get_function_0(struct pci_dev *dev) +{ + unsigned int devfn = PCI_DEVFN(PCI_SLOT(dev->devfn), 0); // Look for function 0 + + return pci_get_domain_bus_and_slot(pci_domain_nr(dev->bus), + dev->bus->number, devfn); +} + +static void read_serial(struct pci_dev *dev, struct ocxl_fn_config *fn) +{ + u32 low, high; + int pos; + + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DSN); + if (pos) { + pci_read_config_dword(dev, pos + 0x04, &low); + pci_read_config_dword(dev, pos + 0x08, &high); + + fn->serial = low | ((u64)high) << 32; + + return; + } + + if (PCI_FUNC(dev->devfn) != 0) { + struct pci_dev *related = get_function_0(dev); + + if (!related) { + fn->serial = 0; + return; + } + + read_serial(related, fn); + pci_dev_put(related); + return; + } + + fn->serial = 0; +} + static void read_pasid(struct pci_dev *dev, struct ocxl_fn_config *fn) { u16 val; @@ -208,6 +253,7 @@ int ocxl_config_read_function(struct pci_dev *dev, struct ocxl_fn_config *fn) int rc; read_pasid(dev, fn); + read_serial(dev, fn); rc = read_dvsec_tl(dev, fn); if (rc) { diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index 6f7c02f0d5e3..9843051c3c5b 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -46,6 +46,7 @@ struct ocxl_fn_config { int dvsec_afu_info_pos; /* offset of the AFU information DVSEC */ s8 max_pasid_log; s8 max_afu_index; + u64 serial; }; enum ocxl_endian { From patchwork Tue Dec 3 03:46:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270337 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AFBD109A for ; Tue, 3 Dec 2019 03:49:08 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 637B220726 for ; Tue, 3 Dec 2019 03:49:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 637B220726 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id AEBEA10097F13; Mon, 2 Dec 2019 19:52:09 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0b-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E60FF10113687 for ; Mon, 2 Dec 2019 19:51:59 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kckW088004 for ; Mon, 2 Dec 2019 22:48:36 -0500 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wkm47b7u4-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:35 -0500 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHuf63307976 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 635DCA4057; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 11606A4053; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id BDAD8A03D3; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 09/27] ocxl: Free detached contexts in ocxl_context_detach_all() Date: Tue, 3 Dec 2019 14:46:37 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0008-0000-0000-0000033C0FDE X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0009-0000-0000-00004A5B289A Message-Id: <20191203034655.51561-10-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 malwarescore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 impostorscore=0 adultscore=0 suspectscore=3 mlxlogscore=821 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: YTRM7KVAVS54ZFEHLQ3BTVS46QVGNBXE X-Message-ID-Hash: YTRM7KVAVS54ZFEHLQ3BTVS46QVGNBXE X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva ocxl_context_detach_all() is called from ocxl_function_close(), so there is no reason to leave the contexts allocated, as the caller can do nothing useful with them at that point. This also has the side-effect of freeing any allocated IRQs within the context. Signed-off-by: Alastair D'Silva --- drivers/misc/ocxl/context.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/misc/ocxl/context.c b/drivers/misc/ocxl/context.c index 994563a078eb..6cb36ef96e09 100644 --- a/drivers/misc/ocxl/context.c +++ b/drivers/misc/ocxl/context.c @@ -259,10 +259,11 @@ void ocxl_context_detach_all(struct ocxl_afu *afu) { struct ocxl_context *ctx; int tmp; + int rc; mutex_lock(&afu->contexts_lock); idr_for_each_entry(&afu->contexts_idr, ctx, tmp) { - ocxl_context_detach(ctx); + rc = ocxl_context_detach(ctx); /* * We are force detaching - remove any active mmio * mappings so userspace cannot interfere with the @@ -274,6 +275,9 @@ void ocxl_context_detach_all(struct ocxl_afu *afu) if (ctx->mapping) unmap_mapping_range(ctx->mapping, 0, 0, 1); mutex_unlock(&ctx->mapping_lock); + + if (rc != -EBUSY) + ocxl_context_free(ctx); } mutex_unlock(&afu->contexts_lock); } From patchwork Tue Dec 3 03:46:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57FF6159A for ; Tue, 3 Dec 2019 03:48:46 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 40E2B20862 for ; Tue, 3 Dec 2019 03:48:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 40E2B20862 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B97F610113689; Mon, 2 Dec 2019 19:51:58 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5F50710113669 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmjH001165 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 2wm6myr931-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:29 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:26 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHAa50790640 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B97D74C046; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 132AD4C044; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id D21B6A03DC; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 10/27] nvdimm: Add driver for OpenCAPI Storage Class Memory Date: Tue, 3 Dec 2019 14:46:38 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0012-0000-0000-000003701288 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0013-0000-0000-000021ABCCBD Message-Id: <20191203034655.51561-11-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 spamscore=0 phishscore=0 suspectscore=4 adultscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: PGX356QGOLR2G5AZ2JOMUZ4SRB6LNYSV X-Message-ID-Hash: PGX356QGOLR2G5AZ2JOMUZ4SRB6LNYSV X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This driver exposes LPC memory on OpenCAPI SCM cards as an NVDIMM, allowing the existing nvram infrastructure to be used. Namespace metadata is stored on the media itself, so scm_reserve_metadata() maps 1 section's worth of PMEM storage at the start to hold this. The rest of the PMEM range is registered with libnvdimm as an nvdimm. scm_ndctl_config_read/write/size() provide callbacks to libnvdimm to access the metadata. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/Kconfig | 2 + drivers/nvdimm/Makefile | 2 +- drivers/nvdimm/ocxl/Kconfig | 15 + drivers/nvdimm/ocxl/Makefile | 7 + drivers/nvdimm/ocxl/scm.c | 519 +++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.h | 28 ++ 6 files changed, 572 insertions(+), 1 deletion(-) create mode 100644 drivers/nvdimm/ocxl/Kconfig create mode 100644 drivers/nvdimm/ocxl/Makefile create mode 100644 drivers/nvdimm/ocxl/scm.c create mode 100644 drivers/nvdimm/ocxl/scm_internal.h diff --git a/drivers/nvdimm/Kconfig b/drivers/nvdimm/Kconfig index 36af7af6b7cf..d1bab36da61c 100644 --- a/drivers/nvdimm/Kconfig +++ b/drivers/nvdimm/Kconfig @@ -130,4 +130,6 @@ config NVDIMM_TEST_BUILD core devm_memremap_pages() implementation and other infrastructure. +source "drivers/nvdimm/ocxl/Kconfig" + endif diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile index 29203f3d3069..e33492128042 100644 --- a/drivers/nvdimm/Makefile +++ b/drivers/nvdimm/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o +obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o ocxl/ obj-$(CONFIG_BLK_DEV_PMEM) += nd_pmem.o obj-$(CONFIG_ND_BTT) += nd_btt.o obj-$(CONFIG_ND_BLK) += nd_blk.o diff --git a/drivers/nvdimm/ocxl/Kconfig b/drivers/nvdimm/ocxl/Kconfig new file mode 100644 index 000000000000..24099b300f5e --- /dev/null +++ b/drivers/nvdimm/ocxl/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +if LIBNVDIMM + +config OCXL_SCM + tristate "OpenCAPI Storage Class Memory" + depends on LIBNVDIMM && PPC_POWERNV && PCI && EEH + select ZONE_DEVICE + select OCXL + help + Exposes devices that implement the OpenCAPI Storage Class Memory + specification as persistent memory regions. + + Select N if unsure. + +endif diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile new file mode 100644 index 000000000000..74a1bd98848e --- /dev/null +++ b/drivers/nvdimm/ocxl/Makefile @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0 + +ccflags-$(CONFIG_PPC_WERROR) += -Werror + +obj-$(CONFIG_OCXL_SCM) += ocxlscm.o + +ocxlscm-y := scm.o diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c new file mode 100644 index 000000000000..571058a9e7b8 --- /dev/null +++ b/drivers/nvdimm/ocxl/scm.c @@ -0,0 +1,519 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2019 IBM Corp. + +/* + * A driver for Storage Class Memory, connected via OpenCAPI + */ + +#include +#include +#include +#include +#include +#include "scm_internal.h" + + +static const struct pci_device_id scm_pci_tbl[] = { + { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x0625), }, + { } +}; + +MODULE_DEVICE_TABLE(pci, scm_pci_tbl); + +#define SCM_NUM_MINORS 256 // Total to reserve + +static dev_t scm_dev; +static struct class *scm_class; +static struct mutex minors_idr_lock; +static struct idr minors_idr; + +static const struct attribute_group *scm_pmem_attribute_groups[] = { + &nvdimm_bus_attribute_group, + NULL, +}; + +static const struct attribute_group *scm_pmem_region_attribute_groups[] = { + &nd_region_attribute_group, + &nd_device_attribute_group, + &nd_mapping_attribute_group, + &nd_numa_attribute_group, + NULL, +}; + +/** + * scm_ndctl_config_write() - Handle a ND_CMD_SET_CONFIG_DATA command from ndctl + * @scm_data: the SCM metadata + * @command: the incoming data to write + * Return: 0 on success, negative on failure + */ +static int scm_ndctl_config_write(struct scm_data *scm_data, + struct nd_cmd_set_config_hdr *command) +{ + if (command->in_offset + command->in_length > SCM_LABEL_AREA_SIZE) + return -EINVAL; + + memcpy_flushcache(scm_data->metadata_addr + command->in_offset, command->in_buf, + command->in_length); + + return 0; +} + +/** + * scm_ndctl_config_read() - Handle a ND_CMD_GET_CONFIG_DATA command from ndctl + * @scm_data: the SCM metadata + * @command: the read request + * Return: 0 on success, negative on failure + */ +static int scm_ndctl_config_read(struct scm_data *scm_data, + struct nd_cmd_get_config_data_hdr *command) +{ + if (command->in_offset + command->in_length > SCM_LABEL_AREA_SIZE) + return -EINVAL; + + memcpy_mcsafe(command->out_buf, scm_data->metadata_addr + command->in_offset, + command->in_length); + + return 0; +} + +/** + * scm_ndctl_config_size() - Handle a ND_CMD_GET_CONFIG_SIZE command from ndctl + * @scm_data: the SCM metadata + * @command: the read request + * Return: 0 on success, negative on failure + */ +static int scm_ndctl_config_size(struct nd_cmd_get_config_size *command) +{ + command->status = 0; + command->config_size = SCM_LABEL_AREA_SIZE; + command->max_xfer = PAGE_SIZE; + + return 0; +} + +static int scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, + struct nvdimm *nvdimm, + unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) +{ + struct scm_data *scm_data = container_of(nd_desc, struct scm_data, bus_desc); + + switch (cmd) { + case ND_CMD_GET_CONFIG_SIZE: + *cmd_rc = scm_ndctl_config_size(buf); + return 0; + + case ND_CMD_GET_CONFIG_DATA: + *cmd_rc = scm_ndctl_config_read(scm_data, buf); + return 0; + + case ND_CMD_SET_CONFIG_DATA: + *cmd_rc = scm_ndctl_config_write(scm_data, buf); + return 0; + + default: + return -ENOTTY; + } +} + +static ssize_t serial_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct nvdimm *nvdimm = to_nvdimm(dev); + struct scm_data *scm_data = nvdimm_provider_data(nvdimm); + const struct ocxl_fn_config *config = ocxl_function_config(scm_data->ocxl_fn); + + return sprintf(buf, "0x%llx\n", config->serial); +} +static DEVICE_ATTR_RO(serial); + +static struct attribute *scm_dimm_attributes[] = { + &dev_attr_serial.attr, + NULL, +}; + +static umode_t scm_dimm_attr_visible(struct kobject *kobj, + struct attribute *a, int n) +{ + return a->mode; +} + +static const struct attribute_group scm_dimm_attribute_group = { + .name = "ocxl", + .attrs = scm_dimm_attributes, + .is_visible = scm_dimm_attr_visible, +}; + +static const struct attribute_group *scm_dimm_attribute_groups[] = { + &nvdimm_attribute_group, + &nd_device_attribute_group, + &scm_dimm_attribute_group, + NULL, +}; + +/** + * scm_reserve_metadata() - Reserve space for nvdimm metadata + * @scm_data: The SCM device data + * @lpc_mem: The resource representing the LPC memory of the SCM device + */ +static int scm_reserve_metadata(struct scm_data *scm_data, + struct resource *lpc_mem) +{ + scm_data->metadata_addr = devm_memremap(&scm_data->dev, lpc_mem->start, + SCM_LABEL_AREA_SIZE, MEMREMAP_WB); + if (IS_ERR(scm_data->metadata_addr)) + return PTR_ERR(scm_data->metadata_addr); + + return 0; +} + +/** + * scm_register_lpc_mem() - Discover persistent memory on a device and register it with the NVDIMM subsystem + * @scm_data: The SCM device data + * Return: 0 on success + */ +static int scm_register_lpc_mem(struct scm_data *scm_data) +{ + struct nd_region_desc region_desc; + struct nd_mapping_desc nd_mapping_desc; + struct resource *lpc_mem; + const struct ocxl_afu_config *config; + const struct ocxl_fn_config *fn_config; + int rc; + unsigned long nvdimm_cmd_mask = 0; + unsigned long nvdimm_flags = 0; + int target_node; + char serial[16+1]; + + // Set up the reserved metadata area + rc = ocxl_afu_map_lpc_mem(scm_data->ocxl_afu); + if (rc < 0) + return rc; + + lpc_mem = ocxl_afu_lpc_mem(scm_data->ocxl_afu); + if (lpc_mem == NULL || lpc_mem->start == 0) + return -EINVAL; + + config = ocxl_afu_config(scm_data->ocxl_afu); + fn_config = ocxl_function_config(scm_data->ocxl_fn); + + rc = scm_reserve_metadata(scm_data, lpc_mem); + if (rc) + return rc; + + scm_data->bus_desc.attr_groups = scm_pmem_attribute_groups; + scm_data->bus_desc.provider_name = "ocxl-scm"; + scm_data->bus_desc.ndctl = scm_ndctl; + scm_data->bus_desc.module = THIS_MODULE; + + scm_data->nvdimm_bus = nvdimm_bus_register(&scm_data->dev, + &scm_data->bus_desc); + if (!scm_data->nvdimm_bus) + return -EINVAL; + + scm_data->scm_res.start = (u64)lpc_mem->start + SCM_LABEL_AREA_SIZE; + scm_data->scm_res.end = (u64)lpc_mem->start + config->lpc_mem_size - 1; + scm_data->scm_res.name = "SCM persistent memory"; + + set_bit(ND_CMD_GET_CONFIG_SIZE, &nvdimm_cmd_mask); + set_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm_cmd_mask); + set_bit(ND_CMD_SET_CONFIG_DATA, &nvdimm_cmd_mask); + + set_bit(NDD_ALIASING, &nvdimm_flags); + + snprintf(serial, sizeof(serial), "%llx", fn_config->serial); + nd_mapping_desc.nvdimm = nvdimm_create(scm_data->nvdimm_bus, scm_data, + scm_dimm_attribute_groups, + nvdimm_flags, nvdimm_cmd_mask, + 0, NULL); + if (!nd_mapping_desc.nvdimm) + return -ENOMEM; + + if (nvdimm_bus_check_dimm_count(scm_data->nvdimm_bus, 1)) + return -EINVAL; + + nd_mapping_desc.start = scm_data->scm_res.start; + nd_mapping_desc.size = resource_size(&scm_data->scm_res); + nd_mapping_desc.position = 0; + + scm_data->nd_set.cookie1 = fn_config->serial + 1; // allow for empty serial + scm_data->nd_set.cookie2 = fn_config->serial + 1; + + target_node = of_node_to_nid(scm_data->pdev->dev.of_node); + + memset(®ion_desc, 0, sizeof(region_desc)); + region_desc.res = &scm_data->scm_res; + region_desc.attr_groups = scm_pmem_region_attribute_groups; + region_desc.numa_node = NUMA_NO_NODE; + region_desc.target_node = target_node; + region_desc.num_mappings = 1; + region_desc.mapping = &nd_mapping_desc; + region_desc.nd_set = &scm_data->nd_set; + + set_bit(ND_REGION_PAGEMAP, ®ion_desc.flags); + /* + * NB: libnvdimm copies the data from ndr_desc into it's own + * structures so passing a stack pointer is fine. + */ + scm_data->nd_region = nvdimm_pmem_region_create(scm_data->nvdimm_bus, + ®ion_desc); + if (!scm_data->nd_region) + return -EINVAL; + + dev_info(&scm_data->dev, + "Onlining %lluMB of persistent memory\n", + nd_mapping_desc.size / SZ_1M); + + return 0; +} + +/** + * allocate_scm_minor() - Allocate a minor number to use for an SCM device + * @scm_data: The SCM device to associate the minor with + * Return: the allocated minor number + */ +static int allocate_scm_minor(struct scm_data *scm_data) +{ + int minor; + + mutex_lock(&minors_idr_lock); + minor = idr_alloc(&minors_idr, scm_data, 0, SCM_NUM_MINORS, GFP_KERNEL); + mutex_unlock(&minors_idr_lock); + return minor; +} + +static void free_scm_minor(struct scm_data *scm_data) +{ + mutex_lock(&minors_idr_lock); + idr_remove(&minors_idr, MINOR(scm_data->dev.devt)); + mutex_unlock(&minors_idr_lock); +} + +/** + * free_scm() - Free all members of an SCM struct + * @scm_data: the SCM metadata to clear + */ +static void free_scm(struct scm_data *scm_data) +{ + int rc; + + if (scm_data->nvdimm_bus) + nvdimm_bus_unregister(scm_data->nvdimm_bus); + + free_scm_minor(scm_data); + + if (scm_data->metadata_addr) + devm_memunmap(&scm_data->dev, scm_data->metadata_addr); + + if (scm_data->ocxl_context) { + rc = ocxl_context_detach(scm_data->ocxl_context); + if (rc == -EBUSY) + dev_warn(&scm_data->dev, "Timeout detaching ocxl context\n"); + else + ocxl_context_free(scm_data->ocxl_context); + + } + + if (scm_data->ocxl_afu) + ocxl_afu_put(scm_data->ocxl_afu); + + if (scm_data->ocxl_fn) + ocxl_function_close(scm_data->ocxl_fn); + + kfree(scm_data); +} + +/** + * free_scm_dev - Free an SCM device + * @dev: The device struct + */ +static void free_scm_dev(struct device *dev) +{ + struct scm_data *scm_data = container_of(dev, struct scm_data, dev); + + free_scm(scm_data); +} + +/** + * scm_register - Register an SCM device with the kernel + * @scm_data: the SCM metadata + * Return: 0 on success, negative on failure + */ +static int scm_register(struct scm_data *scm_data) +{ + int rc; + int minor = allocate_scm_minor(scm_data); + + if (minor < 0) + return minor; + + scm_data->dev.release = free_scm_dev; + rc = dev_set_name(&scm_data->dev, "ocxl-scm%d", minor); + if (rc < 0) + return rc; + + scm_data->dev.devt = MKDEV(MAJOR(scm_dev), minor); + scm_data->dev.class = scm_class; + scm_data->dev.parent = &scm_data->pdev->dev; + + rc = device_register(&scm_data->dev); + return rc; +} + +/** + * scm_remove() - Free an OpenCAPI Storage Class Memory device + * @pdev: the PCI device information struct + */ +static void scm_remove(struct pci_dev *pdev) +{ + if (PCI_FUNC(pdev->devfn) == 0) { + struct scm_function_0 *scm_func_0 = pci_get_drvdata(pdev); + + if (scm_func_0) { + ocxl_function_close(scm_func_0->ocxl_fn); + scm_func_0->ocxl_fn = NULL; + } + } else { + struct scm_data *scm_data = pci_get_drvdata(pdev); + + if (scm_data) + device_unregister(&scm_data->dev); + } +} + +/** + * scm_probe_function_0 - Set up function 0 for an OpenCAPI Storage Class Memory device + * This is important as it enables templates higher than 0 across all other functions, + * which in turn enables higher bandwidth accesses + * which in turn enables higher bandwidth accesses + * @pdev: the PCI device information struct + * Return: 0 on success, negative on failure + */ +static int scm_probe_function_0(struct pci_dev *pdev) +{ + struct scm_function_0 *scm_func_0 = NULL; + struct ocxl_fn *fn; + + scm_func_0 = kzalloc(sizeof(*scm_func_0), GFP_KERNEL); + if (!scm_func_0) + return -ENOMEM; + + scm_func_0->pdev = pdev; + fn = ocxl_function_open(pdev); + if (IS_ERR(fn)) { + kfree(scm_func_0); + dev_err(&pdev->dev, "failed to open OCXL function\n"); + return PTR_ERR(fn); + } + scm_func_0->ocxl_fn = fn; + + pci_set_drvdata(pdev, scm_func_0); + + return 0; +} + +/** + * scm_probe - Init an OpenCAPI Storage Class Memory device + * @pdev: the PCI device information struct + * @ent: The entry from scm_pci_tbl + * Return: 0 on success, negative on failure + */ +static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) +{ + struct scm_data *scm_data = NULL; + + if (PCI_FUNC(pdev->devfn) == 0) + return scm_probe_function_0(pdev); + else if (PCI_FUNC(pdev->devfn) != 1) + return 0; + + scm_data = kzalloc(sizeof(*scm_data), GFP_KERNEL); + if (!scm_data) { + dev_err(&pdev->dev, "Could not allocate SCM metadata\n"); + goto err; + } + scm_data->pdev = pdev; + + pci_set_drvdata(pdev, scm_data); + + scm_data->ocxl_fn = ocxl_function_open(pdev); + if (IS_ERR(scm_data->ocxl_fn)) { + kfree(scm_data); + scm_data = NULL; + pci_set_drvdata(pdev, NULL); + dev_err(&pdev->dev, "failed to open OCXL function\n"); + goto err; + } + + scm_data->ocxl_afu = ocxl_function_fetch_afu(scm_data->ocxl_fn, 0); + if (scm_data->ocxl_afu == NULL) { + dev_err(&pdev->dev, "Could not get OCXL AFU from function\n"); + goto err; + } + + ocxl_afu_get(scm_data->ocxl_afu); + + if (scm_register(scm_data) < 0) { + dev_err(&pdev->dev, "Could not register SCM device with the kernel\n"); + goto err; + } + + // Resources allocated below here are cleaned up in the release handler + + if (ocxl_context_alloc(&scm_data->ocxl_context, scm_data->ocxl_afu, NULL)) { + dev_err(&pdev->dev, "Could not allocate OCXL context\n"); + goto err; + } + + if (ocxl_context_attach(scm_data->ocxl_context, 0, NULL)) { + dev_err(&pdev->dev, "Could not attach ocxl context\n"); + goto err; + } + + if (scm_register_lpc_mem(scm_data)) { + dev_err(&pdev->dev, "Could not register OCXL SCM memory with libnvdimm\n"); + goto err; + } + + return 0; + +err: + /* + * Further cleanup is done in the release handler via free_scm() + * This allows us to keep the character device live to handle IOCTLs to + * investigate issues if the card has an error + */ + + dev_err(&pdev->dev, + "Error detected, will not register storage class memory\n"); + return -ENXIO; +} + +static struct pci_driver scm_pci_driver = { + .name = "ocxl-scm", + .id_table = scm_pci_tbl, + .probe = scm_probe, + .remove = scm_remove, + .shutdown = scm_remove, +}; + +static int __init scm_init(void) +{ + int rc = 0; + + rc = pci_register_driver(&scm_pci_driver); + if (rc) + return rc; + + return 0; +} + +static void scm_exit(void) +{ + pci_unregister_driver(&scm_pci_driver); +} + +module_init(scm_init); +module_exit(scm_exit); + +MODULE_DESCRIPTION("Storage Class Memory"); +MODULE_LICENSE("GPL"); diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h new file mode 100644 index 000000000000..6340012e0f8a --- /dev/null +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2019 IBM Corp. + +#include +#include +#include +#include + +#define SCM_LABEL_AREA_SIZE (1UL << PA_SECTION_SHIFT) + +struct scm_function_0 { + struct pci_dev *pdev; + struct ocxl_fn *ocxl_fn; +}; + +struct scm_data { + struct device dev; + struct pci_dev *pdev; + struct ocxl_fn *ocxl_fn; + struct nd_interleave_set nd_set; + struct nvdimm_bus_descriptor bus_desc; + struct nvdimm_bus *nvdimm_bus; + struct ocxl_afu *ocxl_afu; + struct ocxl_context *ocxl_context; + void *metadata_addr; + struct resource scm_res; + struct nd_region *nd_region; +}; From patchwork Tue Dec 3 03:46:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9F366C1 for ; Tue, 3 Dec 2019 03:48:43 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B234E2071F for ; Tue, 3 Dec 2019 03:48:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B234E2071F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7115010113682; Mon, 2 Dec 2019 19:51:57 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5711410113666 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kjmF021073 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wnehxj6t8-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHrh28704874 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B644011C050; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13AB711C04C; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id EB68FA03DD; Tue, 3 Dec 2019 14:48:12 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 11/27] nvdimm/ocxl: Add register addresses & status values to header Date: Tue, 3 Dec 2019 14:46:39 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0012-0000-0000-00000370128A X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0013-0000-0000-000021ABCCBE Message-Id: <20191203034655.51561-12-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=1 impostorscore=0 mlxlogscore=868 priorityscore=1501 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: LW6XD5JNVBZR6VKKKLVXFF577H6LOGPZ X-Message-ID-Hash: LW6XD5JNVBZR6VKKKLVXFF577H6LOGPZ X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva These values have been taken from the device specifications. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm_internal.h | 72 ++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 6340012e0f8a..d6ab361f5de9 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -6,6 +6,78 @@ #include #include +#define GLOBAL_MMIO_CHI 0x000 +#define GLOBAL_MMIO_CHIC 0x008 +#define GLOBAL_MMIO_CHIE 0x010 +#define GLOBAL_MMIO_CHIEC 0x018 +#define GLOBAL_MMIO_HCI 0x020 +#define GLOBAL_MMIO_HCIC 0x028 +#define GLOBAL_MMIO_IMA0_OHP 0x040 +#define GLOBAL_MMIO_IMA0_CFP 0x048 +#define GLOBAL_MMIO_IMA1_OHP 0x050 +#define GLOBAL_MMIO_IMA1_CFP 0x058 +#define GLOBAL_MMIO_ACMA_CREQO 0x100 +#define GLOBAL_MMIO_ACMA_CRSPO 0x104 +#define GLOBAL_MMIO_ACMA_CDBO 0x108 +#define GLOBAL_MMIO_ACMA_CDBS 0x10c +#define GLOBAL_MMIO_NSCMA_CREQO 0x120 +#define GLOBAL_MMIO_NSCMA_CRSPO 0x124 +#define GLOBAL_MMIO_NSCMA_CDBO 0x128 +#define GLOBAL_MMIO_NSCMA_CDBS 0x12c +#define GLOBAL_MMIO_CSTS 0x140 +#define GLOBAL_MMIO_FWVER 0x148 +#define GLOBAL_MMIO_CCAP0 0x160 +#define GLOBAL_MMIO_CCAP1 0x168 + +#define GLOBAL_MMIO_CHI_ACRA BIT_ULL(0) +#define GLOBAL_MMIO_CHI_NSCRA BIT_ULL(1) +#define GLOBAL_MMIO_CHI_CRDY BIT_ULL(4) +#define GLOBAL_MMIO_CHI_CFFS BIT_ULL(5) +#define GLOBAL_MMIO_CHI_MA BIT_ULL(6) +#define GLOBAL_MMIO_CHI_ELA BIT_ULL(7) +#define GLOBAL_MMIO_CHI_CDA BIT_ULL(8) +#define GLOBAL_MMIO_CHI_CHFS BIT_ULL(9) + +#define GLOBAL_MMIO_CHI_ALL (GLOBAL_MMIO_CHI_ACRA | \ + GLOBAL_MMIO_CHI_NSCRA | \ + GLOBAL_MMIO_CHI_CRDY | \ + GLOBAL_MMIO_CHI_CFFS | \ + GLOBAL_MMIO_CHI_MA | \ + GLOBAL_MMIO_CHI_ELA | \ + GLOBAL_MMIO_CHI_CDA | \ + GLOBAL_MMIO_CHI_CHFS) + +#define GLOBAL_MMIO_HCI_ACRW BIT_ULL(0) +#define GLOBAL_MMIO_HCI_NSCRW BIT_ULL(1) +#define GLOBAL_MMIO_HCI_AFU_RESET BIT_ULL(2) +#define GLOBAL_MMIO_HCI_FW_DEBUG BIT_ULL(3) +#define GLOBAL_MMIO_HCI_CONTROLLER_DUMP BIT_ULL(4) +#define GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED BIT_ULL(5) +#define GLOBAL_MMIO_HCI_REQ_HEALTH_PERF BIT_ULL(6) + +#define ADMIN_COMMAND_HEARTBEAT 0x00u +#define ADMIN_COMMAND_SHUTDOWN 0x01u +#define ADMIN_COMMAND_FW_UPDATE 0x02u +#define ADMIN_COMMAND_FW_DEBUG 0x03u +#define ADMIN_COMMAND_ERRLOG 0x04u +#define ADMIN_COMMAND_SMART 0x05u +#define ADMIN_COMMAND_CONTROLLER_STATS 0x06u +#define ADMIN_COMMAND_CONTROLLER_DUMP 0x07u +#define ADMIN_COMMAND_CMD_CAPS 0x08u +#define ADMIN_COMMAND_MAX 0x08u + +#define STATUS_SUCCESS 0x00 +#define STATUS_MEM_UNAVAILABLE 0x20 +#define STATUS_BAD_OPCODE 0x50 +#define STATUS_BAD_REQUEST_PARM 0x51 +#define STATUS_BAD_DATA_PARM 0x52 +#define STATUS_DEBUG_BLOCKED 0x70 +#define STATUS_FAIL 0xFF + +#define STATUS_FW_UPDATE_BLOCKED 0x21 +#define STATUS_FW_ARG_INVALID 0x51 +#define STATUS_FW_INVALID 0x52 + #define SCM_LABEL_AREA_SIZE (1UL << PA_SECTION_SHIFT) struct scm_function_0 { From patchwork Tue Dec 3 03:46:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61BA1109A for ; Tue, 3 Dec 2019 03:48:40 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A6112071F for ; Tue, 3 Dec 2019 03:48:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A6112071F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 04AB41011367B; Mon, 2 Dec 2019 19:51:56 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5324A10113660 for ; Mon, 2 Dec 2019 19:51:52 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kocs077089 for ; Mon, 2 Dec 2019 22:48:29 -0500 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wknuapy75-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:29 -0500 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHb640894600 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AF9F14C04A; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 14F4D4C04E; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 02A3AA03DE; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 12/27] nvdimm/ocxl: Read the capability registers & wait for device ready Date: Tue, 3 Dec 2019 14:46:40 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0008-0000-0000-0000033C0FDD X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0009-0000-0000-00004A5B289B Message-Id: <20191203034655.51561-13-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 bulkscore=0 adultscore=0 mlxlogscore=999 phishscore=0 spamscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=1 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: MPWTEXKTLADXUBES3J7N6CQJHCAR6DLR X-Message-ID-Hash: MPWTEXKTLADXUBES3J7N6CQJHCAR6DLR X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch reads timeouts & firmware version from the controller, and uses those timeouts to wait for the controller to report that it is ready before handing the memory over to libnvdimm. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/Makefile | 2 +- drivers/nvdimm/ocxl/scm.c | 84 ++++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.c | 19 +++++++ drivers/nvdimm/ocxl/scm_internal.h | 24 +++++++++ 4 files changed, 128 insertions(+), 1 deletion(-) create mode 100644 drivers/nvdimm/ocxl/scm_internal.c diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile index 74a1bd98848e..9b6e31f0eb3e 100644 --- a/drivers/nvdimm/ocxl/Makefile +++ b/drivers/nvdimm/ocxl/Makefile @@ -4,4 +4,4 @@ ccflags-$(CONFIG_PPC_WERROR) += -Werror obj-$(CONFIG_OCXL_SCM) += ocxlscm.o -ocxlscm-y := scm.o +ocxlscm-y := scm.o scm_internal.o diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 571058a9e7b8..8088f65c289e 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -7,6 +7,7 @@ #include #include +#include #include #include #include @@ -266,6 +267,30 @@ static int scm_register_lpc_mem(struct scm_data *scm_data) return 0; } +/** + * scm_is_usable() - Is a controller usable? + * @scm_data: a pointer to the SCM device data + * Return: true if the controller is usable + */ +static bool scm_is_usable(const struct scm_data *scm_data) +{ + u64 chi = 0; + int rc = scm_chi(scm_data, &chi); + + if (!(chi & GLOBAL_MMIO_CHI_CRDY)) { + dev_err(&scm_data->dev, "SCM controller is not ready.\n"); + return false; + } + + if (!(chi & GLOBAL_MMIO_CHI_MA)) { + dev_err(&scm_data->dev, + "SCM controller does not have memory available.\n"); + return false; + } + + return true; +} + /** * allocate_scm_minor() - Allocate a minor number to use for an SCM device * @scm_data: The SCM device to associate the minor with @@ -380,6 +405,48 @@ static void scm_remove(struct pci_dev *pdev) } } +/** + * read_device_metadata() - Retrieve config information from the AFU and save it for future use + * @scm_data: the SCM metadata + * Return: 0 on success, negative on failure + */ +static int read_device_metadata(struct scm_data *scm_data) +{ + u64 val; + int rc; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, GLOBAL_MMIO_CCAP0, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + scm_data->scm_revision = val & 0xFFFF; + scm_data->read_latency = (val >> 32) & 0xFF; + scm_data->readiness_timeout = (val >> 48) & 0xff; + scm_data->memory_available_timeout = val >> 52; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, GLOBAL_MMIO_CCAP1, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + scm_data->max_controller_dump_size = val & 0xFFFFFFFF; + + // Extract firmware version text + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, GLOBAL_MMIO_FWVER, + OCXL_HOST_ENDIAN, (u64 *)scm_data->fw_version); + if (rc) + return rc; + + scm_data->fw_version[8] = '\0'; + + dev_info(&scm_data->dev, + "Firmware version '%s' SCM revision %d:%d\n", scm_data->fw_version, + scm_data->scm_revision >> 4, scm_data->scm_revision & 0x0F); + + return 0; +} + /** * scm_probe_function_0 - Set up function 0 for an OpenCAPI Storage Class Memory device * This is important as it enables templates higher than 0 across all other functions, @@ -420,6 +487,8 @@ static int scm_probe_function_0(struct pci_dev *pdev) static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { struct scm_data *scm_data = NULL; + int elapsed; + u16 timeout; if (PCI_FUNC(pdev->devfn) == 0) return scm_probe_function_0(pdev); @@ -469,6 +538,21 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (read_device_metadata(scm_data)) { + dev_err(&pdev->dev, "Could not read SCM device metadata\n"); + goto err; + } + + elapsed = 0; + timeout = scm_data->readiness_timeout + scm_data->memory_available_timeout; + while (!scm_is_usable(scm_data)) { + if (elapsed++ > timeout) { + dev_warn(&scm_data->dev, "SCM ready timeout.\n"); + goto err; + } + + msleep(1000); + } if (scm_register_lpc_mem(scm_data)) { dev_err(&pdev->dev, "Could not register OCXL SCM memory with libnvdimm\n"); goto err; diff --git a/drivers/nvdimm/ocxl/scm_internal.c b/drivers/nvdimm/ocxl/scm_internal.c new file mode 100644 index 000000000000..72d3c0e7d846 --- /dev/null +++ b/drivers/nvdimm/ocxl/scm_internal.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2019 IBM Corp. + +#include +#include +#include "scm_internal.h" + +int scm_chi(const struct scm_data *scm_data, u64 *chi) +{ + u64 val; + int rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, GLOBAL_MMIO_CHI, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + *chi = val; + + return 0; +} diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index d6ab361f5de9..584450f55e30 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -97,4 +97,28 @@ struct scm_data { void *metadata_addr; struct resource scm_res; struct nd_region *nd_region; + char fw_version[8+1]; + + u32 max_controller_dump_size; + u16 scm_revision; // major/minor + u8 readiness_timeout; /* The worst case time (in seconds) that the host shall + * wait for the controller to become operational following a reset (CHI.CRDY). + */ + u8 memory_available_timeout; /* The worst case time (in seconds) that the host shall + * wait for memory to become available following a reset (CHI.MA). + */ + + u16 read_latency; /* The nominal measure of latency (in nanoseconds) + * associated with an unassisted read of a memory block. + * This represents the capability of the raw media technology without assistance + */ }; + +/** + * scm_chi() - Get the value of the CHI register + * @scm_data: The SCM metadata + * @chi: returns the CHI value + * + * Returns 0 on success, negative on error + */ +int scm_chi(const struct scm_data *scm_data, u64 *chi); From patchwork Tue Dec 3 03:46:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C375A159A for ; Tue, 3 Dec 2019 03:48:40 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC5FE2071F for ; Tue, 3 Dec 2019 03:48:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC5FE2071F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 26D561011367E; Mon, 2 Dec 2019 19:51:56 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D9D4710113660 for ; Mon, 2 Dec 2019 19:51:52 -0800 (PST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33klVe001076 for ; Mon, 2 Dec 2019 22:48:29 -0500 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0b-001b2d01.pphosted.com with ESMTP id 2wm6myr92n-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:29 -0500 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:26 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHc547710208 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AAED3AE056; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13FA0AE051; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 178A2A03DF; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 13/27] nvdimm/ocxl: Add support for Admin commands Date: Tue, 3 Dec 2019 14:46:41 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0008-0000-0000-0000033C0FDF X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0009-0000-0000-00004A5B289C Message-Id: <20191203034655.51561-14-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 spamscore=0 phishscore=0 suspectscore=1 adultscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: PQPRX5GQRNGERYYMJJRQT7BLMCLHMU36 X-Message-ID-Hash: PQPRX5GQRNGERYYMJJRQT7BLMCLHMU36 X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch requests the metadata required to issue admin commands, as well as some helper functions to construct and check the completion of the commands. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 67 +++++++++++++ drivers/nvdimm/ocxl/scm_internal.c | 152 +++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.h | 62 ++++++++++++ 3 files changed, 281 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 8088f65c289e..1e175f3c3cf2 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -267,6 +267,58 @@ static int scm_register_lpc_mem(struct scm_data *scm_data) return 0; } +/** + * scm_extract_command_metadata() - Extract command data from MMIO & save it for further use + * @scm_data: a pointer to the SCM device data + * @offset: The base address of the command data structures (address of CREQO) + * @command_metadata: A pointer to the command metadata to populate + * Return: 0 on success, negative on failure + */ +static int scm_extract_command_metadata(struct scm_data *scm_data, u32 offset, + struct command_metadata *command_metadata) +{ + int rc; + u64 tmp; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, offset, OCXL_LITTLE_ENDIAN, + &tmp); + if (rc) + return rc; + + command_metadata->request_offset = tmp >> 32; + command_metadata->response_offset = tmp & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, offset + 8, OCXL_LITTLE_ENDIAN, + &tmp); + if (rc) + return rc; + + command_metadata->data_offset = tmp >> 32; + command_metadata->data_size = tmp & 0xFFFFFFFF; + + command_metadata->id = 0; + + return 0; +} + +/** + * scm_setup_command_metadata() - Set up the command metadata + * @scm_data: a pointer to the SCM device data + */ +static int scm_setup_command_metadata(struct scm_data *scm_data) +{ + int rc; + + mutex_init(&scm_data->admin_command.lock); + + rc = scm_extract_command_metadata(scm_data, GLOBAL_MMIO_ACMA_CREQO, + &scm_data->admin_command); + if (rc) + return rc; + + return 0; +} + /** * scm_is_usable() - Is a controller usable? * @scm_data: a pointer to the SCM device data @@ -276,6 +328,8 @@ static bool scm_is_usable(const struct scm_data *scm_data) { u64 chi = 0; int rc = scm_chi(scm_data, &chi); + if (rc) + return false; if (!(chi & GLOBAL_MMIO_CHI_CRDY)) { dev_err(&scm_data->dev, "SCM controller is not ready.\n"); @@ -502,6 +556,14 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) } scm_data->pdev = pdev; + scm_data->timeouts[ADMIN_COMMAND_ERRLOG] = 2000; // ms + scm_data->timeouts[ADMIN_COMMAND_HEARTBEAT] = 100; // ms + scm_data->timeouts[ADMIN_COMMAND_SMART] = 100; // ms + scm_data->timeouts[ADMIN_COMMAND_CONTROLLER_DUMP] = 1000; // ms + scm_data->timeouts[ADMIN_COMMAND_CONTROLLER_STATS] = 100; // ms + scm_data->timeouts[ADMIN_COMMAND_SHUTDOWN] = 1000; // ms + scm_data->timeouts[ADMIN_COMMAND_FW_UPDATE] = 16000; // ms + pci_set_drvdata(pdev, scm_data); scm_data->ocxl_fn = ocxl_function_open(pdev); @@ -543,6 +605,11 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (scm_setup_command_metadata(scm_data)) { + dev_err(&pdev->dev, "Could not read OCXL command matada\n"); + goto err; + } + elapsed = 0; timeout = scm_data->readiness_timeout + scm_data->memory_available_timeout; while (!scm_is_usable(scm_data)) { diff --git a/drivers/nvdimm/ocxl/scm_internal.c b/drivers/nvdimm/ocxl/scm_internal.c index 72d3c0e7d846..7b11b56863fb 100644 --- a/drivers/nvdimm/ocxl/scm_internal.c +++ b/drivers/nvdimm/ocxl/scm_internal.c @@ -17,3 +17,155 @@ int scm_chi(const struct scm_data *scm_data, u64 *chi) return 0; } + +static int scm_command_request(const struct scm_data *scm_data, + struct command_metadata *cmd, u8 op_code) +{ + u64 val = op_code; + int rc; + u8 i; + + cmd->op_code = op_code; + cmd->id++; + + val |= ((u64)cmd->id) << 16; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, cmd->request_offset, + OCXL_LITTLE_ENDIAN, val); + if (rc) + return rc; + + for (i = 0x08; i <= 0x38; i += 0x08) { + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + cmd->request_offset + i, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + return rc; + } + + return 0; +} + +int scm_admin_command_request(struct scm_data *scm_data, u8 op_code) +{ + u64 val; + int rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, GLOBAL_MMIO_CHI, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + return scm_command_request(scm_data, &scm_data->admin_command, op_code); +} + +static int scm_command_response(const struct scm_data *scm_data, + const struct command_metadata *cmd) +{ + u64 val; + u16 id; + u8 status; + int rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + cmd->response_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + status = val & 0xff; + id = (val >> 16) & 0xffff; + + if (id != cmd->id) { + dev_warn(&scm_data->dev, + "Expected response for command %d, but received response for command %d instead.\n", + cmd->id, id); + } + + return status; +} + +int scm_admin_response(const struct scm_data *scm_data) +{ + return scm_command_response(scm_data, &scm_data->admin_command); +} + + +int scm_admin_command_execute(const struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_HCI_ACRW); +} + +static bool scm_admin_command_complete(const struct scm_data *scm_data) +{ + u64 val = 0; + + int rc = scm_chi(scm_data, &val); + + WARN_ON(rc); + + return (val & GLOBAL_MMIO_CHI_ACRA) != 0; +} + +int scm_admin_command_complete_timeout(const struct scm_data *scm_data, + int command) +{ + u32 timeout = scm_data->timeouts[command]; + // 32 is the next power of 2 greater than the 20ms minimum for msleep +#define TIMEOUT_SLEEP_MILLIS 32 + timeout /= TIMEOUT_SLEEP_MILLIS; + if (!timeout) + timeout = SCM_DEFAULT_TIMEOUT / TIMEOUT_SLEEP_MILLIS; + + while (timeout-- > 0) { + if (scm_admin_command_complete(scm_data)) + return 0; + msleep(TIMEOUT_SLEEP_MILLIS); + } + + if (scm_admin_command_complete(scm_data)) + return 0; + + return -EBUSY; +} + +int scm_admin_response_handled(const struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_CHIC, + OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_CHI_ACRA); +} + +void scm_warn_status(const struct scm_data *scm_data, const char *message, + u8 status) +{ + const char *text = "Unknown"; + + switch (status) { + case STATUS_SUCCESS: + text = "Success"; + break; + + case STATUS_MEM_UNAVAILABLE: + text = "Persistent memory unavailable"; + break; + + case STATUS_BAD_OPCODE: + text = "Bad opcode"; + break; + + case STATUS_BAD_REQUEST_PARM: + text = "Bad request parameter"; + break; + + case STATUS_BAD_DATA_PARM: + text = "Bad data parameter"; + break; + + case STATUS_DEBUG_BLOCKED: + text = "Debug action blocked"; + break; + + case STATUS_FAIL: + text = "Failed"; + break; + } + + dev_warn(&scm_data->dev, "%s: %s (%x)\n", message, text, status); +} diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 584450f55e30..9bff684cd069 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -6,6 +6,8 @@ #include #include +#define SCM_DEFAULT_TIMEOUT 100 + #define GLOBAL_MMIO_CHI 0x000 #define GLOBAL_MMIO_CHIC 0x008 #define GLOBAL_MMIO_CHIE 0x010 @@ -80,6 +82,16 @@ #define SCM_LABEL_AREA_SIZE (1UL << PA_SECTION_SHIFT) +struct command_metadata { + u32 request_offset; + u32 response_offset; + u32 data_offset; + u32 data_size; + struct mutex lock; + u16 id; + u8 op_code; +}; + struct scm_function_0 { struct pci_dev *pdev; struct ocxl_fn *ocxl_fn; @@ -95,9 +107,11 @@ struct scm_data { struct ocxl_afu *ocxl_afu; struct ocxl_context *ocxl_context; void *metadata_addr; + struct command_metadata admin_command; struct resource scm_res; struct nd_region *nd_region; char fw_version[8+1]; + u32 timeouts[ADMIN_COMMAND_MAX+1]; u32 max_controller_dump_size; u16 scm_revision; // major/minor @@ -122,3 +136,51 @@ struct scm_data { * Returns 0 on success, negative on error */ int scm_chi(const struct scm_data *scm_data, u64 *chi); + +/** + * scm_admin_command_request() - Issue an admin command request + * @scm_data: a pointer to the SCM device data + * @op_code: The op-code for the command + * + * Returns an identifier for the command, or negative on error + */ +int scm_admin_command_request(struct scm_data *scm_data, u8 op_code); + +/** + * scm_admin_response() - Validate an admin response + * @scm_data: a pointer to the SCM device data + * Returns the status code of the command, or negative on error + */ +int scm_admin_response(const struct scm_data *scm_data); + +/** + * scm_admin_command_execute() - Notify the controller to start processing a pending admin command + * @scm_data: a pointer to the SCM device data + * Returns 0 on success, negative on error + */ +int scm_admin_command_execute(const struct scm_data *scm_data); + +/** + * scm_admin_command_complete_timeout() - Wait for an admin command to finish executing + * @scm_data: a pointer to the SCM device data + * @command: the admin command to wait for completion (determines the timeout) + * Returns 0 on success, -EBUSY on timeout + */ +int scm_admin_command_complete_timeout(const struct scm_data *scm_data, + int command); + +/** + * scm_admin_response_handled() - Notify the controller that the admin response has been handled + * @scm_data: a pointer to the SCM device data + * Returns 0 on success, negative on failure + */ +int scm_admin_response_handled(const struct scm_data *scm_data); + +/** + * scm_warn_status() - Emit a kernel warning showing a command status. + * @scm_data: a pointer to the SCM device data + * @message: A message to accompany the warning + * @status: The command status + */ +void scm_warn_status(const struct scm_data *scm_data, const char *message, + u8 status); From patchwork Tue Dec 3 03:46:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 022926C1 for ; Tue, 3 Dec 2019 03:48:35 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF3682071F for ; Tue, 3 Dec 2019 03:48:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF3682071F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9A4AC1011366C; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 1065E1011363B for ; Mon, 2 Dec 2019 19:51:52 -0800 (PST) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kko8001762 for ; Mon, 2 Dec 2019 22:48:29 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wn3pd3evq-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:28 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:18 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mHRK51839012 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:17 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ABAABAE057; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1561FAE053; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 2E613A03E0; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 14/27] nvdimm/ocxl: Add support for near storage commands Date: Tue, 3 Dec 2019 14:46:42 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0028-0000-0000-000003C3CC74 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0029-0000-0000-00002486E433 Message-Id: <20191203034655.51561-15-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 priorityscore=1501 mlxscore=0 adultscore=0 impostorscore=0 suspectscore=1 bulkscore=0 lowpriorityscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: ACEUP5YPRLEBTHR7JR4NB6MAHCIDQXPX X-Message-ID-Hash: ACEUP5YPRLEBTHR7JR4NB6MAHCIDQXPX X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva Similar to the previous patch, this adds support for near storage commands. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 6 +++++ drivers/nvdimm/ocxl/scm_internal.c | 41 ++++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.h | 38 +++++++++++++++++++++++++++ 3 files changed, 85 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 1e175f3c3cf2..6c16ca7fabfa 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -310,12 +310,18 @@ static int scm_setup_command_metadata(struct scm_data *scm_data) int rc; mutex_init(&scm_data->admin_command.lock); + mutex_init(&scm_data->ns_command.lock); rc = scm_extract_command_metadata(scm_data, GLOBAL_MMIO_ACMA_CREQO, &scm_data->admin_command); if (rc) return rc; + rc = scm_extract_command_metadata(scm_data, GLOBAL_MMIO_NSCMA_CREQO, + &scm_data->ns_command); + if (rc) + return rc; + return 0; } diff --git a/drivers/nvdimm/ocxl/scm_internal.c b/drivers/nvdimm/ocxl/scm_internal.c index 7b11b56863fb..c405f1d8afb8 100644 --- a/drivers/nvdimm/ocxl/scm_internal.c +++ b/drivers/nvdimm/ocxl/scm_internal.c @@ -132,6 +132,47 @@ int scm_admin_response_handled(const struct scm_data *scm_data) OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_CHI_ACRA); } +int scm_ns_command_request(struct scm_data *scm_data, u8 op_code) +{ + u64 val; + int rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, GLOBAL_MMIO_CHI, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + if (!(val & GLOBAL_MMIO_CHI_NSCRA)) + return -EBUSY; + + return scm_command_request(scm_data, &scm_data->ns_command, op_code); +} + +int scm_ns_response(const struct scm_data *scm_data) +{ + return scm_command_response(scm_data, &scm_data->ns_command); +} + +int scm_ns_command_execute(const struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_HCI_NSCRW); +} + +bool scm_ns_command_complete(const struct scm_data *scm_data) +{ + u64 val = 0; + int rc = scm_chi(scm_data, &val); + + WARN_ON(rc); + + return (val & GLOBAL_MMIO_CHI_NSCRA) != 0; +} + +int scm_ns_response_handled(const struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_CHIC, + OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_CHI_NSCRA); +} + void scm_warn_status(const struct scm_data *scm_data, const char *message, u8 status) { diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 9bff684cd069..9575996a89e7 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -108,6 +108,7 @@ struct scm_data { struct ocxl_context *ocxl_context; void *metadata_addr; struct command_metadata admin_command; + struct command_metadata ns_command; struct resource scm_res; struct nd_region *nd_region; char fw_version[8+1]; @@ -176,6 +177,42 @@ int scm_admin_command_complete_timeout(const struct scm_data *scm_data, */ int scm_admin_response_handled(const struct scm_data *scm_data); +/** + * scm_ns_command_request() - Issue a near storage command request + * @scm_data: a pointer to the SCM device data + * @op_code: The op-code for the command + * Returns an identifier for the command, or negative on error + */ +int scm_ns_command_request(struct scm_data *scm_data, u8 op_code); + +/** + * scm_ns_response() - Validate a near storage response + * @scm_data: a pointer to the SCM device data + * Returns the status code of the command, or negative on error + */ +int scm_ns_response(const struct scm_data *scm_data); + +/** + * scm_ns_command_execute() - Notify the controller to start processing a pending near storage command + * @scm_data: a pointer to the SCM device data + * Returns 0 on success, negative on error + */ +int scm_ns_command_execute(const struct scm_data *scm_data); + +/** + * scm_ns_command_complete() - Is a near storage command executing + * scm_data: a pointer to the SCM device data + * Returns true if the previous admin command has completed + */ +bool scm_ns_command_complete(const struct scm_data *scm_data); + +/** + * scm_ns_response_handled() - Notify the controller that the near storage response has been handled + * scm_data: a pointer to the SCM device data + * Returns 0 on success, negative on failure + */ +int scm_ns_response_handled(const struct scm_data *scm_data); + /** * scm_warn_status() - Emit a kernel warning showing a command status. * @scm_data: a pointer to the SCM device data @@ -184,3 +221,4 @@ int scm_admin_response_handled(const struct scm_data *scm_data); */ void scm_warn_status(const struct scm_data *scm_data, const char *message, u8 status); + From patchwork Tue Dec 3 03:46:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09F896C1 for ; Tue, 3 Dec 2019 03:48:55 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E6FF92073F for ; Tue, 3 Dec 2019 03:48:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6FF92073F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 6EDFF1011369B; Mon, 2 Dec 2019 19:52:01 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 057F41011366B for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33koAJ076599 for ; Mon, 2 Dec 2019 22:48:31 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6s60fpn-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:20 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mJEj46334272 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1C32A4204C; Tue, 3 Dec 2019 03:48:19 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 75F574203F; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 3B162A03E1; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 15/27] nvdimm/ocxl: Register a character device for userspace to interact with Date: Tue, 3 Dec 2019 14:46:43 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-4275-0000-0000-0000038A31E2 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-4276-0000-0000-0000389DCDDB Message-Id: <20191203034655.51561-16-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 mlxlogscore=999 suspectscore=3 clxscore=1015 priorityscore=1501 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: 6PJ5AWDI2ON5GFBVVPRX7UVQLVK3HKVP X-Message-ID-Hash: 6PJ5AWDI2ON5GFBVVPRX7UVQLVK3HKVP X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch introduces a character device (/dev/ocxl-scmX) which further patches will use to interact with userspace. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 114 ++++++++++++++++++++++++++++- drivers/nvdimm/ocxl/scm_internal.h | 2 + 2 files changed, 115 insertions(+), 1 deletion(-) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 6c16ca7fabfa..c313a473a28e 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include "scm_internal.h" @@ -386,6 +387,9 @@ static void free_scm(struct scm_data *scm_data) free_scm_minor(scm_data); + if (scm_data->cdev.owner) + cdev_del(&scm_data->cdev); + if (scm_data->metadata_addr) devm_memunmap(&scm_data->dev, scm_data->metadata_addr); @@ -444,6 +448,70 @@ static int scm_register(struct scm_data *scm_data) return rc; } +static void scm_put(struct scm_data *scm_data) +{ + put_device(&scm_data->dev); +} + +static struct scm_data *scm_get(struct scm_data *scm_data) +{ + return (get_device(&scm_data->dev) == NULL) ? NULL : scm_data; +} + +static struct scm_data *find_and_get_scm(dev_t devno) +{ + struct scm_data *scm_data; + int minor = MINOR(devno); + /* + * We don't declare an RCU critical section here, as our AFU + * is protected by a reference counter on the device. By the time the + * minor number of a device is removed from the idr, the ref count of + * the device is already at 0, so no user API will access that AFU and + * this function can't return it. + */ + scm_data = idr_find(&minors_idr, minor); + if (scm_data) + scm_get(scm_data); + return scm_data; +} + +static int scm_file_open(struct inode *inode, struct file *file) +{ + struct scm_data *scm_data; + + scm_data = find_and_get_scm(inode->i_rdev); + if (!scm_data) + return -ENODEV; + + file->private_data = scm_data; + return 0; +} + +static int scm_file_release(struct inode *inode, struct file *file) +{ + struct scm_data *scm_data = file->private_data; + + scm_put(scm_data); + return 0; +} + +static const struct file_operations scm_fops = { + .owner = THIS_MODULE, + .open = scm_file_open, + .release = scm_file_release, +}; + +/** + * scm_create_cdev() - Create the chardev in /dev for this scm device + * @scm_data: the SCM metadata + * Return: 0 on success, negative on failure + */ +static int scm_create_cdev(struct scm_data *scm_data) +{ + cdev_init(&scm_data->cdev, &scm_fops); + return cdev_add(&scm_data->cdev, scm_data->dev.devt, 1); +} + /** * scm_remove() - Free an OpenCAPI Storage Class Memory device * @pdev: the PCI device information struct @@ -616,6 +684,11 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (scm_create_cdev(scm_data)) { + dev_err(&pdev->dev, "Could not create SCM character device\n"); + goto err; + } + elapsed = 0; timeout = scm_data->readiness_timeout + scm_data->memory_available_timeout; while (!scm_is_usable(scm_data)) { @@ -653,20 +726,59 @@ static struct pci_driver scm_pci_driver = { .shutdown = scm_remove, }; +static int scm_file_init(void) +{ + int rc; + + mutex_init(&minors_idr_lock); + idr_init(&minors_idr); + + rc = alloc_chrdev_region(&scm_dev, 0, SCM_NUM_MINORS, "ocxl-scm"); + if (rc) { + idr_destroy(&minors_idr); + pr_err("Unable to allocate scm major number: %d\n", rc); + return rc; + } + + scm_class = class_create(THIS_MODULE, "ocxl-scm"); + if (IS_ERR(scm_class)) { + idr_destroy(&minors_idr); + pr_err("Unable to create ocxl-scm class\n"); + unregister_chrdev_region(scm_dev, SCM_NUM_MINORS); + return PTR_ERR(scm_class); + } + + return 0; +} + +static void scm_file_exit(void) +{ + class_destroy(scm_class); + unregister_chrdev_region(scm_dev, SCM_NUM_MINORS); + idr_destroy(&minors_idr); +} + static int __init scm_init(void) { int rc = 0; - rc = pci_register_driver(&scm_pci_driver); + rc = scm_file_init(); if (rc) return rc; + rc = pci_register_driver(&scm_pci_driver); + if (rc) { + scm_file_exit(); + return rc; + } + return 0; } static void scm_exit(void) { pci_unregister_driver(&scm_pci_driver); + scm_file_exit(); } module_init(scm_init); diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 9575996a89e7..57491dbee1a4 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -2,6 +2,7 @@ // Copyright 2019 IBM Corp. #include +#include #include #include #include @@ -100,6 +101,7 @@ struct scm_function_0 { struct scm_data { struct device dev; struct pci_dev *pdev; + struct cdev cdev; struct ocxl_fn *ocxl_fn; struct nd_interleave_set nd_set; struct nvdimm_bus_descriptor bus_desc; From patchwork Tue Dec 3 03:46:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 468AB109A for ; Tue, 3 Dec 2019 03:48:46 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F91320733 for ; Tue, 3 Dec 2019 03:48:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F91320733 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 8D53D10113686; Mon, 2 Dec 2019 19:51:57 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 714CD10113666 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33klkl032879 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6snr95b-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:25 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mI1Z30146586 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:18 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 82817AE051; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E249CAE04D; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 4411CA03E2; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 16/27] nvdimm/ocxl: Implement the Read Error Log command Date: Tue, 3 Dec 2019 14:46:44 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0012-0000-0000-000003701289 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0013-0000-0000-000021ABCCBF Message-Id: <20191203034655.51561-17-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 mlxscore=0 suspectscore=3 clxscore=1015 malwarescore=0 adultscore=0 impostorscore=0 phishscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: IVM4ZUD2K6JTNN7SJDRVUNIEW546T2Q7 X-Message-ID-Hash: IVM4ZUD2K6JTNN7SJDRVUNIEW546T2Q7 X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva The read error log command extracts information from the controller's internal error log. This patch exposes this information in 2 ways: - During probe, if an error occurs & a log is available, print it to the console - After probe, make the error log available to userspace via an IOCTL. Userspace is notified of pending error logs in a later patch ("nvdimm/ocxl: Forward events to userspace") Signed-off-by: Alastair D'Silva Reported-by: kbuild test robot --- drivers/nvdimm/ocxl/scm.c | 270 +++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.h | 1 + include/uapi/nvdimm/ocxl-scm.h | 46 +++++ 3 files changed, 317 insertions(+) create mode 100644 include/uapi/nvdimm/ocxl-scm.h diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index c313a473a28e..0bbe1a14291e 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -495,10 +495,220 @@ static int scm_file_release(struct inode *inode, struct file *file) return 0; } +/** + * scm_error_log_header_parse() - Parse the first 64 bits of the error log command response + * @scm_data: the SCM metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int scm_error_log_header_parse(struct scm_data *scm_data, u16 *length) +{ + int rc; + u64 val; + + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFF; + + if (data_identifier != 0x454C) { + dev_err(&scm_data->dev, + "Bad data identifier for error log data, expected 'EL', got '%2s' (%#x), data_length=%u\n", + (char *)&data_identifier, + (unsigned int)data_identifier, data_length); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int scm_error_log_offset_0x08(struct scm_data *scm_data, + u32 *log_identifier, u32 *program_ref_code) +{ + int rc; + u64 val; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + *log_identifier = val >> 32; + *program_ref_code = val & 0xFFFFFFFF; + + return 0; +} + +static int scm_read_error_log(struct scm_data *scm_data, + struct scm_ioctl_error_log *log, bool buf_is_user) +{ + u64 val; + u16 user_buf_length; + u16 buf_length; + u16 i; + int rc; + + if (log->buf_size % 8) + return -EINVAL; + + rc = scm_chi(scm_data, &val); + if (rc) + goto out; + + if (!(val & GLOBAL_MMIO_CHI_ELA)) + return -EAGAIN; + + user_buf_length = log->buf_size; + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_ERRLOG); + if (rc) + goto out; + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_complete_timeout(scm_data, ADMIN_COMMAND_ERRLOG); + if (rc < 0) { + dev_warn(&scm_data->dev, "Read error log timed out\n"); + goto out; + } + + rc = scm_admin_response(scm_data); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + scm_warn_status(scm_data, "Unexpected status from retrieve error log", rc); + goto out; + } + + + rc = scm_error_log_header_parse(scm_data, &log->buf_size); + if (rc) + goto out; + // log->buf_size now contains the scm buffer size, not the user size + + rc = scm_error_log_offset_0x08(scm_data, &log->log_identifier, + &log->program_reference_code); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x10, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + log->error_log_type = val >> 56; + log->action_flags = (log->error_log_type == SCM_ERROR_LOG_TYPE_GENERAL) ? + (val >> 32) & 0xFFFFFF : 0; + log->power_on_seconds = val & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x18, + OCXL_LITTLE_ENDIAN, &log->timestamp); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x20, + OCXL_HOST_ENDIAN, &log->wwid[0]); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x28, + OCXL_HOST_ENDIAN, &log->wwid[1]); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x30, + OCXL_HOST_ENDIAN, (u64 *)log->fw_revision); + if (rc) + goto out; + log->fw_revision[8] = '\0'; + + buf_length = (user_buf_length < log->buf_size) ? + user_buf_length : log->buf_size; + for (i = 0; i < buf_length + 0x48; i += 8) { + u64 val; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, &val); + if (rc) + goto out; + + if (buf_is_user) { + if (copy_to_user(&log->buf[i], &val, sizeof(u64))) { + rc = -EFAULT; + goto out; + } + } else + log->buf[i] = val; + } + + rc = scm_admin_response_handled(scm_data); + if (rc) + goto out; + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; + +} + +static int scm_ioctl_error_log(struct scm_data *scm_data, + struct scm_ioctl_error_log __user *uarg) +{ + struct scm_ioctl_error_log args; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + rc = scm_read_error_log(scm_data, &args, true); + if (rc) + return rc; + + if (copy_to_user(uarg, &args, sizeof(args))) + return -EFAULT; + + return 0; +} + +static long scm_file_ioctl(struct file *file, unsigned int cmd, + unsigned long args) +{ + struct scm_data *scm_data = file->private_data; + int rc = -EINVAL; + + switch (cmd) { + case SCM_IOCTL_ERROR_LOG: + rc = scm_ioctl_error_log(scm_data, + (struct scm_ioctl_error_log __user *)args); + break; + } + return rc; +} + static const struct file_operations scm_fops = { .owner = THIS_MODULE, .open = scm_file_open, .release = scm_file_release, + .unlocked_ioctl = scm_file_ioctl, + .compat_ioctl = scm_file_ioctl, }; /** @@ -575,6 +785,60 @@ static int read_device_metadata(struct scm_data *scm_data) return 0; } +static const char *scm_decode_error_log_type(u8 error_log_type) +{ + switch (error_log_type) { + case 0x00: + return "general"; + case 0x01: + return "predictive failure"; + case 0x02: + return "thermal warning"; + case 0x03: + return "data loss"; + case 0x04: + return "health & performance"; + default: + return "unknown"; + } +} + +static void scm_dump_error_log(struct scm_data *scm_data) +{ + struct scm_ioctl_error_log log; + u32 buf_size; + u8 *buf; + int rc; + + if (scm_data->admin_command.data_size == 0) + return; + + buf_size = scm_data->admin_command.data_size - 0x48; + buf = kzalloc(buf_size, GFP_KERNEL); + if (!buf) + return; + + log.buf = buf; + log.buf_size = buf_size; + + rc = scm_read_error_log(scm_data, &log, false); + if (rc < 0) + goto out; + + dev_warn(&scm_data->dev, + "SCM Error log: WWID=0x%016llx%016llx LID=0x%x PRC=%x type=0x%x %s, Uptime=%u seconds timestamp=0x%llx\n", + log.wwid[0], log.wwid[1], + log.log_identifier, log.program_reference_code, + log.error_log_type, + scm_decode_error_log_type(log.error_log_type), + log.power_on_seconds, log.timestamp); + print_hex_dump(KERN_WARNING, "buf", DUMP_PREFIX_OFFSET, 16, 1, buf, + log.buf_size, false); + +out: + kfree(buf); +} + /** * scm_probe_function_0 - Set up function 0 for an OpenCAPI Storage Class Memory device * This is important as it enables templates higher than 0 across all other functions, @@ -617,6 +881,7 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) struct scm_data *scm_data = NULL; int elapsed; u16 timeout; + u64 chi; if (PCI_FUNC(pdev->devfn) == 0) return scm_probe_function_0(pdev); @@ -707,6 +972,11 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return 0; err: + if (scm_data && + (scm_chi(scm_data, &chi) == 0) && + (chi & GLOBAL_MMIO_CHI_ELA)) + scm_dump_error_log(scm_data); + /* * Further cleanup is done in the release handler via free_scm() * This allows us to keep the character device live to handle IOCTLs to diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 57491dbee1a4..9bf8fcf30ea6 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #define SCM_DEFAULT_TIMEOUT 100 diff --git a/include/uapi/nvdimm/ocxl-scm.h b/include/uapi/nvdimm/ocxl-scm.h new file mode 100644 index 000000000000..b34dd1ba06ff --- /dev/null +++ b/include/uapi/nvdimm/ocxl-scm.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* Copyright 2017 IBM Corp. */ +#ifndef _UAPI_OCXL_SCM_H +#define _UAPI_OCXL_SCM_H + +#include +#include + +#define SCM_ERROR_LOG_ACTION_RESET (1 << (32-32)) +#define SCM_ERROR_LOG_ACTION_CHKFW (1 << (53-32)) +#define SCM_ERROR_LOG_ACTION_REPLACE (1 << (54-32)) +#define SCM_ERROR_LOG_ACTION_DUMP (1 << (55-32)) + +#define SCM_ERROR_LOG_TYPE_GENERAL (0x00) +#define SCM_ERROR_LOG_TYPE_PREDICTIVE_FAILURE (0x01) +#define SCM_ERROR_LOG_TYPE_THERMAL_WARNING (0x02) +#define SCM_ERROR_LOG_TYPE_DATA_LOSS (0x03) +#define SCM_ERROR_LOG_TYPE_HEALTH_PERFORMANCE (0x04) + +struct scm_ioctl_error_log { + __u32 log_identifier; // out + __u32 program_reference_code; // out + __u32 action_flags; //out, recommended course of action + __u32 power_on_seconds; // out, Number of seconds the controller has been on when the error occurred + __u64 timestamp; // out, relative time since the current IPL + __u64 wwid[2]; // out, the NAA formatted WWID associated with the controller + char fw_revision[8+1]; // out, firmware revision as null terminated text + __u16 buf_size; /* in/out, buffer size provided/required. + * If required is greater than provided, the buffer + * will be truncated to the amount provided. If its + * less, then only the required bytes will be populated. + * If it is 0, then there are no more error log entries. + */ + __u8 error_log_type; + __u8 reserved1; + __u32 reserved2; + __u64 reserved3[2]; + __u8 *buf; // pointer to output buffer +}; + +/* ioctl numbers */ +#define SCM_MAGIC 0x5C +/* SCM devices */ +#define SCM_IOCTL_ERROR_LOG _IOWR(SCM_MAGIC, 0x01, struct scm_ioctl_error_log) + +#endif /* _UAPI_OCXL_SCM_H */ From patchwork Tue Dec 3 03:46:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C01C6C1 for ; Tue, 3 Dec 2019 03:48:49 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E93242071F for ; Tue, 3 Dec 2019 03:48:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E93242071F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id D1EBB1011368C; Mon, 2 Dec 2019 19:51:58 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 836C010113669 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33lGvv111334 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6v00yfx-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mI2F44630180 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:18 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8C2A1A4040; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EBBFEA4057; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 4DCB9A03E3; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 17/27] nvdimm/ocxl: Add controller dump IOCTLs Date: Tue, 3 Dec 2019 14:46:45 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0020-0000-0000-00000392CC5A X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0021-0000-0000-000021E9EAA9 Message-Id: <20191203034655.51561-18-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=12 malwarescore=0 clxscore=1015 priorityscore=1501 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 suspectscore=1 lowpriorityscore=0 mlxlogscore=88 mlxscore=12 spamscore=12 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: ZS7YZEVXEI5AZFZ6HR4SWBECC5GHHLOK X-Message-ID-Hash: ZS7YZEVXEI5AZFZ6HR4SWBECC5GHHLOK X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch adds IOCTLs to allow userspace to request & fetch dumps of the internal controller state. This is useful during debugging or when a fatal error on the controller has occurred. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 132 +++++++++++++++++++++++++++++++++ include/uapi/nvdimm/ocxl-scm.h | 15 ++++ 2 files changed, 147 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 0bbe1a14291e..a520f209d626 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -688,6 +688,124 @@ static int scm_ioctl_error_log(struct scm_data *scm_data, return 0; } +static int scm_ioctl_controller_dump_data(struct scm_data *scm_data, + struct scm_ioctl_controller_dump_data __user *uarg) +{ + struct scm_ioctl_controller_dump_data args; + u16 i; + u64 val; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + if (args.buf_size % 8) + return -EINVAL; + + if (args.buf_size > scm_data->admin_command.data_size) + return -EINVAL; + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_CONTROLLER_DUMP); + if (rc) + goto out; + + val = ((u64)args.offset) << 32; + val |= args.buf_size; + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + 0x08, + OCXL_LITTLE_ENDIAN, val); + if (rc) + goto out; + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_complete_timeout(scm_data, + ADMIN_COMMAND_CONTROLLER_DUMP); + if (rc < 0) { + dev_warn(&scm_data->dev, "Controller dump timed out\n"); + goto out; + } + + rc = scm_admin_response(scm_data); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + scm_warn_status(scm_data, + "Unexpected status from retrieve error log", + rc); + goto out; + } + + for (i = 0; i < args.buf_size; i += 8) { + u64 val; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, &val); + if (rc) + goto out; + + if (copy_to_user(&args.buf[i], &val, sizeof(u64))) { + rc = -EFAULT; + goto out; + } + } + + if (copy_to_user(uarg, &args, sizeof(args))) { + rc = -EFAULT; + goto out; + } + + rc = scm_admin_response_handled(scm_data); + if (rc) + goto out; + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; +} + +int scm_request_controller_dump(struct scm_data *scm_data) +{ + int rc; + u64 busy = 1; + + rc = ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_CHIC, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_CHI_CDA); + + + rc = ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_CONTROLLER_DUMP); + if (rc) + return rc; + + while (busy) { + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, &busy); + if (rc) + return rc; + + busy &= GLOBAL_MMIO_HCI_CONTROLLER_DUMP; + cond_resched(); + } + + return 0; +} + +static int scm_ioctl_controller_dump_complete(struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED); +} + static long scm_file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { @@ -699,7 +817,21 @@ static long scm_file_ioctl(struct file *file, unsigned int cmd, rc = scm_ioctl_error_log(scm_data, (struct scm_ioctl_error_log __user *)args); break; + + case SCM_IOCTL_CONTROLLER_DUMP: + rc = scm_request_controller_dump(scm_data); + break; + + case SCM_IOCTL_CONTROLLER_DUMP_DATA: + rc = scm_ioctl_controller_dump_data(scm_data, + (struct scm_ioctl_controller_dump_data __user *)args); + break; + + case SCM_IOCTL_CONTROLLER_DUMP_COMPLETE: + rc = scm_ioctl_controller_dump_complete(scm_data); + break; } + return rc; } diff --git a/include/uapi/nvdimm/ocxl-scm.h b/include/uapi/nvdimm/ocxl-scm.h index b34dd1ba06ff..abd2dc9ea112 100644 --- a/include/uapi/nvdimm/ocxl-scm.h +++ b/include/uapi/nvdimm/ocxl-scm.h @@ -38,9 +38,24 @@ struct scm_ioctl_error_log { __u8 *buf; // pointer to output buffer }; +struct scm_ioctl_controller_dump_data { + __u8 *buf; // pointer to output buffer + __u16 buf_size; /* in/out, buffer size provided/required. + * If required is greater than provided, the buffer + * will be truncated to the amount provided. If its + * less, then only the required bytes will be populated. + * If it is 0, then there is no more dump data available. + */ + __u32 offset; // in, Offset within the dump + __u64 reserved[8]; +}; + /* ioctl numbers */ #define SCM_MAGIC 0x5C /* SCM devices */ #define SCM_IOCTL_ERROR_LOG _IOWR(SCM_MAGIC, 0x01, struct scm_ioctl_error_log) +#define SCM_IOCTL_CONTROLLER_DUMP _IO(SCM_MAGIC, 0x02) +#define SCM_IOCTL_CONTROLLER_DUMP_DATA _IOWR(SCM_MAGIC, 0x03, struct scm_ioctl_controller_dump_data) +#define SCM_IOCTL_CONTROLLER_DUMP_COMPLETE _IO(SCM_MAGIC, 0x04) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Tue Dec 3 03:46:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A1E1109A for ; Tue, 3 Dec 2019 03:48:51 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 21EA620726 for ; Tue, 3 Dec 2019 03:48:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21EA620726 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 2F68D1011368D; Mon, 2 Dec 2019 19:52:00 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 9EEE010113669 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmRn076422 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6s60fp8-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:26 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:20 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mJe930998776 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2870F4C052; Tue, 3 Dec 2019 03:48:19 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 802D44C046; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 5663FA03E6; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 18/27] nvdimm/ocxl: Add an IOCTL to report controller statistics Date: Tue, 3 Dec 2019 14:46:46 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0016-0000-0000-000002D0158D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0017-0000-0000-000033320CA8 Message-Id: <20191203034655.51561-19-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 adultscore=0 mlxlogscore=722 suspectscore=1 clxscore=1015 priorityscore=1501 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: LXC5UBUGU3PBPOROXZ52HXKVERSTQWSM X-Message-ID-Hash: LXC5UBUGU3PBPOROXZ52HXKVERSTQWSM X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva The controller can report a number of statistics that are useful in evaluating the performance and reliability of the card. This patch exposes this information via an IOCTL. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 185 +++++++++++++++++++++++++++++++++ include/uapi/nvdimm/ocxl-scm.h | 17 +++ 2 files changed, 202 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index a520f209d626..54a2bac1cab7 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -806,6 +806,186 @@ static int scm_ioctl_controller_dump_complete(struct scm_data *scm_data) GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED); } +/** + * scm_controller_stats_header_parse() - Parse the first 64 bits of the controller stats admin command response + * @scm_data: the SCM metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int scm_controller_stats_header_parse(struct scm_data *scm_data, + u32 *length) +{ + int rc; + u64 val; + + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFFFFFF; + + if (data_identifier != 0x4353) { + dev_err(&scm_data->dev, + "Bad data identifier for controller stats, expected 'CS', got '%-.*s'\n", + 2, (char *)&data_identifier); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int scm_ioctl_controller_stats(struct scm_data *scm_data, + struct scm_ioctl_controller_stats __user *uarg) +{ + struct scm_ioctl_controller_stats args; + u32 length; + int rc; + u64 val; + + memset(&args, '\0', sizeof(args)); + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_CONTROLLER_STATS); + if (rc) + goto out; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + 0x08, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out; + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + + rc = scm_admin_command_complete_timeout(scm_data, + ADMIN_COMMAND_CONTROLLER_STATS); + if (rc < 0) { + dev_warn(&scm_data->dev, "Controller stats timed out\n"); + goto out; + } + + rc = scm_admin_response(scm_data); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + scm_warn_status(scm_data, + "Unexpected status from controller stats", rc); + goto out; + } + + rc = scm_controller_stats_header_parse(scm_data, &length); + if (rc) + goto out; + + if (length != 0x140) + scm_warn_status(scm_data, + "Unexpected length for controller stats data, expected 0x140, got 0x%x", + length); + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x08, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + args.reset_count = val >> 32; + args.reset_uptime = val & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x10, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + args.power_on_uptime = val >> 32; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x08, + OCXL_LITTLE_ENDIAN, &args.host_load_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x10, + OCXL_LITTLE_ENDIAN, &args.host_store_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x18, + OCXL_LITTLE_ENDIAN, &args.media_read_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x20, + OCXL_LITTLE_ENDIAN, &args.media_write_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x28, + OCXL_LITTLE_ENDIAN, &args.cache_hit_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x30, + OCXL_LITTLE_ENDIAN, &args.cache_miss_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x38, + OCXL_LITTLE_ENDIAN, &args.media_read_latency); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x40, + OCXL_LITTLE_ENDIAN, &args.media_write_latency); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x48, + OCXL_LITTLE_ENDIAN, &args.cache_read_latency); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + 0x08 + 0x40 + 0x50, + OCXL_LITTLE_ENDIAN, &args.cache_write_latency); + if (rc) + goto out; + + if (copy_to_user(uarg, &args, sizeof(args))) { + rc = -EFAULT; + goto out; + } + + rc = scm_admin_response_handled(scm_data); + if (rc) + goto out; + + rc = 0; + goto out; + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; +} + static long scm_file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { @@ -830,6 +1010,11 @@ static long scm_file_ioctl(struct file *file, unsigned int cmd, case SCM_IOCTL_CONTROLLER_DUMP_COMPLETE: rc = scm_ioctl_controller_dump_complete(scm_data); break; + + case SCM_IOCTL_CONTROLLER_STATS: + rc = scm_ioctl_controller_stats(scm_data, + (struct scm_ioctl_controller_stats __user *)args); + break; } return rc; diff --git a/include/uapi/nvdimm/ocxl-scm.h b/include/uapi/nvdimm/ocxl-scm.h index abd2dc9ea112..0a5de46c5acd 100644 --- a/include/uapi/nvdimm/ocxl-scm.h +++ b/include/uapi/nvdimm/ocxl-scm.h @@ -50,6 +50,22 @@ struct scm_ioctl_controller_dump_data { __u64 reserved[8]; }; +struct scm_ioctl_controller_stats { + __u32 reset_count; + __u32 reset_uptime; // seconds + __u32 power_on_uptime; // seconds + __u64 host_load_count; + __u64 host_store_count; + __u64 media_read_count; + __u64 media_write_count; + __u64 cache_hit_count; + __u64 cache_miss_count; + __u64 media_read_latency; // nanoseconds + __u64 media_write_latency; // nanoseconds + __u64 cache_read_latency; // nanoseconds + __u64 cache_write_latency; // nanoseconds +}; + /* ioctl numbers */ #define SCM_MAGIC 0x5C /* SCM devices */ @@ -57,5 +73,6 @@ struct scm_ioctl_controller_dump_data { #define SCM_IOCTL_CONTROLLER_DUMP _IO(SCM_MAGIC, 0x02) #define SCM_IOCTL_CONTROLLER_DUMP_DATA _IOWR(SCM_MAGIC, 0x03, struct scm_ioctl_controller_dump_data) #define SCM_IOCTL_CONTROLLER_DUMP_COMPLETE _IO(SCM_MAGIC, 0x04) +#define SCM_IOCTL_CONTROLLER_STATS _IO(SCM_MAGIC, 0x05) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Tue Dec 3 03:46:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96FE66C1 for ; Tue, 3 Dec 2019 03:49:03 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7FD9E206E1 for ; Tue, 3 Dec 2019 03:49:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7FD9E206E1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 3CE8110113673; Mon, 2 Dec 2019 19:52:06 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 51E3210113664 for ; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33klKt078530 for ; Mon, 2 Dec 2019 22:48:31 -0500 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6rr0pd3-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:28 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mIal46202948 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:18 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 91B76A404D; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F0202A4059; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 5F1CDA03E7; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 19/27] nvdimm/ocxl: Forward events to userspace Date: Tue, 3 Dec 2019 14:46:47 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0020-0000-0000-00000392CC5B X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0021-0000-0000-000021E9EAAA Message-Id: <20191203034655.51561-20-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=3 adultscore=0 spamscore=0 priorityscore=1501 mlxscore=0 bulkscore=5 mlxlogscore=896 lowpriorityscore=5 impostorscore=0 phishscore=0 clxscore=1015 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: S6GVFEF3RPLGLE65IHI6QR2BZM2AKY4K X-Message-ID-Hash: S6GVFEF3RPLGLE65IHI6QR2BZM2AKY4K X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva Some of the interrupts that the card generates are better handled by the userspace daemon, in particular: Controller Hardware/Firmware Fatal Controller Dump Available Error Log available This patch allows a userspace application to register an eventfd with the driver via SCM_IOCTL_EVENTFD to receive notifications of these interrupts. Userspace can then identify what events have occurred by calling SCM_IOCTL_EVENT_CHECK and checking against the SCM_IOCTL_EVENT_FOO masks. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 216 +++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.h | 5 + include/uapi/nvdimm/ocxl-scm.h | 16 +++ 3 files changed, 237 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 54a2bac1cab7..854787950334 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -382,11 +383,22 @@ static void free_scm(struct scm_data *scm_data) { int rc; + // Disable doorbells + (void)ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_CHIEC, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_CHI_ALL); + if (scm_data->nvdimm_bus) nvdimm_bus_unregister(scm_data->nvdimm_bus); free_scm_minor(scm_data); + if (scm_data->irq_addr[1]) + iounmap(scm_data->irq_addr[1]); + + if (scm_data->irq_addr[0]) + iounmap(scm_data->irq_addr[0]); + if (scm_data->cdev.owner) cdev_del(&scm_data->cdev); @@ -491,6 +503,11 @@ static int scm_file_release(struct inode *inode, struct file *file) { struct scm_data *scm_data = file->private_data; + if (scm_data->ev_ctx) { + eventfd_ctx_put(scm_data->ev_ctx); + scm_data->ev_ctx = NULL; + } + scm_put(scm_data); return 0; } @@ -986,6 +1003,51 @@ static int scm_ioctl_controller_stats(struct scm_data *scm_data, return rc; } +static int scm_ioctl_eventfd(struct scm_data *scm_data, + struct scm_ioctl_eventfd __user *uarg) +{ + struct scm_ioctl_eventfd args; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + if (scm_data->ev_ctx) + return -EINVAL; + + scm_data->ev_ctx = eventfd_ctx_fdget(args.eventfd); + if (!scm_data->ev_ctx) + return -EFAULT; + + return 0; +} + +static int scm_ioctl_event_check(struct scm_data *scm_data, u64 __user *uarg) +{ + u64 val = 0; + int rc; + u64 chi = 0; + + rc = scm_chi(scm_data, &chi); + if (rc < 0) + return rc; + + if (chi & GLOBAL_MMIO_CHI_ELA) + val |= SCM_IOCTL_EVENT_ERROR_LOG_AVAILABLE; + + if (chi & GLOBAL_MMIO_CHI_CDA) + val |= SCM_IOCTL_EVENT_CONTROLLER_DUMP_AVAILABLE; + + if (chi & GLOBAL_MMIO_CHI_CFFS) + val |= SCM_IOCTL_EVENT_FIRMWARE_FATAL; + + if (chi & GLOBAL_MMIO_CHI_CHFS) + val |= SCM_IOCTL_EVENT_HARDWARE_FATAL; + + rc = copy_to_user((u64 __user *) uarg, &val, sizeof(val)); + + return rc; +} + static long scm_file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { @@ -1015,6 +1077,15 @@ static long scm_file_ioctl(struct file *file, unsigned int cmd, rc = scm_ioctl_controller_stats(scm_data, (struct scm_ioctl_controller_stats __user *)args); break; + + case SCM_IOCTL_EVENTFD: + rc = scm_ioctl_eventfd(scm_data, + (struct scm_ioctl_eventfd __user *)args); + break; + + case SCM_IOCTL_EVENT_CHECK: + rc = scm_ioctl_event_check(scm_data, (u64 __user *)args); + break; } return rc; @@ -1156,6 +1227,146 @@ static void scm_dump_error_log(struct scm_data *scm_data) kfree(buf); } +static irqreturn_t scm_imn0_handler(void *private) +{ + struct scm_data *scm_data = private; + u64 chi = 0; + + (void)scm_chi(scm_data, &chi); + + if (chi & GLOBAL_MMIO_CHI_ELA) { + dev_warn(&scm_data->dev, "Error log is available\n"); + + if (scm_data->ev_ctx) + eventfd_signal(scm_data->ev_ctx, 1); + } + + if (chi & GLOBAL_MMIO_CHI_CDA) { + dev_warn(&scm_data->dev, "Controller dump is available\n"); + + if (scm_data->ev_ctx) + eventfd_signal(scm_data->ev_ctx, 1); + } + + + return IRQ_HANDLED; +} + +static irqreturn_t scm_imn1_handler(void *private) +{ + struct scm_data *scm_data = private; + u64 chi = 0; + + (void)scm_chi(scm_data, &chi); + + if (chi & (GLOBAL_MMIO_CHI_CFFS | GLOBAL_MMIO_CHI_CHFS)) { + dev_err(&scm_data->dev, + "Controller status is fatal, chi=0x%llx, going offline\n", chi); + + if (scm_data->nvdimm_bus) { + nvdimm_bus_unregister(scm_data->nvdimm_bus); + scm_data->nvdimm_bus = NULL; + } + + if (scm_data->ev_ctx) + eventfd_signal(scm_data->ev_ctx, 1); + } + + return IRQ_HANDLED; +} + + +/** + * scm_setup_irq() - Set up the IRQs for the SCM device + * @scm_data: the SCM metadata + * Return: 0 on success, negative on failure + */ +static int scm_setup_irq(struct scm_data *scm_data) +{ + int rc; + u64 irq_addr; + + rc = ocxl_afu_irq_alloc(scm_data->ocxl_context, &scm_data->irq_id[0]); + if (rc) + return rc; + + rc = ocxl_irq_set_handler(scm_data->ocxl_context, scm_data->irq_id[0], + scm_imn0_handler, NULL, scm_data); + + irq_addr = ocxl_afu_irq_get_addr(scm_data->ocxl_context, scm_data->irq_id[0]); + if (!irq_addr) + return -EINVAL; + + scm_data->irq_addr[0] = ioremap(irq_addr, PAGE_SIZE); + if (!scm_data->irq_addr[0]) + return -EINVAL; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, GLOBAL_MMIO_IMA0_OHP, + OCXL_LITTLE_ENDIAN, + (u64)scm_data->irq_addr[0]); + if (rc) + goto out_irq0; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, GLOBAL_MMIO_IMA0_CFP, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out_irq0; + + rc = ocxl_afu_irq_alloc(scm_data->ocxl_context, &scm_data->irq_id[1]); + if (rc) + goto out_irq0; + + + rc = ocxl_irq_set_handler(scm_data->ocxl_context, scm_data->irq_id[1], + scm_imn1_handler, NULL, scm_data); + if (rc) + goto out_irq0; + + irq_addr = ocxl_afu_irq_get_addr(scm_data->ocxl_context, scm_data->irq_id[1]); + if (!irq_addr) { + rc = -EFAULT; + goto out_irq0; + } + + scm_data->irq_addr[1] = ioremap(irq_addr, PAGE_SIZE); + if (!scm_data->irq_addr[1]) { + rc = -EINVAL; + goto out_irq0; + } + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, GLOBAL_MMIO_IMA1_OHP, + OCXL_LITTLE_ENDIAN, + (u64)scm_data->irq_addr[1]); + if (rc) + goto out_irq1; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, GLOBAL_MMIO_IMA1_CFP, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out_irq1; + + // Enable doorbells + rc = ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_CHIE, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_CHI_ELA | GLOBAL_MMIO_CHI_CDA | + GLOBAL_MMIO_CHI_CFFS | GLOBAL_MMIO_CHI_CHFS | + GLOBAL_MMIO_CHI_NSCRA); + if (rc) + goto out_irq1; + + return 0; + +out_irq1: + iounmap(scm_data->irq_addr[1]); + scm_data->irq_addr[1] = NULL; + +out_irq0: + iounmap(scm_data->irq_addr[0]); + scm_data->irq_addr[0] = NULL; + + return rc; +} + /** * scm_probe_function_0 - Set up function 0 for an OpenCAPI Storage Class Memory device * This is important as it enables templates higher than 0 across all other functions, @@ -1261,6 +1472,11 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (scm_setup_irq(scm_data)) { + dev_err(&pdev->dev, "Could not set up OCXL IRQs for SCM\n"); + goto err; + } + if (scm_setup_command_metadata(scm_data)) { dev_err(&pdev->dev, "Could not read OCXL command matada\n"); goto err; diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 9bf8fcf30ea6..693fd59f8bde 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -104,6 +104,10 @@ struct scm_data { struct pci_dev *pdev; struct cdev cdev; struct ocxl_fn *ocxl_fn; +#define SCM_IRQ_COUNT 2 + int irq_id[SCM_IRQ_COUNT]; + struct dev_pagemap irq_pgmap[SCM_IRQ_COUNT]; + void *irq_addr[SCM_IRQ_COUNT]; struct nd_interleave_set nd_set; struct nvdimm_bus_descriptor bus_desc; struct nvdimm_bus *nvdimm_bus; @@ -114,6 +118,7 @@ struct scm_data { struct command_metadata ns_command; struct resource scm_res; struct nd_region *nd_region; + struct eventfd_ctx *ev_ctx; char fw_version[8+1]; u32 timeouts[ADMIN_COMMAND_MAX+1]; diff --git a/include/uapi/nvdimm/ocxl-scm.h b/include/uapi/nvdimm/ocxl-scm.h index 0a5de46c5acd..e86ffb02d31f 100644 --- a/include/uapi/nvdimm/ocxl-scm.h +++ b/include/uapi/nvdimm/ocxl-scm.h @@ -66,6 +66,20 @@ struct scm_ioctl_controller_stats { __u64 cache_write_latency; // nanoseconds }; +struct scm_ioctl_eventfd { + __s32 eventfd; + __u32 reserved; +}; + +#ifndef BIT_ULL +#define BIT_ULL(nr) (1ULL << (nr)) +#endif + +#define SCM_IOCTL_EVENT_CONTROLLER_DUMP_AVAILABLE BIT_ULL(0) +#define SCM_IOCTL_EVENT_ERROR_LOG_AVAILABLE BIT_ULL(1) +#define SCM_IOCTL_EVENT_HARDWARE_FATAL BIT_ULL(2) +#define SCM_IOCTL_EVENT_FIRMWARE_FATAL BIT_ULL(3) + /* ioctl numbers */ #define SCM_MAGIC 0x5C /* SCM devices */ @@ -74,5 +88,7 @@ struct scm_ioctl_controller_stats { #define SCM_IOCTL_CONTROLLER_DUMP_DATA _IOWR(SCM_MAGIC, 0x03, struct scm_ioctl_controller_dump_data) #define SCM_IOCTL_CONTROLLER_DUMP_COMPLETE _IO(SCM_MAGIC, 0x04) #define SCM_IOCTL_CONTROLLER_STATS _IO(SCM_MAGIC, 0x05) +#define SCM_IOCTL_EVENTFD _IOW(SCM_MAGIC, 0x06, struct scm_ioctl_eventfd) +#define SCM_IOCTL_EVENT_CHECK _IOR(SCM_MAGIC, 0x07, __u64) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Tue Dec 3 03:46:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8FE86C1 for ; Tue, 3 Dec 2019 03:48:59 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B16B220726 for ; Tue, 3 Dec 2019 03:48:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B16B220726 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id DCAFA1011369D; Mon, 2 Dec 2019 19:52:04 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 19CF710113666 for ; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmWF093431 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 2wm6g99bqn-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mIbm54526190 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:18 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8DACA52050; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id E53B052052; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 68547A03E8; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 20/27] nvdimm/ocxl: Add an IOCTL to request controller health & perf data Date: Tue, 3 Dec 2019 14:46:48 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-4275-0000-0000-0000038A31E0 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-4276-0000-0000-0000389DCDD9 Message-Id: <20191203034655.51561-21-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1015 lowpriorityscore=0 bulkscore=0 phishscore=0 priorityscore=1501 impostorscore=0 suspectscore=1 spamscore=0 mlxscore=0 adultscore=0 mlxlogscore=746 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: Y2RSPFOLBARCVBG7NLIBJ3PNYEA5TWTE X-Message-ID-Hash: Y2RSPFOLBARCVBG7NLIBJ3PNYEA5TWTE X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva When health & performance data is requested from the controller, it responds with an error log containing the requested information. This patch allows the request to me issued via an IOCTL. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 16 ++++++++++++++++ include/uapi/nvdimm/ocxl-scm.h | 1 + 2 files changed, 17 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 854787950334..d482b3213a02 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -1048,6 +1048,18 @@ static int scm_ioctl_event_check(struct scm_data *scm_data, u64 __user *uarg) return rc; } +/** + * scm_req_controller_health_perf() - Request controller health & performance data + * @scm_data: the SCM metadata + * Return: 0 on success, negative on failure + */ +int scm_req_controller_health_perf(struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_REQ_HEALTH_PERF); +} + static long scm_file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { @@ -1086,6 +1098,10 @@ static long scm_file_ioctl(struct file *file, unsigned int cmd, case SCM_IOCTL_EVENT_CHECK: rc = scm_ioctl_event_check(scm_data, (u64 __user *)args); break; + + case SCM_IOCTL_REQUEST_HEALTH: + rc = scm_req_controller_health_perf(scm_data); + break; } return rc; diff --git a/include/uapi/nvdimm/ocxl-scm.h b/include/uapi/nvdimm/ocxl-scm.h index e86ffb02d31f..55a7ad59d614 100644 --- a/include/uapi/nvdimm/ocxl-scm.h +++ b/include/uapi/nvdimm/ocxl-scm.h @@ -90,5 +90,6 @@ struct scm_ioctl_eventfd { #define SCM_IOCTL_CONTROLLER_STATS _IO(SCM_MAGIC, 0x05) #define SCM_IOCTL_EVENTFD _IOW(SCM_MAGIC, 0x06, struct scm_ioctl_eventfd) #define SCM_IOCTL_EVENT_CHECK _IOR(SCM_MAGIC, 0x07, __u64) +#define SCM_IOCTL_REQUEST_HEALTH _IO(SCM_MAGIC, 0x08) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Tue Dec 3 03:46:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5420F6C1 for ; Tue, 3 Dec 2019 03:48:52 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3C9E720733 for ; Tue, 3 Dec 2019 03:48:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C9E720733 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 42E9810113696; Mon, 2 Dec 2019 19:52:00 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id A46E210113666 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kk7K006770 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6cy9se7-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:20 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mJmA39256506 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 24ABEAE051; Tue, 3 Dec 2019 03:48:19 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 842B4AE055; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 6FC63A03E9; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 21/27] nvdimm/ocxl: Support firmware update via sysfs Date: Tue, 3 Dec 2019 14:46:49 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0028-0000-0000-000003C3CC75 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0029-0000-0000-00002486E435 Message-Id: <20191203034655.51561-22-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 priorityscore=1501 suspectscore=1 bulkscore=0 phishscore=0 mlxlogscore=999 adultscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: R7GSYEVWTFR2I6HHBDPBX5QUVEQNKGZL X-Message-ID-Hash: R7GSYEVWTFR2I6HHBDPBX5QUVEQNKGZL X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch allows the firmware of an OpenCAPI SCM card to be update by writing a firmware file to a file in sysfs. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/Makefile | 2 +- drivers/nvdimm/ocxl/scm.c | 5 + drivers/nvdimm/ocxl/scm_internal.c | 25 +++++ drivers/nvdimm/ocxl/scm_internal.h | 14 +++ drivers/nvdimm/ocxl/scm_sysfs.c | 163 +++++++++++++++++++++++++++++ 5 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 drivers/nvdimm/ocxl/scm_sysfs.c diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile index 9b6e31f0eb3e..b172cef39de4 100644 --- a/drivers/nvdimm/ocxl/Makefile +++ b/drivers/nvdimm/ocxl/Makefile @@ -4,4 +4,4 @@ ccflags-$(CONFIG_PPC_WERROR) += -Werror obj-$(CONFIG_OCXL_SCM) += ocxlscm.o -ocxlscm-y := scm.o scm_internal.o +ocxlscm-y := scm.o scm_internal.o scm_sysfs.o diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index d482b3213a02..8a30c887b5ed 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -1503,6 +1503,11 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (scm_sysfs_add(scm_data)) { + dev_err(&pdev->dev, "Could not create SCM sysfs entries\n"); + goto err; + } + elapsed = 0; timeout = scm_data->readiness_timeout + scm_data->memory_available_timeout; while (!scm_is_usable(scm_data)) { diff --git a/drivers/nvdimm/ocxl/scm_internal.c b/drivers/nvdimm/ocxl/scm_internal.c index c405f1d8afb8..8fc849610eaa 100644 --- a/drivers/nvdimm/ocxl/scm_internal.c +++ b/drivers/nvdimm/ocxl/scm_internal.c @@ -210,3 +210,28 @@ void scm_warn_status(const struct scm_data *scm_data, const char *message, dev_warn(&scm_data->dev, "%s: %s (%x)\n", message, text, status); } + +void scm_warn_status_fw_update(const struct scm_data *scm_data, + const char *message, u8 status) +{ + const char *text; + + switch (status) { + case STATUS_FW_UPDATE_BLOCKED: + text = "Firmware update is blocked, please try again later"; + break; + + case STATUS_FW_ARG_INVALID: + text = "Internal error in SCM firmware update mechanism"; + break; + + case STATUS_FW_INVALID: + text = "Firmware content is invalid, please verify firmware update file"; + break; + + default: + return scm_warn_status(scm_data, message, status); + } + + dev_warn(&scm_data->dev, "%s: %s (%x)\n", message, text, status); +} diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 693fd59f8bde..af19813a7f75 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -137,6 +137,12 @@ struct scm_data { */ }; +/** + * Create sysfs entries for an SCM device + * scm_data: The SCM metadata + */ +int scm_sysfs_add(struct scm_data *scm_data); + /** * scm_chi() - Get the value of the CHI register * @scm_data: The SCM metadata @@ -230,3 +236,11 @@ int scm_ns_response_handled(const struct scm_data *scm_data); void scm_warn_status(const struct scm_data *scm_data, const char *message, u8 status); +/** + * scm_warn_status_fw_update() - Emit a kernel warning showing a command status. + * @scm_data: a pointer to the SCM device data + * @message: A message to accompany the warning + * @status: The command status + */ +void scm_warn_status_fw_update(const struct scm_data *scm_data, + const char *message, u8 status); diff --git a/drivers/nvdimm/ocxl/scm_sysfs.c b/drivers/nvdimm/ocxl/scm_sysfs.c new file mode 100644 index 000000000000..a04e8a74d0c5 --- /dev/null +++ b/drivers/nvdimm/ocxl/scm_sysfs.c @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2018 IBM Corp. + +#include +#include +#include +#include +#include "scm_internal.h" + +static ssize_t fw_version_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + struct scm_data *scm_data = container_of(device, struct scm_data, dev); + + return scnprintf(buf, PAGE_SIZE, "%s\n", scm_data->fw_version); +} + +#define SCM_FWUPDATE_BLOCK_SIZE 32768 + +/** + * scm_update_firmware() - Write a 32kB block of data to firmware + * The block may be less than 32kB if it is the last one + * + * scm_data the SCM device metadata + * offset: the offset of the start of the block + * buf: the block data + * size: the size of the block + */ +static ssize_t scm_update_firmware(struct scm_data *scm_data, size_t offset, + const char *buf, size_t size) +{ + int rc; + size_t i; + u64 val; + + if (size > SCM_FWUPDATE_BLOCK_SIZE) + return -EINVAL; + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_FW_UPDATE); + if (rc) + return rc; + + val = (((u64)offset) << 32) | size; + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + 8, + OCXL_LITTLE_ENDIAN, val); + if (rc) + return rc; + + for (i = 0; i < size; i += 8) { + val = *(u64 *)(buf + i); + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, val); + if (rc) + return rc; + } + + rc = scm_admin_command_execute(scm_data); + if (rc) + return rc; + + rc = scm_admin_command_complete_timeout(scm_data, + ADMIN_COMMAND_FW_UPDATE); + if (rc < 0) { + dev_err(&scm_data->dev, "Firmware update timeout\n"); + return rc; + } + + rc = scm_admin_response(scm_data); + if (rc < 0) + return rc; + if (rc != STATUS_SUCCESS) { + scm_warn_status_fw_update(scm_data, "FW Update", rc); + return rc; + } + + return 0; +} + +/* + * Parse out a firmware filename from sysfs, retrieve it's contents and write it + * to the SCM device firmware storage + */ +static ssize_t fw_update_filename_store(struct device *device, + struct device_attribute *attr, + const char *buf, size_t size) +{ + char path[NAME_MAX+1]; + const char *end; + const struct firmware *firmware = NULL; + size_t offset; + int rc; + struct scm_data *scm_data = container_of(device, struct scm_data, dev); + + if (!capable(CAP_SYS_ADMIN)) + return -EACCES; + + end = strnchr(buf, size, '\n'); + if (end == NULL) + end = buf + strnlen(buf, size); + + if ((end - buf) > NAME_MAX) { + dev_err(device, "Firmware filename '%-.*s' too long\n", + (int)(end - buf), buf); + return -EIO; + } + + memcpy(path, buf, end - buf); + path[end - buf] = '\0'; + + if (request_firmware(&firmware, path, device)) { + dev_err(device, "Firmware file %s not found\n", path); + return -EIO; + } + + if (firmware->size % 8) { + release_firmware(firmware); + dev_err(device, "Firmware '%s' should be a multiple of 8 bytes", path); + return -EINVAL; + } + + mutex_lock(&scm_data->admin_command.lock); + + for (offset = 0; offset < firmware->size; offset += SCM_FWUPDATE_BLOCK_SIZE) { + size_t remainder = firmware->size - offset; + size_t block_size; + + block_size = (remainder > SCM_FWUPDATE_BLOCK_SIZE) ? + SCM_FWUPDATE_BLOCK_SIZE : remainder; + rc = scm_update_firmware(scm_data, offset, + firmware->data + offset, block_size); + if (rc) { + mutex_unlock(&scm_data->admin_command.lock); + return -EFAULT; + } + } + + mutex_unlock(&scm_data->admin_command.lock); + + return size; +} + +static struct device_attribute scm_attrs[] = { + __ATTR_RO(fw_version), + __ATTR_WO(fw_update_filename), +}; + +int scm_sysfs_add(struct scm_data *scm_data) +{ + int i, rc; + + for (i = 0; i < ARRAY_SIZE(scm_attrs); i++) { + rc = device_create_file(&scm_data->dev, &scm_attrs[i]); + if (rc) { + for (; --i >= 0;) + device_remove_file(&scm_data->dev, &scm_attrs[i]); + + return rc; + } + } + return 0; +} From patchwork Tue Dec 3 03:46:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270321 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 04A496C1 for ; Tue, 3 Dec 2019 03:49:01 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E1DAF20803 for ; Tue, 3 Dec 2019 03:49:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E1DAF20803 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id F2FD510097F09; Mon, 2 Dec 2019 19:52:04 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2E85210113669 for ; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33knSw084158 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6xb0p1h-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:28 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:20 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mJrE24707224 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 33C694C040; Tue, 3 Dec 2019 03:48:19 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 857924C04A; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 83126A03EA; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 22/27] nvdimm/ocxl: Implement the heartbeat command Date: Tue, 3 Dec 2019 14:46:50 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0008-0000-0000-0000033C0FE0 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0009-0000-0000-00004A5B289D Message-Id: <20191203034655.51561-23-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 suspectscore=1 lowpriorityscore=0 phishscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 adultscore=0 mlxlogscore=905 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: QZZ6PGXKLB2W2A2QC4SGJTEIKJVNN7L4 X-Message-ID-Hash: QZZ6PGXKLB2W2A2QC4SGJTEIKJVNN7L4 X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva The heartbeat admin command is a simple admin command that exercises the communication mechanisms within the controller. This patch issues a heartbeat command to the card during init to ensure we can communicate with the card's crontroller. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 43 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 8a30c887b5ed..e8b34262f397 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -353,6 +353,44 @@ static bool scm_is_usable(const struct scm_data *scm_data) return true; } +/** + * scm_heartbeat() - Issue a heartbeat command to the controller + * @scm_data: a pointer to the SCM device data + * Return: 0 if the controller responded correctly, negative on error + */ +static int scm_heartbeat(struct scm_data *scm_data) +{ + int rc; + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_HEARTBEAT); + if (rc) + goto out; + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_complete_timeout(scm_data, ADMIN_COMMAND_HEARTBEAT); + if (rc < 0) { + dev_err(&scm_data->dev, "Heartbeat timeout\n"); + goto out; + } + + rc = scm_admin_response(scm_data); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) + scm_warn_status(scm_data, "Unexpected status from heartbeat", rc); + + rc = scm_admin_response_handled(scm_data); + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; +} + /** * allocate_scm_minor() - Allocate a minor number to use for an SCM device * @scm_data: The SCM device to associate the minor with @@ -1508,6 +1546,11 @@ static int scm_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (scm_heartbeat(scm_data)) { + dev_err(&pdev->dev, "SCM Heartbeat failed\n"); + goto err; + } + elapsed = 0; timeout = scm_data->readiness_timeout + scm_data->memory_available_timeout; while (!scm_is_usable(scm_data)) { From patchwork Tue Dec 3 03:46:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16443109A for ; Tue, 3 Dec 2019 03:49:06 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F384D20726 for ; Tue, 3 Dec 2019 03:49:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F384D20726 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 796E510113667; Mon, 2 Dec 2019 19:52:06 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0b-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 75EF610113688 for ; Mon, 2 Dec 2019 19:51:58 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kc7C087993 for ; Mon, 2 Dec 2019 22:48:35 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wkm47b7vv-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:34 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:26 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mIc063111208 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:18 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 97C144C04E; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0002A4C044; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:17 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 8DC01A03EB; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 23/27] nvdimm/ocxl: Add debug IOCTLs Date: Tue, 3 Dec 2019 14:46:51 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-4275-0000-0000-0000038A31DF X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-4276-0000-0000-0000389DCDDA Message-Id: <20191203034655.51561-24-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 malwarescore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 impostorscore=0 adultscore=0 suspectscore=1 mlxlogscore=999 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: GJJJJFYI357YFAAC6HARQY54MFN2MJXN X-Message-ID-Hash: GJJJJFYI357YFAAC6HARQY54MFN2MJXN X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva These IOCTLs provide low level access to the card to aid in debugging controller/FPGA firmware. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/Kconfig | 6 + drivers/nvdimm/ocxl/scm.c | 249 +++++++++++++++++++++++++++++++++ include/uapi/nvdimm/ocxl-scm.h | 32 +++++ 3 files changed, 287 insertions(+) diff --git a/drivers/nvdimm/ocxl/Kconfig b/drivers/nvdimm/ocxl/Kconfig index 24099b300f5e..1df030cdd958 100644 --- a/drivers/nvdimm/ocxl/Kconfig +++ b/drivers/nvdimm/ocxl/Kconfig @@ -12,4 +12,10 @@ config OCXL_SCM Select N if unsure. +config OCXL_SCM_DEBUG + bool "OpenCAPI Storage Class Memory debugging" + depends on OCXL_SCM + help + Enables low level IOCTLs for OpenCAPI SCM firmware development + endif diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index e8b34262f397..a81eb5916eb3 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -1098,6 +1098,235 @@ int scm_req_controller_health_perf(struct scm_data *scm_data) GLOBAL_MMIO_HCI_REQ_HEALTH_PERF); } +#ifdef CONFIG_OCXL_SCM_DEBUG +/** + * scm_enable_fwdebug() - Enable FW debug on the controller + * @scm_data: a pointer to the SCM device data + * Return: 0 on success, negative on failure + */ +static int scm_enable_fwdebug(const struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_FW_DEBUG); +} + +/** + * scm_disable_fwdebug() - Disable FW debug on the controller + * @scm_data: a pointer to the SCM device data + * Return: 0 on success, negative on failure + */ +static int scm_disable_fwdebug(const struct scm_data *scm_data) +{ + return ocxl_global_mmio_set64(scm_data->ocxl_afu, GLOBAL_MMIO_HCIC, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_FW_DEBUG); +} + +static int scm_ioctl_fwdebug(struct scm_data *scm_data, + struct scm_ioctl_fwdebug __user *uarg) +{ + struct scm_ioctl_fwdebug args; + u64 val; + int i; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + // Buffer size must be a multiple of 8 + if ((args.buf_size & 0x07)) + return -EINVAL; + + if (args.buf_size > scm_data->admin_command.data_size) + return -EINVAL; + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_enable_fwdebug(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_FW_DEBUG); + if (rc) + goto out; + + // Write DebugAction & FunctionCode + val = ((u64)args.debug_action << 56) | ((u64)args.function_code << 40); + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + 0x08, + OCXL_LITTLE_ENDIAN, val); + if (rc) + goto out; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + 0x10, + OCXL_LITTLE_ENDIAN, args.debug_parameter_1); + if (rc) + goto out; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + 0x18, + OCXL_LITTLE_ENDIAN, args.debug_parameter_2); + if (rc) + goto out; + + for (i = 0x20; i < 0x38; i += 0x08) + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.request_offset + i, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out; + + + // Populate admin command buffer + if (args.buf_size) { + for (i = 0; i < args.buf_size; i += sizeof(u64)) { + u64 val; + + if (copy_from_user(&val, &args.buf[i], sizeof(u64))) + return -EFAULT; + + rc = ocxl_global_mmio_write64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, val); + if (rc) + goto out; + } + } + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_complete_timeout(scm_data, + scm_data->timeouts[ADMIN_COMMAND_FW_DEBUG]); + if (rc < 0) + goto out; + + rc = scm_admin_response(scm_data); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + scm_warn_status(scm_data, "Unexpected status from FW Debug", rc); + goto out; + } + + if (args.buf_size) { + for (i = 0; i < args.buf_size; i += sizeof(u64)) { + u64 val; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, &val); + if (rc) + goto out; + + if (copy_to_user(&args.buf[i], &val, sizeof(u64))) { + rc = -EFAULT; + goto out; + } + } + } + + rc = scm_admin_response_handled(scm_data); + if (rc) + goto out; + + rc = scm_disable_fwdebug(scm_data); + if (rc) + goto out; + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; +} + +static int scm_ioctl_shutdown(struct scm_data *scm_data) +{ + int rc; + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_SHUTDOWN); + if (rc) + goto out; + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_complete_timeout(scm_data, ADMIN_COMMAND_SHUTDOWN); + if (rc < 0) { + dev_warn(&scm_data->dev, "Shutdown timed out\n"); + goto out; + } + + rc = 0; + goto out; + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; +} + +static int scm_ioctl_mmio_write(struct scm_data *scm_data, + struct scm_ioctl_mmio __user *uarg) +{ + struct scm_ioctl_mmio args; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + return ocxl_global_mmio_write64(scm_data->ocxl_afu, args.address, + OCXL_LITTLE_ENDIAN, args.val); +} + +static int scm_ioctl_mmio_read(struct scm_data *scm_data, + struct scm_ioctl_mmio __user *uarg) +{ + struct scm_ioctl_mmio args; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, args.address, + OCXL_LITTLE_ENDIAN, &args.val); + if (rc) + return rc; + + if (copy_to_user(uarg, &args, sizeof(args))) + return -EFAULT; + + return 0; +} +#else /* CONFIG_OCXL_SCM_DEBUG */ +static int scm_ioctl_fwdebug(struct scm_data *scm_data, + struct scm_ioctl_fwdebug __user *uarg) +{ + return -EPERM; +} + +static int scm_ioctl_shutdown(struct scm_data *scm_data) +{ + return -EPERM; +} + +static int scm_ioctl_mmio_write(struct scm_data *scm_data, + struct scm_ioctl_mmio __user *uarg) +{ + return -EPERM; +} + +static int scm_ioctl_mmio_read(struct scm_data *scm_data, + struct scm_ioctl_mmio __user *uarg) +{ + return -EPERM; +} +#endif /* CONFIG_OCXL_SCM_DEBUG */ + static long scm_file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { @@ -1140,6 +1369,26 @@ static long scm_file_ioctl(struct file *file, unsigned int cmd, case SCM_IOCTL_REQUEST_HEALTH: rc = scm_req_controller_health_perf(scm_data); break; + + case SCM_IOCTL_FWDEBUG: + rc = scm_ioctl_fwdebug(scm_data, + (struct scm_ioctl_fwdebug __user *)args); + break; + + case SCM_IOCTL_SHUTDOWN: + rc = scm_ioctl_shutdown(scm_data); + break; + + case SCM_IOCTL_MMIO_WRITE: + rc = scm_ioctl_mmio_write(scm_data, + (struct scm_ioctl_mmio __user *)args); + break; + + case SCM_IOCTL_MMIO_READ: + rc = scm_ioctl_mmio_read(scm_data, + (struct scm_ioctl_mmio __user *)args); + break; + } return rc; diff --git a/include/uapi/nvdimm/ocxl-scm.h b/include/uapi/nvdimm/ocxl-scm.h index 55a7ad59d614..6e0f25c5f9f3 100644 --- a/include/uapi/nvdimm/ocxl-scm.h +++ b/include/uapi/nvdimm/ocxl-scm.h @@ -6,6 +6,28 @@ #include #include +enum scm_fwdebug_action { + SCM_FWDEBUG_READ_CONTROLLER_MEMORY = 0x01, + SCM_FWDEBUG_WRITE_CONTROLLER_MEMORY = 0x02, + SCM_FWDEBUG_ENABLE_FUNCTION = 0x03, + SCM_FWDEBUG_DISABLE_FUNCTION = 0x04, + SCM_FWDEBUG_GET_PEL = 0x05, // Retrieve Persistent Error Log +}; + +struct scm_ioctl_buffer_info { + __u32 admin_command_buffer_size; // out + __u32 near_storage_buffer_size; // out +}; + +struct scm_ioctl_fwdebug { // All args are inputs + enum scm_fwdebug_action debug_action; + __u16 function_code; + __u16 buf_size; // Size of optional data buffer + __u64 debug_parameter_1; + __u64 debug_parameter_2; + __u8 *buf; // Pointer to optional in/out data buffer +}; + #define SCM_ERROR_LOG_ACTION_RESET (1 << (32-32)) #define SCM_ERROR_LOG_ACTION_CHKFW (1 << (53-32)) #define SCM_ERROR_LOG_ACTION_REPLACE (1 << (54-32)) @@ -66,6 +88,11 @@ struct scm_ioctl_controller_stats { __u64 cache_write_latency; // nanoseconds }; +struct scm_ioctl_mmio { + __u64 address; // Offset in global MMIO space + __u64 val; // value to write/was read +}; + struct scm_ioctl_eventfd { __s32 eventfd; __u32 reserved; @@ -92,4 +119,9 @@ struct scm_ioctl_eventfd { #define SCM_IOCTL_EVENT_CHECK _IOR(SCM_MAGIC, 0x07, __u64) #define SCM_IOCTL_REQUEST_HEALTH _IO(SCM_MAGIC, 0x08) +#define SCM_IOCTL_FWDEBUG _IOWR(SCM_MAGIC, 0xf0, struct scm_ioctl_fwdebug) +#define SCM_IOCTL_MMIO_WRITE _IOW(SCM_MAGIC, 0xf1, struct scm_ioctl_mmio) +#define SCM_IOCTL_MMIO_READ _IOWR(SCM_MAGIC, 0xf2, struct scm_ioctl_mmio) +#define SCM_IOCTL_SHUTDOWN _IO(SCM_MAGIC, 0xf3) + #endif /* _UAPI_OCXL_SCM_H */ From patchwork Tue Dec 3 03:46:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF684109A for ; Tue, 3 Dec 2019 03:49:04 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B87FD20726 for ; Tue, 3 Dec 2019 03:49:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B87FD20726 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 5423910097F0F; Mon, 2 Dec 2019 19:52:06 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 4264410113673 for ; Mon, 2 Dec 2019 19:51:55 -0800 (PST) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kkw9021117 for ; Mon, 2 Dec 2019 22:48:32 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wnehxj6um-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:32 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:28 -0000 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:20 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mJeB45023540 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2DBB64204F; Tue, 3 Dec 2019 03:48:19 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 873FC42041; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 96F42A03EE; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 24/27] nvdimm/ocxl: Implement Overwrite Date: Tue, 3 Dec 2019 14:46:52 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-4275-0000-0000-0000038A31E1 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-4276-0000-0000-0000389DCDDC Message-Id: <20191203034655.51561-25-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=1 impostorscore=0 mlxlogscore=999 priorityscore=1501 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: E6CFGWA4QFSIB4RPU4BM6D5AGZ47NK2A X-Message-ID-Hash: E6CFGWA4QFSIB4RPU4BM6D5AGZ47NK2A X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva The near storage command 'Secure Erase' overwrites all data on the media. This patch hooks it up to the security function 'overwrite'. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 164 ++++++++++++++++++++++++++++- drivers/nvdimm/ocxl/scm_internal.c | 1 + drivers/nvdimm/ocxl/scm_internal.h | 17 +++ 3 files changed, 180 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index a81eb5916eb3..8deb7862793c 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -169,6 +169,86 @@ static int scm_reserve_metadata(struct scm_data *scm_data, return 0; } +/** + * scm_overwrite() - Overwrite all data on the card + * @scm_data: The SCM device data + * Return: 0 on success + */ +int scm_overwrite(struct scm_data *scm_data) +{ + int rc; + + mutex_lock(&scm_data->ns_command.lock); + + rc = scm_ns_command_request(scm_data, NS_COMMAND_SECURE_ERASE); + if (rc) + goto out; + + rc = scm_ns_command_execute(scm_data); + if (rc) + goto out; + + scm_data->overwrite_state = SCM_OVERWRITE_BUSY; + + return 0; + +out: + mutex_unlock(&scm_data->ns_command.lock); + return rc; +} + +/** + * scm_secop_overwrite() - Overwrite all data on the card + * @nvdimm: The nvdimm representation of the SCM device to start the overwrite on + * @key_data: Unused (no security key implementation) + * Return: 0 on success + */ +static int scm_secop_overwrite(struct nvdimm *nvdimm, + const struct nvdimm_key_data *key_data) +{ + struct scm_data *scm_data = nvdimm_provider_data(nvdimm); + + return scm_overwrite(scm_data); +} + +/** + * scm_secop_query_overwrite() - Get the current overwrite state + * @nvdimm: The nvdimm representation of the SCM device to start the overwrite on + * Return: 0 if successful or idle, -EBUSY if busy, -EFAULT if failed + */ +static int scm_secop_query_overwrite(struct nvdimm *nvdimm) +{ + struct scm_data *scm_data = nvdimm_provider_data(nvdimm); + + if (scm_data->overwrite_state == SCM_OVERWRITE_BUSY) + return -EBUSY; + + if (scm_data->overwrite_state == SCM_OVERWRITE_FAILED) + return -EFAULT; + + return 0; +} + +/** + * scm_secop_get_flags() - return the security flags for the SCM device + */ +static unsigned long scm_secop_get_flags(struct nvdimm *nvdimm, + enum nvdimm_passphrase_type ptype) +{ + struct scm_data *scm_data = nvdimm_provider_data(nvdimm); + + if (scm_data->overwrite_state == SCM_OVERWRITE_BUSY) + return BIT(NVDIMM_SECURITY_OVERWRITE); + + return BIT(NVDIMM_SECURITY_DISABLED); +} + +static const struct nvdimm_security_ops sec_ops = { + .get_flags = scm_secop_get_flags, + .overwrite = scm_secop_overwrite, + .query_overwrite = scm_secop_query_overwrite, +}; + /** * scm_register_lpc_mem() - Discover persistent memory on a device and register it with the NVDIMM subsystem * @scm_data: The SCM device data @@ -224,10 +304,10 @@ static int scm_register_lpc_mem(struct scm_data *scm_data) set_bit(NDD_ALIASING, &nvdimm_flags); snprintf(serial, sizeof(serial), "%llx", fn_config->serial); - nd_mapping_desc.nvdimm = nvdimm_create(scm_data->nvdimm_bus, scm_data, + nd_mapping_desc.nvdimm = __nvdimm_create(scm_data->nvdimm_bus, scm_data, scm_dimm_attribute_groups, nvdimm_flags, nvdimm_cmd_mask, - 0, NULL); + 0, NULL, serial, &sec_ops); if (!nd_mapping_desc.nvdimm) return -ENOMEM; @@ -1530,6 +1610,83 @@ static void scm_dump_error_log(struct scm_data *scm_data) kfree(buf); } +static void scm_handle_nscra_doorbell(struct scm_data *scm_data) +{ + int rc; + + if (scm_data->ns_command.op_code == NS_COMMAND_SECURE_ERASE) { + u64 success, attempted; + + + rc = scm_ns_response(scm_data); + if (rc < 0) { + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + mutex_unlock(&scm_data->ns_command.lock); + return; + } + if (rc != STATUS_SUCCESS) + scm_warn_status(scm_data, "Unexpected status from overwrite", rc); + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->ns_command.response_offset + + NS_RESPONSE_SECURE_ERASE_ACCESSIBLE_SUCCESS, + OCXL_HOST_ENDIAN, &success); + if (rc) { + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + mutex_unlock(&scm_data->ns_command.lock); + return; + } + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->ns_command.response_offset + + NS_RESPONSE_SECURE_ERASE_ACCESSIBLE_ATTEMPTED, + OCXL_HOST_ENDIAN, &attempted); + if (rc) { + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + mutex_unlock(&scm_data->ns_command.lock); + return; + } + + scm_data->overwrite_state = SCM_OVERWRITE_SUCCESS; + if (success != attempted) + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + + dev_info(&scm_data->dev, + "Overwritten %llu/%llu accessible pages", success, attempted); + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->ns_command.response_offset + + NS_RESPONSE_SECURE_ERASE_DEFECTIVE_SUCCESS, + OCXL_HOST_ENDIAN, &success); + if (rc) { + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + mutex_unlock(&scm_data->ns_command.lock); + return; + } + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->ns_command.response_offset + + NS_RESPONSE_SECURE_ERASE_DEFECTIVE_ATTEMPTED, + OCXL_HOST_ENDIAN, &attempted); + if (rc) { + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + mutex_unlock(&scm_data->ns_command.lock); + return; + } + + if (success != attempted) + scm_data->overwrite_state = SCM_OVERWRITE_FAILED; + + dev_info(&scm_data->dev, + "Overwritten %llu/%llu defective pages", success, attempted); + + scm_ns_response_handled(scm_data); + + mutex_unlock(&scm_data->ns_command.lock); + return; + } +} + static irqreturn_t scm_imn0_handler(void *private) { struct scm_data *scm_data = private; @@ -1537,6 +1694,9 @@ static irqreturn_t scm_imn0_handler(void *private) (void)scm_chi(scm_data, &chi); + if (chi & GLOBAL_MMIO_CHI_NSCRA) + scm_handle_nscra_doorbell(scm_data); + if (chi & GLOBAL_MMIO_CHI_ELA) { dev_warn(&scm_data->dev, "Error log is available\n"); diff --git a/drivers/nvdimm/ocxl/scm_internal.c b/drivers/nvdimm/ocxl/scm_internal.c index 8fc849610eaa..db919a23c69b 100644 --- a/drivers/nvdimm/ocxl/scm_internal.c +++ b/drivers/nvdimm/ocxl/scm_internal.c @@ -173,6 +173,7 @@ int scm_ns_response_handled(const struct scm_data *scm_data) OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_CHI_NSCRA); } + void scm_warn_status(const struct scm_data *scm_data, const char *message, u8 status) { diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index af19813a7f75..4a29088612a9 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -70,6 +70,15 @@ #define ADMIN_COMMAND_CMD_CAPS 0x08u #define ADMIN_COMMAND_MAX 0x08u +#define NS_COMMAND_SECURE_ERASE 0x20ull + +#define NS_RESPONSE_SECURE_ERASE_ACCESSIBLE_SUCCESS 0x20 +#define NS_RESPONSE_SECURE_ERASE_ACCESSIBLE_ATTEMPTED 0x28 +#define NS_RESPONSE_SECURE_ERASE_DEFECTIVE_SUCCESS 0x30 +#define NS_RESPONSE_SECURE_ERASE_DEFECTIVE_ATTEMPTED 0x38 + + + #define STATUS_SUCCESS 0x00 #define STATUS_MEM_UNAVAILABLE 0x20 #define STATUS_BAD_OPCODE 0x50 @@ -99,6 +108,13 @@ struct scm_function_0 { struct ocxl_fn *ocxl_fn; }; +enum overwrite_state { + SCM_OVERWRITE_IDLE = 0, + SCM_OVERWRITE_BUSY, + SCM_OVERWRITE_SUCCESS, + SCM_OVERWRITE_FAILED +}; + struct scm_data { struct device dev; struct pci_dev *pdev; @@ -116,6 +132,7 @@ struct scm_data { void *metadata_addr; struct command_metadata admin_command; struct command_metadata ns_command; + enum overwrite_state overwrite_state; struct resource scm_res; struct nd_region *nd_region; struct eventfd_ctx *ev_ctx; From patchwork Tue Dec 3 03:46:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCB5D6C1 for ; Tue, 3 Dec 2019 03:48:53 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5B8720733 for ; Tue, 3 Dec 2019 03:48:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5B8720733 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 591DE10113699; Mon, 2 Dec 2019 19:52:00 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E1B7310113666 for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kmA2083980 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wm6xb0p18-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mIIj24707220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC467AE051; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F3AEAE053; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id A91CAA03EF; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 25/27] nvdimm/ocxl: Expose SMART data via ndctl Date: Tue, 3 Dec 2019 14:46:53 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0028-0000-0000-000003C3CC76 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0029-0000-0000-00002486E434 Message-Id: <20191203034655.51561-26-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 suspectscore=1 lowpriorityscore=0 phishscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 adultscore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: KI4HKCRYHC7HW6LXYXIEFRGOTTNEFBBN X-Message-ID-Hash: KI4HKCRYHC7HW6LXYXIEFRGOTTNEFBBN X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva This patch retrieves proprietary formatted SMART data and makes it available via ndctl. A later contribution will be made to ndctl to parse this data. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/scm.c | 156 +++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/scm_internal.h | 21 ++++ 2 files changed, 177 insertions(+) diff --git a/drivers/nvdimm/ocxl/scm.c b/drivers/nvdimm/ocxl/scm.c index 8deb7862793c..77b9e68870a3 100644 --- a/drivers/nvdimm/ocxl/scm.c +++ b/drivers/nvdimm/ocxl/scm.c @@ -94,6 +94,157 @@ static int scm_ndctl_config_size(struct nd_cmd_get_config_size *command) return 0; } +static int read_smart_attrib(struct scm_data *scm_data, u16 offset, + struct scm_smart_attribs *attribs) +{ + u64 val; + int rc; + struct scm_smart_attrib *attrib; + u8 attrib_id; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, offset, OCXL_LITTLE_ENDIAN, + &val); + if (rc) + return rc; + + attrib_id = (val >> 56) & 0xff; + switch (attrib_id) { + case SCM_SMART_ATTR_POWER_ON_HOURS: + attrib = &attribs->power_on_hours; + break; + + case SCM_SMART_ATTR_TEMPERATURE: + attrib = &attribs->temperature; + break; + + case SCM_SMART_ATTR_LIFE_REMAINING: + attrib = &attribs->life_remaining; + break; + + default: + dev_warn(&scm_data->dev, "Unknown smart attrib '%d'", attrib_id); + return -ENOENT; + } + + attrib->id = attrib_id; + attrib->attribute_flags = (val >> 40) & 0xffff; + attrib->current_val = (val >> 32) & 0xff; + attrib->threshold_val = (val >> 24) & 0xff; + attrib->worst_val = (val >> 16) & 0xff; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, offset + 0x08, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + attrib->raw_val = val; + + return 0; +} + +/** + * scm_smart_header_parse() - Parse the first 64 bits of the SMART admin command response + * @scm_data: the SCM metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int scm_smart_header_parse(struct scm_data *scm_data, u32 *length) +{ + int rc; + u64 val; + + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(scm_data->ocxl_afu, + scm_data->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFFFFFF; + + if (data_identifier != 0x534D) { + dev_err(&scm_data->dev, + "Bad data identifier for smart data, expected 'SM', got '%-.*s'\n", + 2, (char *)&data_identifier); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int scm_smart_update(struct scm_data *scm_data) +{ + u32 length, i; + int rc; + + mutex_lock(&scm_data->admin_command.lock); + + rc = scm_admin_command_request(scm_data, ADMIN_COMMAND_SMART); + if (rc) + goto out; + + rc = scm_admin_command_execute(scm_data); + if (rc) + goto out; + + rc = scm_admin_command_complete_timeout(scm_data, ADMIN_COMMAND_SMART); + if (rc < 0) { + dev_err(&scm_data->dev, "SMART timeout\n"); + goto out; + } + + rc = scm_admin_response(scm_data); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + scm_warn_status(scm_data, "Unexpected status from SMART", rc); + goto out; + } + + rc = scm_smart_header_parse(scm_data, &length); + if (rc) + goto out; + + length /= 0x10; // Length now contains the number of attributes + + for (i = 0; i < length; i++) + read_smart_attrib(scm_data, + scm_data->admin_command.data_offset + 0x08 + i * 0x10, + &scm_data->smart); + + rc = scm_admin_response_handled(scm_data); + if (rc) + goto out; + + rc = 0; + goto out; + +out: + mutex_unlock(&scm_data->admin_command.lock); + return rc; +} + +static int scm_ndctl_smart(struct scm_data *scm_data, void *buf, + unsigned int buf_len) +{ + int rc; + + if (buf_len != sizeof(scm_data->smart)) + return -EINVAL; + + rc = scm_smart_update(scm_data); + if (rc) + return rc; + + memcpy(buf, &scm_data->smart, buf_len); + + return 0; +} + + static int scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm, unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) @@ -101,6 +252,10 @@ static int scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct scm_data *scm_data = container_of(nd_desc, struct scm_data, bus_desc); switch (cmd) { + case ND_CMD_SMART: + *cmd_rc = scm_ndctl_smart(scm_data, buf, buf_len); + return 0; + case ND_CMD_GET_CONFIG_SIZE: *cmd_rc = scm_ndctl_config_size(buf); return 0; @@ -300,6 +455,7 @@ static int scm_register_lpc_mem(struct scm_data *scm_data) set_bit(ND_CMD_GET_CONFIG_SIZE, &nvdimm_cmd_mask); set_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm_cmd_mask); set_bit(ND_CMD_SET_CONFIG_DATA, &nvdimm_cmd_mask); + set_bit(ND_CMD_SMART, &nvdimm_cmd_mask); set_bit(NDD_ALIASING, &nvdimm_flags); diff --git a/drivers/nvdimm/ocxl/scm_internal.h b/drivers/nvdimm/ocxl/scm_internal.h index 4a29088612a9..d593fefe38d5 100644 --- a/drivers/nvdimm/ocxl/scm_internal.h +++ b/drivers/nvdimm/ocxl/scm_internal.h @@ -115,6 +115,26 @@ enum overwrite_state { SCM_OVERWRITE_FAILED }; +#define SCM_SMART_ATTR_POWER_ON_HOURS 0x09 +#define SCM_SMART_ATTR_TEMPERATURE 0xC2 +#define SCM_SMART_ATTR_LIFE_REMAINING 0xCA + +struct scm_smart_attrib { + __u8 id; /* See defines above */ + __u16 attribute_flags; + __u8 current_val; + __u8 threshold_val; + __u8 worst_val; + __u8 reserved; + __u64 raw_val; +}; + +struct scm_smart_attribs { + struct scm_smart_attrib power_on_hours; + struct scm_smart_attrib temperature; + struct scm_smart_attrib life_remaining; +}; + struct scm_data { struct device dev; struct pci_dev *pdev; @@ -136,6 +156,7 @@ struct scm_data { struct resource scm_res; struct nd_region *nd_region; struct eventfd_ctx *ev_ctx; + struct scm_smart_attribs smart; char fw_version[8+1]; u32 timeouts[ADMIN_COMMAND_MAX+1]; From patchwork Tue Dec 3 03:46:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CC7F6C1 for ; Tue, 3 Dec 2019 03:49:02 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25FC52073C for ; Tue, 3 Dec 2019 03:49:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25FC52073C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 2190B10097F0D; Mon, 2 Dec 2019 19:52:05 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5BAC21011363B for ; Mon, 2 Dec 2019 19:51:54 -0800 (PST) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kkYw021125 for ; Mon, 2 Dec 2019 22:48:31 -0500 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2wnehxj6u6-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:31 -0500 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:19 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mIqZ24707218 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:18 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 90108AE057; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F513AE056; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id BDBD5A03F0; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 26/27] powerpc: Enable OpenCAPI Storage Class Memory driver on bare metal Date: Tue, 3 Dec 2019 14:46:54 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0016-0000-0000-000002D0158C X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0017-0000-0000-000033320CA7 Message-Id: <20191203034655.51561-27-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 spamscore=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=1 impostorscore=0 mlxlogscore=535 priorityscore=1501 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: Z7NBN5CGX22K4BJTZUTUNIXYF73R65OK X-Message-ID-Hash: Z7NBN5CGX22K4BJTZUTUNIXYF73R65OK X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva Enable OpenCAPI Storage Class Memory driver on bare metal Signed-off-by: Alastair D'Silva --- arch/powerpc/configs/powernv_defconfig | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/powerpc/configs/powernv_defconfig b/arch/powerpc/configs/powernv_defconfig index 6658cceb928c..a8f46aece8a4 100644 --- a/arch/powerpc/configs/powernv_defconfig +++ b/arch/powerpc/configs/powernv_defconfig @@ -352,3 +352,7 @@ CONFIG_KVM_BOOK3S_64=m CONFIG_KVM_BOOK3S_64_HV=m CONFIG_VHOST_NET=m CONFIG_PRINTK_TIME=y +CONFIG_OCXL_SCM=m +CONFIG_DEV_DAX=m +CONFIG_DEV_DAX_PMEM=m +CONFIG_FS_DAX=m From patchwork Tue Dec 3 03:46:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11270315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E5BF86C1 for ; Tue, 3 Dec 2019 03:48:57 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CF0432080F for ; Tue, 3 Dec 2019 03:48:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF0432080F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=au1.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B7ED810097F08; Mon, 2 Dec 2019 19:52:01 -0800 (PST) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=alastair@au1.ibm.com; receiver= Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 03F4B1011366A for ; Mon, 2 Dec 2019 19:51:53 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB33kl5j093296 for ; Mon, 2 Dec 2019 22:48:30 -0500 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0b-001b2d01.pphosted.com with ESMTP id 2wm6g99bqf-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Dec 2019 22:48:30 -0500 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Dec 2019 03:48:27 -0000 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Dec 2019 03:48:20 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id xB33mJs837224800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Dec 2019 03:48:19 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2BDE24204D; Tue, 3 Dec 2019 03:48:19 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C7B1642045; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 3 Dec 2019 03:48:18 +0000 (GMT) Received: from adsilva.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id D07FBA03F1; Tue, 3 Dec 2019 14:48:13 +1100 (AEDT) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v2 27/27] MAINTAINERS: Add myself & nvdimm/ocxl to ocxl Date: Tue, 3 Dec 2019 14:46:55 +1100 X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191203034655.51561-1-alastair@au1.ibm.com> References: <20191203034655.51561-1-alastair@au1.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19120303-0016-0000-0000-000002D0158E X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19120303-0017-0000-0000-000033320CA9 Message-Id: <20191203034655.51561-28-alastair@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-02_06:2019-11-29,2019-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 clxscore=1015 lowpriorityscore=0 bulkscore=0 phishscore=0 priorityscore=1501 impostorscore=0 suspectscore=1 spamscore=0 mlxscore=0 adultscore=0 mlxlogscore=615 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1912030032 Message-ID-Hash: MKCZPDVT6BLOCQ2P2DJ5HKZQ3KWRRNXL X-Message-ID-Hash: MKCZPDVT6BLOCQ2P2DJ5HKZQ3KWRRNXL X-MailFrom: alastair@au1.ibm.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@list s.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Alastair D'Silva The nvdimm/ocxl driver will be maintained as part of the ppc tree. I'm also adding myself as an author of the driver & contributor to the generic ocxl driver. Signed-off-by: Alastair D'Silva --- MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 9d3a5c54a41d..e9152cf27176 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -11724,13 +11724,16 @@ F: tools/objtool/ OCXL (Open Coherent Accelerator Processor Interface OpenCAPI) DRIVER M: Frederic Barrat M: Andrew Donnellan +M: Alastair D'Silva L: linuxppc-dev@lists.ozlabs.org S: Supported F: arch/powerpc/platforms/powernv/ocxl.c F: arch/powerpc/include/asm/pnv-ocxl.h F: drivers/misc/ocxl/ +F: drivers/nvdimm/ocxl/ F: include/misc/ocxl* F: include/uapi/misc/ocxl.h +F: include/uapi/nvdimm/ocxl-scm.h F: Documentation/userspace-api/accelerators/ocxl.rst OMAP AUDIO SUPPORT