From patchwork Fri Mar 27 07:11:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461809 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B00941668 for ; Fri, 27 Mar 2020 07:12:33 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F25720717 for ; Fri, 27 Mar 2020 07:12:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="hCzyPFbC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F25720717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 37A2B10FC3BB0; Fri, 27 Mar 2020 00:13:17 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 43E5210FC3614 for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id E40E22DC682D; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=NLU6a+GZmpbGM7ZZlM+S8RXczj/+GapNI1yMP3rrpMk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hCzyPFbCqeOcmtzW6SY05RhBcPIxs8zbC5tLYywSkBDdyzkBfaz87rhMNSBwMnoYp Zs73azLmq1wWh2TxHbBd8GSmjvxZSVtaA79yHebEbjTy/YXjZzrWCDDU5bDpxki2pB rN2yPnnECyv0AP/0MpPXrv8ZnHvyARjDQkGZauduLjpYcpvGKthonv2K2P7ZEGeT3r BWtkFZekdnExUu8PcpN9hof6O6UVI8aP0qrYDNZVkshtTLRvZbGfsRPrhKuotRnefo jCpgo90BgMbqz2oDrTKep24o4Co6T59jViwsTP8ee/qv0Xlz/j8EBmv20t8dT4DnuM nbJOUSeitii2d2dnPk723FdeI/+4gos4LDyKZz01DH5KbYRzCR+459OyUX++I1fKNK hqsTtTXeKLOz71rQf2BzGHmPkcWw7K5r3OBgi/R4EN56O38sqsK6yoBQvHi2orUs6j vVIvNEYUU+xmK+zTlYLatt9vnPH8wa81do/yIaNuL5o0jYorLHklmBJz3Yg8hb1eP3 SJwPR8jJWB9+JkFp+pvthgTGgzLA8uozWXIEBpG6zASx0q7OCBnaODBeZqQSg5PLwn N1U5Xz2nG6uQ/mLENf2G4yEIrWdeJS7o8GXcUJwgjqe7dXHyrcZl9/+ZiO0yHu3jQb L2zJCEa24D4PbxYl88zmfV2s= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Aa045934; Fri, 27 Mar 2020 18:12:14 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 01/25] powerpc/powernv: Add OPAL calls for LPC memory alloc/release Date: Fri, 27 Mar 2020 18:11:38 +1100 Message-Id: <20200327071202.2159885-2-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:14 +1100 (AEDT) Message-ID-Hash: FBMDK4LTXTOX3PWM6ZBJQPFHRPFTBWKM X-Message-ID-Hash: FBMDK4LTXTOX3PWM6ZBJQPFHRPFTBWKM X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Add OPAL calls for LPC memory alloc/release Signed-off-by: Alastair D'Silva Acked-by: Andrew Donnellan Acked-by: Frederic Barrat --- arch/powerpc/include/asm/opal-api.h | 2 ++ arch/powerpc/include/asm/opal.h | 2 ++ arch/powerpc/platforms/powernv/opal-call.c | 2 ++ 3 files changed, 6 insertions(+) diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index c1f25a760eb1..9298e603001b 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -208,6 +208,8 @@ #define OPAL_HANDLE_HMI2 166 #define OPAL_NX_COPROC_INIT 167 #define OPAL_XIVE_GET_VP_STATE 170 +#define OPAL_NPU_MEM_ALLOC 171 +#define OPAL_NPU_MEM_RELEASE 172 #define OPAL_MPIPL_UPDATE 173 #define OPAL_MPIPL_REGISTER_TAG 174 #define OPAL_MPIPL_QUERY_TAG 175 diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index 9986ac34b8e2..301fea46c7ca 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -39,6 +39,8 @@ int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t bdfn, uint64_t PE_handle); int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, long cap, uint64_t rate_phys, uint32_t size); +int64_t opal_npu_mem_alloc(u64 phb_id, u32 bdfn, u64 size, __be64 *bar); +int64_t opal_npu_mem_release(u64 phb_id, u32 bdfn); int64_t opal_console_write(int64_t term_number, __be64 *length, const uint8_t *buffer); diff --git a/arch/powerpc/platforms/powernv/opal-call.c b/arch/powerpc/platforms/powernv/opal-call.c index 5cd0f52d258f..f26e58b72c04 100644 --- a/arch/powerpc/platforms/powernv/opal-call.c +++ b/arch/powerpc/platforms/powernv/opal-call.c @@ -287,6 +287,8 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR); OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64); OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE); OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT); +OPAL_CALL(opal_npu_mem_alloc, OPAL_NPU_MEM_ALLOC); +OPAL_CALL(opal_npu_mem_release, OPAL_NPU_MEM_RELEASE); OPAL_CALL(opal_mpipl_update, OPAL_MPIPL_UPDATE); OPAL_CALL(opal_mpipl_register_tag, OPAL_MPIPL_REGISTER_TAG); OPAL_CALL(opal_mpipl_query_tag, OPAL_MPIPL_QUERY_TAG); From patchwork Fri Mar 27 07:11:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461819 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E9CE1667 for ; Fri, 27 Mar 2020 07:12:41 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5DA9020717 for ; Fri, 27 Mar 2020 07:12:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="ZprMO5g5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5DA9020717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id D09B210FC3BCE; Fri, 27 Mar 2020 00:13:18 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 6D6CF10FC36EE for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 9F2462DC6831; Fri, 27 Mar 2020 18:12:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293142; bh=HPdDJ0enxYWuoioAjwzepK5FMG95uTKM1kU7cOk1haY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZprMO5g5LCg0oRdH2K3uXz8l4c6L9W1d9Mxd3y9LbAARGZu9kt6uIz9A3MEUOedWV g7DqMLDrr2m/TH6d6yrJtFgEOg9BlrSv1R2NLozkVnU6OkgsrVmJx9BBn8aI/Sn62O w/5CiI+YX0ODTMBr8YVhJA07fuOcUVGHda3b5a9mGukeXcANsMAjZc4Gg4MUBM/PoJ Jnr9+eXE1sX7vtXdpSE1woIPFaLHakSy5NHhRbC+k3KhNv5RH6TZ6A/u+iDAiiRCE5 N06lRZtlEVFMlb5wR4Jid5B9ENUsU6ZZPEVYF/txRgiW4MKAS+CcjtkONcUOSmYaD9 ppYmaM2ef6S8tshDUjxJypHcy1+sz3bZZrRF+2nIuWjpwN2hZGDxRVKHsZfKZhVqQF sblZp70rrN6e/kpXRz/acmkv2yO8Fa77dlC3P84DeB/nV2oDsJDuyI07z13vZ/Bmuj o26AIi8KDpxo1HfGM10O/KoPFg3TtC7iqbL2WVTgVmat/MkiiY9qOLW8neZfNnPbUY Yfawn1JU84te5v7VgBv+fKxvfyGbsXV1RLhC09q4r5qhGolccVrWWCbRAXVplfwLiE ko6h2lMIzDuDXQc0NNcn6PH9aJ9muG0dIOvlWkH8pN4W1vzD3LNkEswpP0E7Hqf3M9 jksprRXa8oR2y8sUhkAlZwgY= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ab045934; Fri, 27 Mar 2020 18:12:14 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 02/25] mm/memory_hotplug: Allow check_hotplug_memory_addressable to be called from drivers Date: Fri, 27 Mar 2020 18:11:39 +1100 Message-Id: <20200327071202.2159885-3-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:14 +1100 (AEDT) Message-ID-Hash: XTPP6VGPMVPOIHEPPTG4NZGOMO6ZRAH2 X-Message-ID-Hash: XTPP6VGPMVPOIHEPPTG4NZGOMO6ZRAH2 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: When setting up OpenCAPI connected persistent memory, the range check may not be performed until quite late (or perhaps not at all, if the user does not establish a DAX device). This patch makes the range check callable so we can perform the check while probing the OpenCAPI Persistent Memory device. Signed-off-by: Alastair D'Silva Reviewed-by: Andrew Donnellan --- include/linux/memory_hotplug.h | 5 +++++ mm/memory_hotplug.c | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index f4d59155f3d4..9a19ae0d7e31 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -337,6 +337,11 @@ static inline void __remove_memory(int nid, u64 start, u64 size) {} extern void set_zone_contiguous(struct zone *zone); extern void clear_zone_contiguous(struct zone *zone); +#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE +int check_hotplug_memory_addressable(unsigned long pfn, + unsigned long nr_pages); +#endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */ + extern void __ref free_area_init_core_hotplug(int nid); extern int __add_memory(int nid, u64 start, u64 size); extern int add_memory(int nid, u64 start, u64 size); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 0a54ffac8c68..14945f033594 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -276,8 +276,8 @@ static int check_pfn_span(unsigned long pfn, unsigned long nr_pages, return 0; } -static int check_hotplug_memory_addressable(unsigned long pfn, - unsigned long nr_pages) +int check_hotplug_memory_addressable(unsigned long pfn, + unsigned long nr_pages) { const u64 max_addr = PFN_PHYS(pfn + nr_pages) - 1; From patchwork Fri Mar 27 07:11:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461815 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A3B6159A for ; Fri, 27 Mar 2020 07:12:39 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DBB9820717 for ; Fri, 27 Mar 2020 07:12:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="kC/+/w08" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBB9820717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id A8B0210FC3BC7; Fri, 27 Mar 2020 00:13:18 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 57FA110FC3614 for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 7A6E42DC682A; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=+19ERjJVKb04vm4biVoKIgTzaFpr9UkwgZhCwlBFNZ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kC/+/w081uhGq50IGkocqjNtFwFT1jfxjvZzedjpb8Zv7xxpRYbzhpwskvozUByYX t1UJjPknJOIK7/+aPNJSLi6XUVox4iV1VJGSrm9Uj3vTDCzhS9i/TOfK53MveelFAp bko/gKfzyjxtjuza/Zzfg3Lazv0GZ3jjrrXWCwg1vWGqkSIUVixWnbTR0MM7DkIYv3 zFps93hBiRICgFLlR41gopwn4qGl98D54wGKy1XO2IDsZXDn+oys9f7szHubl2erxM 98hazA7s2Hun5cwyF6UOwhIe8q33x097i9e5n+Rf5dLtx5rgNYv7jDOjsRKzeUUWlG QJtJdRZQpQ31k0EmXeQHJNxR5CcCfaSVkJEql1R4IJPneJz+k6VO55PzRXId+c+3Px jsmelTImM2q0z1bIl+aQUc38mZExstNx7apQSEylv5bAFmaRgbbpPWuz/9an1KZub8 Qzb9tJxzQmy5RvMKkAtuGKgUxYeA09K4wgOVNYnoBytc1Q07/DzuM4Np1Eq88rqcTp 81BizxxTLWlPoA9bTIfVasmLRUIxGJ5ZA+2wwklqRXTwAd2lZZB446Y4Rfva5ONbNx 2lC2CPiWB8qxGj796Eq1C0STbZHYpgpbtTNAlfrufYLT7q3aruf3ZSbLGO9v8DSjI7 /UOdhgZ/mEaSk8QieEOClM90= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ac045934; Fri, 27 Mar 2020 18:12:14 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 03/25] powerpc/powernv: Map & release OpenCAPI LPC memory Date: Fri, 27 Mar 2020 18:11:40 +1100 Message-Id: <20200327071202.2159885-4-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:14 +1100 (AEDT) Message-ID-Hash: VDTBKZR36JH7B6XH6ZNXL2MXSM3WVHG4 X-Message-ID-Hash: VDTBKZR36JH7B6XH6ZNXL2MXSM3WVHG4 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch adds OPAL calls to powernv so that the OpenCAPI driver can map & release LPC (Lowest Point of Coherency) memory. Signed-off-by: Alastair D'Silva Reviewed-by: Andrew Donnellan --- arch/powerpc/include/asm/pnv-ocxl.h | 2 ++ arch/powerpc/platforms/powernv/ocxl.c | 43 +++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/arch/powerpc/include/asm/pnv-ocxl.h b/arch/powerpc/include/asm/pnv-ocxl.h index 7de82647e761..560a19bb71b7 100644 --- a/arch/powerpc/include/asm/pnv-ocxl.h +++ b/arch/powerpc/include/asm/pnv-ocxl.h @@ -32,5 +32,7 @@ extern int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle) extern int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr); extern void pnv_ocxl_free_xive_irq(u32 irq); +u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size); +void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev); #endif /* _ASM_PNV_OCXL_H */ diff --git a/arch/powerpc/platforms/powernv/ocxl.c b/arch/powerpc/platforms/powernv/ocxl.c index 8c65aacda9c8..f13119a7c026 100644 --- a/arch/powerpc/platforms/powernv/ocxl.c +++ b/arch/powerpc/platforms/powernv/ocxl.c @@ -475,6 +475,49 @@ void pnv_ocxl_spa_release(void *platform_data) } EXPORT_SYMBOL_GPL(pnv_ocxl_spa_release); +u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size) +{ + struct pci_controller *hose = pci_bus_to_host(pdev->bus); + struct pnv_phb *phb = hose->private_data; + u32 bdfn = pci_dev_id(pdev); + __be64 base_addr_be64; + u64 base_addr; + int rc; + + rc = opal_npu_mem_alloc(phb->opal_id, bdfn, size, &base_addr_be64); + if (rc) { + dev_warn(&pdev->dev, + "OPAL could not allocate LPC memory, rc=%d\n", rc); + return 0; + } + + base_addr = be64_to_cpu(base_addr_be64); + +#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE + rc = check_hotplug_memory_addressable(base_addr >> PAGE_SHIFT, + size >> PAGE_SHIFT); + if (rc) + return 0; +#endif + + return base_addr; +} +EXPORT_SYMBOL_GPL(pnv_ocxl_platform_lpc_setup); + +void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev) +{ + struct pci_controller *hose = pci_bus_to_host(pdev->bus); + struct pnv_phb *phb = hose->private_data; + u32 bdfn = pci_dev_id(pdev); + int rc; + + rc = opal_npu_mem_release(phb->opal_id, bdfn); + if (rc) + dev_warn(&pdev->dev, + "OPAL reported rc=%d when releasing LPC memory\n", rc); +} +EXPORT_SYMBOL_GPL(pnv_ocxl_platform_lpc_release); + int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle) { struct spa_data *data = (struct spa_data *) platform_data; From patchwork Fri Mar 27 07:11:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461813 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B6A0159A for ; Fri, 27 Mar 2020 07:12:37 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A2BB20717 for ; Fri, 27 Mar 2020 07:12:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="dDzEJ12M" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A2BB20717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7F20210FC3BBF; Fri, 27 Mar 2020 00:13:17 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 4E9C510FC3763 for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id F39C02DC682F; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=cTN8FMRoX34pEnqAaScxUQo064CptcomhYQubar6PL4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dDzEJ12MPCicxaVAdHkydps6s4bYG45dmh7GrJNLgOaIVhimCK4WsnAr28LxgF9nz SZ1C4w5tiF/ghQDzxSyJQQ4ZgL86wWzX423PKNWVFIFj724JhV5EldneRKE4rj1LGR 61kGSrR6SXUtYsCr43AHIHhqj5D1VpoOx5qbkeXK5g2YJlwCRk53lmHuZyHWFsMv5C UIev+ORU4xZrQ+nJStJe5NkumRYilZyw4g2UzKJq9I4iulDlTSkXR/IVoMnbBPlFlS 8FOnA4O9Q1Tn0zCEfOEGNYlZtINAXQFQRc9M9qM7imsIc/NTffHIa/hFYLBjGg47hz nBYcC2u3t8f52WVvJECJjIRV41puWvpmjrLjYPcMR2UG1JllWLEE4+EeK6I6xcpdJi gbT009RMoO44xqVORSk51rxQkr4y9PA1eXvMYxFTsby9DNUVwVL6kDGK48B5w8TbpK JcwCR9+3Qv0bF66b5pGwDBjYVyo1DYAjYC24hN28JGsUeDHjls4olMODbGP6DL3jAq O84KFtHng5DI3fXHjRvXAhVNzdPtu/Y/PBccmFoEnnsvqn1YapgeRb4gkqh2EyJPCM aVv9XZAGwUCwF27wFKECjzydTo1r+WX/+DJoNsp0Of3K7qTbKQnc2X9fCyLSk3dkt7 r7OXXTocQUPjuAp4QH4snO5U= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ad045934; Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 04/25] ocxl: Remove unnecessary externs Date: Fri, 27 Mar 2020 18:11:41 +1100 Message-Id: <20200327071202.2159885-5-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) Message-ID-Hash: QGN2VULHY34FMYJZGC6IZUL4I4CXYQ6T X-Message-ID-Hash: QGN2VULHY34FMYJZGC6IZUL4I4CXYQ6T X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Function declarations don't need externs, remove the existing ones so they are consistent with newer code Signed-off-by: Alastair D'Silva Acked-by: Andrew Donnellan Acked-by: Frederic Barrat --- arch/powerpc/include/asm/pnv-ocxl.h | 40 ++++++++++++++--------------- include/misc/ocxl.h | 6 ++--- 2 files changed, 22 insertions(+), 24 deletions(-) diff --git a/arch/powerpc/include/asm/pnv-ocxl.h b/arch/powerpc/include/asm/pnv-ocxl.h index 560a19bb71b7..205efc41a33c 100644 --- a/arch/powerpc/include/asm/pnv-ocxl.h +++ b/arch/powerpc/include/asm/pnv-ocxl.h @@ -9,29 +9,27 @@ #define PNV_OCXL_TL_BITS_PER_RATE 4 #define PNV_OCXL_TL_RATE_BUF_SIZE ((PNV_OCXL_TL_MAX_TEMPLATE+1) * PNV_OCXL_TL_BITS_PER_RATE / 8) -extern int pnv_ocxl_get_actag(struct pci_dev *dev, u16 *base, u16 *enabled, - u16 *supported); -extern int pnv_ocxl_get_pasid_count(struct pci_dev *dev, int *count); +int pnv_ocxl_get_actag(struct pci_dev *dev, u16 *base, u16 *enabled, u16 *supported); +int pnv_ocxl_get_pasid_count(struct pci_dev *dev, int *count); -extern int pnv_ocxl_get_tl_cap(struct pci_dev *dev, long *cap, +int pnv_ocxl_get_tl_cap(struct pci_dev *dev, long *cap, char *rate_buf, int rate_buf_size); -extern int pnv_ocxl_set_tl_conf(struct pci_dev *dev, long cap, - uint64_t rate_buf_phys, int rate_buf_size); - -extern int pnv_ocxl_get_xsl_irq(struct pci_dev *dev, int *hwirq); -extern void pnv_ocxl_unmap_xsl_regs(void __iomem *dsisr, void __iomem *dar, - void __iomem *tfc, void __iomem *pe_handle); -extern int pnv_ocxl_map_xsl_regs(struct pci_dev *dev, void __iomem **dsisr, - void __iomem **dar, void __iomem **tfc, - void __iomem **pe_handle); - -extern int pnv_ocxl_spa_setup(struct pci_dev *dev, void *spa_mem, int PE_mask, - void **platform_data); -extern void pnv_ocxl_spa_release(void *platform_data); -extern int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle); - -extern int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr); -extern void pnv_ocxl_free_xive_irq(u32 irq); +int pnv_ocxl_set_tl_conf(struct pci_dev *dev, long cap, + uint64_t rate_buf_phys, int rate_buf_size); + +int pnv_ocxl_get_xsl_irq(struct pci_dev *dev, int *hwirq); +void pnv_ocxl_unmap_xsl_regs(void __iomem *dsisr, void __iomem *dar, + void __iomem *tfc, void __iomem *pe_handle); +int pnv_ocxl_map_xsl_regs(struct pci_dev *dev, void __iomem **dsisr, + void __iomem **dar, void __iomem **tfc, + void __iomem **pe_handle); + +int pnv_ocxl_spa_setup(struct pci_dev *dev, void *spa_mem, int PE_mask, void **platform_data); +void pnv_ocxl_spa_release(void *platform_data); +int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle); + +int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr); +void pnv_ocxl_free_xive_irq(u32 irq); u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size); void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev); diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index 06dd5839e438..0a762e387418 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -173,7 +173,7 @@ int ocxl_context_detach(struct ocxl_context *ctx); * * Returns 0 on success, negative on failure */ -extern int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); +int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); /** * Frees an IRQ associated with an AFU context @@ -182,7 +182,7 @@ extern int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); * * Returns 0 on success, negative on failure */ -extern int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); +int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); /** * Gets the address of the trigger page for an IRQ @@ -193,7 +193,7 @@ extern int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); * * returns the trigger page address, or 0 if the IRQ is not valid */ -extern u64 ocxl_afu_irq_get_addr(struct ocxl_context *ctx, int irq_id); +u64 ocxl_afu_irq_get_addr(struct ocxl_context *ctx, int irq_id); /** * Provide a callback to be called when an IRQ is triggered From patchwork Fri Mar 27 07:11:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461811 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8C37159A for ; Fri, 27 Mar 2020 07:12:35 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7B9B20714 for ; Fri, 27 Mar 2020 07:12:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="haqzFNC1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7B9B20714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 671D210FC3BBB; Fri, 27 Mar 2020 00:13:17 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 445E410FC36EE for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 308892DC6830; Fri, 27 Mar 2020 18:12:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=2ddCHH2P/L90a2vCtlV2AzxdbZFdXmepYfCwpHedjjE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=haqzFNC13xamkbQf9t3YMfUf0hZ5HTJ8Gm3EYFajsSrUCxPpevss37X7MSXLoFfSA e5jqimITlwRpfYmnZ+HTZonYEY/G+ksa5amlQB6XPT//9bhSazgWRo7LjoNLBw8U/P hyBBGgg645E5eHOEOIdABgXNYtr7v1A4S0TmKEYSn6kHfSSNR9rU7d/9m3BUhx7+4W uvl3i3Sq14QD2ZzQHC7Zv9isFhR/cSg+0pEvJN+aM2mtfxbhT2aluxn113WyFZhhnY WmY9T1Wh+t/Oydhzb78zzW71g8yE6dpCSqCjol8lc4+WgXGfZeCp+LZVFC451r6EBs m8yTywyzG0ghfUuSGGp0Zy5M8mYu1SSnkPtXcBcqD4P/BqV3f8HTGMxBDBYTTc5JMy gOw3Vt6tjimak+wxSpdvNcTrQZ4gkXmoXSzWRCKvrMWz03rfk8nyAja6EcRg4M4rlY XsiLBGRrDyrKmnVYGs4612pIbhQ6eI3Ksjqmn33y9VAvqE2sTsUWjh9LqZKI5mxccJ 0DV5CWAoOS6JyoYsMw9D6uWj2ICW6nbgA/iCpGoFPe3Tq9Z3Wt2rL6TR8Af5S19Gg3 Czv74NZNJZ7eJQ7EuRVN2V3YXpYVNGxuasa8pXk59Isua/t3pOYvNZ9uguGwDJxdTw /+XUMzPabxFP/ARjBigR/uzE= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ae045934; Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 05/25] ocxl: Address kernel doc errors & warnings Date: Fri, 27 Mar 2020 18:11:42 +1100 Message-Id: <20200327071202.2159885-6-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) Message-ID-Hash: BYRN5I7XEKXDVR4ARBEQX7TGZD4G5KCB X-Message-ID-Hash: BYRN5I7XEKXDVR4ARBEQX7TGZD4G5KCB X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch addresses warnings and errors from the kernel doc scripts for the OpenCAPI driver. It also makes minor tweaks to make the docs more consistent. Signed-off-by: Alastair D'Silva Acked-by: Andrew Donnellan --- drivers/misc/ocxl/config.c | 24 ++++---- drivers/misc/ocxl/ocxl_internal.h | 9 +-- include/misc/ocxl.h | 96 ++++++++++++------------------- 3 files changed, 55 insertions(+), 74 deletions(-) diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c index c8e19bfb5ef9..a62e3d7db2bf 100644 --- a/drivers/misc/ocxl/config.c +++ b/drivers/misc/ocxl/config.c @@ -273,16 +273,16 @@ static int read_afu_info(struct pci_dev *dev, struct ocxl_fn_config *fn, } /** - * Read the template version from the AFU - * dev: the device for the AFU - * fn: the AFU offsets - * len: outputs the template length - * version: outputs the major<<8,minor version + * read_template_version() - Read the template version from the AFU + * @dev: the device for the AFU + * @fn: the AFU offsets + * @len: outputs the template length + * @version: outputs the major<<8,minor version * * Returns 0 on success, negative on failure */ static int read_template_version(struct pci_dev *dev, struct ocxl_fn_config *fn, - u16 *len, u16 *version) + u16 *len, u16 *version) { u32 val32; u8 major, minor; @@ -476,16 +476,16 @@ static int validate_afu(struct pci_dev *dev, struct ocxl_afu_config *afu) } /** - * Populate AFU metadata regarding LPC memory - * dev: the device for the AFU - * fn: the AFU offsets - * afu: the AFU struct to populate the LPC metadata into + * read_afu_lpc_memory_info() - Populate AFU metadata regarding LPC memory + * @dev: the device for the AFU + * @fn: the AFU offsets + * @afu: the AFU struct to populate the LPC metadata into * * Returns 0 on success, negative on failure */ static int read_afu_lpc_memory_info(struct pci_dev *dev, - struct ocxl_fn_config *fn, - struct ocxl_afu_config *afu) + struct ocxl_fn_config *fn, + struct ocxl_afu_config *afu) { int rc; u32 val32; diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h index 345bf843a38e..198e4e4bc51d 100644 --- a/drivers/misc/ocxl/ocxl_internal.h +++ b/drivers/misc/ocxl/ocxl_internal.h @@ -122,11 +122,12 @@ int ocxl_config_check_afu_index(struct pci_dev *dev, struct ocxl_fn_config *fn, int afu_idx); /** - * Update values within a Process Element + * ocxl_link_update_pe() - Update values within a Process Element + * @link_handle: the link handle associated with the process element + * @pasid: the PASID for the AFU context + * @tid: the new thread id for the process element * - * link_handle: the link handle associated with the process element - * pasid: the PASID for the AFU context - * tid: the new thread id for the process element + * Returns 0 on success */ int ocxl_link_update_pe(void *link_handle, int pasid, __u16 tid); diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index 0a762e387418..357ef1aadbc0 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -62,8 +62,7 @@ struct ocxl_context; // Device detection & initialisation /** - * Open an OpenCAPI function on an OpenCAPI device - * + * ocxl_function_open() - Open an OpenCAPI function on an OpenCAPI device * @dev: The PCI device that contains the function * * Returns an opaque pointer to the function, or an error pointer (check with IS_ERR) @@ -71,8 +70,7 @@ struct ocxl_context; struct ocxl_fn *ocxl_function_open(struct pci_dev *dev); /** - * Get the list of AFUs associated with a PCI function device - * + * ocxl_function_afu_list() - Get the list of AFUs associated with a PCI function device * Returns a list of struct ocxl_afu * * * @fn: The OpenCAPI function containing the AFUs @@ -80,8 +78,7 @@ struct ocxl_fn *ocxl_function_open(struct pci_dev *dev); struct list_head *ocxl_function_afu_list(struct ocxl_fn *fn); /** - * Fetch an AFU instance from an OpenCAPI function - * + * ocxl_function_fetch_afu() - Fetch an AFU instance from an OpenCAPI function * @fn: The OpenCAPI function to get the AFU from * @afu_idx: The index of the AFU to get * @@ -92,23 +89,20 @@ struct list_head *ocxl_function_afu_list(struct ocxl_fn *fn); struct ocxl_afu *ocxl_function_fetch_afu(struct ocxl_fn *fn, u8 afu_idx); /** - * Take a reference to an AFU - * + * ocxl_afu_get() - Take a reference to an AFU * @afu: The AFU to increment the reference count on */ void ocxl_afu_get(struct ocxl_afu *afu); /** - * Release a reference to an AFU - * + * ocxl_afu_put() - Release a reference to an AFU * @afu: The AFU to decrement the reference count on */ void ocxl_afu_put(struct ocxl_afu *afu); /** - * Get the configuration information for an OpenCAPI function - * + * ocxl_function_config() - Get the configuration information for an OpenCAPI function * @fn: The OpenCAPI function to get the config for * * Returns the function config, or NULL on error @@ -116,8 +110,7 @@ void ocxl_afu_put(struct ocxl_afu *afu); const struct ocxl_fn_config *ocxl_function_config(struct ocxl_fn *fn); /** - * Close an OpenCAPI function - * + * ocxl_function_close() - Close an OpenCAPI function * This will free any AFUs previously retrieved from the function, and * detach and associated contexts. The contexts must by freed by the caller. * @@ -129,8 +122,7 @@ void ocxl_function_close(struct ocxl_fn *fn); // Context allocation /** - * Allocate an OpenCAPI context - * + * ocxl_context_alloc() - Allocate an OpenCAPI context * @context: The OpenCAPI context to allocate, must be freed with ocxl_context_free * @afu: The AFU the context belongs to * @mapping: The mapping to unmap when the context is closed (may be NULL) @@ -139,14 +131,13 @@ int ocxl_context_alloc(struct ocxl_context **context, struct ocxl_afu *afu, struct address_space *mapping); /** - * Free an OpenCAPI context - * + * ocxl_context_free() - Free an OpenCAPI context * @ctx: The OpenCAPI context to free */ void ocxl_context_free(struct ocxl_context *ctx); /** - * Grant access to an MM to an OpenCAPI context + * ocxl_context_attach() - Grant access to an MM to an OpenCAPI context * @ctx: The OpenCAPI context to attach * @amr: The value of the AMR register to restrict access * @mm: The mm to attach to the context @@ -157,7 +148,7 @@ int ocxl_context_attach(struct ocxl_context *ctx, u64 amr, struct mm_struct *mm); /** - * Detach an MM from an OpenCAPI context + * ocxl_context_detach() - Detach an MM from an OpenCAPI context * @ctx: The OpenCAPI context to attach * * Returns 0 on success, negative on failure @@ -167,7 +158,7 @@ int ocxl_context_detach(struct ocxl_context *ctx); // AFU IRQs /** - * Allocate an IRQ associated with an AFU context + * ocxl_afu_irq_alloc() - Allocate an IRQ associated with an AFU context * @ctx: the AFU context * @irq_id: out, the IRQ ID * @@ -176,7 +167,7 @@ int ocxl_context_detach(struct ocxl_context *ctx); int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); /** - * Frees an IRQ associated with an AFU context + * ocxl_afu_irq_free() - Frees an IRQ associated with an AFU context * @ctx: the AFU context * @irq_id: the IRQ ID * @@ -185,7 +176,7 @@ int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); /** - * Gets the address of the trigger page for an IRQ + * ocxl_afu_irq_get_addr() - Gets the address of the trigger page for an IRQ * This can then be provided to an AFU which will write to that * page to trigger the IRQ. * @ctx: The AFU context that the IRQ is associated with @@ -196,7 +187,7 @@ int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); u64 ocxl_afu_irq_get_addr(struct ocxl_context *ctx, int irq_id); /** - * Provide a callback to be called when an IRQ is triggered + * ocxl_irq_set_handler() - Provide a callback to be called when an IRQ is triggered * @ctx: The AFU context that the IRQ is associated with * @irq_id: The IRQ ID * @handler: the callback to be called when the IRQ is triggered @@ -213,8 +204,7 @@ int ocxl_irq_set_handler(struct ocxl_context *ctx, int irq_id, // AFU Metadata /** - * Get a pointer to the config for an AFU - * + * ocxl_afu_config() - Get a pointer to the config for an AFU * @afu: a pointer to the AFU to get the config for * * Returns a pointer to the AFU config @@ -222,27 +212,24 @@ int ocxl_irq_set_handler(struct ocxl_context *ctx, int irq_id, struct ocxl_afu_config *ocxl_afu_config(struct ocxl_afu *afu); /** - * Assign opaque hardware specific information to an OpenCAPI AFU. - * - * @dev: The PCI device associated with the OpenCAPI device + * ocxl_afu_set_private() - Assign opaque hardware specific information to an OpenCAPI AFU. + * @afu: The OpenCAPI AFU * @private: the opaque hardware specific information to assign to the driver */ void ocxl_afu_set_private(struct ocxl_afu *afu, void *private); /** - * Fetch the hardware specific information associated with an external OpenCAPI - * AFU. This may be consumed by an external OpenCAPI driver. - * - * @afu: The AFU + * ocxl_afu_get_private() - Fetch the hardware specific information associated with + * an external OpenCAPI AFU. This may be consumed by an external OpenCAPI driver. + * @afu: The OpenCAPI AFU * * Returns the opaque pointer associated with the device, or NULL if not set */ -void *ocxl_afu_get_private(struct ocxl_afu *dev); +void *ocxl_afu_get_private(struct ocxl_afu *afu); // Global MMIO /** - * Read a 32 bit value from global MMIO - * + * ocxl_global_mmio_read32() - Read a 32 bit value from global MMIO * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -251,11 +238,10 @@ void *ocxl_afu_get_private(struct ocxl_afu *dev); * Returns 0 for success, negative on error */ int ocxl_global_mmio_read32(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u32 *val); + enum ocxl_endian endian, u32 *val); /** - * Read a 64 bit value from global MMIO - * + * ocxl_global_mmio_read64() - Read a 64 bit value from global MMIO * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -264,11 +250,10 @@ int ocxl_global_mmio_read32(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_read64(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u64 *val); + enum ocxl_endian endian, u64 *val); /** - * Write a 32 bit value to global MMIO - * + * ocxl_global_mmio_write32() - Write a 32 bit value to global MMIO * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -277,11 +262,10 @@ int ocxl_global_mmio_read64(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_write32(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u32 val); + enum ocxl_endian endian, u32 val); /** - * Write a 64 bit value to global MMIO - * + * ocxl_global_mmio_write64() - Write a 64 bit value to global MMIO * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -290,11 +274,10 @@ int ocxl_global_mmio_write32(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_write64(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u64 val); + enum ocxl_endian endian, u64 val); /** - * Set bits in a 32 bit global MMIO register - * + * ocxl_global_mmio_set32() - Set bits in a 32 bit global MMIO register * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -303,11 +286,10 @@ int ocxl_global_mmio_write64(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_set32(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u32 mask); + enum ocxl_endian endian, u32 mask); /** - * Set bits in a 64 bit global MMIO register - * + * ocxl_global_mmio_set64() - Set bits in a 64 bit global MMIO register * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -316,11 +298,10 @@ int ocxl_global_mmio_set32(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_set64(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u64 mask); + enum ocxl_endian endian, u64 mask); /** - * Set bits in a 32 bit global MMIO register - * + * ocxl_global_mmio_clear32() - Set bits in a 32 bit global MMIO register * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -329,11 +310,10 @@ int ocxl_global_mmio_set64(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_clear32(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u32 mask); + enum ocxl_endian endian, u32 mask); /** - * Set bits in a 64 bit global MMIO register - * + * ocxl_global_mmio_clear64() - Set bits in a 64 bit global MMIO register * @afu: The AFU * @offset: The Offset from the start of MMIO * @endian: the endianness that the MMIO data is in @@ -342,7 +322,7 @@ int ocxl_global_mmio_clear32(struct ocxl_afu *afu, size_t offset, * Returns 0 for success, negative on error */ int ocxl_global_mmio_clear64(struct ocxl_afu *afu, size_t offset, - enum ocxl_endian endian, u64 mask); + enum ocxl_endian endian, u64 mask); // Functions left here are for compatibility with the cxlflash driver From patchwork Fri Mar 27 07:11:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464655 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0729692C for ; Mon, 30 Mar 2020 05:53:14 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE97E20786 for ; Mon, 30 Mar 2020 05:53:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="AUp+mlMk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE97E20786 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9D45010FC3791; Sun, 29 Mar 2020 22:54:03 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id D7C5A10FC3787 for ; Sun, 29 Mar 2020 22:54:01 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 0CD9B2DC685F; Mon, 30 Mar 2020 16:52:52 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547573; bh=JCkxqdhDpbqnh8PRykYbOapzLDhQpaFdQTq2avtnUbk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AUp+mlMku4qydmYRmIQAz0mHtgW/QWdocLSpZmVj8gEVvUW0kV4JR5lWebXhN5UXf S45Os+VecBpgDQFNUPlPIYXuVSFytq7eOZr/9Mr+wCXi23AfDkwqy/5i9Ey1707t4r innVeW33PDa3HOA9nzLJ9T7Q2SdCOzUdxdWbtW9lRGfaUN1sesyWSQR+lxlim45wvg O5+qPjj4wXBAhnO0PLxbtzB7b9YN4wZDlQflV6mMgXniUrHrXz1RhmBYlopcvdKHP7 kEC0wemMdjz4K6l2g/eSA7yNMIqeRV6CnbDTcgzgK9uoyA0K4NgSEJkG13xKQx8IbT 2eX40QC7fIyb7bgjbYd9UG0oWR+Cmh7SkzJBz/VbMzOIykZgXgMFkq/Dfq/mpzPcTl f5PlgYNRzlM262ad4hILfnl/danLaHgpo1/mFyhwHYVDyKTu6vCwl62EwA0Lhk/UPn T58ZhVy9t7ddJN+Ws/1u/Lqzpird3mG83JGhn0JkjW+WaYswOWFx1HLxKPNqHrMK/J NvHprrmROvUqRV9pSaOWik29SdcxMd+24+KWQZWF4NMNavF9XMw69bhjFIsAUHrC7l oWQLrYbGwAd1EN+0gwYjA3nQ4ygL0t7JOlBEleXmRayk0do9PBVp1k4lUab+TbmNlC wjaYarf5IsxL+Nzdn4fhC0VI= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Af045934; Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 06/25] ocxl: Tally up the LPC memory on a link & allow it to be mapped Date: Fri, 27 Mar 2020 18:11:43 +1100 Message-Id: <20200327071202.2159885-7-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) Message-ID-Hash: L4RLPEQAQTJCVIYTU5BDZ7T2ODK2O2RC X-Message-ID-Hash: L4RLPEQAQTJCVIYTU5BDZ7T2ODK2O2RC X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: OpenCAPI LPC memory is allocated per link, but each link supports multiple AFUs, and each AFU can have LPC memory assigned to it. This patch tallys the memory for all AFUs on a link, allowing it to be mapped in a single operation after the AFUs have been enumerated. Signed-off-by: Alastair D'Silva --- drivers/misc/ocxl/core.c | 10 ++++++ drivers/misc/ocxl/link.c | 60 +++++++++++++++++++++++++++++++ drivers/misc/ocxl/ocxl_internal.h | 33 +++++++++++++++++ 3 files changed, 103 insertions(+) diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c index b7a09b21ab36..2531c6cf19a0 100644 --- a/drivers/misc/ocxl/core.c +++ b/drivers/misc/ocxl/core.c @@ -230,8 +230,18 @@ static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev) if (rc) goto err_free_pasid; + if (afu->config.lpc_mem_size || afu->config.special_purpose_mem_size) { + rc = ocxl_link_add_lpc_mem(afu->fn->link, afu->config.lpc_mem_offset, + afu->config.lpc_mem_size + + afu->config.special_purpose_mem_size); + if (rc) + goto err_free_mmio; + } + return 0; +err_free_mmio: + unmap_mmio_areas(afu); err_free_pasid: reclaim_afu_pasid(afu); err_free_actag: diff --git a/drivers/misc/ocxl/link.c b/drivers/misc/ocxl/link.c index 58d111afd9f6..af119d3ef79a 100644 --- a/drivers/misc/ocxl/link.c +++ b/drivers/misc/ocxl/link.c @@ -84,6 +84,11 @@ struct ocxl_link { int dev; atomic_t irq_available; struct spa *spa; + struct mutex lpc_mem_lock; /* protects lpc_mem & lpc_mem_sz */ + u64 lpc_mem_sz; /* Total amount of LPC memory presented on the link */ + u64 lpc_mem; + int lpc_consumers; + void *platform_data; }; static struct list_head links_list = LIST_HEAD_INIT(links_list); @@ -396,6 +401,8 @@ static int alloc_link(struct pci_dev *dev, int PE_mask, struct ocxl_link **out_l if (rc) goto err_spa; + mutex_init(&link->lpc_mem_lock); + /* platform specific hook */ rc = pnv_ocxl_spa_setup(dev, link->spa->spa_mem, PE_mask, &link->platform_data); @@ -711,3 +718,56 @@ void ocxl_link_free_irq(void *link_handle, int hw_irq) atomic_inc(&link->irq_available); } EXPORT_SYMBOL_GPL(ocxl_link_free_irq); + +int ocxl_link_add_lpc_mem(void *link_handle, u64 offset, u64 size) +{ + struct ocxl_link *link = (struct ocxl_link *)link_handle; + + // Check for overflow + if (offset > (offset + size)) + return -EINVAL; + + mutex_lock(&link->lpc_mem_lock); + link->lpc_mem_sz = max(link->lpc_mem_sz, offset + size); + + mutex_unlock(&link->lpc_mem_lock); + + return 0; +} + +u64 ocxl_link_lpc_map(void *link_handle, struct pci_dev *pdev) +{ + struct ocxl_link *link = (struct ocxl_link *)link_handle; + + mutex_lock(&link->lpc_mem_lock); + + if (!link->lpc_mem) + link->lpc_mem = pnv_ocxl_platform_lpc_setup(pdev, link->lpc_mem_sz); + + if (link->lpc_mem) + link->lpc_consumers++; + mutex_unlock(&link->lpc_mem_lock); + + return link->lpc_mem; +} + +void ocxl_link_lpc_release(void *link_handle, struct pci_dev *pdev) +{ + struct ocxl_link *link = (struct ocxl_link *)link_handle; + + mutex_lock(&link->lpc_mem_lock); + + if (!link->lpc_mem) { + mutex_unlock(&link->lpc_mem_lock); + return; + } + + WARN_ON(--link->lpc_consumers < 0); + + if (link->lpc_consumers == 0) { + pnv_ocxl_platform_lpc_release(pdev); + link->lpc_mem = 0; + } + + mutex_unlock(&link->lpc_mem_lock); +} diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h index 198e4e4bc51d..2d7575225bd7 100644 --- a/drivers/misc/ocxl/ocxl_internal.h +++ b/drivers/misc/ocxl/ocxl_internal.h @@ -142,4 +142,37 @@ int ocxl_irq_offset_to_id(struct ocxl_context *ctx, u64 offset); u64 ocxl_irq_id_to_offset(struct ocxl_context *ctx, int irq_id); void ocxl_afu_irq_free_all(struct ocxl_context *ctx); +/** + * ocxl_link_add_lpc_mem() - Increment the amount of memory required by an OpenCAPI link + * + * @link_handle: The OpenCAPI link handle + * @offset: The offset of the memory to add + * @size: The number of bytes to increment memory on the link by + * + * Returns 0 on success, -EINVAL on overflow + */ +int ocxl_link_add_lpc_mem(void *link_handle, u64 offset, u64 size); + +/** + * ocxl_link_lpc_map() - Map the LPC memory for an OpenCAPI device + * Since LPC memory belongs to a link, the whole LPC memory available + * on the link must be mapped in order to make it accessible to a device. + * @link_handle: The OpenCAPI link handle + * @pdev: A device that is on the link + * + * Returns the address of the mapped LPC memory, or 0 on error + */ +u64 ocxl_link_lpc_map(void *link_handle, struct pci_dev *pdev); + +/** + * ocxl_link_lpc_release() - Release the LPC memory device for an OpenCAPI device + * + * Offlines LPC memory on an OpenCAPI link for a device. If this is the + * last device on the link to release the memory, unmap it from the link. + * + * @link_handle: The OpenCAPI link handle + * @pdev: A device that is on the link + */ +void ocxl_link_lpc_release(void *link_handle, struct pci_dev *pdev); + #endif /* _OCXL_INTERNAL_H_ */ From patchwork Fri Mar 27 07:11:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461799 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 93D761668 for ; Fri, 27 Mar 2020 07:12:27 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 732E420714 for ; Fri, 27 Mar 2020 07:12:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="l0xv6Qkr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 732E420714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id E8FAD10FC3BA6; Fri, 27 Mar 2020 00:13:15 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id D6F7410FC3614 for ; Fri, 27 Mar 2020 00:13:11 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 738C32DC6829; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=iql+P5xKXAK7D5i2p3DQ+jr9oIISWrdh7woNYcXPetQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l0xv6Qkr8VzM9ZG5lMS/tOu1JFQYeHTOxng+UyItI7K+BTfAdHAn5jl0Vc98MdH5L svJHiChQtX5o8cTO3dd/fdihbcTq1JBv0lcp1/0c3Ilk5aLLpJOnQ1FQ15klXidcI9 HzlXm5cZbwJbsCl6Byo4R9vTIo8oGh+UAYH/OsCvx1PSCVjVWx1x/VXegbQyjMOR51 V4LP4dGgNfczNXy3uzX+jsLMyTr269zBNIgIk44+ayuhFugOXkoTzpijNadY64HMyY 8BwDM3tMr1Bi5xMuveddrROnBXS/aYhpLzKyHqCTZjwaY0YlGyMCNhgkHXqdKkM8/5 0ZIjsg3Sg0ou01WLekF/gpkzxV28dn8yvkIO0OcGrhzyxu4kmlzPyFEp1rskt5UCkv N9C0oEHQG0GJR+e65zOkTmC8UkwdqUsHqI9kYTpXuJcrbPDSDPrsEljoPuI/NAibsH GIm4KPDgwMdhe2CXHCLovQFSBQww6wbR+rGrzrWcwkr7P7ps+fjPRJYXnZjA8AkvVk L//ADFtFERH1s3KlT4Xs2wPEdyiR6A3bFiTqt/0itt2K+YOTFn/gufrnwp3JV/eQJ5 XLE9vu40yNlig92V8LtkILAEFfP0NQDPOGhMQca160KxhQYz8l7pPQdHYe+9lxRZBJ DGCfKUohqC6Ydh4+l1CuVtcI= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ag045934; Fri, 27 Mar 2020 18:12:15 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 07/25] ocxl: Add functions to map/unmap LPC memory Date: Fri, 27 Mar 2020 18:11:44 +1100 Message-Id: <20200327071202.2159885-8-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) Message-ID-Hash: QUFYV3PYVQB5UDLFGEYTLZIAU3UW7UK2 X-Message-ID-Hash: QUFYV3PYVQB5UDLFGEYTLZIAU3UW7UK2 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Add functions to map/unmap LPC memory Signed-off-by: Alastair D'Silva Acked-by: Frederic Barrat --- drivers/misc/ocxl/core.c | 51 +++++++++++++++++++++++++++++++ drivers/misc/ocxl/ocxl_internal.h | 3 ++ include/misc/ocxl.h | 21 +++++++++++++ 3 files changed, 75 insertions(+) diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c index 2531c6cf19a0..75ff14e3882a 100644 --- a/drivers/misc/ocxl/core.c +++ b/drivers/misc/ocxl/core.c @@ -210,6 +210,56 @@ static void unmap_mmio_areas(struct ocxl_afu *afu) release_fn_bar(afu->fn, afu->config.global_mmio_bar); } +int ocxl_afu_map_lpc_mem(struct ocxl_afu *afu) +{ + struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent); + + if ((afu->config.lpc_mem_size + afu->config.special_purpose_mem_size) == 0) + return 0; + + afu->lpc_base_addr = ocxl_link_lpc_map(afu->fn->link, dev); + if (afu->lpc_base_addr == 0) + return -EINVAL; + + if (afu->config.lpc_mem_size > 0) { + afu->lpc_res.start = afu->lpc_base_addr + afu->config.lpc_mem_offset; + afu->lpc_res.end = afu->lpc_res.start + afu->config.lpc_mem_size - 1; + } + + if (afu->config.special_purpose_mem_size > 0) { + afu->special_purpose_res.start = afu->lpc_base_addr + + afu->config.special_purpose_mem_offset; + afu->special_purpose_res.end = afu->special_purpose_res.start + + afu->config.special_purpose_mem_size - 1; + } + + return 0; +} +EXPORT_SYMBOL_GPL(ocxl_afu_map_lpc_mem); + +struct resource *ocxl_afu_lpc_mem(struct ocxl_afu *afu) +{ + return &afu->lpc_res; +} +EXPORT_SYMBOL_GPL(ocxl_afu_lpc_mem); + +static void unmap_lpc_mem(struct ocxl_afu *afu) +{ + struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent); + + if (afu->lpc_res.start || afu->special_purpose_res.start) { + void *link = afu->fn->link; + + // only release the link when the the last consumer calls release + ocxl_link_lpc_release(link, dev); + + afu->lpc_res.start = 0; + afu->lpc_res.end = 0; + afu->special_purpose_res.start = 0; + afu->special_purpose_res.end = 0; + } +} + static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev) { int rc; @@ -251,6 +301,7 @@ static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev) static void deconfigure_afu(struct ocxl_afu *afu) { + unmap_lpc_mem(afu); unmap_mmio_areas(afu); reclaim_afu_pasid(afu); reclaim_afu_actag(afu); diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h index 2d7575225bd7..7b975a89db7b 100644 --- a/drivers/misc/ocxl/ocxl_internal.h +++ b/drivers/misc/ocxl/ocxl_internal.h @@ -52,6 +52,9 @@ struct ocxl_afu { void __iomem *global_mmio_ptr; u64 pp_mmio_start; void *private; + u64 lpc_base_addr; /* Covers both LPC & special purpose memory */ + struct resource lpc_res; + struct resource special_purpose_res; }; enum ocxl_context_status { diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index 357ef1aadbc0..d8b0b4d46bfb 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -203,6 +203,27 @@ int ocxl_irq_set_handler(struct ocxl_context *ctx, int irq_id, // AFU Metadata +/** + * ocxl_afu_map_lpc_mem() - Map the LPC system & special purpose memory for an AFU + * Do not call this during device discovery, as there may me multiple + * devices on a link, and the memory is mapped for the whole link, not + * just one device. It should only be called after all devices have + * registered their memory on the link. + * + * @afu: The AFU that has the LPC memory to map + * + * Returns 0 on success, negative on failure + */ +int ocxl_afu_map_lpc_mem(struct ocxl_afu *afu); + +/** + * ocxl_afu_lpc_mem() - Get the physical address range of LPC memory for an AFU + * @afu: The AFU associated with the LPC memory + * + * Returns a pointer to the resource struct for the physical address range + */ +struct resource *ocxl_afu_lpc_mem(struct ocxl_afu *afu); + /** * ocxl_afu_config() - Get a pointer to the config for an AFU * @afu: a pointer to the AFU to get the config for From patchwork Fri Mar 27 07:11:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8600A1667 for ; Fri, 27 Mar 2020 07:12:31 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 64ED620705 for ; Fri, 27 Mar 2020 07:12:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="gd3j0Duo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 64ED620705 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 0AFDB10FC3BAF; Fri, 27 Mar 2020 00:13:16 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id DA7EC10FC36EE for ; Fri, 27 Mar 2020 00:13:11 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 5E12D2DC67FE; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=bEfzXRR0cTBf+yBtWTvEdtXlD5xDUP18rOvIDgiLves=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gd3j0DuoeTtlWmyHUBUazlQ1VFusMIA9aMaYhizfgDpSKt92Sy7qNXsG96bPw/XxL 9QhMWe73iJYRcKatLqQyH20jrk2zgVkqk6EZ71FermuqUedtDX+/a0Nl0KdpTvyQGX TI7AMhz5XxEnv3yAqHVIW9qqxXNSXJloUqB4fVEgjJLPiAyElMs9/sGcUgkKv9wRiT pVzWhJmqwR4it0c5Jr1sRPCU+d7RO8j9kbV60yXlW6plmXpocT8DiJaRVac54Kl5bx iykZ94QQC1lJiJhk6vSgbqe3hvFn4jc4B7ggDCo3TAcwMB0jDRWxoQ9t48F2xrqoxc Uc7oM3knlUNd38QNUCVGzKSnmF0yYLUaLZOpbB6nKsaWWmft6E6+3oQTcHtQjB8Ja/ f9bmXSMg7fOd70wY0ezYnuB6dFA38AdcPEhax50weTdofK/DbxqRRk+Nn/8pWP3THe yfQFCXqYYY7dSVktejI9B3aDo17PWhaE3+ZCC1FDLqmXe9TesqrQPPeQ8B/8Lnk3jm +jdLoxFPBf78IBZA6+JcRjRAKDCxDsrWUAW8qhRPf595Us6aILG5DUC852KzjPL4me SGVCPNuhRq371astBd/3M8cq2ByQeoJvs2+NtXwgbRW4GSgBLQwfL719g3fE041XB8 93xnLU1sO4dAMbvZvsnBeSS8= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ah045934; Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 08/25] ocxl: Emit a log message showing how much LPC memory was detected Date: Fri, 27 Mar 2020 18:11:45 +1100 Message-Id: <20200327071202.2159885-9-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) Message-ID-Hash: U3DGGMOIJDBO3HATEQV3IXSMQMB6HHXX X-Message-ID-Hash: U3DGGMOIJDBO3HATEQV3IXSMQMB6HHXX X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch emits a message showing how much LPC memory & special purpose memory was detected on an OCXL device. Signed-off-by: Alastair D'Silva Acked-by: Frederic Barrat Acked-by: Andrew Donnellan --- drivers/misc/ocxl/config.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c index a62e3d7db2bf..69cca341d446 100644 --- a/drivers/misc/ocxl/config.c +++ b/drivers/misc/ocxl/config.c @@ -568,6 +568,10 @@ static int read_afu_lpc_memory_info(struct pci_dev *dev, afu->special_purpose_mem_size = total_mem_size - lpc_mem_size; } + + dev_info(&dev->dev, "Probed LPC memory of %#llx bytes and special purpose memory of %#llx bytes\n", + afu->lpc_mem_size, afu->special_purpose_mem_size); + return 0; } From patchwork Fri Mar 27 07:11:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461795 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DAB81668 for ; Fri, 27 Mar 2020 07:12:25 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CBDB20714 for ; Fri, 27 Mar 2020 07:12:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="e6a1tDVW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5CBDB20714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id CAF8C10FC3BA5; Fri, 27 Mar 2020 00:13:14 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id BFFE410FC36EE for ; Fri, 27 Mar 2020 00:13:11 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id BA61C2DC682C; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=HzmhbcogFky5JzzjdOPameps3zKSn5YAqkFn77Y+Dc4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e6a1tDVWViG1+jiUIoTsiKrTGAyowuWFrA0PCTXk1vQKOWbpPReJad1CLZS9xKu8k EFRmZjIT9KBKqhJkrXzSnapItH1lRVZy7d4h5SogWLOpKIlosNd+cPQnK9TV3acuos zvH+Z0Jlut8IdWJOA4nF0J+YX9ygI9d5FS8ft8Ug6UxQqC8bWv6iiR3IjAo4/tCohC q6050U+9dAQa8V/fLtJsr4zD3J9cetIFwB2t9713wyqkcLVixRwfD1DhTZ6rrH9Gyv 3fE9mD/sde6wZE8pV1MUt+saA/POMeHIwdVv6C+e47d/WERR3wVblA+1tpoOAZ2brc QUZbjNYVZRNoVsybJoN8DiEnEeoxsuQStYY6tLmid9RvXWGFSoH3cdanT8y1/OqKLe 46GQ4nBoOr41PkguF0htNeCUAgxiuWpgTfoTavWhM1SgpEVR/EyN7kN9W7/S1Dcbo0 Qe7o34WtZWKvlH5TdnhdnIt9/lMP7k8XveBbJuZMAJZ2Cv1YDeBTlAVY+huyKrNSMD SiWAECLmjomCHrfZsez028Ik+Ie2blO70vTcKyUoMtX8jHzd2vInEfjMo8B5jgYIbN MvO4F6oDy8dvsMAqHvNygUygzyX0lLS6O/099IdPj6gQvdo5SHyLURRfpB+Orwi2kV n8/DM6ski3PsH/O+rPuOm8L8= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ai045934; Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 09/25] ocxl: Save the device serial number in ocxl_fn Date: Fri, 27 Mar 2020 18:11:46 +1100 Message-Id: <20200327071202.2159885-10-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) Message-ID-Hash: X2M6FS4LAXDXEE4TPKECKGSI7VY5SPTZ X-Message-ID-Hash: X2M6FS4LAXDXEE4TPKECKGSI7VY5SPTZ X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch retrieves the serial number of the card and makes it available to consumers of the ocxl driver via the ocxl_fn struct. Signed-off-by: Alastair D'Silva Acked-by: Frederic Barrat Acked-by: Andrew Donnellan --- drivers/misc/ocxl/config.c | 46 ++++++++++++++++++++++++++++++++++++++ include/misc/ocxl.h | 1 + 2 files changed, 47 insertions(+) diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c index 69cca341d446..2b835e57daca 100644 --- a/drivers/misc/ocxl/config.c +++ b/drivers/misc/ocxl/config.c @@ -71,6 +71,51 @@ static int find_dvsec_afu_ctrl(struct pci_dev *dev, u8 afu_idx) return 0; } +/** + * get_function_0() - Find a related PCI device (function 0) + * @device: PCI device to match + * + * Returns a pointer to the related device, or null if not found + */ +static struct pci_dev *get_function_0(struct pci_dev *dev) +{ + unsigned int devfn = PCI_DEVFN(PCI_SLOT(dev->devfn), 0); + + return pci_get_domain_bus_and_slot(pci_domain_nr(dev->bus), + dev->bus->number, devfn); +} + +static void read_serial(struct pci_dev *dev, struct ocxl_fn_config *fn) +{ + u32 low, high; + int pos; + + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DSN); + if (pos) { + pci_read_config_dword(dev, pos + 0x04, &low); + pci_read_config_dword(dev, pos + 0x08, &high); + + fn->serial = low | ((u64)high) << 32; + + return; + } + + if (PCI_FUNC(dev->devfn) != 0) { + struct pci_dev *related = get_function_0(dev); + + if (!related) { + fn->serial = 0; + return; + } + + read_serial(related, fn); + pci_dev_put(related); + return; + } + + fn->serial = 0; +} + static void read_pasid(struct pci_dev *dev, struct ocxl_fn_config *fn) { u16 val; @@ -208,6 +253,7 @@ int ocxl_config_read_function(struct pci_dev *dev, struct ocxl_fn_config *fn) int rc; read_pasid(dev, fn); + read_serial(dev, fn); rc = read_dvsec_tl(dev, fn); if (rc) { diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index d8b0b4d46bfb..b8514dc64bd0 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -46,6 +46,7 @@ struct ocxl_fn_config { int dvsec_afu_info_pos; /* offset of the AFU information DVSEC */ s8 max_pasid_log; s8 max_afu_index; + u64 serial; }; enum ocxl_endian { From patchwork Fri Mar 27 07:11:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461817 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3132A1668 for ; Fri, 27 Mar 2020 07:12:40 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0F45C20717 for ; Fri, 27 Mar 2020 07:12:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="F4h29DRh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F45C20717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id BCBA510FC3BCB; Fri, 27 Mar 2020 00:13:18 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 6B4CE10FC3614 for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id AE1BE2DC682B; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=wz/42EdWYu+4cxF4TVIz60iDaZdQN/2nycuXkQuthlY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F4h29DRhjvN8N3XO0FtfABSCnVYfNMjo6BubHZ31RTohvOub6kanPQvJzFi5EIv/5 AE/6Mqi81qb6pGsJCAuY1XqXjYTUSdekKFVIbBZ7PBc42TC1aCQOVGKrqhQsGus/jE oq3I39fAuWJn+kXL/kdJYqHtJHiK0UBSHC2Fudu10FHe5/b9dR68M2qUDuzvGq3fU+ +lffHLLHf/GZxbGuPd+6DvrD//6aGGd2Dukuo5C4aBLOgj1TvyYe3/G9izTcynMEvM syYwM88/QXyNWjHangopg8qZwH5VrhxPaVGkLqPRNw8K0UjaY7e9L+jci94ovAhHpw yd4ubXwNXbW65Q4+wAipn1dqfISbfuzKc5qzY0K6M53u7qEBuU7MLHe9gemcSl/798 cGmtU1MHUqqp7CaVkc5y/FuES2PzU6GPKlD93l8FpMy57ROLLp285y5Hej/3Hxbpu5 RuTgabj+Qg9KLE1xYGWUrUDaAMg/bqhhQ3D7WS2NbkpVbzyIUtfhpuXnkmNWFk4qBw 6YVtf4jpdztL56DZFZNdrSGkmKMi64dTjZaRYKvY14TWCvb1aBsYxcVIKqpSzMULGJ t1hJsHyYdzzhWDrBQWv/gSBNOW/2HyB6l2Hp3GdiCYR4zRq0rejFhnr56V66GylcyE BwVscwQ9Oh0gX8sDpWdx2DaQ= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Aj045934; Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 10/25] nvdimm: Add driver for OpenCAPI Persistent Memory Date: Fri, 27 Mar 2020 18:11:47 +1100 Message-Id: <20200327071202.2159885-11-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) Message-ID-Hash: YUEMGLIZ772HBRPGQK2QE5RNCWYKVZPS X-Message-ID-Hash: YUEMGLIZ772HBRPGQK2QE5RNCWYKVZPS X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This driver exposes LPC memory on OpenCAPI pmem cards as an NVDIMM, allowing the existing nvram infrastructure to be used. Namespace metadata is stored on the media itself, so scm_reserve_metadata() maps 1 section's worth of PMEM storage at the start to hold this. The rest of the PMEM range is registered with libnvdimm as an nvdimm. ndctl_config_read/write/size() provide callbacks to libnvdimm to access the metadata. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/Kconfig | 2 + drivers/nvdimm/Makefile | 1 + drivers/nvdimm/ocxl/Kconfig | 15 ++ drivers/nvdimm/ocxl/Makefile | 7 + drivers/nvdimm/ocxl/main.c | 476 +++++++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/ocxlpmem.h | 23 ++ 6 files changed, 524 insertions(+) create mode 100644 drivers/nvdimm/ocxl/Kconfig create mode 100644 drivers/nvdimm/ocxl/Makefile create mode 100644 drivers/nvdimm/ocxl/main.c create mode 100644 drivers/nvdimm/ocxl/ocxlpmem.h diff --git a/drivers/nvdimm/Kconfig b/drivers/nvdimm/Kconfig index b7d1eb38b27d..368328637182 100644 --- a/drivers/nvdimm/Kconfig +++ b/drivers/nvdimm/Kconfig @@ -131,4 +131,6 @@ config NVDIMM_TEST_BUILD core devm_memremap_pages() implementation and other infrastructure. +source "drivers/nvdimm/ocxl/Kconfig" + endif diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile index 29203f3d3069..bc02be11c794 100644 --- a/drivers/nvdimm/Makefile +++ b/drivers/nvdimm/Makefile @@ -33,3 +33,4 @@ libnvdimm-$(CONFIG_NVDIMM_KEYS) += security.o TOOLS := ../../tools TEST_SRC := $(TOOLS)/testing/nvdimm/test obj-$(CONFIG_NVDIMM_TEST_BUILD) += $(TEST_SRC)/iomap.o +obj-$(CONFIG_LIBNVDIMM) += ocxl/ diff --git a/drivers/nvdimm/ocxl/Kconfig b/drivers/nvdimm/ocxl/Kconfig new file mode 100644 index 000000000000..c5d927520920 --- /dev/null +++ b/drivers/nvdimm/ocxl/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +if LIBNVDIMM + +config OCXL_PMEM + tristate "OpenCAPI Persistent Memory" + depends on LIBNVDIMM && PPC_POWERNV && PCI && EEH && ZONE_DEVICE && OCXL + help + Exposes devices that implement the OpenCAPI Storage Class Memory + specification as persistent memory regions. You may also want + DEV_DAX, DEV_DAX_PMEM & FS_DAX if you plan on using DAX devices + stacked on top of this driver. + + Select N if unsure. + +endif diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile new file mode 100644 index 000000000000..e0e8ade1987a --- /dev/null +++ b/drivers/nvdimm/ocxl/Makefile @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0 + +ccflags-$(CONFIG_PPC_WERROR) += -Werror + +obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o + +ocxlpmem-y := main.o \ No newline at end of file diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c new file mode 100644 index 000000000000..c0066fedf9cc --- /dev/null +++ b/drivers/nvdimm/ocxl/main.c @@ -0,0 +1,476 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2020 IBM Corp. + +/* + * A driver for OpenCAPI devices that implement the Storage Class + * Memory specification. + */ + +#include +#include +#include +#include +#include +#include "ocxlpmem.h" + +static const struct pci_device_id pci_tbl[] = { + { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x0625), }, + { } +}; + +MODULE_DEVICE_TABLE(pci, pci_tbl); + +#define NUM_MINORS 256 // Total to reserve + +static dev_t ocxlpmem_dev; +static struct class *ocxlpmem_class; +static struct mutex minors_idr_lock; +static struct idr minors_idr; + +/** + * ndctl_config_write() - Handle a ND_CMD_SET_CONFIG_DATA command from ndctl + * @ocxlpmem: the device metadata + * @command: the incoming data to write + * Return: 0 on success, negative on failure + */ +static int ndctl_config_write(struct ocxlpmem *ocxlpmem, + struct nd_cmd_set_config_hdr *command) +{ + if (command->in_offset + command->in_length > LABEL_AREA_SIZE) + return -EINVAL; + + memcpy_flushcache(ocxlpmem->metadata_addr + command->in_offset, + command->in_buf, command->in_length); + + return 0; +} + +/** + * ndctl_config_read() - Handle a ND_CMD_GET_CONFIG_DATA command from ndctl + * @ocxlpmem: the device metadata + * @command: the read request + * Return: 0 on success, negative on failure + */ +static int ndctl_config_read(struct ocxlpmem *ocxlpmem, + struct nd_cmd_get_config_data_hdr *command) +{ + if (command->in_offset + command->in_length > LABEL_AREA_SIZE) + return -EINVAL; + + memcpy_mcsafe(command->out_buf, + ocxlpmem->metadata_addr + command->in_offset, + command->in_length); + + return 0; +} + +/** + * ndctl_config_size() - Handle a ND_CMD_GET_CONFIG_SIZE command from ndctl + * @command: the read request + * Return: 0 on success, negative on failure + */ +static int ndctl_config_size(struct nd_cmd_get_config_size *command) +{ + command->status = 0; + command->config_size = LABEL_AREA_SIZE; + command->max_xfer = PAGE_SIZE; + + return 0; +} + +static int ndctl(struct nvdimm_bus_descriptor *nd_desc, + struct nvdimm *nvdimm, + unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) +{ + struct ocxlpmem *ocxlpmem = container_of(nd_desc, + struct ocxlpmem, bus_desc); + + switch (cmd) { + case ND_CMD_GET_CONFIG_SIZE: + *cmd_rc = ndctl_config_size(buf); + return 0; + + case ND_CMD_GET_CONFIG_DATA: + *cmd_rc = ndctl_config_read(ocxlpmem, buf); + return 0; + + case ND_CMD_SET_CONFIG_DATA: + *cmd_rc = ndctl_config_write(ocxlpmem, buf); + return 0; + + default: + return -ENOTTY; + } +} + +/** + * reserve_metadata() - Reserve space for nvdimm metadata + * @ocxlpmem: the device metadata + * @lpc_mem: The resource representing the LPC memory of the OpenCAPI device + */ +static int reserve_metadata(struct ocxlpmem *ocxlpmem, + struct resource *lpc_mem) +{ + ocxlpmem->metadata_addr = devm_memremap(&ocxlpmem->dev, lpc_mem->start, + LABEL_AREA_SIZE, MEMREMAP_WB); + if (IS_ERR(ocxlpmem->metadata_addr)) + return PTR_ERR(ocxlpmem->metadata_addr); + + return 0; +} + +/** + * register_lpc_mem() - Discover persistent memory on a device and register it with the NVDIMM subsystem + * @ocxlpmem: the device metadata + * Return: 0 on success + */ +static int register_lpc_mem(struct ocxlpmem *ocxlpmem) +{ + struct nd_region_desc region_desc; + struct nd_mapping_desc nd_mapping_desc; + struct resource *lpc_mem; + const struct ocxl_afu_config *config; + const struct ocxl_fn_config *fn_config; + int rc; + unsigned long nvdimm_cmd_mask = 0; + unsigned long nvdimm_flags = 0; + int target_node; + char serial[16 + 1]; + + // Set up the reserved metadata area + rc = ocxl_afu_map_lpc_mem(ocxlpmem->ocxl_afu); + if (rc < 0) + return rc; + + lpc_mem = ocxl_afu_lpc_mem(ocxlpmem->ocxl_afu); + if (!lpc_mem || !lpc_mem->start) + return -EINVAL; + + config = ocxl_afu_config(ocxlpmem->ocxl_afu); + fn_config = ocxl_function_config(ocxlpmem->ocxl_fn); + + rc = reserve_metadata(ocxlpmem, lpc_mem); + if (rc) + return rc; + + ocxlpmem->bus_desc.provider_name = "ocxlpmem"; + ocxlpmem->bus_desc.ndctl = ndctl; + ocxlpmem->bus_desc.module = THIS_MODULE; + + ocxlpmem->nvdimm_bus = nvdimm_bus_register(&ocxlpmem->dev, + &ocxlpmem->bus_desc); + if (!ocxlpmem->nvdimm_bus) + return -EINVAL; + + ocxlpmem->pmem_res.start = (u64)lpc_mem->start + LABEL_AREA_SIZE; + ocxlpmem->pmem_res.end = (u64)lpc_mem->start + config->lpc_mem_size - 1; + ocxlpmem->pmem_res.name = "OpenCAPI persistent memory"; + + set_bit(ND_CMD_GET_CONFIG_SIZE, &nvdimm_cmd_mask); + set_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm_cmd_mask); + set_bit(ND_CMD_SET_CONFIG_DATA, &nvdimm_cmd_mask); + + set_bit(NDD_ALIASING, &nvdimm_flags); + + snprintf(serial, sizeof(serial), "%llx", fn_config->serial); + nd_mapping_desc.nvdimm = nvdimm_create(ocxlpmem->nvdimm_bus, ocxlpmem, + NULL, nvdimm_flags, + nvdimm_cmd_mask, 0, NULL); + if (!nd_mapping_desc.nvdimm) + return -ENOMEM; + + if (nvdimm_bus_check_dimm_count(ocxlpmem->nvdimm_bus, 1)) + return -EINVAL; + + nd_mapping_desc.start = ocxlpmem->pmem_res.start; + nd_mapping_desc.size = resource_size(&ocxlpmem->pmem_res); + nd_mapping_desc.position = 0; + + ocxlpmem->nd_set.cookie1 = fn_config->serial; + ocxlpmem->nd_set.cookie2 = fn_config->serial; + + target_node = of_node_to_nid(ocxlpmem->pdev->dev.of_node); + + memset(®ion_desc, 0, sizeof(region_desc)); + region_desc.res = &ocxlpmem->pmem_res; + region_desc.numa_node = NUMA_NO_NODE; + region_desc.target_node = target_node; + region_desc.num_mappings = 1; + region_desc.mapping = &nd_mapping_desc; + region_desc.nd_set = &ocxlpmem->nd_set; + + set_bit(ND_REGION_PAGEMAP, ®ion_desc.flags); + /* + * NB: libnvdimm copies the data from ndr_desc into it's own + * structures so passing a stack pointer is fine. + */ + ocxlpmem->nd_region = nvdimm_pmem_region_create(ocxlpmem->nvdimm_bus, + ®ion_desc); + if (!ocxlpmem->nd_region) + return -EINVAL; + + dev_info(&ocxlpmem->dev, + "Onlining %lluMB of persistent memory\n", + nd_mapping_desc.size / SZ_1M); + + return 0; +} + +/** + * allocate_minor() - Allocate a minor number to use for an OpenCAPI pmem device + * @ocxlpmem: the device metadata + * Return: the allocated minor number + */ +static int allocate_minor(struct ocxlpmem *ocxlpmem) +{ + int minor; + + mutex_lock(&minors_idr_lock); + minor = idr_alloc(&minors_idr, ocxlpmem, 0, NUM_MINORS, GFP_KERNEL); + mutex_unlock(&minors_idr_lock); + return minor; +} + +static void free_minor(struct ocxlpmem *ocxlpmem) +{ + mutex_lock(&minors_idr_lock); + idr_remove(&minors_idr, MINOR(ocxlpmem->dev.devt)); + mutex_unlock(&minors_idr_lock); +} + +/** + * free_ocxlpmem() - Free all members of an ocxlpmem struct + * @ocxlpmem: the device struct to clear + */ +static void free_ocxlpmem(struct ocxlpmem *ocxlpmem) +{ + int rc; + + free_minor(ocxlpmem); + + if (ocxlpmem->ocxl_context) { + rc = ocxl_context_detach(ocxlpmem->ocxl_context); + if (rc == -EBUSY) + dev_warn(&ocxlpmem->dev, "Timeout detaching ocxl context\n"); + else + ocxl_context_free(ocxlpmem->ocxl_context); + } + + if (ocxlpmem->ocxl_afu) + ocxl_afu_put(ocxlpmem->ocxl_afu); + + if (ocxlpmem->ocxl_fn) + ocxl_function_close(ocxlpmem->ocxl_fn); + + pci_dev_put(ocxlpmem->pdev); + + kfree(ocxlpmem); +} + +/** + * free_ocxlpmem_dev() - Free an OpenCAPI persistent memory device + * @dev: The device struct + */ +static void free_ocxlpmem_dev(struct device *dev) +{ + struct ocxlpmem *ocxlpmem = container_of(dev, struct ocxlpmem, dev); + + free_ocxlpmem(ocxlpmem); +} + +/** + * ocxlpmem_register() - Register an OpenCAPI pmem device with the kernel + * @ocxlpmem: the device metadata + * Return: 0 on success, negative on failure + */ +static int ocxlpmem_register(struct ocxlpmem *ocxlpmem) +{ + int rc; + int minor = allocate_minor(ocxlpmem); + + if (minor < 0) + return minor; + + ocxlpmem->dev.release = free_ocxlpmem_dev; + rc = dev_set_name(&ocxlpmem->dev, "ocxlpmem%d", minor); + if (rc < 0) + return rc; + + ocxlpmem->dev.devt = MKDEV(MAJOR(ocxlpmem_dev), minor); + ocxlpmem->dev.class = ocxlpmem_class; + ocxlpmem->dev.parent = &ocxlpmem->pdev->dev; + + return device_register(&ocxlpmem->dev); +} + +/** + * ocxlpmem_remove() - Free an OpenCAPI persistent memory device + * @pdev: the PCI device information struct + */ +static void remove(struct pci_dev *pdev) +{ + if (PCI_FUNC(pdev->devfn) == 0) { + struct ocxl_fn *func0 = pci_get_drvdata(pdev); + + if (func0) + ocxl_function_close(func0); + } else { + struct ocxlpmem *ocxlpmem = pci_get_drvdata(pdev); + + if (!ocxlpmem) + return; + + if (ocxlpmem->nvdimm_bus) + nvdimm_bus_unregister(ocxlpmem->nvdimm_bus); + + device_unregister(&ocxlpmem->dev); + } +} + +/** + * probe_function0() - Set up function 0 for an OpenCAPI persistent memory device + * This is important as it enables templates higher than 0 across all other + * functions, which in turn enables higher bandwidth accesses + * @pdev: the PCI device information struct + * Return: 0 on success, negative on failure + */ +static int probe_function0(struct pci_dev *pdev) +{ + struct ocxl_fn *fn; + + fn = ocxl_function_open(pdev); + if (IS_ERR(fn)) { + dev_err(&pdev->dev, "failed to open OCXL function\n"); + return PTR_ERR(fn); + } + + pci_set_drvdata(pdev, fn); + + return 0; +} + +/** + * probe() - Init an OpenCAPI persistent memory device + * @pdev: the PCI device information struct + * @ent: The entry from ocxlpmem_pci_tbl + * Return: 0 on success, negative on failure + */ +static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) +{ + struct ocxlpmem *ocxlpmem; + int rc; + + if (PCI_FUNC(pdev->devfn) == 0) + return probe_function0(pdev); + else if (PCI_FUNC(pdev->devfn) != 1) + return 0; + + ocxlpmem = kzalloc(sizeof(*ocxlpmem), GFP_KERNEL); + if (!ocxlpmem) { + rc = -ENOMEM; + goto err; + } + + ocxlpmem->pdev = pci_dev_get(pdev); + + pci_set_drvdata(pdev, ocxlpmem); + + ocxlpmem->ocxl_fn = ocxl_function_open(pdev); + if (IS_ERR(ocxlpmem->ocxl_fn)) { + rc = PTR_ERR(ocxlpmem->ocxl_fn); + dev_err(&pdev->dev, "failed to open OCXL function\n"); + goto err_unregistered; + } + + ocxlpmem->ocxl_afu = ocxl_function_fetch_afu(ocxlpmem->ocxl_fn, 0); + if (!ocxlpmem->ocxl_afu) { + rc = -ENXIO; + dev_err(&pdev->dev, "Could not get OCXL AFU from function\n"); + goto err_unregistered; + } + + ocxl_afu_get(ocxlpmem->ocxl_afu); + + // Resources allocated below here are cleaned up in the release handler + + rc = ocxlpmem_register(ocxlpmem); + if (rc) { + dev_err(&pdev->dev, + "Could not register OpenCAPI persistent memory device with the kernel\n"); + goto err; + } + + rc = ocxl_context_alloc(&ocxlpmem->ocxl_context, ocxlpmem->ocxl_afu, + NULL); + if (rc) { + dev_err(&pdev->dev, "Could not allocate OCXL context\n"); + goto err; + } + + rc = ocxl_context_attach(ocxlpmem->ocxl_context, 0, NULL); + if (rc) { + dev_err(&pdev->dev, "Could not attach ocxl context\n"); + goto err; + } + + rc = register_lpc_mem(ocxlpmem); + if (rc) { + dev_err(&pdev->dev, + "Could not register OpenCAPI persistent memory with libnvdimm\n"); + goto err; + } + + return 0; + +err_unregistered: + if (!IS_ERR(ocxlpmem->ocxl_fn)) + ocxl_function_close(ocxlpmem->ocxl_fn); + pci_dev_put(ocxlpmem->pdev); + kfree(ocxlpmem); + pci_set_drvdata(pdev, NULL); + +err: + /* + * Further cleanup is done in the release handler via free_ocxlpmem() + * This allows us to keep the character device live to handle IOCTLs to + * investigate issues if the card has an error + */ + + dev_err(&pdev->dev, + "Error detected, will not register OpenCAPI persistent memory\n"); + return 0; +} + +static struct pci_driver pci_driver = { + .name = "ocxlpmem", + .id_table = pci_tbl, + .probe = probe, + .remove = remove, + .shutdown = remove, +}; + +static int __init ocxlpmem_init(void) +{ + int rc; + + mutex_init(&minors_idr_lock); + idr_init(&minors_idr); + + rc = pci_register_driver(&pci_driver); + if (rc) + return rc; + + return 0; +} + +static void ocxlpmem_exit(void) +{ + pci_unregister_driver(&pci_driver); +} + +module_init(ocxlpmem_init); +module_exit(ocxlpmem_exit); + +MODULE_DESCRIPTION("OpenCAPI Persistent Memory"); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Alastair D'Silva "); diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h new file mode 100644 index 000000000000..03fe7a264281 --- /dev/null +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2020 IBM Corp. + +#include +#include +#include +#include + +#define LABEL_AREA_SIZE BIT_ULL(PA_SECTION_SHIFT) + +struct ocxlpmem { + struct device dev; + struct pci_dev *pdev; + struct ocxl_fn *ocxl_fn; + struct nd_interleave_set nd_set; + struct nvdimm_bus_descriptor bus_desc; + struct nvdimm_bus *nvdimm_bus; + struct ocxl_afu *ocxl_afu; + struct ocxl_context *ocxl_context; + void *metadata_addr; + struct resource pmem_res; + struct nd_region *nd_region; +}; From patchwork Fri Mar 27 07:11:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461805 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7E881668 for ; Fri, 27 Mar 2020 07:12:31 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9549D20705 for ; Fri, 27 Mar 2020 07:12:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="ToWrV+3Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9549D20705 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 24A1410FC3BB3; Fri, 27 Mar 2020 00:13:16 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 12D3210FC3614 for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id EB93C2DC682E; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293141; bh=vpE3Z7AWSr9my143HyVtO4yHV91rl8rmhtd6A3TKrQs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ToWrV+3ZvZNUE2ymKS5AQKjdsoM8VnN0qS7Mcg9YpzMua6UKfnzZyjTa2CjCdkbAx KA4DnOTootSPbswhgz91nw0o0biwbDxGGtU0moHxcS4X33+W5cUSY031Xl/IrjDcZd k3akP1KgKpgAFMucLlxQ3QSayVxS+wFdH5r/CbdTgtEm2Lm0kz0rlKGVn2QnbTTmUY UkMI0q1W76+Cvy4N74Irk+FktDx/FAGYFcmEmjYiX7Nqu9M/8R0gPLHdVNTj3r5m56 z8ExH4vvUt4YZDpKoUykg9YeTGV2GWmofs8LL315gf708HqmzIocIMebbvWxygke/J 32agGl9J6xWPkVtSWxa+am6KzeJxoO0/g4PK3Ng2KA76AixIloAhpOJc7kNPGwysHV T7Z+WgI/ICr/00Hx0uOy+52nbSf4jEphgh3ufk7wZnG00NaTJJ565EWv8n93D9nR/C 8gE9SepGUhdH3IliID4vYmthn4YIuB85FAXfMAQ6NpMPmma20AplulMTz4Potr3Qw/ krKKVXInJ8PzHdWHNiOFeLBHEa011pvUyKeZs2wdc6f8SoFaBDJl2Fl1zKiU3R5MjB e4zLtL+yqH7gkqhY5wwcPBO/0Knua2Y3YlppmvVOY4DDN5PsenvhIhiww1FHkOXtoP NiMUvB+XldjBvNPEI3v1PTTo= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ak045934; Fri, 27 Mar 2020 18:12:16 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 11/25] powerpc: Enable the OpenCAPI Persistent Memory driver for powernv_defconfig Date: Fri, 27 Mar 2020 18:11:48 +1100 Message-Id: <20200327071202.2159885-12-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) Message-ID-Hash: 4WTEAWCSDIX7AN3MINYCHBJZ2PHLX37P X-Message-ID-Hash: 4WTEAWCSDIX7AN3MINYCHBJZ2PHLX37P X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch enables the OpenCAPI Persistent Memory driver, as well as DAX support, for the 'powernv' defconfig. DAX is not a strict requirement for the functioning of the driver, but it is likely that a user will want to create a DAX device on top of their persistent memory device. Signed-off-by: Alastair D'Silva Reviewed-by: Andrew Donnellan --- arch/powerpc/configs/powernv_defconfig | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/powerpc/configs/powernv_defconfig b/arch/powerpc/configs/powernv_defconfig index 71749377d164..921d77bbd3d2 100644 --- a/arch/powerpc/configs/powernv_defconfig +++ b/arch/powerpc/configs/powernv_defconfig @@ -348,3 +348,8 @@ CONFIG_KVM_BOOK3S_64=m CONFIG_KVM_BOOK3S_64_HV=m CONFIG_VHOST_NET=m CONFIG_PRINTK_TIME=y +CONFIG_ZONE_DEVICE=y +CONFIG_OCXL_PMEM=m +CONFIG_DEV_DAX=m +CONFIG_DEV_DAX_PMEM=m +CONFIG_FS_DAX=y From patchwork Fri Mar 27 07:11:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D28421668 for ; Mon, 30 Mar 2020 05:53:09 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AF7F220786 for ; Mon, 30 Mar 2020 05:53:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="TXqe6QDq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF7F220786 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7E29310FC378B; Sun, 29 Mar 2020 22:53:59 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 55F9010FC378A for ; Sun, 29 Mar 2020 22:53:57 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id AF2322DC685E; Mon, 30 Mar 2020 16:52:51 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547572; bh=DG24uTFgm3ncmUTPxNdi4/JJAgeS8enEWAzD03RxOZg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TXqe6QDqrC9mr09B5wF/29o0TTisxmFcpb5xQh1cZb3/AR1YXvRJoY3sx1z5LcBrJ t+zJPG4Xc2mBEfI9YJ7ZlVHEqpnEIjgtG+oJxMMcOxiWX5uzkfFNWyD8/6zjasfBTB rqVrg2JP746Asy6Z/qaq9NMnTrtWQVnCkBaPFUTOag7RLSOQ+9ZtmG2HohPm8pWRHB SfhVnebyKrHl4UmR+bvyFhsVxhMHGgcIY7wtxBko3oFk4oPb63fkDyyLz5sSLjH2qY aTAC1kB9J2U44M2VoXiD8s9EsJGtTe8Fk/lfb3T6+EGaPceKk9iT6/jkKMukuv4Skj 6/6s6vgnQWdmcMo5HODCL7TUdBesiiHnc2KpFVyZPTgZ6BNryXNj+mmuNJdV4HLtlr P3w8SP9EHHAq4QIMmgJXJfzAtLXITdqEcvLHiO7YX1AhrZqRQRzFTrYNI5c//gmrqg is5EJ/rlAXvxvbSgXvQ3fkGOdaje6XLIU1B/fA0oWC0K4UzVpxsgeva6a891KuU/qT 5v1DJFbFVYhn/OiuiOgTT+XCkQnZ3c1Vs7td9zkVBOOqt56O61tzdqtWQdGqlJTOIw spFT5kwK2XD1UnFYSz49rNfHBpGn5H4Ezzhmxgd/eUnWxy+gjeNaS8tPjgnmay81N5 9azCa0l7R+D6MSdd46z0bfig= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Al045934; Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 12/25] nvdimm/ocxl: Add register addresses & status values to the header Date: Fri, 27 Mar 2020 18:11:49 +1100 Message-Id: <20200327071202.2159885-13-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) Message-ID-Hash: VIVGRC3AGWX4MWFASSCISHVCEZLTWZJ2 X-Message-ID-Hash: VIVGRC3AGWX4MWFASSCISHVCEZLTWZJ2 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: These values have been taken from the device specifications. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/ocxlpmem.h | 73 ++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index 03fe7a264281..322387873b4b 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -8,6 +8,79 @@ #define LABEL_AREA_SIZE BIT_ULL(PA_SECTION_SHIFT) +#define GLOBAL_MMIO_CHI 0x000 +#define GLOBAL_MMIO_CHIC 0x008 +#define GLOBAL_MMIO_CHIE 0x010 +#define GLOBAL_MMIO_CHIEC 0x018 +#define GLOBAL_MMIO_HCI 0x020 +#define GLOBAL_MMIO_HCIC 0x028 +#define GLOBAL_MMIO_IMA0_OHP 0x040 +#define GLOBAL_MMIO_IMA0_CFP 0x048 +#define GLOBAL_MMIO_IMA1_OHP 0x050 +#define GLOBAL_MMIO_IMA1_CFP 0x058 +#define GLOBAL_MMIO_ACMA_CREQO 0x100 +#define GLOBAL_MMIO_ACMA_CRSPO 0x104 +#define GLOBAL_MMIO_ACMA_CDBO 0x108 +#define GLOBAL_MMIO_ACMA_CDBS 0x10c +#define GLOBAL_MMIO_NSCMA_CREQO 0x120 +#define GLOBAL_MMIO_NSCMA_CRSPO 0x124 +#define GLOBAL_MMIO_NSCMA_CDBO 0x128 +#define GLOBAL_MMIO_NSCMA_CDBS 0x12c +#define GLOBAL_MMIO_CSTS 0x140 +#define GLOBAL_MMIO_FWVER 0x148 +#define GLOBAL_MMIO_CCAP0 0x160 +#define GLOBAL_MMIO_CCAP1 0x168 + +#define GLOBAL_MMIO_CHI_ACRA BIT_ULL(0) +#define GLOBAL_MMIO_CHI_NSCRA BIT_ULL(1) +#define GLOBAL_MMIO_CHI_CRDY BIT_ULL(4) +#define GLOBAL_MMIO_CHI_CFFS BIT_ULL(5) +#define GLOBAL_MMIO_CHI_MA BIT_ULL(6) +#define GLOBAL_MMIO_CHI_ELA BIT_ULL(7) +#define GLOBAL_MMIO_CHI_CDA BIT_ULL(8) +#define GLOBAL_MMIO_CHI_CHFS BIT_ULL(9) + +#define GLOBAL_MMIO_CHI_ALL (GLOBAL_MMIO_CHI_ACRA | \ + GLOBAL_MMIO_CHI_NSCRA | \ + GLOBAL_MMIO_CHI_CRDY | \ + GLOBAL_MMIO_CHI_CFFS | \ + GLOBAL_MMIO_CHI_MA | \ + GLOBAL_MMIO_CHI_ELA | \ + GLOBAL_MMIO_CHI_CDA | \ + GLOBAL_MMIO_CHI_CHFS) + +#define GLOBAL_MMIO_HCI_ACRW BIT_ULL(0) // ACRW +#define GLOBAL_MMIO_HCI_NSCRW BIT_ULL(1) // NSCRW +#define GLOBAL_MMIO_HCI_AFU_RESET BIT_ULL(2) // AR +#define GLOBAL_MMIO_HCI_FW_DEBUG BIT_ULL(3) // FDE +#define GLOBAL_MMIO_HCI_CONTROLLER_DUMP BIT_ULL(4) // CD +#define GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED BIT_ULL(5) // CDC +#define GLOBAL_MMIO_HCI_REQ_HEALTH_PERF BIT_ULL(6) // CHPD + +#define ADMIN_COMMAND_HEARTBEAT 0x00u +#define ADMIN_COMMAND_SHUTDOWN 0x01u +#define ADMIN_COMMAND_FW_UPDATE 0x02u +#define ADMIN_COMMAND_FW_DEBUG 0x03u +#define ADMIN_COMMAND_ERRLOG 0x04u +#define ADMIN_COMMAND_SMART 0x05u +#define ADMIN_COMMAND_CONTROLLER_STATS 0x06u +#define ADMIN_COMMAND_CONTROLLER_DUMP 0x07u +#define ADMIN_COMMAND_CMD_CAPS 0x08u +#define ADMIN_COMMAND_MAX 0x08u + +#define STATUS_SUCCESS 0x00 +#define STATUS_MEM_UNAVAILABLE 0x20 +#define STATUS_BLOCKED_BG_TASK 0x21 +#define STATUS_BAD_OPCODE 0x50 +#define STATUS_BAD_REQUEST_PARM 0x51 +#define STATUS_BAD_DATA_PARM 0x52 +#define STATUS_DEBUG_BLOCKED 0x70 +#define STATUS_FAIL 0xFF + +#define STATUS_FW_UPDATE_BLOCKED STATUS_BLOCKED_BG_TASK +#define STATUS_FW_ARG_INVALID STATUS_BAD_REQUEST_PARM +#define STATUS_FW_INVALID STATUS_BAD_DATA_PARM + struct ocxlpmem { struct device dev; struct pci_dev *pdev; From patchwork Fri Mar 27 07:11:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11463871 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 14F3E913 for ; Sun, 29 Mar 2020 02:28:03 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D762520723 for ; Sun, 29 Mar 2020 02:28:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="fvbnSejt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D762520723 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 6BEEB1003E9A2; Sat, 28 Mar 2020 19:28:52 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 9A0031007B1C4 for ; Sat, 28 Mar 2020 19:28:49 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id EB3472DC6834; Fri, 27 Mar 2020 18:12:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293142; bh=inrMR42ADZy1WNP4o1RXdEGSTOHSv7zVqzMowRo0hwA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fvbnSejt9QKrOXK21Vhhf5qPVmU6c3uisBfNLm1Wj2yoYsJBcgk7zYHbmCuRoR46v argMRyl7VugKyct1EIjELvdNG5xLWmVf6kmXgg9UXhZFOu+tDwmuAjb6U9id4WwXrG hIuujDU9qh3YYuhXSU7NMD9YnQVSxc4fzKG5p6OVh+70ri7BwAOPVuj6weise2ZEi4 fGbNI78NO99yV0ijEoDCM0bMQpygNkk80e1TMqrEByDMe6w8RS8LAkFQh26KkH7mkj 8rV6cyjxk2kTynoAKMJyS2DempRxkb90KiKmPbUOWZjpmM9PWG3P8u8Orcu3J4jSPm lvsb5bbKlztRdCpYRw4FiY0vAQcLK+nWt96vvQlzeTZ5Y76IKAsGsmv+H1zaz5dJqH x3gbKZc7MEPS9176j7um/5VUdJZv+Jo1QPFpGsBduhbY/6dcl6n+wH1vQSFIsRblQF lf4L3ROfB4VKVKiS0n4ethjH2kn42F2uD24Go9+NxOGT1jo2VRyl1EzWDyRIhcWSCw OGDXteEqG1a9F+E9jAz5YpAa7Y132/6QZJYkK94wmmFsezphVL1uGMU1vQf4O9QwyY D+Gif/x4De7Rx5zTgpz946jJHSr9MAwSOckxaVoLJSTo5mcW66RpJhFRaKJAqihjow B0cAcJITzqr7CiapV6y/2kOI= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Am045934; Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 13/25] nvdimm/ocxl: Read the capability registers & wait for device ready Date: Fri, 27 Mar 2020 18:11:50 +1100 Message-Id: <20200327071202.2159885-14-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) Message-ID-Hash: RKYEMGPY3Z3BUYCVF2MG7HOSOYAQBSH4 X-Message-ID-Hash: RKYEMGPY3Z3BUYCVF2MG7HOSOYAQBSH4 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch reads timeouts & firmware version from the controller, and uses those timeouts to wait for the controller to report that it is ready before handing the memory over to libnvdimm. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/Makefile | 2 +- drivers/nvdimm/ocxl/main.c | 85 +++++++++++++++++++++++++ drivers/nvdimm/ocxl/ocxlpmem.h | 29 +++++++++ drivers/nvdimm/ocxl/ocxlpmem_internal.c | 19 ++++++ 4 files changed, 134 insertions(+), 1 deletion(-) create mode 100644 drivers/nvdimm/ocxl/ocxlpmem_internal.c diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile index e0e8ade1987a..bab97082e062 100644 --- a/drivers/nvdimm/ocxl/Makefile +++ b/drivers/nvdimm/ocxl/Makefile @@ -4,4 +4,4 @@ ccflags-$(CONFIG_PPC_WERROR) += -Werror obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o -ocxlpmem-y := main.o \ No newline at end of file +ocxlpmem-y := main.o ocxlpmem_internal.o diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index c0066fedf9cc..be76acd33d74 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -8,6 +8,7 @@ #include #include +#include #include #include #include @@ -327,6 +328,50 @@ static void remove(struct pci_dev *pdev) } } +/** + * read_device_metadata() - Retrieve config information from the AFU and save it for future use + * @ocxlpmem: the device metadata + * Return: 0 on success, negative on failure + */ +static int read_device_metadata(struct ocxlpmem *ocxlpmem) +{ + u64 val; + int rc; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CCAP0, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + ocxlpmem->scm_revision = val & 0xFFFF; + ocxlpmem->read_latency = (val >> 32) & 0xFFFF; + ocxlpmem->readiness_timeout = (val >> 48) & 0x0F; + ocxlpmem->memory_available_timeout = val >> 52; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CCAP1, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + ocxlpmem->max_controller_dump_size = val & 0xFFFFFFFF; + + // Extract firmware version text + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_FWVER, + OCXL_HOST_ENDIAN, + (u64 *)ocxlpmem->fw_version); + if (rc) + return rc; + + ocxlpmem->fw_version[8] = '\0'; + + dev_info(&ocxlpmem->dev, + "Firmware version '%s' SCM revision %d:%d\n", + ocxlpmem->fw_version, ocxlpmem->scm_revision >> 4, + ocxlpmem->scm_revision & 0x0F); + + return 0; +} + /** * probe_function0() - Set up function 0 for an OpenCAPI persistent memory device * This is important as it enables templates higher than 0 across all other @@ -359,6 +404,9 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) { struct ocxlpmem *ocxlpmem; int rc; + u64 chi; + u16 elapsed, timeout; + bool ready = false; if (PCI_FUNC(pdev->devfn) == 0) return probe_function0(pdev); @@ -413,6 +461,43 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + rc = read_device_metadata(ocxlpmem); + if (rc) { + dev_err(&pdev->dev, "Could not read metadata\n"); + goto err; + } + + elapsed = 0; + timeout = ocxlpmem->readiness_timeout + + ocxlpmem->memory_available_timeout; + + while (true) { + rc = ocxlpmem_chi(ocxlpmem, &chi); + ready = (chi & (GLOBAL_MMIO_CHI_CRDY | GLOBAL_MMIO_CHI_MA)) == + (GLOBAL_MMIO_CHI_CRDY | GLOBAL_MMIO_CHI_MA); + + if (ready) + break; + + if (elapsed++ > timeout) { + dev_err(&ocxlpmem->dev, + "OpenCAPI Persistent Memory ready timeout.\n"); + + if (!(chi & GLOBAL_MMIO_CHI_CRDY)) + dev_err(&ocxlpmem->dev, + "controller is not ready.\n"); + + if (!(chi & GLOBAL_MMIO_CHI_MA)) + dev_err(&ocxlpmem->dev, + "controller does not have memory available.\n"); + + rc = -ENXIO; + goto err; + } + + msleep(1000); + } + rc = register_lpc_mem(ocxlpmem); if (rc) { dev_err(&pdev->dev, diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index 322387873b4b..3eadbe19f6d0 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -93,4 +93,33 @@ struct ocxlpmem { void *metadata_addr; struct resource pmem_res; struct nd_region *nd_region; + char fw_version[8 + 1]; + + u32 max_controller_dump_size; + u16 scm_revision; // major/minor + u8 readiness_timeout; /* The worst case time (in seconds) that the host + * shall wait for the controller to become + * operational following a reset (CHI.CRDY). + */ + u8 memory_available_timeout; /* The worst case time (in seconds) that + * the host shall wait for memory to + * become available following a reset + * (CHI.MA). + */ + + u16 read_latency; /* The nominal measure of latency (in nanoseconds) + * associated with an unassisted read of a memory + * block. + * This represents the capability of the raw media + * technology without assistance + */ }; + +/** + * ocxlpmem_chi() - Get the value of the CHI register + * @ocxlpmem: the device metadata + * @chi: returns the CHI value + * + * Returns 0 on success, negative on error + */ +int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi); diff --git a/drivers/nvdimm/ocxl/ocxlpmem_internal.c b/drivers/nvdimm/ocxl/ocxlpmem_internal.c new file mode 100644 index 000000000000..5578169b7515 --- /dev/null +++ b/drivers/nvdimm/ocxl/ocxlpmem_internal.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0+ +// Copyright 2020 IBM Corp. + +#include +#include +#include "ocxlpmem.h" + +int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi) +{ + u64 val; + int rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHI, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + *chi = val; + + return 0; +} From patchwork Fri Mar 27 07:11:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11461821 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E52E4159A for ; Fri, 27 Mar 2020 07:12:42 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C478B20717 for ; Fri, 27 Mar 2020 07:12:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="QF7zCKhO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C478B20717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id E729E10FC3BB6; Fri, 27 Mar 2020 00:13:19 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 8667510FC3614 for ; Fri, 27 Mar 2020 00:13:12 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id B91E72DC6832; Fri, 27 Mar 2020 18:12:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585293142; bh=cqaf1e/QB+IAchm8eT268pDK8d/xM41B6FFZ8OqT9TI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QF7zCKhOO9JvoXlvxy4VnsDJX+RwAyK4FeUeePRbrO8WnTpKijCUVL+iCF4MaMQrG ODUD+yy/GvqqYRFbFAhuAmr95mcjJ3FrGBL9IcJd82LSzVZ6n505dXIjRK5nQ19haz XIhzLF1o3GbzKvzl62NLKo1bgpWPTET1DgVMEk0NxV+85UIOszyAYm/eJ50+9LjziC HHEFwR7hJlwcRcSjs80Q7YUKX+EZslhdHnIDDsluGeo620FtT09A0WHYXlWT/GQ7Rh lslYpbukXPF+GkOfYhgHSNQB9T9TdVuv2OJf1C5MlMagEloNAEW4ZHT6JQivBWi+m+ Y9ZuIyQHe7Z240X832lKU/bC2AGYKCBupxW52G0BdEtNwVSGXzTeN+fntXETw1ZCZn 3Xs2pTUDHwyD7WdoNyUorO3WE4BkVVTgr2kOeQW3kxbhnLMSi5DvBP4VJUfWfEsCy0 tLQZiWPZey9bX9WqPZQXFch/xoGYoG/79znqmFlQdFC8GyITraJlBO/GehHDStGyD3 DAqwT0+8NGQvuPhjx9yfDN31nTaBzytXIuvHtEniZvZ6CpAlv+tLVc4uh2A7+ev0kv zluDpBRC7bvTnT8TKZbFkT/oEMeKPYxN1VMFytlWUlBlYMmxbyG6ii3I2N166ErNDF H99oxckYP+MvCqnW37EbNJj8= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4An045934; Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 14/25] nvdimm/ocxl: Add support for Admin commands Date: Fri, 27 Mar 2020 18:11:51 +1100 Message-Id: <20200327071202.2159885-15-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:17 +1100 (AEDT) Message-ID-Hash: OCOIZBDOINK5GUAYBQRETTOY5EZP5YH4 X-Message-ID-Hash: OCOIZBDOINK5GUAYBQRETTOY5EZP5YH4 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Admin commands for these devices are the primary means of interacting with the device controller to provide functionality beyond the load/store capabilities offered via the NPU. For example, SMART data, firmware update, and device error logs are implemented via admin commands. This patch requests the metadata required to issue admin commands, as well as some helper functions to construct and check the completion of the commands. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 65 ++++++ drivers/nvdimm/ocxl/ocxlpmem.h | 50 ++++- drivers/nvdimm/ocxl/ocxlpmem_internal.c | 261 ++++++++++++++++++++++++ 3 files changed, 375 insertions(+), 1 deletion(-) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index be76acd33d74..8db573036423 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -217,6 +217,58 @@ static int register_lpc_mem(struct ocxlpmem *ocxlpmem) return 0; } +/** + * extract_command_metadata() - Extract command data from MMIO & save it for further use + * @ocxlpmem: the device metadata + * @offset: The base address of the command data structures (address of CREQO) + * @command_metadata: A pointer to the command metadata to populate + * Return: 0 on success, negative on failure + */ +static int extract_command_metadata(struct ocxlpmem *ocxlpmem, u32 offset, + struct command_metadata *command_metadata) +{ + int rc; + u64 tmp; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, offset, + OCXL_LITTLE_ENDIAN, &tmp); + if (rc) + return rc; + + command_metadata->request_offset = tmp >> 32; + command_metadata->response_offset = tmp & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, offset + 8, + OCXL_LITTLE_ENDIAN, &tmp); + if (rc) + return rc; + + command_metadata->data_offset = tmp >> 32; + command_metadata->data_size = tmp & 0xFFFFFFFF; + + command_metadata->id = 0; + + return 0; +} + +/** + * setup_command_metadata() - Set up the command metadata + * @ocxlpmem: the device metadata + */ +static int setup_command_metadata(struct ocxlpmem *ocxlpmem) +{ + int rc; + + mutex_init(&ocxlpmem->admin_command.lock); + + rc = extract_command_metadata(ocxlpmem, GLOBAL_MMIO_ACMA_CREQO, + &ocxlpmem->admin_command); + if (rc) + return rc; + + return 0; +} + /** * allocate_minor() - Allocate a minor number to use for an OpenCAPI pmem device * @ocxlpmem: the device metadata @@ -421,6 +473,14 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) ocxlpmem->pdev = pci_dev_get(pdev); + ocxlpmem->timeouts[ADMIN_COMMAND_ERRLOG] = 2000; // ms + ocxlpmem->timeouts[ADMIN_COMMAND_HEARTBEAT] = 100; // ms + ocxlpmem->timeouts[ADMIN_COMMAND_SMART] = 100; // ms + ocxlpmem->timeouts[ADMIN_COMMAND_CONTROLLER_DUMP] = 1000; // ms + ocxlpmem->timeouts[ADMIN_COMMAND_CONTROLLER_STATS] = 100; // ms + ocxlpmem->timeouts[ADMIN_COMMAND_SHUTDOWN] = 1000; // ms + ocxlpmem->timeouts[ADMIN_COMMAND_FW_UPDATE] = 16000; // ms + pci_set_drvdata(pdev, ocxlpmem); ocxlpmem->ocxl_fn = ocxl_function_open(pdev); @@ -467,6 +527,11 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + if (setup_command_metadata(ocxlpmem)) { + dev_err(&pdev->dev, "Could not read command metadata\n"); + goto err; + } + elapsed = 0; timeout = ocxlpmem->readiness_timeout + ocxlpmem->memory_available_timeout; diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index 3eadbe19f6d0..b72b3f909fc3 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -7,6 +7,7 @@ #include #define LABEL_AREA_SIZE BIT_ULL(PA_SECTION_SHIFT) +#define DEFAULT_TIMEOUT 100 #define GLOBAL_MMIO_CHI 0x000 #define GLOBAL_MMIO_CHIC 0x008 @@ -57,6 +58,7 @@ #define GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED BIT_ULL(5) // CDC #define GLOBAL_MMIO_HCI_REQ_HEALTH_PERF BIT_ULL(6) // CHPD +// must be maintained with admin_command_names in ocxlpmem_internal.c #define ADMIN_COMMAND_HEARTBEAT 0x00u #define ADMIN_COMMAND_SHUTDOWN 0x01u #define ADMIN_COMMAND_FW_UPDATE 0x02u @@ -68,6 +70,13 @@ #define ADMIN_COMMAND_CMD_CAPS 0x08u #define ADMIN_COMMAND_MAX 0x08u +#define NS_COMMAND_SECURE_ERASE 0x20ull + +#define NS_RESPONSE_SECURE_ERASE_ACCESSIBLE_SUCCESS 0x20 +#define NS_RESPONSE_SECURE_ERASE_ACCESSIBLE_ATTEMPTED 0x28 +#define NS_RESPONSE_SECURE_ERASE_DEFECTIVE_SUCCESS 0x30 +#define NS_RESPONSE_SECURE_ERASE_DEFECTIVE_ATTEMPTED 0x38 + #define STATUS_SUCCESS 0x00 #define STATUS_MEM_UNAVAILABLE 0x20 #define STATUS_BLOCKED_BG_TASK 0x21 @@ -81,6 +90,16 @@ #define STATUS_FW_ARG_INVALID STATUS_BAD_REQUEST_PARM #define STATUS_FW_INVALID STATUS_BAD_DATA_PARM +struct command_metadata { + u32 request_offset; + u32 response_offset; + u32 data_offset; + u32 data_size; + struct mutex lock; /* locks access to this command */ + u16 id; + u8 op_code; +}; + struct ocxlpmem { struct device dev; struct pci_dev *pdev; @@ -91,10 +110,11 @@ struct ocxlpmem { struct ocxl_afu *ocxl_afu; struct ocxl_context *ocxl_context; void *metadata_addr; + struct command_metadata admin_command; struct resource pmem_res; struct nd_region *nd_region; char fw_version[8 + 1]; - + u32 timeouts[ADMIN_COMMAND_MAX + 1]; u32 max_controller_dump_size; u16 scm_revision; // major/minor u8 readiness_timeout; /* The worst case time (in seconds) that the host @@ -123,3 +143,31 @@ struct ocxlpmem { * Returns 0 on success, negative on error */ int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi); + +/** + * admin_command_execute() - Execute an admin command and wait for completion + * + * Additional MMIO registers (dependent on the command) may + * need to be initialized + * + * @ocxlpmem: the device metadata + * @op_code: the code for the admin command + * Returns 0 on success, -EINVAL for a bad op code, -EBUSY on timeout + */ +int admin_command_execute(struct ocxlpmem *ocxlpmem, u8 op_code); + +/** + * admin_response_handled() - Notify the controller that the admin response has been handled + * @ocxlpmem: the device metadata + * Returns 0 on success, negative on failure + */ +int admin_response_handled(const struct ocxlpmem *ocxlpmem); + +/** + * warn_status() - Emit a kernel warning showing a command status. + * @ocxlpmem: the device metadata + * @message: A message to accompany the warning + * @status: The command status + */ +void warn_status(const struct ocxlpmem *ocxlpmem, const char *message, + u8 status); diff --git a/drivers/nvdimm/ocxl/ocxlpmem_internal.c b/drivers/nvdimm/ocxl/ocxlpmem_internal.c index 5578169b7515..7470a6ab3b08 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem_internal.c +++ b/drivers/nvdimm/ocxl/ocxlpmem_internal.c @@ -17,3 +17,264 @@ int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi) return 0; } + +#define COMMAND_REQUEST_SIZE (8 * sizeof(u64)) +/** + * scm_command_request() - Set up a command request + * @cmd: The metadata for the type of command to be issued + * @op_code: the op code for the command + * @valid_bytes: the number of bytes in the header to preserve (these must be set before calling) + */ +static int scm_command_request(const struct ocxlpmem *ocxlpmem, + struct command_metadata *cmd, u8 op_code, + u8 valid_bytes) +{ + u64 val = op_code; + int rc; + u8 i; + + cmd->op_code = op_code; + cmd->id++; + + val |= ((u64)cmd->id) << 16; + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, cmd->request_offset, + OCXL_LITTLE_ENDIAN, val); + if (rc) + return rc; + + for (i = valid_bytes; i < COMMAND_REQUEST_SIZE; i += sizeof(u64)) { + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + cmd->request_offset + i, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + return rc; + } + + return 0; +} + +/** + * admin_command_request() - Issue an admin command request + * @ocxlpmem: the device metadata + * @op_code: The op-code for the command + * + * Returns an identifier for the command, or negative on error + */ +static int admin_command_request(struct ocxlpmem *ocxlpmem, u8 op_code) +{ + u8 valid_bytes = sizeof(u64); + + switch (op_code) { + case ADMIN_COMMAND_HEARTBEAT: + case ADMIN_COMMAND_SHUTDOWN: + case ADMIN_COMMAND_ERRLOG: + case ADMIN_COMMAND_CMD_CAPS: + valid_bytes += 0; + break; + case ADMIN_COMMAND_FW_UPDATE: + case ADMIN_COMMAND_SMART: + case ADMIN_COMMAND_CONTROLLER_STATS: + case ADMIN_COMMAND_CONTROLLER_DUMP: + valid_bytes += sizeof(u64); + break; + case ADMIN_COMMAND_FW_DEBUG: + valid_bytes += 3 * sizeof(u64); + break; + default: + return -EINVAL; + } + + return scm_command_request(ocxlpmem, &ocxlpmem->admin_command, op_code, + valid_bytes); +} + +static int command_response(const struct ocxlpmem *ocxlpmem, + const struct command_metadata *cmd) +{ + u64 val; + u16 id; + u8 status; + int rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + cmd->response_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + status = val & 0xff; + id = (val >> 16) & 0xffff; + + if (id != cmd->id) { + dev_err(&ocxlpmem->dev, + "Expected response for command %d, but received response for command %d instead.\n", + cmd->id, id); + return -EBUSY; + } + + return status; +} + +/** + * admin_response() - Validate an admin response + * @ocxlpmem: the device metadata + * Returns the status code of the command, or negative on error + */ +static int admin_response(const struct ocxlpmem *ocxlpmem) +{ + return command_response(ocxlpmem, &ocxlpmem->admin_command); +} + +/** + * admin_command_exec() - Notify the controller to start processing a pending admin command + * @ocxlpmem: the device metadata + * Returns 0 on success, negative on error + */ +static int admin_command_exec(const struct ocxlpmem *ocxlpmem) +{ + return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_HCI_ACRW); +} + +static bool admin_command_complete(const struct ocxlpmem *ocxlpmem) +{ + u64 val = 0; + + int rc = ocxlpmem_chi(ocxlpmem, &val); + + WARN_ON(rc); + + return (val & GLOBAL_MMIO_CHI_ACRA) != 0; +} + +/** + * admin_command_complete_timeout() - Wait for an admin command to finish executing + * @ocxlpmem: the device metadata + * @command: the admin command to wait for completion (determines the timeout) + * Returns 0 on success, -EBUSY on timeout + */ +static int admin_command_complete_timeout(const struct ocxlpmem *ocxlpmem, + int command) +{ + unsigned long timeout = jiffies + + msecs_to_jiffies(ocxlpmem->timeouts[command]); + + // 32 is the next power of 2 greater than the 20ms minimum for msleep +#define TIMEOUT_SLEEP_MILLIS 32 + do { + if (admin_command_complete(ocxlpmem)) + return 0; + msleep(TIMEOUT_SLEEP_MILLIS); + } while (time_before(jiffies, timeout)); + + if (admin_command_complete(ocxlpmem)) + return 0; + + return -EBUSY; +} + +// Must be maintained with ADMIN_COMMAND_* in ocxlpmem.h +static const char * const admin_command_names[] = { + "heartbeat", + "shutdown", + "firmware update", + "firmware debug", + "retrieve error log", + "retrieve SMART data", + "controller statistics", + "controller dump", + "command capabilities", +}; + +/** + * admin_command_name() - get the name of an admin command + * @ocxlpmem: the device metadata + * @op_code: the code for the admin command + * Returns a string representing the name of the command + */ +static const char *admin_command_name(u8 op_code) +{ + if (op_code > ADMIN_COMMAND_MAX) + return "unknown command"; + + return admin_command_names[op_code]; +} + +/** + * admin_command_execute() - Execute an admin command and wait for completion + * + * Additional MMIO registers (dependent on the command) may + * need to be initialized + * + * @ocxlpmem: the device metadata + * @op_code: the code for the admin command + * Returns 0 on success, -EINVAL for a bad op code, -EBUSY on timeout + */ +int admin_command_execute(struct ocxlpmem *ocxlpmem, u8 op_code) +{ + int rc; + + if (op_code > ADMIN_COMMAND_MAX) + return -EINVAL; + + rc = admin_command_request(ocxlpmem, op_code); + if (rc) + return rc; + + rc = admin_command_exec(ocxlpmem); + if (rc) + return rc; + + rc = admin_command_complete_timeout(ocxlpmem, op_code); + if (rc < 0) { + dev_warn(&ocxlpmem->dev, "%s timed out\n", + admin_command_name(op_code)); + return rc; + } + + return admin_response(ocxlpmem); +} + +int admin_response_handled(const struct ocxlpmem *ocxlpmem) +{ + // writing to the CHIC register clears the bit in CHI + return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIC, + OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_CHI_ACRA); +} + +void warn_status(const struct ocxlpmem *ocxlpmem, const char *message, + u8 status) +{ + const char *text = "Unknown"; + + switch (status) { + case STATUS_SUCCESS: + text = "Success"; + break; + + case STATUS_MEM_UNAVAILABLE: + text = "Persistent memory unavailable"; + break; + + case STATUS_BAD_OPCODE: + text = "Bad opcode"; + break; + + case STATUS_BAD_REQUEST_PARM: + text = "Bad request parameter"; + break; + + case STATUS_BAD_DATA_PARM: + text = "Bad data parameter"; + break; + + case STATUS_DEBUG_BLOCKED: + text = "Debug action blocked"; + break; + + case STATUS_FAIL: + text = "Failed"; + break; + } + + dev_warn(&ocxlpmem->dev, "%s: %s (%x)\n", message, text, status); +} From patchwork Fri Mar 27 07:11:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464659 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A4131668 for ; Mon, 30 Mar 2020 05:53:19 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED457208FE for ; Mon, 30 Mar 2020 05:53:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="DcslQM+q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED457208FE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B70D710FC379F; Sun, 29 Mar 2020 22:54:08 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 4557210FC3792 for ; Sun, 29 Mar 2020 22:54:06 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 32B862DC6860; Mon, 30 Mar 2020 16:52:54 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547574; bh=UtTeCR+k2Y01sGISBjlacY9KQsMxyTaomboXrU4wwyo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DcslQM+qGci9qChZilaQA/DfS7pIdT/bNqHpbmQhKkju0pgXMUz8JD90Cz+H1bOZ/ amHXQtG9t9iPoAPZuinrZD7AwVpmAv1jXyLxSdfobhFwpgpUM0K6qXy+IAmaP+HmlE TNOYm4ybNsk99pct5O+F75N7mUe3GbKAMkrk8zM0bR7/KJdH31MXZ2m7M39Aa0aERC hhBsydX+iRUw3CN/F63pYnhDAkKdFRLC7xbyoEEdnEDLEc7Igxlv0Of84H5ld7cps0 V0O9+JZZ2QIz8gyRAvAlapeKh5K8RWoV3B9FGc30JME4hgJGuzP427kc5vGzXjEH5K owUqeLtXwXVJDdQmdwFJCqkbEpULxf9gwx8QvS5clbc3TJNV4IZ6zQDxSjW7H/+yni p7+SHPCkLZZZhNXt04DvM8qX26uChSVK6PmzZI5tnv7hQSlx4XtQxD11dePDdL42r1 fDYadY3krb5KndhtNhvCfr1WBs7e97qb4o94/aLW6iqXGP3/FzACm/xRJT/ctSx5kc y/c+jC1pb1s+PJWQOXQuxvUmL1yb7VAP+LmMsMHoSQ9prJ0i+wFPws4FtxKpAkFTyi 0meqZfMRKeTShg5TSQuGOSjNVPBiPoAXGTDoW4TqYnZ+7azYSrqrMhWVYgnLWN69xR 8KL4NoIcEOzKu9p7busd2CuE= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ao045934; Fri, 27 Mar 2020 18:12:18 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 15/25] nvdimm/ocxl: Register a character device for userspace to interact with Date: Fri, 27 Mar 2020 18:11:52 +1100 Message-Id: <20200327071202.2159885-16-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:18 +1100 (AEDT) Message-ID-Hash: EWOP2SZ3VFOJLPPRIZ7IOVWSXOJFJU4K X-Message-ID-Hash: EWOP2SZ3VFOJLPPRIZ7IOVWSXOJFJU4K X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch introduces a character device (/dev/ocxlpmemX) which further patches will use to interact with userspace, such as error logs, controller stats and card debug functionality. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 117 ++++++++++++++++++++++++++++++++- drivers/nvdimm/ocxl/ocxlpmem.h | 2 + 2 files changed, 117 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index 8db573036423..9b85fcd3f1c9 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include "ocxlpmem.h" @@ -356,6 +357,67 @@ static int ocxlpmem_register(struct ocxlpmem *ocxlpmem) return device_register(&ocxlpmem->dev); } +static void ocxlpmem_put(struct ocxlpmem *ocxlpmem) +{ + put_device(&ocxlpmem->dev); +} + +static struct ocxlpmem *ocxlpmem_get(struct ocxlpmem *ocxlpmem) +{ + return (!get_device(&ocxlpmem->dev)) ? NULL : ocxlpmem; +} + +static struct ocxlpmem *find_and_get_ocxlpmem(dev_t devno) +{ + struct ocxlpmem *ocxlpmem; + int minor = MINOR(devno); + + mutex_lock(&minors_idr_lock); + ocxlpmem = idr_find(&minors_idr, minor); + if (ocxlpmem) + ocxlpmem_get(ocxlpmem); + mutex_unlock(&minors_idr_lock); + + return ocxlpmem; +} + +static int file_open(struct inode *inode, struct file *file) +{ + struct ocxlpmem *ocxlpmem; + + ocxlpmem = find_and_get_ocxlpmem(inode->i_rdev); + if (!ocxlpmem) + return -ENODEV; + + file->private_data = ocxlpmem; + return 0; +} + +static int file_release(struct inode *inode, struct file *file) +{ + struct ocxlpmem *ocxlpmem = file->private_data; + + ocxlpmem_put(ocxlpmem); + return 0; +} + +static const struct file_operations fops = { + .owner = THIS_MODULE, + .open = file_open, + .release = file_release, +}; + +/** + * create_cdev() - Create the chardev in /dev for the device + * @ocxlpmem: the SCM metadata + * Return: 0 on success, negative on failure + */ +static int create_cdev(struct ocxlpmem *ocxlpmem) +{ + cdev_init(&ocxlpmem->cdev, &fops); + return cdev_add(&ocxlpmem->cdev, ocxlpmem->dev.devt, 1); +} + /** * ocxlpmem_remove() - Free an OpenCAPI persistent memory device * @pdev: the PCI device information struct @@ -376,6 +438,13 @@ static void remove(struct pci_dev *pdev) if (ocxlpmem->nvdimm_bus) nvdimm_bus_unregister(ocxlpmem->nvdimm_bus); + /* + * Remove the cdev early to prevent a race against userspace + * via the char dev + */ + if (ocxlpmem->cdev.owner) + cdev_del(&ocxlpmem->cdev); + device_unregister(&ocxlpmem->dev); } } @@ -527,11 +596,18 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } - if (setup_command_metadata(ocxlpmem)) { + rc = setup_command_metadata(ocxlpmem); + if (rc) { dev_err(&pdev->dev, "Could not read command metadata\n"); goto err; } + rc = create_cdev(ocxlpmem); + if (rc) { + dev_err(&pdev->dev, "Could not create character device\n"); + goto err; + } + elapsed = 0; timeout = ocxlpmem->readiness_timeout + ocxlpmem->memory_available_timeout; @@ -599,6 +675,36 @@ static struct pci_driver pci_driver = { .shutdown = remove, }; +static int file_init(void) +{ + int rc; + + rc = alloc_chrdev_region(&ocxlpmem_dev, 0, NUM_MINORS, "ocxlpmem"); + if (rc) { + idr_destroy(&minors_idr); + pr_err("Unable to allocate OpenCAPI persistent memory major number: %d\n", + rc); + return rc; + } + + ocxlpmem_class = class_create(THIS_MODULE, "ocxlpmem"); + if (IS_ERR(ocxlpmem_class)) { + idr_destroy(&minors_idr); + pr_err("Unable to create ocxlpmem class\n"); + unregister_chrdev_region(ocxlpmem_dev, NUM_MINORS); + return PTR_ERR(ocxlpmem_class); + } + + return 0; +} + +static void file_exit(void) +{ + class_destroy(ocxlpmem_class); + unregister_chrdev_region(ocxlpmem_dev, NUM_MINORS); + idr_destroy(&minors_idr); +} + static int __init ocxlpmem_init(void) { int rc; @@ -606,16 +712,23 @@ static int __init ocxlpmem_init(void) mutex_init(&minors_idr_lock); idr_init(&minors_idr); - rc = pci_register_driver(&pci_driver); + rc = file_init(); if (rc) return rc; + rc = pci_register_driver(&pci_driver); + if (rc) { + file_exit(); + return rc; + } + return 0; } static void ocxlpmem_exit(void) { pci_unregister_driver(&pci_driver); + file_exit(); } module_init(ocxlpmem_init); diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index b72b3f909fc3..ee3bd651f254 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -2,6 +2,7 @@ // Copyright 2020 IBM Corp. #include +#include #include #include #include @@ -103,6 +104,7 @@ struct command_metadata { struct ocxlpmem { struct device dev; struct pci_dev *pdev; + struct cdev cdev; struct ocxl_fn *ocxl_fn; struct nd_interleave_set nd_set; struct nvdimm_bus_descriptor bus_desc; From patchwork Fri Mar 27 07:11:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11466985 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B934914B4 for ; Tue, 31 Mar 2020 08:59:36 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 96F1520787 for ; Tue, 31 Mar 2020 08:59:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="Kalaxq5z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96F1520787 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 4AF5010FC3893; Tue, 31 Mar 2020 02:00:26 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 3B49210FC3886 for ; Tue, 31 Mar 2020 02:00:24 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 4BCB92DC6849; Mon, 30 Mar 2020 16:53:02 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547583; bh=4YGi5cIVTJtyQ3RAuYeIobqzlKnYrZhy+tDNyNbmbBI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kalaxq5zVXcqXTV+7cShrpQDd+DPjZ0pPj3T9DCz6pKgtR1jC4i1QdZ4SEX5u98e8 mKVtVWsG0B4kjkBeXSf6ADl11fwAI/lJGptO7Tf+KvP3OWculqcOsJiOhWP/8WY0Mj YR9egnccj5YS2nT1mNHiRYvl+hfcbxFCe2QQnIAm0pnpRLBt5DP3TOBWlcReFrz7mR Vpy6p0JeKxz7QzJyGl5TtfstdMotqraAwA3yafv3XyqrYpXf0mzjw/Ypq6sRf+jyoD 3AclM8QlsKopRJ5p03zihQ4NjOKFUTGcAblvp3FJrml3mxvQgH8cNtpmsD+i/2ni4b YiUOoudKtzWsjftqEx3J/ZkF49wCnIG9Jdz6uT10iPY2K371GsYcxNmQOqSNfmOQ/r W1LReKTP0Rpv1vxZAyq4KSphvzy4rOjiQwSTpdVwVmA1yMFA412VOuZaAUD4pBudtt aLsEbJ/C7y/SDEGhxgz+KPhJByXpDmpN64EBQ8Q5oWbrBtYktHR4O6mR69Fmty4HeM aOfCKx0hdaJQ3THwIrsnPhX7aWWvs0U1KMN9vVKbgO1aezYhhjSdR+FEItYQdhRnUp OHl1LYLkwGFzo+zpxHnP+j8wleROH4fR17cfK6tCa/NmWejDWYK23qZP1t+SqmGftV sCXgC3b2hOmfKrutn8Fut04Y= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ap045934; Fri, 27 Mar 2020 18:12:18 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 16/25] nvdimm/ocxl: Implement the Read Error Log command Date: Fri, 27 Mar 2020 18:11:53 +1100 Message-Id: <20200327071202.2159885-17-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:18 +1100 (AEDT) Message-ID-Hash: 3WC4BUKBZ3W7FXD4O4YS2F6ZEZHR5KAG X-Message-ID-Hash: 3WC4BUKBZ3W7FXD4O4YS2F6ZEZHR5KAG X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: The read error log command extracts information from the controller's internal error log. This patch exposes this information in 2 ways: - During probe, if an error occurs & a log is available, print it to the console - After probe, make the error log available to userspace via an IOCTL. Userspace is notified of pending error logs in a later patch ("powerpc/powernv/pmem: Forward events to userspace") Signed-off-by: Alastair D'Silva --- .../userspace-api/ioctl/ioctl-number.rst | 1 + drivers/nvdimm/ocxl/main.c | 240 ++++++++++++++++++ include/uapi/nvdimm/ocxlpmem.h | 46 ++++ 3 files changed, 287 insertions(+) create mode 100644 include/uapi/nvdimm/ocxlpmem.h diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 9425377615ce..ba0ce7dca643 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -340,6 +340,7 @@ Code Seq# Include File Comments 0xC0 00-0F linux/usb/iowarrior.h 0xCA 00-0F uapi/misc/cxl.h 0xCA 10-2F uapi/misc/ocxl.h +0xCA 30-3F uapi/nvdimm/ocxlpmem.h OpenCAPI Persistent Memory 0xCA 80-BF uapi/scsi/cxlflash_ioctl.h 0xCB 00-1F CBM serial IEC bus in development: diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index 9b85fcd3f1c9..e6be0029f658 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "ocxlpmem.h" static const struct pci_device_id pci_tbl[] = { @@ -401,10 +402,190 @@ static int file_release(struct inode *inode, struct file *file) return 0; } +/** + * error_log_header_parse() - Parse the first 64 bits of the error log command response + * @ocxlpmem: the device metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int error_log_header_parse(struct ocxlpmem *ocxlpmem, u16 *length) +{ + int rc; + u64 val; + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFF; + + if (data_identifier != 0x454C) { // 'EL' + dev_err(&ocxlpmem->dev, + "Bad data identifier for error log data, expected 'EL', got '%2s' (%#x), data_length=%u\n", + (char *)&data_identifier, + (unsigned int)data_identifier, data_length); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int read_error_log(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxlpmem_error_log *log, + bool buf_is_user) +{ + u64 val; + u16 user_buf_length; + u16 buf_length; + u64 *buf = (u64 *)log->buf_ptr; + u16 i; + int rc; + + if (log->buf_size % 8) + return -EINVAL; + + rc = ocxlpmem_chi(ocxlpmem, &val); + if (rc) + return rc; + + if (!(val & GLOBAL_MMIO_CHI_ELA)) + return -EAGAIN; + + user_buf_length = log->buf_size; + + mutex_lock(&ocxlpmem->admin_command.lock); + + rc = admin_command_execute(ocxlpmem, ADMIN_COMMAND_ERRLOG); + if (rc != STATUS_SUCCESS) { + warn_status(ocxlpmem, + "Unexpected status from retrieve error log", rc); + goto out; + } + + rc = error_log_header_parse(ocxlpmem, &log->buf_size); + if (rc) + goto out; + // log->buf_size now contains the returned buffer size, not the user size + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + 0x08, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + log->log_identifier = val >> 32; + log->program_reference_code = val & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + 0x10, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + log->error_log_type = val >> 56; + log->action_flags = (log->error_log_type == OCXLPMEM_ERROR_LOG_TYPE_GENERAL) ? + (val >> 32) & 0xFFFFFF : 0; + log->power_on_seconds = val & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + 0x18, + OCXL_LITTLE_ENDIAN, &log->timestamp); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + 0x20, + OCXL_LITTLE_ENDIAN, &log->wwid[0]); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + 0x28, + OCXL_LITTLE_ENDIAN, &log->wwid[1]); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + 0x30, + OCXL_HOST_ENDIAN, (u64 *)log->fw_revision); + if (rc) + goto out; + log->fw_revision[8] = '\0'; + + buf_length = (user_buf_length < log->buf_size) ? + user_buf_length : log->buf_size; + for (i = 0; i < buf_length / (sizeof(u64)); i++) { + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + + i * sizeof(u64), + OCXL_HOST_ENDIAN, &val); + if (rc) + goto out; + + if (buf_is_user) { + if (copy_to_user((u64 __user *)&buf[i], &val, + sizeof(u64))) { + rc = -EFAULT; + goto out; + } + } else { + buf[i] = val; + } + } + + rc = admin_response_handled(ocxlpmem); + if (rc) + goto out; + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + +static int ioctl_error_log(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxlpmem_error_log __user *uarg) +{ + struct ioctl_ocxlpmem_error_log args; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + rc = read_error_log(ocxlpmem, &args, true); + if (rc) + return rc; + + if (copy_to_user(uarg, &args, sizeof(args))) + return -EFAULT; + + return 0; +} + +static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) +{ + struct ocxlpmem *ocxlpmem = file->private_data; + int rc = -EINVAL; + + switch (cmd) { + case IOCTL_OCXLPMEM_ERROR_LOG: + rc = ioctl_error_log(ocxlpmem, + (struct ioctl_ocxlpmem_error_log __user *)args); + break; + } + return rc; +} + static const struct file_operations fops = { .owner = THIS_MODULE, .open = file_open, .release = file_release, + .unlocked_ioctl = file_ioctl, + .compat_ioctl = file_ioctl, }; /** @@ -493,6 +674,60 @@ static int read_device_metadata(struct ocxlpmem *ocxlpmem) return 0; } +static const char *decode_error_log_type(u8 error_log_type) +{ + switch (error_log_type) { + case 0x00: + return "general"; + case 0x01: + return "predictive failure"; + case 0x02: + return "thermal warning"; + case 0x03: + return "data loss"; + case 0x04: + return "health & performance"; + default: + return "unknown"; + } +} + +static void dump_error_log(struct ocxlpmem *ocxlpmem) +{ + struct ioctl_ocxlpmem_error_log log; + u32 buf_size; + u8 *buf; + int rc; + + if (ocxlpmem->admin_command.data_size == 0) + return; + + buf_size = ocxlpmem->admin_command.data_size - 0x48; + buf = kzalloc(buf_size, GFP_KERNEL); + if (!buf) + return; + + log.buf_ptr = (u64)buf; + log.buf_size = buf_size; + + rc = read_error_log(ocxlpmem, &log, false); + if (rc < 0) + goto out; + + dev_warn(&ocxlpmem->dev, + "OCXL PMEM Error log: WWID=0x%016llx%016llx LID=0x%x PRC=%x type=0x%x %s, Uptime=%u seconds timestamp=0x%llx\n", + log.wwid[0], log.wwid[1], + log.log_identifier, log.program_reference_code, + log.error_log_type, + decode_error_log_type(log.error_log_type), + log.power_on_seconds, log.timestamp); + print_hex_dump(KERN_WARNING, "buf", DUMP_PREFIX_OFFSET, 16, 1, buf, + log.buf_size, false); + +out: + kfree(buf); +} + /** * probe_function0() - Set up function 0 for an OpenCAPI persistent memory device * This is important as it enables templates higher than 0 across all other @@ -656,6 +891,11 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) pci_set_drvdata(pdev, NULL); err: + if (ocxlpmem && + (ocxlpmem_chi(ocxlpmem, &chi) == 0) && + (chi & GLOBAL_MMIO_CHI_ELA)) + dump_error_log(ocxlpmem); + /* * Further cleanup is done in the release handler via free_ocxlpmem() * This allows us to keep the character device live to handle IOCTLs to diff --git a/include/uapi/nvdimm/ocxlpmem.h b/include/uapi/nvdimm/ocxlpmem.h new file mode 100644 index 000000000000..5d3a03ea1e08 --- /dev/null +++ b/include/uapi/nvdimm/ocxlpmem.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */ +/* Copyright 2020 IBM Corp. */ +#ifndef _UAPI_OCXL_SCM_H +#define _UAPI_OCXL_SCM_H + +#include +#include + +#define OCXLPMEM_ERROR_LOG_ACTION_RESET (1 << (32 - 32)) +#define OCXLPMEM_ERROR_LOG_ACTION_CHKFW (1 << (53 - 32)) +#define OCXLPMEM_ERROR_LOG_ACTION_REPLACE (1 << (54 - 32)) +#define OCXLPMEM_ERROR_LOG_ACTION_DUMP (1 << (55 - 32)) + +#define OCXLPMEM_ERROR_LOG_TYPE_GENERAL (0x00) +#define OCXLPMEM_ERROR_LOG_TYPE_PREDICTIVE_FAILURE (0x01) +#define OCXLPMEM_ERROR_LOG_TYPE_THERMAL_WARNING (0x02) +#define OCXLPMEM_ERROR_LOG_TYPE_DATA_LOSS (0x03) +#define OCXLPMEM_ERROR_LOG_TYPE_HEALTH_PERFORMANCE (0x04) + +struct ioctl_ocxlpmem_error_log { + __u32 log_identifier; /* out */ + __u32 program_reference_code; /* out */ + __u32 action_flags; /* out, recommended course of action */ + __u32 power_on_seconds; /* out, Number of seconds the controller has been on when the error occurred */ + __u64 timestamp; /* out, relative time since the current IPL */ + __u64 wwid[2]; /* out, the NAA formatted WWID associated with the controller */ + char fw_revision[8 + 1]; /* out, firmware revision as null terminated text */ + __u8 reserved0[7]; + __u16 buf_size; /* in/out, buffer size provided/required. + * If required is greater than provided, the buffer + * will be truncated to the amount provided. If its + * less, then only the required bytes will be populated. + * If it is 0, then there are no more error log entries. + */ + __u8 error_log_type; + __u8 reserved1[5]; + __u64 buf_ptr; /* coerced pointer to output buffer */ + __u64 reserved2[2]; +}; + +/* ioctl numbers */ +#define OCXLPMEM_MAGIC 0xCA +/* OpenCAPI Persistent memory devices */ +#define IOCTL_OCXLPMEM_ERROR_LOG _IOWR(OCXLPMEM_MAGIC, 0x30, struct ioctl_ocxlpmem_error_log) + +#endif /* _UAPI_OCXL_SCM_H */ From patchwork Fri Mar 27 07:11:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11466989 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E67681805 for ; Tue, 31 Mar 2020 08:59:40 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C279D20CC7 for ; Tue, 31 Mar 2020 08:59:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="U1kOlzbc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C279D20CC7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 783F91003E9A2; Tue, 31 Mar 2020 02:00:30 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 0574C1003E9A2 for ; Tue, 31 Mar 2020 02:00:29 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id B7AE42DC6862; Mon, 30 Mar 2020 16:52:56 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547577; bh=4KPkiBrQsWsmYoMNYfJ6CRmop/RAcelGM+uqf1omm5w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U1kOlzbcYmJhAT6doXL3ZKT3NhzezXd8Adxt/k4bk3c5thHUwDs4uREOOI27nDbjV kz7wvAggzCzWys4yKv+Ogcmt6/5Xdt9SsfLHWQF8grRtdLRT0WX/t5vVGNP4IKLfmd 0OJ6goSBTWtgeELXe2wvVXCEBP8BPlBWhnarkWKw2TBMkblAjgbBrXi6MX+n4yYihQ dmDNxIcQsURnxUxNDaqYNmWSf8sX9jYzxgEr9mOcOdh+ftNrra5tB4sEvIoRSwUlDw uMFeVRAM8uPL/gLJDsihGP+IwD/CNrHB+LUCWkgTvZXFavZo/9B9iQxb3DaBImYffQ jo7EPoX5eiZwotzQVqXUFxRvkM+fZfQ74YOsi/XxTetqDGOUGjafjGqU19V4ebr5jm HWOTZq4PS7Q7pbrdYwUnOijgKH8UQtksQChY1yHdcL4NGjFA5Y1UGeCuX85c2XJ9B0 KpGqivVPZaBjc0uhHo6Js2SVbTCI+fzt/dayrT+l1foyzt4RyFv94nEU1mFlS4WU+W LU30wVxtLy7xCwMpJZSq5+OPSiqbwuSc4Bxgsy1k3iHqIweQig/kC7sPU9skD2N3gm 1GfzJKiT1EdnZZe7fvViYU5C11/8T7ZHhoNrCFR8YehNvbOj0S07rLu3M1BOGZ26rG Y1QXb2QA0t6LcEx40ICwDKDY= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Aq045934; Fri, 27 Mar 2020 18:12:18 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 17/25] nvdimm/ocxl: Add controller dump IOCTLs Date: Fri, 27 Mar 2020 18:11:54 +1100 Message-Id: <20200327071202.2159885-18-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:18 +1100 (AEDT) Message-ID-Hash: L4T4P4EHBO5IJU7NN4ZDQNKNHUYQAYSX X-Message-ID-Hash: L4T4P4EHBO5IJU7NN4ZDQNKNHUYQAYSX X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch adds IOCTLs to allow userspace to request & fetch dumps of the internal controller state. This is useful during debugging or when a fatal error on the controller has occurred. The expected flow of operations are: 1. IOCTL_OCXL_PMEM_CONTROLLER_DUMP to request the controller to take a dump. This IOCTL will complete after the dump is available for collection. 2. IOCTL_OCXL_PMEM_CONTROLLER_DUMP_DATA called repeatedly to fetch chunks from the buffer 3. IOCTL_OCXL_PMEM_CONTROLLER_DUMP_COMPLETE to notify the controller that it can free any internal resources used for the dump Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 161 +++++++++++++++++++++++++++++++++ include/uapi/nvdimm/ocxlpmem.h | 16 ++++ 2 files changed, 177 insertions(+) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index e6be0029f658..d0db358ded43 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -566,6 +566,153 @@ static int ioctl_error_log(struct ocxlpmem *ocxlpmem, return 0; } +/** + * controller_dump_header_parse() - Parse the first 64 bits of the controller dump command response + * @ocxlpmem: the device metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int controller_dump_header_parse(struct ocxlpmem *ocxlpmem, u16 *length) +{ + int rc; + u64 val; + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFF; + + if (data_identifier != 0x4344) { // 'CD' + dev_err(&ocxlpmem->dev, + "Bad data identifier for error log data, expected 'CD', got '%2s' (%#x), data_length=%u\n", + (char *)&data_identifier, + (unsigned int)data_identifier, data_length); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int ioctl_controller_dump_data(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxlpmem_controller_dump_data __user *uarg) +{ + struct ioctl_ocxlpmem_controller_dump_data args; + u64 __user *buf; + u16 i, buf_size; + u64 val; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + if (args.buf_size % sizeof(u64)) + return -EINVAL; + + if (args.buf_size > ocxlpmem->admin_command.data_size) + return -EINVAL; + + buf = (u64 *)args.buf_ptr; + + mutex_lock(&ocxlpmem->admin_command.lock); + + val = ((u64)args.offset) << 32; + val |= args.buf_size; + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.request_offset + 0x08, + OCXL_LITTLE_ENDIAN, val); + if (rc) + goto out; + + rc = admin_command_execute(ocxlpmem, ADMIN_COMMAND_CONTROLLER_DUMP); + if (rc) + goto out; + if (rc != STATUS_SUCCESS) { + warn_status(ocxlpmem, + "Unexpected status from controller dump", + rc); + goto out; + } + + rc = controller_dump_header_parse(ocxlpmem, &buf_size); + if (rc) + goto out; + + buf_size = min((u16)(buf_size + sizeof(u64)), args.buf_size); + + for (i = 0; i < buf_size / sizeof(u64); i++) { + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + + i * sizeof(u64), + OCXL_HOST_ENDIAN, &val); + if (rc) + goto out; + + if (copy_to_user(&buf[i], &val, sizeof(u64))) { + rc = -EFAULT; + goto out; + } + } + + args.buf_size = buf_size; + + if (copy_to_user(uarg, &args, sizeof(args))) { + rc = -EFAULT; + goto out; + } + + rc = admin_response_handled(ocxlpmem); + if (rc) + goto out; + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + +int request_controller_dump(struct ocxlpmem *ocxlpmem) +{ + int rc; + u64 busy = 1; + + rc = ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIC, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_CHI_CDA); + if (rc) + return rc; + + rc = ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_CONTROLLER_DUMP); + if (rc) + return rc; + + while (busy) { + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, &busy); + if (rc) + return rc; + + busy &= GLOBAL_MMIO_HCI_CONTROLLER_DUMP; + cond_resched(); + } + + return 0; +} + +static int ioctl_controller_dump_complete(struct ocxlpmem *ocxlpmem) +{ + return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED); +} + static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { struct ocxlpmem *ocxlpmem = file->private_data; @@ -576,7 +723,21 @@ static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) rc = ioctl_error_log(ocxlpmem, (struct ioctl_ocxlpmem_error_log __user *)args); break; + + case IOCTL_OCXLPMEM_CONTROLLER_DUMP: + rc = request_controller_dump(ocxlpmem); + break; + + case IOCTL_OCXLPMEM_CONTROLLER_DUMP_DATA: + rc = ioctl_controller_dump_data(ocxlpmem, + (struct ioctl_ocxlpmem_controller_dump_data __user *)args); + break; + + case IOCTL_OCXLPMEM_CONTROLLER_DUMP_COMPLETE: + rc = ioctl_controller_dump_complete(ocxlpmem); + break; } + return rc; } diff --git a/include/uapi/nvdimm/ocxlpmem.h b/include/uapi/nvdimm/ocxlpmem.h index 5d3a03ea1e08..05e2b3f7b27c 100644 --- a/include/uapi/nvdimm/ocxlpmem.h +++ b/include/uapi/nvdimm/ocxlpmem.h @@ -38,9 +38,25 @@ struct ioctl_ocxlpmem_error_log { __u64 reserved2[2]; }; +struct ioctl_ocxlpmem_controller_dump_data { + __u64 buf_ptr; /* coerced pointer to output buffer */ + __u16 buf_size; /* in/out, buffer size provided/required. + * If required is greater than provided, the buffer + * will be truncated to the amount provided. If its + * less, then only the required bytes will be populated. + * If it is 0, then there is no more dump data available. + */ + __u16 reserved0; + __u32 offset; /* in, Offset within the dump */ + __u64 reserved[8]; +}; + /* ioctl numbers */ #define OCXLPMEM_MAGIC 0xCA /* OpenCAPI Persistent memory devices */ #define IOCTL_OCXLPMEM_ERROR_LOG _IOWR(OCXLPMEM_MAGIC, 0x30, struct ioctl_ocxlpmem_error_log) +#define IOCTL_OCXLPMEM_CONTROLLER_DUMP _IO(OCXLPMEM_MAGIC, 0x31) +#define IOCTL_OCXLPMEM_CONTROLLER_DUMP_DATA _IOWR(OCXLPMEM_MAGIC, 0x32, struct ioctl_ocxlpmem_controller_dump_data) +#define IOCTL_OCXLPMEM_CONTROLLER_DUMP_COMPLETE _IO(OCXLPMEM_MAGIC, 0x33) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Fri Mar 27 07:11:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11466981 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9F2D114B4 for ; Tue, 31 Mar 2020 08:59:29 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7D71820787 for ; Tue, 31 Mar 2020 08:59:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="H7bb13Em" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D71820787 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 28E9E10FC3891; Tue, 31 Mar 2020 02:00:19 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 3D20E10FC388F for ; Tue, 31 Mar 2020 02:00:17 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id BFE3E2DC6864; Mon, 30 Mar 2020 16:52:59 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547580; bh=c5YZBYO+yginh2wx+CLNIvIUORNFUjOABhnB4YY2rcQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H7bb13EmE2VTz/xNCZFvHIKXPP3TeNvxDkuS5IRaBtVx57jSBYN2U2LYEtdHLKsJ3 w4c2EU9+WAnUbQQnn2xp5vsvkGT1oVzYp2Z/e9qG1S8bNA5QKVc7iJZO9lmsj3Qm4k YIxHdAmnT7hBPI3FtB6+r7Ep+eYY2O7BpcwMIEw17UqxBr0Ip3WA1RahjZq5SGA5Ao I23uOXicZiWYalDfMaAP2NyfLEbCxHY0EdojnqMAP/6/Tf+OvBKt2S3ukyx4Pr7N3r 6phOUFhEe0kGDr8SDz4olli9RICvKa9rm5WkiTQCTRUSDQb79fCJRSVqKQf6L7vh2K CmuLTORzjDMeh9R9eyh8U6bRW8ofSvDfs5LP7/FcGukxTIsv83kPhH7UTN/r2SoUx4 0TAiI6GwCc26okqemO+Q+PxHryoajtYkGAdoiaHm03kfiZinubmAkuDC9YGhE8celo fgVtJABORQ9OQtMPr394ZBKILzYYqr1Lx6qdvZrvB2OLIfP7Qu/LC0d1Ggjro7FRFo nfEOFx2RlxtY/oMHljhuxCRnhvchR1YYea6iUPgWi01teicDbR1nfWmsbl5z+TTcD4 uyXvwNNJ98kh/UfzbI6Nzah+vwR8aHhe7kl+nvFRZHKmQQknMxkZ+Q/HK/UX9vOF4N yWRGXpaYIXGr/tLjjgQJvnFQ= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ar045934; Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 18/25] nvdimm/ocxl: Add an IOCTL to report controller statistics Date: Fri, 27 Mar 2020 18:11:55 +1100 Message-Id: <20200327071202.2159885-19-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) Message-ID-Hash: CZSE6DLAUTVKKG2RGDVDNCQPD5QX6RRQ X-Message-ID-Hash: CZSE6DLAUTVKKG2RGDVDNCQPD5QX6RRQ X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: The controller can report a number of statistics that are useful in evaluating the performance and reliability of the card. This patch exposes this information via an IOCTL. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 220 +++++++++++++++++++++++++++++++++ include/uapi/nvdimm/ocxlpmem.h | 21 ++++ 2 files changed, 241 insertions(+) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index d0db358ded43..0040fc09cceb 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -713,6 +713,221 @@ static int ioctl_controller_dump_complete(struct ocxlpmem *ocxlpmem) GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED); } +/** + * controller_stats_header_parse() - Parse the first 64 bits of the controller stats admin command response + * @ocxlpmem: the device metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int controller_stats_header_parse(struct ocxlpmem *ocxlpmem, + u32 *length) +{ + int rc; + u64 val; + + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFFFFFF; + + if (data_identifier != 0x4353) { // 'CS' + dev_err(&ocxlpmem->dev, + "Bad data identifier for error log data, expected 'CS', got '%2s' (%#x), data_length=%u\n", + (char *)&data_identifier, + (unsigned int)data_identifier, data_length); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int ioctl_controller_stats(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxlpmem_controller_stats __user *uarg) +{ + struct ioctl_ocxlpmem_controller_stats args; + u32 length; + int rc; + u64 val; + + memset(&args, '\0', sizeof(args)); + + mutex_lock(&ocxlpmem->admin_command.lock); + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.request_offset + 0x08, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out; + + rc = admin_command_execute(ocxlpmem, ADMIN_COMMAND_CONTROLLER_STATS); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + warn_status(ocxlpmem, + "Unexpected status from controller stats", rc); + goto out; + } + + rc = controller_stats_header_parse(ocxlpmem, &length); + if (rc) + goto out; + + if (length != 0x140) // Documented length of controller stats response + warn_status(ocxlpmem, + "Unexpected length for controller stats data, expected 0x140, got 0x%x", + length); + +#define SPID1_OFFSET (ocxlpmem->admin_command.data_offset + 0x08) +#define SPID2_OFFSET (ocxlpmem->admin_command.data_offset + 0x48) +#define RESET_INFO (SPID1_OFFSET + 0x08) +#define UPTIME_LIFE (SPID1_OFFSET + 0x10) +#define CRITICAL_UTIL (SPID2_OFFSET + 0x08) +#define HOST_LOAD_COUNT (SPID2_OFFSET + 0x10) +#define HOST_STORE_COUNT (SPID2_OFFSET + 0x18) +#define HOST_LOAD_DURATION (SPID2_OFFSET + 0x20) +#define HOST_STORE_DURATION (SPID2_OFFSET + 0x28) +#define MEDIA_READ_COUNT (SPID2_OFFSET + 0x50) +#define MEDIA_WRITE_COUNT (SPID2_OFFSET + 0x58) +#define MEDIA_READ_DURATION (SPID2_OFFSET + 0x60) +#define MEDIA_WRITE_DURATION (SPID2_OFFSET + 0x68) +#define CACHE_READ_HIT_COUNT (SPID2_OFFSET + 0x90) +#define CACHE_WRITE_HIT_COUNT (SPID2_OFFSET + 0x98) +#define FAST_WRITE_COUNT (SPID2_OFFSET + 0xA0) + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, SPID1_OFFSET, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + if ((val >> 56) != 0x01) { // Check the parameter ID + rc = -ENODEV; + goto out; + } + if ((val & 0xFFFFFFFF) != 0x38) { // Check the length + rc = -ENODEV; + goto out; + } + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, RESET_INFO, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + args.reset_count = val >> 32; + args.reset_uptime = val & 0xFFFFFFFF; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, UPTIME_LIFE, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + args.power_on_uptime = val >> 32; + args.life_remaining = val & 0xFF; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, SPID2_OFFSET, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + if ((val >> 56) != 0x02) { // Check the parameter ID + rc = -ENODEV; + goto out; + } + if ((val & 0xFFFFFFFF) != 0xF8) { // Check the length + rc = -ENODEV; + goto out; + } + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, CRITICAL_UTIL, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + goto out; + + args.critical_resource_utilization = val & 0xff; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, HOST_LOAD_COUNT, + OCXL_LITTLE_ENDIAN, + &args.host_load_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, HOST_STORE_COUNT, + OCXL_LITTLE_ENDIAN, + &args.host_store_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, HOST_LOAD_DURATION, + OCXL_LITTLE_ENDIAN, + &args.host_load_duration); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, HOST_STORE_DURATION, + OCXL_LITTLE_ENDIAN, + &args.host_store_duration); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, MEDIA_READ_COUNT, + OCXL_LITTLE_ENDIAN, + &args.media_read_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, MEDIA_WRITE_COUNT, + OCXL_LITTLE_ENDIAN, + &args.media_write_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, MEDIA_READ_DURATION, + OCXL_LITTLE_ENDIAN, + &args.media_read_duration); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, MEDIA_WRITE_DURATION, + OCXL_LITTLE_ENDIAN, + &args.media_write_duration); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, CACHE_READ_HIT_COUNT, + OCXL_LITTLE_ENDIAN, + &args.cache_read_hit_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, CACHE_WRITE_HIT_COUNT, + OCXL_LITTLE_ENDIAN, + &args.cache_write_hit_count); + if (rc) + goto out; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, FAST_WRITE_COUNT, + OCXL_LITTLE_ENDIAN, + &args.fast_write_count); + if (rc) + goto out; + + if (copy_to_user(uarg, &args, sizeof(args))) { + rc = -EFAULT; + goto out; + } + + rc = admin_response_handled(ocxlpmem); + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { struct ocxlpmem *ocxlpmem = file->private_data; @@ -736,6 +951,11 @@ static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) case IOCTL_OCXLPMEM_CONTROLLER_DUMP_COMPLETE: rc = ioctl_controller_dump_complete(ocxlpmem); break; + + case IOCTL_OCXLPMEM_CONTROLLER_STATS: + rc = ioctl_controller_stats(ocxlpmem, + (struct ioctl_ocxlpmem_controller_stats __user *)args); + break; } return rc; diff --git a/include/uapi/nvdimm/ocxlpmem.h b/include/uapi/nvdimm/ocxlpmem.h index 05e2b3f7b27c..ca3a7098fa9d 100644 --- a/include/uapi/nvdimm/ocxlpmem.h +++ b/include/uapi/nvdimm/ocxlpmem.h @@ -51,6 +51,26 @@ struct ioctl_ocxlpmem_controller_dump_data { __u64 reserved[8]; }; +struct ioctl_ocxlpmem_controller_stats { + __u32 reset_count; + __u32 reset_uptime; /* seconds */ + __u32 power_on_uptime; /* seconds */ + __u8 life_remaining; /* percentage, 0-100 */ + __u8 critical_resource_utilization; /* percentage, 0-100 */ + __u8 reserved[2]; + __u64 host_load_count; + __u64 host_store_count; + __u64 host_load_duration; /* nanoseconds, total */ + __u64 host_store_duration; /* nanoseconds, total */ + __u64 media_read_count; + __u64 media_write_count; + __u64 media_read_duration; /* nanoseconds, total */ + __u64 media_write_duration; /* nanoseconds, total */ + __u64 cache_read_hit_count; + __u64 cache_write_hit_count; + __u64 fast_write_count; +}; + /* ioctl numbers */ #define OCXLPMEM_MAGIC 0xCA /* OpenCAPI Persistent memory devices */ @@ -58,5 +78,6 @@ struct ioctl_ocxlpmem_controller_dump_data { #define IOCTL_OCXLPMEM_CONTROLLER_DUMP _IO(OCXLPMEM_MAGIC, 0x31) #define IOCTL_OCXLPMEM_CONTROLLER_DUMP_DATA _IOWR(OCXLPMEM_MAGIC, 0x32, struct ioctl_ocxlpmem_controller_dump_data) #define IOCTL_OCXLPMEM_CONTROLLER_DUMP_COMPLETE _IO(OCXLPMEM_MAGIC, 0x33) +#define IOCTL_OCXLPMEM_CONTROLLER_STATS _IO(OCXLPMEM_MAGIC, 0x34) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Fri Mar 27 07:11:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11466977 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D98F814B4 for ; Tue, 31 Mar 2020 08:59:19 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC37120848 for ; Tue, 31 Mar 2020 08:59:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="WFsUJw7F" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC37120848 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 03CF210FC388A; Tue, 31 Mar 2020 02:00:09 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 2849E1003E9A2 for ; Tue, 31 Mar 2020 02:00:06 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 167402DC6865; Mon, 30 Mar 2020 16:53:00 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547581; bh=vRLE2b0DEucnyqflmU5elmtDiv/1IFmmCdakHAT0FxQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WFsUJw7F0hIvPb6ZSnt7GAZpXr/E2Z9FYSbulLN05P/O+C0TaVzvwKh5K8dIeXpiB 1y8bnd3fGf1XVk3oXKsSI3fdeQmG2Me70bYNsdEBHoW8J42QpayBFlESBkFy7hhRgO 1IqLDbk9/270vDZqLI04gDZnK4Zk7KL6KQgSKfRaDWPSQEL53mUKtqvNVX19talTTY 5GQaF52LoakCZPynBwsvCdUq1XJovsBZLYme3hHOkaqo4pdalXmKJe8+4lLbuHvUMZ Y7X6oVZUKnMsZB4sbxShJLs6L/G/IAZ1MCptBoex4XCq0mx4W+x8nrgnfKDuSTbMDn fghqXsNGY5v14b0EDv0f+f9cfcMrCP/enxq03xCTHsDWAGRtNCbzS6nG4UR0xKQYyA /hNaj4Uy20cpBY5Y57b+0zCilg4+GOdZucdF3SA+TMPC9K0gq8ClMn8f1w/lp/Mn+h Jn7wjsWP505+/8Mn94Lb1USkIdTadhAnDQW+OrDzezfqlVWBThppZJKxi2C397wF2Q QzR6dAX2U3BPMQKNyjQoHMqnwmWm2ieepCeTAkAmErwjiwtz69TRaanTh+Xf61MbSy T8KOniORK8+jCPalfJasMUw6I1GNvWxqrhhlYo69FnhBtCnNWJTfrVf6ZW1N4+lYUi BpfFwmmdfaCKrHJ2RWcBpCE8= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4As045934; Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 19/25] nvdimm/ocxl: Forward events to userspace Date: Fri, 27 Mar 2020 18:11:56 +1100 Message-Id: <20200327071202.2159885-20-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) Message-ID-Hash: 4MYDKY6BJ7ILMLOGKYLS663AIK6TAF5O X-Message-ID-Hash: 4MYDKY6BJ7ILMLOGKYLS663AIK6TAF5O X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Some of the interrupts that the card generates are better handled by the userspace daemon, in particular: Controller Hardware/Firmware Fatal Controller Dump Available Error Log available This patch allows a userspace application to register an eventfd with the driver via SCM_IOCTL_EVENTFD to receive notifications of these interrupts. Userspace can then identify what events have occurred by calling SCM_IOCTL_EVENT_CHECK and checking against the SCM_IOCTL_EVENT_FOO masks. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 220 +++++++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/ocxlpmem.h | 4 + include/uapi/nvdimm/ocxlpmem.h | 12 ++ 3 files changed, 236 insertions(+) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index 0040fc09cceb..cb6cdc9eb899 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -301,8 +302,19 @@ static void free_ocxlpmem(struct ocxlpmem *ocxlpmem) { int rc; + // Disable doorbells + (void)ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIEC, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_CHI_ALL); + free_minor(ocxlpmem); + if (ocxlpmem->irq_addr[1]) + iounmap(ocxlpmem->irq_addr[1]); + + if (ocxlpmem->irq_addr[0]) + iounmap(ocxlpmem->irq_addr[0]); + if (ocxlpmem->ocxl_context) { rc = ocxl_context_detach(ocxlpmem->ocxl_context); if (rc == -EBUSY) @@ -398,6 +410,11 @@ static int file_release(struct inode *inode, struct file *file) { struct ocxlpmem *ocxlpmem = file->private_data; + if (ocxlpmem->ev_ctx) { + eventfd_ctx_put(ocxlpmem->ev_ctx); + ocxlpmem->ev_ctx = NULL; + } + ocxlpmem_put(ocxlpmem); return 0; } @@ -928,6 +945,52 @@ static int ioctl_controller_stats(struct ocxlpmem *ocxlpmem, return rc; } +static int ioctl_eventfd(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxlpmem_eventfd __user *uarg) +{ + struct ioctl_ocxlpmem_eventfd args; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + if (ocxlpmem->ev_ctx) + return -EBUSY; + + ocxlpmem->ev_ctx = eventfd_ctx_fdget(args.eventfd); + if (IS_ERR(ocxlpmem->ev_ctx)) + return PTR_ERR(ocxlpmem->ev_ctx); + + return 0; +} + +static int ioctl_event_check(struct ocxlpmem *ocxlpmem, u64 __user *uarg) +{ + u64 val = 0; + int rc; + u64 chi = 0; + + rc = ocxlpmem_chi(ocxlpmem, &chi); + if (rc < 0) + return rc; + + if (chi & GLOBAL_MMIO_CHI_ELA) + val |= IOCTL_OCXLPMEM_EVENT_ERROR_LOG_AVAILABLE; + + if (chi & GLOBAL_MMIO_CHI_CDA) + val |= IOCTL_OCXLPMEM_EVENT_CONTROLLER_DUMP_AVAILABLE; + + if (chi & GLOBAL_MMIO_CHI_CFFS) + val |= IOCTL_OCXLPMEM_EVENT_FIRMWARE_FATAL; + + if (chi & GLOBAL_MMIO_CHI_CHFS) + val |= IOCTL_OCXLPMEM_EVENT_HARDWARE_FATAL; + + if (copy_to_user((u64 __user *)uarg, &val, sizeof(val))) + return -EFAULT; + + return rc; +} + static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { struct ocxlpmem *ocxlpmem = file->private_data; @@ -956,6 +1019,15 @@ static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) rc = ioctl_controller_stats(ocxlpmem, (struct ioctl_ocxlpmem_controller_stats __user *)args); break; + + case IOCTL_OCXLPMEM_EVENTFD: + rc = ioctl_eventfd(ocxlpmem, + (struct ioctl_ocxlpmem_eventfd __user *)args); + break; + + case IOCTL_OCXLPMEM_EVENT_CHECK: + rc = ioctl_event_check(ocxlpmem, (u64 __user *)args); + break; } return rc; @@ -1109,6 +1181,148 @@ static void dump_error_log(struct ocxlpmem *ocxlpmem) kfree(buf); } +static irqreturn_t imn0_handler(void *private) +{ + struct ocxlpmem *ocxlpmem = private; + u64 chi = 0; + + (void)ocxlpmem_chi(ocxlpmem, &chi); + + if (chi & GLOBAL_MMIO_CHI_ELA) { + dev_warn(&ocxlpmem->dev, "Error log is available\n"); + + if (ocxlpmem->ev_ctx) + eventfd_signal(ocxlpmem->ev_ctx, 1); + } + + if (chi & GLOBAL_MMIO_CHI_CDA) { + dev_warn(&ocxlpmem->dev, "Controller dump is available\n"); + + if (ocxlpmem->ev_ctx) + eventfd_signal(ocxlpmem->ev_ctx, 1); + } + + return IRQ_HANDLED; +} + +static irqreturn_t imn1_handler(void *private) +{ + struct ocxlpmem *ocxlpmem = private; + u64 chi = 0; + + (void)ocxlpmem_chi(ocxlpmem, &chi); + + if (chi & (GLOBAL_MMIO_CHI_CFFS | GLOBAL_MMIO_CHI_CHFS)) { + dev_err(&ocxlpmem->dev, + "Controller status is fatal, chi=0x%llx, going offline\n", + chi); + + if (ocxlpmem->nvdimm_bus) { + nvdimm_bus_unregister(ocxlpmem->nvdimm_bus); + ocxlpmem->nvdimm_bus = NULL; + } + + if (ocxlpmem->ev_ctx) + eventfd_signal(ocxlpmem->ev_ctx, 1); + } + + return IRQ_HANDLED; +} + +/** + * ocxlpmem_setup_irq() - Set up the IRQs for the OpenCAPI Persistent Memory device + * @ocxlpmem: the device metadata + * Return: 0 on success, negative on failure + */ +static int setup_irqs(struct ocxlpmem *ocxlpmem) +{ + int rc; + u64 irq_addr; + + rc = ocxl_afu_irq_alloc(ocxlpmem->ocxl_context, + &ocxlpmem->irq_id[0]); + if (rc) + return rc; + + rc = ocxl_irq_set_handler(ocxlpmem->ocxl_context, ocxlpmem->irq_id[0], + imn0_handler, NULL, ocxlpmem); + + irq_addr = ocxl_afu_irq_get_addr(ocxlpmem->ocxl_context, + ocxlpmem->irq_id[0]); + if (!irq_addr) + return -EFAULT; + + ocxlpmem->irq_addr[0] = ioremap(irq_addr, PAGE_SIZE); + if (!ocxlpmem->irq_addr[0]) + return -ENODEV; + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_IMA0_OHP, + OCXL_LITTLE_ENDIAN, + (u64)ocxlpmem->irq_addr[0]); + if (rc) + goto out_irq0; + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_IMA0_CFP, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out_irq0; + + rc = ocxl_afu_irq_alloc(ocxlpmem->ocxl_context, &ocxlpmem->irq_id[1]); + if (rc) + goto out_irq0; + + rc = ocxl_irq_set_handler(ocxlpmem->ocxl_context, ocxlpmem->irq_id[1], + imn1_handler, NULL, ocxlpmem); + if (rc) + goto out_irq0; + + irq_addr = ocxl_afu_irq_get_addr(ocxlpmem->ocxl_context, + ocxlpmem->irq_id[1]); + if (!irq_addr) { + rc = -EFAULT; + goto out_irq0; + } + + ocxlpmem->irq_addr[1] = ioremap(irq_addr, PAGE_SIZE); + if (!ocxlpmem->irq_addr[1]) { + rc = -ENODEV; + goto out_irq0; + } + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_IMA1_OHP, + OCXL_LITTLE_ENDIAN, + (u64)ocxlpmem->irq_addr[1]); + if (rc) + goto out_irq1; + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_IMA1_CFP, + OCXL_LITTLE_ENDIAN, 0); + if (rc) + goto out_irq1; + + // Enable doorbells + rc = ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIE, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_CHI_ELA | + GLOBAL_MMIO_CHI_CDA | + GLOBAL_MMIO_CHI_CFFS | + GLOBAL_MMIO_CHI_CHFS); + if (rc) + goto out_irq1; + + return 0; + +out_irq1: + iounmap(ocxlpmem->irq_addr[1]); + ocxlpmem->irq_addr[1] = NULL; + +out_irq0: + iounmap(ocxlpmem->irq_addr[0]); + ocxlpmem->irq_addr[0] = NULL; + + return rc; +} + /** * probe_function0() - Set up function 0 for an OpenCAPI persistent memory device * This is important as it enables templates higher than 0 across all other @@ -1212,6 +1426,12 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + rc = setup_irqs(ocxlpmem); + if (rc) { + dev_err(&pdev->dev, "Could not set up OCXL IRQs\n"); + goto err; + } + rc = setup_command_metadata(ocxlpmem); if (rc) { dev_err(&pdev->dev, "Could not read command metadata\n"); diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index ee3bd651f254..01721596f982 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -106,6 +106,9 @@ struct ocxlpmem { struct pci_dev *pdev; struct cdev cdev; struct ocxl_fn *ocxl_fn; +#define SCM_IRQ_COUNT 2 + int irq_id[SCM_IRQ_COUNT]; + void __iomem *irq_addr[SCM_IRQ_COUNT]; struct nd_interleave_set nd_set; struct nvdimm_bus_descriptor bus_desc; struct nvdimm_bus *nvdimm_bus; @@ -117,6 +120,7 @@ struct ocxlpmem { struct nd_region *nd_region; char fw_version[8 + 1]; u32 timeouts[ADMIN_COMMAND_MAX + 1]; + struct eventfd_ctx *ev_ctx; u32 max_controller_dump_size; u16 scm_revision; // major/minor u8 readiness_timeout; /* The worst case time (in seconds) that the host diff --git a/include/uapi/nvdimm/ocxlpmem.h b/include/uapi/nvdimm/ocxlpmem.h index ca3a7098fa9d..d573bd307e35 100644 --- a/include/uapi/nvdimm/ocxlpmem.h +++ b/include/uapi/nvdimm/ocxlpmem.h @@ -71,6 +71,16 @@ struct ioctl_ocxlpmem_controller_stats { __u64 fast_write_count; }; +struct ioctl_ocxlpmem_eventfd { + __s32 eventfd; + __u32 reserved; +}; + +#define IOCTL_OCXLPMEM_EVENT_CONTROLLER_DUMP_AVAILABLE (1ULL << (0)) +#define IOCTL_OCXLPMEM_EVENT_ERROR_LOG_AVAILABLE (1ULL << (1)) +#define IOCTL_OCXLPMEM_EVENT_HARDWARE_FATAL (1ULL << (2)) +#define IOCTL_OCXLPMEM_EVENT_FIRMWARE_FATAL (1ULL << (3)) + /* ioctl numbers */ #define OCXLPMEM_MAGIC 0xCA /* OpenCAPI Persistent memory devices */ @@ -79,5 +89,7 @@ struct ioctl_ocxlpmem_controller_stats { #define IOCTL_OCXLPMEM_CONTROLLER_DUMP_DATA _IOWR(OCXLPMEM_MAGIC, 0x32, struct ioctl_ocxlpmem_controller_dump_data) #define IOCTL_OCXLPMEM_CONTROLLER_DUMP_COMPLETE _IO(OCXLPMEM_MAGIC, 0x33) #define IOCTL_OCXLPMEM_CONTROLLER_STATS _IO(OCXLPMEM_MAGIC, 0x34) +#define IOCTL_OCXLPMEM_EVENTFD _IOW(OCXLPMEM_MAGIC, 0x35, struct ioctl_ocxlpmem_eventfd) +#define IOCTL_OCXLPMEM_EVENT_CHECK _IOR(OCXLPMEM_MAGIC, 0x36, __u64) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Fri Mar 27 07:11:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464643 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6FDD1668 for ; Mon, 30 Mar 2020 05:53:00 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8668520786 for ; Mon, 30 Mar 2020 05:53:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="hoY0Amw8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8668520786 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 4D7CB10FC3789; Sun, 29 Mar 2020 22:53:50 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 8A0F81003ECB6 for ; Sun, 29 Mar 2020 22:53:48 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 7AF382DC6858; Mon, 30 Mar 2020 16:52:48 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547569; bh=g2tKnklWJ8nRzWz5jOb4SQnCXOC8tlA+PKeUIsvkvgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hoY0Amw8E3jMSf124cMI1wOBWdhV16swXKrVuGoVuKJBlh/3/iI04X2bHZTflwSVT rXkGmTHKruF/ots8Q5MKXaWfN0FTpt6tBiDh44ZwOyTu+MjrxVmhH3f+1ZbJvqCY9T 8p4a6Z+1e0cD086gWlUP0filTlBTCQyDL6L4ikJ5KfcZAd/WFDiTUxYmERPNcQ8LO1 dVvD31gngYJE56oQA2OMztgkhZwDMSuUyo4rurny1G2ArnKsFgg755eAnzTE+dDBLB kusCv/GKPBY04EUChSAd95Z301G9lNDqGLp6YwFYrcwEk/0HugQDFLS/6em0wul/3e eJ1lBvVRdTCXPo2EenFtjKu9WVdT3b08+EVkoY7u01Tvsy8RUkHavAhN/EXnGBYy7b 6teojKsAvPV4TXtwVIlKKXPgk1+euhW7uNgDbDCu4wJ6aVdak6PuZFkaydM7I/Io+x GIsf1TuI8RrZUBymhW1SvPuTVpH2iedMZZwDQJKIA9RByNtaLZLszsx59gxS26bFfE cWr0XYAzlnA2pq1V4GBMTtfvLJ2bKIFX84BVuPasQ7aB0Gwk8tem7NaxLJAp910yoF +lXFcuTaHlxiopdYshGLHOXT1lwOv3FyRkv/jAc2cyQrqp7AdMd/4ovUfT2+/bYZ/C er5boYLjD7a0hnoy9MFr6D50= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4At045934; Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 20/25] nvdimm/ocxl: Add an IOCTL to request controller health & perf data Date: Fri, 27 Mar 2020 18:11:57 +1100 Message-Id: <20200327071202.2159885-21-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) Message-ID-Hash: KKLTMUKAWMVSFLJN535HJYYOGRYO7F5T X-Message-ID-Hash: KKLTMUKAWMVSFLJN535HJYYOGRYO7F5T X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: When health & performance data is requested from the controller, it responds with an error log containing the requested information. This patch allows the request to be issued via an IOCTL, the data can later be collected in userspace via the error log IOCTL introduced in a previous patch. Userspace will be notified of pending error logs via an event. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 16 ++++++++++++++++ include/uapi/nvdimm/ocxlpmem.h | 1 + 2 files changed, 17 insertions(+) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index cb6cdc9eb899..a4315472683c 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -991,6 +991,18 @@ static int ioctl_event_check(struct ocxlpmem *ocxlpmem, u64 __user *uarg) return rc; } +/** + * req_controller_health_perf() - Request controller health & performance data + * @ocxlpmem: the device metadata + * Return: 0 on success, negative on failure + */ +int req_controller_health_perf(struct ocxlpmem *ocxlpmem) +{ + return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_REQ_HEALTH_PERF); +} + static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { struct ocxlpmem *ocxlpmem = file->private_data; @@ -1028,6 +1040,10 @@ static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) case IOCTL_OCXLPMEM_EVENT_CHECK: rc = ioctl_event_check(ocxlpmem, (u64 __user *)args); break; + + case IOCTL_OCXLPMEM_REQUEST_HEALTH: + rc = req_controller_health_perf(ocxlpmem); + break; } return rc; diff --git a/include/uapi/nvdimm/ocxlpmem.h b/include/uapi/nvdimm/ocxlpmem.h index d573bd307e35..9c5c8585c1c2 100644 --- a/include/uapi/nvdimm/ocxlpmem.h +++ b/include/uapi/nvdimm/ocxlpmem.h @@ -91,5 +91,6 @@ struct ioctl_ocxlpmem_eventfd { #define IOCTL_OCXLPMEM_CONTROLLER_STATS _IO(OCXLPMEM_MAGIC, 0x34) #define IOCTL_OCXLPMEM_EVENTFD _IOW(OCXLPMEM_MAGIC, 0x35, struct ioctl_ocxlpmem_eventfd) #define IOCTL_OCXLPMEM_EVENT_CHECK _IOR(OCXLPMEM_MAGIC, 0x36, __u64) +#define IOCTL_OCXLPMEM_REQUEST_HEALTH _IO(OCXLPMEM_MAGIC, 0x37) #endif /* _UAPI_OCXL_SCM_H */ From patchwork Fri Mar 27 07:11:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464639 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 062471668 for ; Mon, 30 Mar 2020 05:52:56 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D7BC320786 for ; Mon, 30 Mar 2020 05:52:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="iMjl+0uX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D7BC320786 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7325110FC36F9; Sun, 29 Mar 2020 22:53:45 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id EB14E10FC36CD for ; Sun, 29 Mar 2020 22:53:42 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 277FA2DC684C; Mon, 30 Mar 2020 16:52:46 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547567; bh=/c0sygyGptvZJ7mjxJ2I2ERy9RY2BO4lgAUmGcAOvOI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iMjl+0uXIk2WsRYFK7QMnoeZwiXtbQvngMc30HklM7M/UaYiwbRU0QE9+G/GnjCpF 6TnIi2bluBUjI33oB/IGG++wI53XGTIwCq5K0VlcVhmAbh8fhf/pZM4Ced6P4QxUan 4mCoqUpqEQuPXCesRSY06c5qvKiv1LsP8cfIca3WUYgQYOJ/q/s8LwSNJX9HBkkIhn gDlQa2bD02tm9GPOYMeF42o7Bj/XBxnHbDvb+sZWgfzyMO+XkIa7CR1jZ4i6HRAFM+ YbdnR295TH0kUDCjW5WeNmhPseRb1h1xcrGkr1E7ZJs7ZsylKwzALmMvOAhH9LL5vq 3sSEcN0wjhm82H8oqQYmkz6pFE9Ngz7Yk7lRcCnGpzQVRWYRpVTksaPYJsbD/6F8t2 MetLhu1IrJSl7OBndz4jJBgg6mmUQXdE5qPmzLqjrTwMXPbnNkvRn55MI0MKPAsiA2 v3UhZHqDsS3yj8k9ftzhbgmpM9o3tilShHFLuLa6EEPGAyGhkC0JmK4UJSSU4vuBsl DAAPAo9W0AH+Quua6imoMGuoYzJ2F3ro6L2zDSZSLZw6BC3GtyzRpLXvgZ5C/FjG+f tIZ+HRu9YYQm06ECg8BUrm41v1jxxG21JM4D7E34KQMuRdIJeivVBNTbxAwVDV35Ax wfbVKJDpJWOjuCjbEN7nnoZA= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Au045934; Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 21/25] nvdimm/ocxl: Implement the heartbeat command Date: Fri, 27 Mar 2020 18:11:58 +1100 Message-Id: <20200327071202.2159885-22-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) Message-ID-Hash: 2HW65I34VO4LRQW64BM7GGN6FSAJG3IL X-Message-ID-Hash: 2HW65I34VO4LRQW64BM7GGN6FSAJG3IL X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: The heartbeat admin command is a simple admin command that exercises the communication mechanisms within the controller. This patch issues a heartbeat command to the card during init to ensure we can communicate with the card's controller. Signed-off-by: Alastair D'Silva Reviewed-by: Andrew Donnellan --- drivers/nvdimm/ocxl/main.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index a4315472683c..2fbe3f2f77d9 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -272,6 +272,30 @@ static int setup_command_metadata(struct ocxlpmem *ocxlpmem) return 0; } +/** + * heartbeat() - Issue a heartbeat command to the controller + * @ocxlpmem: the device metadata + * Return: 0 if the controller responded correctly, negative on error + */ +static int heartbeat(struct ocxlpmem *ocxlpmem) +{ + int rc; + + mutex_lock(&ocxlpmem->admin_command.lock); + + rc = admin_command_execute(ocxlpmem, ADMIN_COMMAND_HEARTBEAT); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) + warn_status(ocxlpmem, "Unexpected status from heartbeat", rc); + + (void)admin_response_handled(ocxlpmem); + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + /** * allocate_minor() - Allocate a minor number to use for an OpenCAPI pmem device * @ocxlpmem: the device metadata @@ -1460,6 +1484,12 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err; } + rc = heartbeat(ocxlpmem); + if (rc) { + dev_err(&pdev->dev, "Heartbeat failed\n"); + goto err; + } + elapsed = 0; timeout = ocxlpmem->readiness_timeout + ocxlpmem->memory_available_timeout; From patchwork Fri Mar 27 07:11:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11466993 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0790014B4 for ; Tue, 31 Mar 2020 08:59:47 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D9CBA20848 for ; Tue, 31 Mar 2020 08:59:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="WEAkgqy8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9CBA20848 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 976A810FC3894; Tue, 31 Mar 2020 02:00:36 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 095A310FC3890 for ; Tue, 31 Mar 2020 02:00:34 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 709922DC6863; Mon, 30 Mar 2020 16:52:58 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547579; bh=/ca1KYpFanyYMVbs68dr19xdKxkz41V0vgTQ9aYJwrk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WEAkgqy8sYoMNXNSHtEMHgFyUhgF7eswqGIzw5cTL9d8Ce/mAiuF+j3x6UsLKfeCJ sKdZc2iP7tvcfSfPjHmM5V72eHMWyvPdp7/ls+jgJF3WTd81TsaF4PPVObcERiLV61 hiNs5FJcPjagQHqbTXyR15J5ngFd+eo6SsOLV0X8YkpTsMEm0ICDvFiSfEiuay91kn 4ohxvdABPi+smiQm+O9dLWUxRXXz3jk1VFkHIcC+iC8yftoAlg/6XGbxqSs08xQu9A 55RvHF+bmzPvzR3igzK2spwG9M3j9SURWqI7vlJXLMHk9HAlN1q/5Jdh/G5VOSaHJ1 88QkO+qFKB48gAQ8Z9DkVMS5xIXff17UEwjngE3ppAKkQotDdHDo6o7O8ksTciapJW djlsNB+lf1rFQLBA29PqyfwXkZwEFhVhgTDCCbc0a8ga7B0O85V+VqzuT4ODU2bPdd ocBKT+GnvkXD3nwPqVZ39v0lOsGIAb5Bgly9fNCf1+DugXZQMjoBHNinZ1XefGz02V 93blURIhnMD1Nk1FodnmCjbywYdB6f2mRb+jEwqvcQogdJN7xE+qxXW3kpuvBeU5dX NrokEv2reY/lK0p+vku/YE1ZckI+CDcB+Tjt2inhEyRDDJftojDu7CjT6J9Q/6B0rn VwiJrr14r7AX8jm6PdMRVxvc= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Av045934; Fri, 27 Mar 2020 18:12:19 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 22/25] nvdimm/ocxl: Add debug IOCTLs Date: Fri, 27 Mar 2020 18:11:59 +1100 Message-Id: <20200327071202.2159885-23-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) Message-ID-Hash: BFKLZYRCCOZJEBNSTRTEBGFLGNAVU2UL X-Message-ID-Hash: BFKLZYRCCOZJEBNSTRTEBGFLGNAVU2UL X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: These IOCTLs provide low level access to the card to aid in debugging controller/FPGA firmware. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/Kconfig | 6 + drivers/nvdimm/ocxl/main.c | 198 +++++++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/ocxlpmem.h | 2 +- include/uapi/nvdimm/ocxlpmem.h | 31 ++++++ 4 files changed, 236 insertions(+), 1 deletion(-) diff --git a/drivers/nvdimm/ocxl/Kconfig b/drivers/nvdimm/ocxl/Kconfig index c5d927520920..3f44429d70c9 100644 --- a/drivers/nvdimm/ocxl/Kconfig +++ b/drivers/nvdimm/ocxl/Kconfig @@ -12,4 +12,10 @@ config OCXL_PMEM Select N if unsure. +config OCXL_PMEM_DEBUG + bool "OpenCAPI Persistent Memory debugging" + depends on OCXL_PMEM + help + Enables low level IOCTLs for OpenCAPI Persistent Memory firmware development + endif diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index 2fbe3f2f77d9..2811bf7efbab 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -1027,6 +1027,183 @@ int req_controller_health_perf(struct ocxlpmem *ocxlpmem) GLOBAL_MMIO_HCI_REQ_HEALTH_PERF); } +#ifdef CONFIG_OCXL_PMEM_DEBUG +/** + * enable_fwdebug() - Enable FW debug on the controller + * @ocxlpmem: the device metadata + * Return: 0 on success, negative on failure + */ +static int enable_fwdebug(const struct ocxlpmem *ocxlpmem) +{ + return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_FW_DEBUG); +} + +/** + * disable_fwdebug() - Disable FW debug on the controller + * @ocxlpmem: the device metadata + * Return: 0 on success, negative on failure + */ +static int disable_fwdebug(const struct ocxlpmem *ocxlpmem) +{ + return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCIC, + OCXL_LITTLE_ENDIAN, + GLOBAL_MMIO_HCI_FW_DEBUG); +} + +static int ioctl_fwdebug(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxl_pmem_fwdebug __user *uarg) +{ + struct ioctl_ocxl_pmem_fwdebug args; + u64 val; + int i; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + // Buffer size must be a multiple of 8 + if ((args.buf_size & 0x07)) + return -EINVAL; + + if (args.buf_size > ocxlpmem->admin_command.data_size) + return -EINVAL; + + mutex_lock(&ocxlpmem->admin_command.lock); + + rc = enable_fwdebug(ocxlpmem); + if (rc) + goto out; + + // Write DebugAction & FunctionCode + val = ((u64)args.debug_action << 56) | ((u64)args.function_code << 40); + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.request_offset + 0x08, + OCXL_LITTLE_ENDIAN, val); + if (rc) + goto out; + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.request_offset + 0x10, + OCXL_LITTLE_ENDIAN, + args.debug_parameter_1); + if (rc) + goto out; + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.request_offset + 0x18, + OCXL_LITTLE_ENDIAN, + args.debug_parameter_2); + if (rc) + goto out; + + // Populate admin command buffer + if (args.buf_size) { + for (i = 0; i < args.buf_size; i += sizeof(u64)) { + u64 val; + + if (copy_from_user(&val, &args.buf[i], sizeof(u64))) { + rc = -EFAULT; + goto out; + } + + rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, val); + if (rc) + goto out; + } + } + + rc = admin_command_execute(ocxlpmem, + ocxlpmem->timeouts[ADMIN_COMMAND_FW_DEBUG]); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + warn_status(ocxlpmem, "Unexpected status from FW Debug", rc); + goto out; + } + + if (args.buf_size) { + for (i = 0; i < args.buf_size; i += sizeof(u64)) { + u64 val; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + i, + OCXL_HOST_ENDIAN, &val); + if (rc) + goto out; + + if (copy_to_user(&args.buf[i], &val, sizeof(u64))) { + rc = -EFAULT; + goto out; + } + } + } + + rc = admin_response_handled(ocxlpmem); + if (rc) + goto out; + + rc = disable_fwdebug(ocxlpmem); + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + +static int ioctl_shutdown(struct ocxlpmem *ocxlpmem) +{ + int rc; + + mutex_lock(&ocxlpmem->admin_command.lock); + + rc = admin_command_execute(ocxlpmem, ADMIN_COMMAND_SHUTDOWN); + if (rc) + goto out; + + rc = admin_response_handled(ocxlpmem); + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + +static int ioctl_mmio_write(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxl_pmem_mmio __user *uarg) +{ + struct scm_ioctl_mmio args; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + return ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, args.address, + OCXL_LITTLE_ENDIAN, args.val); +} + +static int ioctl_mmio_read(struct ocxlpmem *ocxlpmem, + struct ioctl_ocxl_pmem_mmio __user *uarg) +{ + struct ioctl_ocxl_pmem_mmio args; + int rc; + + if (copy_from_user(&args, uarg, sizeof(args))) + return -EFAULT; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, args.address, + OCXL_LITTLE_ENDIAN, &args.val); + if (rc) + return rc; + + if (copy_to_user(uarg, &args, sizeof(args))) + return -EFAULT; + + return 0; +} +#endif /* CONFIG_OCXL_PMEM_DEBUG */ + static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) { struct ocxlpmem *ocxlpmem = file->private_data; @@ -1068,6 +1245,27 @@ static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args) case IOCTL_OCXLPMEM_REQUEST_HEALTH: rc = req_controller_health_perf(ocxlpmem); break; + +#ifdef CONFIG_OCXL_PMEM_DEBUG + case IOCTL_OCXL_PMEM_FWDEBUG: + rc = ioctl_fwdebug(ocxlpmem, + (struct ioctl_ocxl_pmem_fwdebug __user *)args); + break; + + case IOCTL_OCXL_PMEM_SHUTDOWN: + rc = ioctl_shutdown(ocxlpmem); + break; + + case IOCTL_OCXL_PMEM_MMIO_WRITE: + rc = ioctl_mmio_write(ocxlpmem, + (struct ioctl_ocxl_pmem_mmio __user *)args); + break; + + case IOCTL_OCXL_PMEM_MMIO_READ: + rc = ioctl_mmio_read(ocxlpmem, + (struct ioctl_ocxl_pmem_mmio __user *)args); + break; +#endif } return rc; diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index 01721596f982..c8794e7775ec 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -158,7 +158,7 @@ int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi); * * @ocxlpmem: the device metadata * @op_code: the code for the admin command - * Returns 0 on success, -EINVAL for a bad op code, -EBUSY on timeout +// * Returns 0 on success, -EINVAL for a bad op code, -EBUSY on timeout */ int admin_command_execute(struct ocxlpmem *ocxlpmem, u8 op_code); diff --git a/include/uapi/nvdimm/ocxlpmem.h b/include/uapi/nvdimm/ocxlpmem.h index 9c5c8585c1c2..374b70690b3a 100644 --- a/include/uapi/nvdimm/ocxlpmem.h +++ b/include/uapi/nvdimm/ocxlpmem.h @@ -93,4 +93,35 @@ struct ioctl_ocxlpmem_eventfd { #define IOCTL_OCXLPMEM_EVENT_CHECK _IOR(OCXLPMEM_MAGIC, 0x36, __u64) #define IOCTL_OCXLPMEM_REQUEST_HEALTH _IO(OCXLPMEM_MAGIC, 0x37) +/* + * Debug IOCTLs, these are only available if the kernel is compiled with + * CONFIG_OCXLPMEM_DEBUG + */ + +#define OCXLPMEM_FWDEBUG_READ_CONTROLLER_MEMORY 0x01 +#define OCXLPMEM_FWDEBUG_WRITE_CONTROLLER_MEMORY 0x02 +#define OCXLPMEM_FWDEBUG_ENABLE_FUNCTION 0x03 +#define OCXLPMEM_FWDEBUG_DISABLE_FUNCTION 0x04 +#define OCXLPMEM_FWDEBUG_GET_PEL 0x05 /* Retrieve Persistent Error Log */ + +struct ioctl_ocxlpmem_fwdebug { /* All args are inputs */ + __u64 debug_parameter_1; + __u64 debug_parameter_2; + __u64 buf; /* Coerced pointer to optional in/out data buffer */ + __u16 debug_action; /* OCXLPMEM_FWDEBUG_... */ + __u16 function_code; + __u16 buf_size; /* Size of optional data buffer */ + __u16 reserved; +}; + +struct ioctl_ocxlpmem_mmio { + __u64 address; /* Offset in global MMIO space */ + __u64 val; /* value to write/was read */ +}; + +#define IOCTL_OCXLPMEM_FWDEBUG _IOWR(OCXLPMEM_MAGIC, 0xf0, struct ioctl_ocxlpmem_fwdebug) +#define IOCTL_OCXLPMEM_MMIO_WRITE _IOW(OCXLPMEM_MAGIC, 0xf1, struct ioctl_ocxlpmem_mmio) +#define IOCTL_OCXLPMEM_MMIO_READ _IOWR(OCXLPMEM_MAGIC, 0xf2, struct ioctl_ocxlpmem_mmio) +#define IOCTL_OCXLPMEM_SHUTDOWN _IO(OCXLPMEM_MAGIC, 0xf3) + #endif /* _UAPI_OCXL_SCM_H */ From patchwork Fri Mar 27 07:12:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464663 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42BB41668 for ; Mon, 30 Mar 2020 05:53:22 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 212042083E for ; Mon, 30 Mar 2020 05:53:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="NrRTkA2b" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 212042083E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id DA16B10FC3883; Sun, 29 Mar 2020 22:54:11 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 4F41510FC3795 for ; Sun, 29 Mar 2020 22:54:09 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 23A0E2DC6861; Mon, 30 Mar 2020 16:52:54 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547576; bh=8s7da4KbOpaV4mqdxyxNi6aCXQRjcGL9fg5XH8aq39A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NrRTkA2bxZwFMeoCf0Wr6Odf6o65lo0QM+ThRM66hf/7Tb+pvtJVuQxJN394NTysC VgilHFKK5xR1f6lDBP0QlOXi2EJtvYeXpa6JFreVlZ03HY/0O6eVn0cvRH8pvqRpyo cdzWDRkWYXg1MPX4tN1fXYDXGdiYygmqECTUp+0YnDPD6LfGx+xx+gategBGRubdAL L0JRYtfskZ/D5Zw0QaK/Tz53JgYEGDdF037VYvOeTQp7vuawto13Te3Rm7pe7J0n3j H6iwZUTYry2yNKsgzOEM59Q5V7lk5UymbYbT6X5deJlY236MnLO6jpfeBIjPT8WeYG zbyXaR3DorudKtfRcOMCYngpMoCzEzYlFjON6mwdALYIvaxuWIpUi0eUv4wErujoca ev2YxG6LJtlTTbFLoAYDap+my1DyfAet2uXwgjOsEAogxT/DBD0646Y0yGqEn2L3J6 hinFHFG49vIm36L/NCftTmxqx7+9UfvO5s+VkIilw2q9UHK+gBNkJiRQVDG1yFm5Rn co+eZdxegSRaD39VqYwEY+iKgvu/Jb1w0cb0ho+kiB03px14BmTNlQk4Tof0LqmFNk 7tVG7Vgv/QEAcjH8bPZwH8sZw7WayGyD+FfKrWAwVv7XeAsc2mBHBKPSE++URXQ1Nz MdXl8DGDe8LMBnwkv+CQ57LQ= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Aw045934; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 23/25] nvdimm/ocxl: Expose SMART data via ndctl Date: Fri, 27 Mar 2020 18:12:00 +1100 Message-Id: <20200327071202.2159885-24-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) Message-ID-Hash: J5ASZDBHX72SLXCVWSZJ3Y5LCNSWWBVD X-Message-ID-Hash: J5ASZDBHX72SLXCVWSZJ3Y5LCNSWWBVD X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch retrieves proprietary formatted SMART data and makes it available via ndctl. A later contribution will be made to ndctl to parse this data. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 113 +++++++++++++++++++++++++++++++++ drivers/nvdimm/ocxl/ocxlpmem.h | 18 ++++++ include/uapi/linux/ndctl.h | 1 + 3 files changed, 132 insertions(+) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index 2811bf7efbab..92b4389e8cbb 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -82,6 +82,114 @@ static int ndctl_config_size(struct nd_cmd_get_config_size *command) return 0; } +/** + * smart_header_parse() - Parse the first 64 bits of the SMART admin command response + * @ocxlpmem: the device metadata + * @length: out, returns the number of bytes in the response (excluding the 64 bit header) + */ +static int smart_header_parse(struct ocxlpmem *ocxlpmem, u32 *length) +{ + int rc; + u64 val; + + u16 data_identifier; + u32 data_length; + + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset, + OCXL_LITTLE_ENDIAN, &val); + if (rc) + return rc; + + data_identifier = val >> 48; + data_length = val & 0xFFFFFFFF; + + if (data_identifier != 0x534D) { // 'SM' + dev_err(&ocxlpmem->dev, + "Bad data identifier for smart data, expected 'SM', got '%2s'\n", + (char *)&data_identifier); + return -EINVAL; + } + + *length = data_length; + return 0; +} + +static int ndctl_smart(struct ocxlpmem *ocxlpmem, struct nd_cmd_pkg *pkg) +{ + u32 length, i; + struct nd_ocxl_smart *out; + int rc; + + mutex_lock(&ocxlpmem->admin_command.lock); + + rc = admin_command_execute(ocxlpmem, ADMIN_COMMAND_SMART); + if (rc < 0) + goto out; + if (rc != STATUS_SUCCESS) { + warn_status(ocxlpmem, "Unexpected status from SMART", rc); + goto out; + } + + rc = smart_header_parse(ocxlpmem, &length); + if (rc) + goto out; + + pkg->nd_fw_size = length + offsetof(struct nd_ocxl_smart, attribs); + + length = min(length, (u32)(pkg->nd_size_out - + offsetof(struct nd_ocxl_smart, attribs))); // bytes + out = (struct nd_ocxl_smart *)pkg->nd_payload; + // Each SMART attribute is 2 * 64 bits + out->count = length / (2 * sizeof(u64)); // attributes + + for (i = 0; i < length; i += sizeof(u64)) { + rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, + ocxlpmem->admin_command.data_offset + + sizeof(u64) + i, + OCXL_LITTLE_ENDIAN, + &out->attribs[i / sizeof(u64)]); + if (rc) + goto out; + } + + rc = admin_response_handled(ocxlpmem); + if (rc) + goto out; + +out: + mutex_unlock(&ocxlpmem->admin_command.lock); + return rc; +} + +static int ndctl_call(struct ocxlpmem *ocxlpmem, void *buf, + unsigned int buf_len) +{ + struct nd_cmd_pkg *pkg = buf; + + if (buf_len < sizeof(struct nd_cmd_pkg)) { + dev_err(&ocxlpmem->dev, "Invalid ND_CALL size=%u\n", buf_len); + return -EINVAL; + } + + if (pkg->nd_family != NVDIMM_FAMILY_OCXL) { + dev_err(&ocxlpmem->dev, "Invalid ND_CALL family=0x%llx\n", + pkg->nd_family); + return -EINVAL; + } + + switch (pkg->nd_command) { + case ND_CMD_OCXL_SMART: + return ndctl_smart(ocxlpmem, pkg); + default: + dev_err(&ocxlpmem->dev, "Invalid ND_CALL command=0x%llx\n", + pkg->nd_command); + return -EINVAL; + } + + return 0; +} + static int ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm, unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) @@ -90,6 +198,10 @@ static int ndctl(struct nvdimm_bus_descriptor *nd_desc, struct ocxlpmem, bus_desc); switch (cmd) { + case ND_CMD_CALL: + *cmd_rc = ndctl_call(ocxlpmem, buf, buf_len); + return 0; + case ND_CMD_GET_CONFIG_SIZE: *cmd_rc = ndctl_config_size(buf); return 0; @@ -173,6 +285,7 @@ static int register_lpc_mem(struct ocxlpmem *ocxlpmem) set_bit(ND_CMD_GET_CONFIG_SIZE, &nvdimm_cmd_mask); set_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm_cmd_mask); set_bit(ND_CMD_SET_CONFIG_DATA, &nvdimm_cmd_mask); + set_bit(ND_CMD_CALL, &nvdimm_cmd_mask); set_bit(NDD_ALIASING, &nvdimm_flags); diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h index c8794e7775ec..ebaf11692c09 100644 --- a/drivers/nvdimm/ocxl/ocxlpmem.h +++ b/drivers/nvdimm/ocxl/ocxlpmem.h @@ -6,6 +6,7 @@ #include #include #include +#include #define LABEL_AREA_SIZE BIT_ULL(PA_SECTION_SHIFT) #define DEFAULT_TIMEOUT 100 @@ -101,6 +102,23 @@ struct command_metadata { u8 op_code; }; +struct nd_ocxl_smart { + __u8 count; + __u8 reserved[7]; + __u64 attribs[0]; +} __packed; + +struct nd_pkg_ocxl { + struct nd_cmd_pkg gen; + union { + struct nd_ocxl_smart smart; + }; +}; + +enum nd_cmd_ocxl { + ND_CMD_OCXL_SMART = 1, +}; + struct ocxlpmem { struct device dev; struct pci_dev *pdev; diff --git a/include/uapi/linux/ndctl.h b/include/uapi/linux/ndctl.h index de5d90212409..2885052e7f40 100644 --- a/include/uapi/linux/ndctl.h +++ b/include/uapi/linux/ndctl.h @@ -244,6 +244,7 @@ struct nd_cmd_pkg { #define NVDIMM_FAMILY_HPE2 2 #define NVDIMM_FAMILY_MSFT 3 #define NVDIMM_FAMILY_HYPERV 4 +#define NVDIMM_FAMILY_OCXL 6 #define ND_IOCTL_CALL _IOWR(ND_IOCTL, ND_CMD_CALL,\ struct nd_cmd_pkg) From patchwork Fri Mar 27 07:12:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464647 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE80D92C for ; Mon, 30 Mar 2020 05:53:06 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AD3442078B for ; Mon, 30 Mar 2020 05:53:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="QzIshqlF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD3442078B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 6563710FC36DF; Sun, 29 Mar 2020 22:53:56 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id 429341003ECB6 for ; Sun, 29 Mar 2020 22:53:53 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id 1A36B2DC685C; Mon, 30 Mar 2020 16:52:49 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547571; bh=7micxHdia2GvvQ4mY81c4jBHOqLOZ0TeAcVaPQleunM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QzIshqlFp53qA4DB8IAUutZabSbcugvEaXNKxzsAZpWDALJWJPENQ0mvFoELV93rV xBDyaQhfw6Fw/7rbXS8HW2uZrWtaFnR2MvhNnl6jlQdn02p7u/PmP6ztacNKbuPyu9 85tmlez/p2Zxt6TIwBFgAqzZRWTRiMnUh4iIZthtCkmm6j6qHrQe0olItNDpyXKcuu yrr563UM5DgGMo6LV7xzIIBif4ulFgC+/HkkjRqErrlhMI7zkwkMcxgPJBNiKUJris O3eIA6Pue08VWVHyK7Rqs/DxlrIVWcyA5LmzmtJO8eECSw88MVxEtfzv905BcPf3kH cMJ/o5fgzi/txiWRyh4RECCqzSOxZASTGF+B4aKQY9yDgE8tWr1RSDdkAgN9l5T7gK XufI3qOWi3HWntScxvoGIljl4WlGsy2jHNnAP16md7cjStgYPxE+MIOngRuFz9G+ot B0ZpigPUtr4kS4a5Zk/lekbvhIphRoTyFEEg27ngbAd4+R0Pdpi/03ray/974q6DRP oj9yVUBjNPWt7kNYNnilPCHMODV6tOeY6nPwnd/5fUVYpIYO/0G0qRYicw3MoB+cMo WNBBCFdNMn/eXciPsYoJgPiHCgAWonEXwlOPPECDtSKn/8ZevQqjdlygOvDl330WrX giiKa7eoY/kFQHdeCMxIcNVE= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4Ax045934; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 24/25] nvdimm/ocxl: Expose the serial number & firmware version in sysfs Date: Fri, 27 Mar 2020 18:12:01 +1100 Message-Id: <20200327071202.2159885-25-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) Message-ID-Hash: CP57YFMUDSRSR7GUH7NPRHVEK2QQBEP2 X-Message-ID-Hash: CP57YFMUDSRSR7GUH7NPRHVEK2QQBEP2 X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: This patch exposes the serial number & firmware version in sysfs, which will be used by ndctl in userspace to help users identify the device. Signed-off-by: Alastair D'Silva --- drivers/nvdimm/ocxl/main.c | 42 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c index 92b4389e8cbb..1f422f0d51ef 100644 --- a/drivers/nvdimm/ocxl/main.c +++ b/drivers/nvdimm/ocxl/main.c @@ -235,6 +235,43 @@ static int reserve_metadata(struct ocxlpmem *ocxlpmem, return 0; } +static ssize_t serial_show(struct device *device, struct device_attribute *attr, + char *buf) +{ + struct nvdimm *nvdimm = to_nvdimm(device); + struct ocxlpmem *ocxlpmem = nvdimm_provider_data(nvdimm); + const struct ocxl_fn_config *fn_config = ocxl_function_config(ocxlpmem->ocxl_fn); + + return scnprintf(buf, PAGE_SIZE, "%llu\n", fn_config->serial); +} +static DEVICE_ATTR_RO(serial); + +static ssize_t fw_version_show(struct device *device, + struct device_attribute *attr, char *buf) +{ + struct nvdimm *nvdimm = to_nvdimm(device); + struct ocxlpmem *ocxlpmem = nvdimm_provider_data(nvdimm); + + return scnprintf(buf, PAGE_SIZE, "%s\n", ocxlpmem->fw_version); +} +static DEVICE_ATTR_RO(fw_version); + +static struct attribute *ocxl_pmem_attrs[] = { + &dev_attr_serial.attr, + &dev_attr_fw_version.attr, + NULL, +}; + +static const struct attribute_group ocxl_pmem_attribute_group = { + .name = "ocxlpmem", + .attrs = ocxl_pmem_attrs, +}; + +static const struct attribute_group *ocxl_pmem_dimm_attribute_groups[] = { + &ocxl_pmem_attribute_group, + NULL, +}; + /** * register_lpc_mem() - Discover persistent memory on a device and register it with the NVDIMM subsystem * @ocxlpmem: the device metadata @@ -291,8 +328,9 @@ static int register_lpc_mem(struct ocxlpmem *ocxlpmem) snprintf(serial, sizeof(serial), "%llx", fn_config->serial); nd_mapping_desc.nvdimm = nvdimm_create(ocxlpmem->nvdimm_bus, ocxlpmem, - NULL, nvdimm_flags, - nvdimm_cmd_mask, 0, NULL); + ocxl_pmem_dimm_attribute_groups, + nvdimm_flags, nvdimm_cmd_mask, 0, + NULL); if (!nd_mapping_desc.nvdimm) return -ENOMEM; From patchwork Fri Mar 27 07:12:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alastair D'Silva X-Patchwork-Id: 11464635 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2DE8D92C for ; Mon, 30 Mar 2020 05:52:52 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C31EE20786 for ; Mon, 30 Mar 2020 05:52:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (4096-bit key) header.d=d-silva.org header.i=@d-silva.org header.b="bMbioP7E" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C31EE20786 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=d-silva.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 5DBD610FC36CC; Sun, 29 Mar 2020 22:53:41 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=66.55.73.32; helo=ushosting.nmnhosting.com; envelope-from=alastair@d-silva.org; receiver= Received: from ushosting.nmnhosting.com (ushosting.nmnhosting.com [66.55.73.32]) by ml01.01.org (Postfix) with ESMTP id DFED110FC363F for ; Sun, 29 Mar 2020 22:53:38 -0700 (PDT) Received: from mail2.nmnhosting.com (unknown [202.169.106.97]) by ushosting.nmnhosting.com (Postfix) with ESMTPS id BB2D52DC6849; Mon, 30 Mar 2020 16:52:45 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=d-silva.org; s=201810a; t=1585547566; bh=usgtpWwfPqN3Po8Erc64Din9rr6Mrn4qoxSsihnBfcI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bMbioP7EtdRIN7Psu7rzMUpwa2C/X6QC4z279/qrlgZqOuYGPlOwaH67cXt1O9uFN yOjDqQ7nMQ+tvZeiC3hFYQ7HkNgHSBbyXfOcwjqnoGGaTF/NmFZa/cZdks4yFx1rMN SY8i1g6ylE1zIvnQ4+BlwncOv6lIg8IPtu7wxpS04cV44QA+KZi2iMEu6bwkTuOfof Y49t6MWeq96LBdLMYXjaSmO54h9U3i1fSUQp5vBqUJ19SIAdr2sm6OAW6Z1Qkuv20U 7r9tsLXd/jVSP2tx5O3YbfGLhX60JbQU/5G2VTu+b3ndMWAb430Gz2hu8VpwC6JFRp 0ESQ++Zwi1ehD6oM/g89XDPAa1J5Rnn890kEAlVRB7y46BhGvxZbPZ4Vw9AkNpJsHB cMV5R2mdHXLLQgCpbDHpG6zziIq/nLlkrhCrNCg5beqdPmiTucYeBODFIHKReDK02p +bTsSrqanwYTuozWj8TiV9BtUYmI6YnNsfpXcEHcUATwh7NBNKNuNIDiN2vKdHC56v 3MJRPUnWhYZg6Q2eB5TAj14uimXz9l5Y0LlG1MynoP2X+JYl/o4p5ZmKb9I2wE1J6z /JOLdd+DdJwF1YHIa2bnFcs514YF3C+C5vK1g2b0TyRT23PrKcfcG1qzvflCD9DxrC nzuS2Tu74wUp1I8IUt9lKT1M= Received: from localhost.lan ([10.0.1.179]) by mail2.nmnhosting.com (8.15.2/8.15.2) with ESMTP id 02R7C4B0045934; Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) (envelope-from alastair@d-silva.org) From: "Alastair D'Silva" To: alastair@d-silva.org Subject: [PATCH v4 25/25] MAINTAINERS: Add myself & nvdimm/ocxl to ocxl Date: Fri, 27 Mar 2020 18:12:02 +1100 Message-Id: <20200327071202.2159885-26-alastair@d-silva.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org> References: <20200327071202.2159885-1-alastair@d-silva.org> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail2.nmnhosting.com [10.0.1.20]); Fri, 27 Mar 2020 18:12:20 +1100 (AEDT) Message-ID-Hash: BSQ7XGRCDX3Y673OUNCVAIDBF45XXMMJ X-Message-ID-Hash: BSQ7XGRCDX3Y673OUNCVAIDBF45XXMMJ X-MailFrom: alastair@d-silva.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Frederic Barrat , Andrew Donnellan , Arnd Bergmann , Greg Kroah-Hartman , Andrew Morton , Mauro Carvalho Chehab , "David S. Miller" , Rob Herring , Anton Blanchard , Krzysztof Kozlowski , Mahesh Salgaonkar , Madhavan Srinivasan , =?utf-8?q?C=C3=A9dric_Le_Go?= =?utf-8?q?ater?= , Anju T Sudhakar , Hari Bathini , Thomas Gleixner , Greg Kurz , Nicholas Piggin , Masahiro Yamada , Alexey Kardashevskiy , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-nvdimm@lists.01.org, linux-mm@kvack.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: The OpenCAPI Persistent Memory driver will be maintained as part ofi the ppc tree. I'm also adding myself as an author of the driver & contributor to the generic ocxl driver. Signed-off-by: Alastair D'Silva --- MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index f8670989ec91..3fb9a9f576a7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12064,13 +12064,16 @@ F: tools/objtool/ OCXL (Open Coherent Accelerator Processor Interface OpenCAPI) DRIVER M: Frederic Barrat M: Andrew Donnellan +M: Alastair D'Silva L: linuxppc-dev@lists.ozlabs.org S: Supported F: arch/powerpc/platforms/powernv/ocxl.c +F: arch/powerpc/platforms/powernv/pmem/* F: arch/powerpc/include/asm/pnv-ocxl.h F: drivers/misc/ocxl/ F: include/misc/ocxl* F: include/uapi/misc/ocxl.h +F: include/uapi/nvdimm/ocxl-pmem.h F: Documentation/userspace-api/accelerators/ocxl.rst OMAP AUDIO SUPPORT