From patchwork Tue Sep 19 10:37:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 13391142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47DDCCD54B0 for ; Tue, 19 Sep 2023 10:37:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5F63810E0B7; Tue, 19 Sep 2023 10:37:56 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5DCC110E0B7 for ; Tue, 19 Sep 2023 10:37:53 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A1C2661595 for ; Tue, 19 Sep 2023 10:37:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 411EDC433C7 for ; Tue, 19 Sep 2023 10:37:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695119871; bh=tesCEbtPcVZgw3Z16K6lPNfPF98JWxFalyPC21OvM+g=; h=From:To:Subject:Date:From; b=AgETvc9pGw3vwqFyXBfv6MFtoBJnOF2XTEJbL+Gu82z9kZWopTyornplh2HJk9TA3 01Zi2xvYlBE96C6M9AImq+Z2uqdw2sPqsrR7Cus4B3fp7NIeIh1CktalAw4XyS36eH 4UayNr/jSAtuGGO0d5DUXS0yM17Ib5HdkVP7V1t+EAAXAf0lEf7wynxsI7KRm6PHgG OWq4lpm9XptXz5Wn7xARoT0jzZN6pQClGUFnGtQiyPFVP4+phwabF6sc5vSDW+2eHV KfeIpxs1iZkCFtGNhb2r2OqHzAl/dT+WRPa74RAFeJy9JjacwCTYQFMSLZfZWHPAVD HPq66qgSCi5mQ== From: Oded Gabbay To: dri-devel@lists.freedesktop.org Subject: [PATCH 1/3] accel/habanalabs: remove unused asic functions Date: Tue, 19 Sep 2023 13:37:45 +0300 Message-Id: <20230919103747.955418-1-ogabbay@kernel.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" asic_dma_{un}map_single() asic-specific functions are no longer called from the common code, so delete these functions. In addition, delete the gaudi2 implementation as they are also not called. Signed-off-by: Oded Gabbay Reviewed-by: Ofir Bitton --- drivers/accel/habanalabs/common/habanalabs.h | 8 -------- drivers/accel/habanalabs/gaudi2/gaudi2.c | 20 -------------------- 2 files changed, 28 deletions(-) diff --git a/drivers/accel/habanalabs/common/habanalabs.h b/drivers/accel/habanalabs/common/habanalabs.h index 3c224942a758..4f667ac46301 100644 --- a/drivers/accel/habanalabs/common/habanalabs.h +++ b/drivers/accel/habanalabs/common/habanalabs.h @@ -1520,8 +1520,6 @@ struct engines_data { * @asic_dma_pool_free: free small DMA allocation from pool. * @cpu_accessible_dma_pool_alloc: allocate CPU PQ packet from DMA pool. * @cpu_accessible_dma_pool_free: free CPU PQ packet from DMA pool. - * @asic_dma_unmap_single: unmap a single DMA buffer - * @asic_dma_map_single: map a single buffer to a DMA * @hl_dma_unmap_sgtable: DMA unmap scatter-gather table. * @cs_parser: parse Command Submission. * @asic_dma_map_sgtable: DMA map scatter-gather table. @@ -1645,12 +1643,6 @@ struct hl_asic_funcs { size_t size, dma_addr_t *dma_handle); void (*cpu_accessible_dma_pool_free)(struct hl_device *hdev, size_t size, void *vaddr); - void (*asic_dma_unmap_single)(struct hl_device *hdev, - dma_addr_t dma_addr, int len, - enum dma_data_direction dir); - dma_addr_t (*asic_dma_map_single)(struct hl_device *hdev, - void *addr, int len, - enum dma_data_direction dir); void (*hl_dma_unmap_sgtable)(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir); diff --git a/drivers/accel/habanalabs/gaudi2/gaudi2.c b/drivers/accel/habanalabs/gaudi2/gaudi2.c index 35db02e5010b..be4a745f8548 100644 --- a/drivers/accel/habanalabs/gaudi2/gaudi2.c +++ b/drivers/accel/habanalabs/gaudi2/gaudi2.c @@ -6731,24 +6731,6 @@ static void gaudi2_cpu_accessible_dma_pool_free(struct hl_device *hdev, size_t s hl_fw_cpu_accessible_dma_pool_free(hdev, size, vaddr); } -static dma_addr_t gaudi2_dma_map_single(struct hl_device *hdev, void *addr, int len, - enum dma_data_direction dir) -{ - dma_addr_t dma_addr; - - dma_addr = dma_map_single(&hdev->pdev->dev, addr, len, dir); - if (unlikely(dma_mapping_error(&hdev->pdev->dev, dma_addr))) - return 0; - - return dma_addr; -} - -static void gaudi2_dma_unmap_single(struct hl_device *hdev, dma_addr_t addr, int len, - enum dma_data_direction dir) -{ - dma_unmap_single(&hdev->pdev->dev, addr, len, dir); -} - static int gaudi2_validate_cb_address(struct hl_device *hdev, struct hl_cs_parser *parser) { struct asic_fixed_properties *asic_prop = &hdev->asic_prop; @@ -11515,8 +11497,6 @@ static const struct hl_asic_funcs gaudi2_funcs = { .asic_dma_pool_free = gaudi2_dma_pool_free, .cpu_accessible_dma_pool_alloc = gaudi2_cpu_accessible_dma_pool_alloc, .cpu_accessible_dma_pool_free = gaudi2_cpu_accessible_dma_pool_free, - .asic_dma_unmap_single = gaudi2_dma_unmap_single, - .asic_dma_map_single = gaudi2_dma_map_single, .hl_dma_unmap_sgtable = hl_dma_unmap_sgtable, .cs_parser = gaudi2_cs_parser, .asic_dma_map_sgtable = hl_dma_map_sgtable, From patchwork Tue Sep 19 10:37:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 13391143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26FFBCD54B0 for ; Tue, 19 Sep 2023 10:38:02 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6BED610E26B; Tue, 19 Sep 2023 10:38:01 +0000 (UTC) Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 04A3610E386 for ; Tue, 19 Sep 2023 10:37:57 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 61900B80303; Tue, 19 Sep 2023 10:37:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E248C433C7; Tue, 19 Sep 2023 10:37:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695119874; bh=iLY8jDKtSVAY6opAQk2w67hTUaRy54MGzl5xyB9e4qw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AslT3HTDZ5tKSBCPcqRK+lcPXB/M1Ph1kjgwHd+IcA3LVMVQfmEbPgA3PcgQMC0Tk uCINC8VDjlcu4uIbsobH/Ou9sERaeL3jVKhi9qS4waZZyGmhaeQe1WX/xsOixM3/cQ wLVhdvSluGl0jBGMJndOf13p+//hKVCW3ZwC2tXtJO5Xn1ibG4OlIoRP7egJDrFXUZ c6ae8SgR+30Td8DKgjrvvOrT3Q/hvSAdKivBoQyQrXyDDYYMYx2D1H6a155oP7T6jY JMFaRJfoAQmqsRgGLQzrVkJqkDOLRwPc8NNU1CSuBFEdcqyz6zbnD7yN8JV/ukgJaR rcDWFRrg/TGgg== From: Oded Gabbay To: dri-devel@lists.freedesktop.org Subject: [PATCH 2/3] accel/habanalabs: add traces for dma mappings Date: Tue, 19 Sep 2023 13:37:46 +0300 Message-Id: <20230919103747.955418-2-ogabbay@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230919103747.955418-1-ogabbay@kernel.org> References: <20230919103747.955418-1-ogabbay@kernel.org> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ohad Sharabi Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Ohad Sharabi In order to get a full picture of DMA mappings (e.g. to track DMAR errors), DMA mappings APIs should be covered. Signed-off-by: Ohad Sharabi Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- include/trace/events/habanalabs.h | 43 +++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/include/trace/events/habanalabs.h b/include/trace/events/habanalabs.h index 951643e6a7a9..0d3667399d01 100644 --- a/include/trace/events/habanalabs.h +++ b/include/trace/events/habanalabs.h @@ -87,6 +87,49 @@ DEFINE_EVENT(habanalabs_dma_alloc_template, habanalabs_dma_free, TP_PROTO(struct device *dev, u64 cpu_addr, u64 dma_addr, size_t size, const char *caller), TP_ARGS(dev, cpu_addr, dma_addr, size, caller)); +DECLARE_EVENT_CLASS(habanalabs_dma_map_template, + TP_PROTO(struct device *dev, u64 phys_addr, u64 dma_addr, size_t len, + enum dma_data_direction dir, const char *caller), + + TP_ARGS(dev, phys_addr, dma_addr, len, dir, caller), + + TP_STRUCT__entry( + __string(dname, dev_name(dev)) + __field(u64, phys_addr) + __field(u64, dma_addr) + __field(u32, len) + __field(int, dir) + __field(const char *, caller) + ), + + TP_fast_assign( + __assign_str(dname, dev_name(dev)); + __entry->phys_addr = phys_addr; + __entry->dma_addr = dma_addr; + __entry->len = len; + __entry->dir = dir; + __entry->caller = caller; + ), + + TP_printk("%s: phys_addr: %#llx, dma_addr: %#llx, len: %#x, dir: %d, caller: %s", + __get_str(dname), + __entry->phys_addr, + __entry->dma_addr, + __entry->len, + __entry->dir, + __entry->caller) +); + +DEFINE_EVENT(habanalabs_dma_map_template, habanalabs_dma_map_page, + TP_PROTO(struct device *dev, u64 phys_addr, u64 dma_addr, size_t len, + enum dma_data_direction dir, const char *caller), + TP_ARGS(dev, phys_addr, dma_addr, len, dir, caller)); + +DEFINE_EVENT(habanalabs_dma_map_template, habanalabs_dma_unmap_page, + TP_PROTO(struct device *dev, u64 phys_addr, u64 dma_addr, size_t len, + enum dma_data_direction dir, const char *caller), + TP_ARGS(dev, phys_addr, dma_addr, len, dir, caller)); + DECLARE_EVENT_CLASS(habanalabs_comms_template, TP_PROTO(struct device *dev, char *op_str), From patchwork Tue Sep 19 10:37:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 13391144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F708CD54AF for ; Tue, 19 Sep 2023 10:38:06 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BE48210E386; Tue, 19 Sep 2023 10:38:05 +0000 (UTC) Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 872A110E26B for ; Tue, 19 Sep 2023 10:37:59 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 31024CE1298; Tue, 19 Sep 2023 10:37:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 99995C433C8; Tue, 19 Sep 2023 10:37:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695119875; bh=NTjLq7+6UbJfoIfUVQks2QbFCC0wzfa7rwllpto2Lu0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cTI40pOumsiPTz3bMH6Fx/zS2IqpuvjHWNIe3ZDbgYoq6S+g6d89GtIs2ZKNjn9q8 6EV10HZgPm/bavO5wXUiGOB8KXe6ty1FS5iHzKwMXREL+gHEKKggvzr0elhyR1geVN trz4Kpt1eneB13kIYad1ZcMBwMC2oCjiOJkFPFWLa4Ot+/1tzXzFNmHMoPmq5khGJt cozzAAfhJtTCPhESQVkywGXCxJyklH+3GJAks0vvycMZDnNo3EMHmJ+m2MKk4AlnBU 9NqFNGXUf3nz8qsEQtehc0TLfiK3d5gDlpoATlATNsKmNEa94FvrsqWw8KFiAKwcZm RKpbzaJ9e/AZg== From: Oded Gabbay To: dri-devel@lists.freedesktop.org Subject: [PATCH 3/3] accel/habanalabs: trace dma map sgtable Date: Tue, 19 Sep 2023 13:37:47 +0300 Message-Id: <20230919103747.955418-3-ogabbay@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230919103747.955418-1-ogabbay@kernel.org> References: <20230919103747.955418-1-ogabbay@kernel.org> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ohad Sharabi Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Ohad Sharabi Traces the DMA [un]map_sgtable using the new traces we added. Signed-off-by: Ohad Sharabi Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- drivers/accel/habanalabs/common/device.c | 46 +++++++++++++++++++- drivers/accel/habanalabs/common/habanalabs.h | 25 +++++++---- drivers/accel/habanalabs/common/memory.c | 4 +- drivers/accel/habanalabs/gaudi/gaudi.c | 6 +-- drivers/accel/habanalabs/gaudi2/gaudi2.c | 4 +- drivers/accel/habanalabs/goya/goya.c | 6 +-- 6 files changed, 71 insertions(+), 20 deletions(-) diff --git a/drivers/accel/habanalabs/common/device.c b/drivers/accel/habanalabs/common/device.c index d31f228533c0..086769a9c551 100644 --- a/drivers/accel/habanalabs/common/device.c +++ b/drivers/accel/habanalabs/common/device.c @@ -188,7 +188,31 @@ void hl_cpu_accessible_dma_pool_free(struct hl_device *hdev, size_t size, void * hdev->asic_funcs->cpu_accessible_dma_pool_free(hdev, size, vaddr); } -int hl_dma_map_sgtable(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir) +int hl_dma_map_sgtable_caller(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir, const char *caller) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct scatterlist *sg; + int rc, i; + + rc = hdev->asic_funcs->dma_map_sgtable(hdev, sgt, dir); + if (rc) + return rc; + + if (!trace_habanalabs_dma_map_page_enabled()) + return 0; + + for_each_sgtable_dma_sg(sgt, sg, i) + trace_habanalabs_dma_map_page(hdev->dev, + page_to_phys(sg_page(sg)), + sg->dma_address - prop->device_dma_offset_for_host_access, + sg->dma_length, dir, caller); + + return 0; +} + +int hl_asic_dma_map_sgtable(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir) { struct asic_fixed_properties *prop = &hdev->asic_prop; struct scatterlist *sg; @@ -206,7 +230,25 @@ int hl_dma_map_sgtable(struct hl_device *hdev, struct sg_table *sgt, enum dma_da return 0; } -void hl_dma_unmap_sgtable(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir) +void hl_dma_unmap_sgtable_caller(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir, const char *caller) +{ + struct asic_fixed_properties *prop = &hdev->asic_prop; + struct scatterlist *sg; + int i; + + hdev->asic_funcs->dma_unmap_sgtable(hdev, sgt, dir); + + if (trace_habanalabs_dma_unmap_page_enabled()) { + for_each_sgtable_dma_sg(sgt, sg, i) + trace_habanalabs_dma_unmap_page(hdev->dev, page_to_phys(sg_page(sg)), + sg->dma_address - prop->device_dma_offset_for_host_access, + sg->dma_length, dir, caller); + } +} + +void hl_asic_dma_unmap_sgtable(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir) { struct asic_fixed_properties *prop = &hdev->asic_prop; struct scatterlist *sg; diff --git a/drivers/accel/habanalabs/common/habanalabs.h b/drivers/accel/habanalabs/common/habanalabs.h index 4f667ac46301..b1b1c51aa5b1 100644 --- a/drivers/accel/habanalabs/common/habanalabs.h +++ b/drivers/accel/habanalabs/common/habanalabs.h @@ -159,6 +159,11 @@ enum hl_mmu_page_table_location { #define hl_asic_dma_pool_free(hdev, vaddr, dma_addr) \ hl_asic_dma_pool_free_caller(hdev, vaddr, dma_addr, __func__) +#define hl_dma_map_sgtable(hdev, sgt, dir) \ + hl_dma_map_sgtable_caller(hdev, sgt, dir, __func__) +#define hl_dma_unmap_sgtable(hdev, sgt, dir) \ + hl_dma_unmap_sgtable_caller(hdev, sgt, dir, __func__) + /* * Reset Flags * @@ -1520,9 +1525,9 @@ struct engines_data { * @asic_dma_pool_free: free small DMA allocation from pool. * @cpu_accessible_dma_pool_alloc: allocate CPU PQ packet from DMA pool. * @cpu_accessible_dma_pool_free: free CPU PQ packet from DMA pool. - * @hl_dma_unmap_sgtable: DMA unmap scatter-gather table. + * @dma_unmap_sgtable: DMA unmap scatter-gather table. + * @dma_map_sgtable: DMA map scatter-gather table. * @cs_parser: parse Command Submission. - * @asic_dma_map_sgtable: DMA map scatter-gather table. * @add_end_of_cb_packets: Add packets to the end of CB, if device requires it. * @update_eq_ci: update event queue CI. * @context_switch: called upon ASID context switch. @@ -1643,12 +1648,11 @@ struct hl_asic_funcs { size_t size, dma_addr_t *dma_handle); void (*cpu_accessible_dma_pool_free)(struct hl_device *hdev, size_t size, void *vaddr); - void (*hl_dma_unmap_sgtable)(struct hl_device *hdev, - struct sg_table *sgt, + void (*dma_unmap_sgtable)(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir); - int (*cs_parser)(struct hl_device *hdev, struct hl_cs_parser *parser); - int (*asic_dma_map_sgtable)(struct hl_device *hdev, struct sg_table *sgt, + int (*dma_map_sgtable)(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir); + int (*cs_parser)(struct hl_device *hdev, struct hl_cs_parser *parser); void (*add_end_of_cb_packets)(struct hl_device *hdev, void *kernel_address, u32 len, u32 original_len, @@ -3670,8 +3674,13 @@ void *hl_asic_dma_pool_zalloc_caller(struct hl_device *hdev, size_t size, gfp_t dma_addr_t *dma_handle, const char *caller); void hl_asic_dma_pool_free_caller(struct hl_device *hdev, void *vaddr, dma_addr_t dma_addr, const char *caller); -int hl_dma_map_sgtable(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir); -void hl_dma_unmap_sgtable(struct hl_device *hdev, struct sg_table *sgt, +int hl_dma_map_sgtable_caller(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir, const char *caller); +void hl_dma_unmap_sgtable_caller(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir, const char *caller); +int hl_asic_dma_map_sgtable(struct hl_device *hdev, struct sg_table *sgt, + enum dma_data_direction dir); +void hl_asic_dma_unmap_sgtable(struct hl_device *hdev, struct sg_table *sgt, enum dma_data_direction dir); int hl_access_sram_dram_region(struct hl_device *hdev, u64 addr, u64 *val, enum debugfs_access_type acc_type, enum pci_region region_type, bool set_dram_bar); diff --git a/drivers/accel/habanalabs/common/memory.c b/drivers/accel/habanalabs/common/memory.c index 33fb78aba812..dff755614a65 100644 --- a/drivers/accel/habanalabs/common/memory.c +++ b/drivers/accel/habanalabs/common/memory.c @@ -244,7 +244,7 @@ static int dma_map_host_va(struct hl_device *hdev, u64 addr, u64 size, *p_userptr = userptr; - rc = hdev->asic_funcs->asic_dma_map_sgtable(hdev, userptr->sgt, DMA_BIDIRECTIONAL); + rc = hl_dma_map_sgtable(hdev, userptr->sgt, DMA_BIDIRECTIONAL); if (rc) { dev_err(hdev->dev, "failed to map sgt with DMA region\n"); goto dma_map_err; @@ -2444,7 +2444,7 @@ void hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) hl_debugfs_remove_userptr(hdev, userptr); if (userptr->dma_mapped) - hdev->asic_funcs->hl_dma_unmap_sgtable(hdev, userptr->sgt, userptr->dir); + hl_dma_unmap_sgtable(hdev, userptr->sgt, userptr->dir); unpin_user_pages_dirty_lock(userptr->pages, userptr->npages, true); kvfree(userptr->pages); diff --git a/drivers/accel/habanalabs/gaudi/gaudi.c b/drivers/accel/habanalabs/gaudi/gaudi.c index 29fb71c8b47f..31e04af46876 100644 --- a/drivers/accel/habanalabs/gaudi/gaudi.c +++ b/drivers/accel/habanalabs/gaudi/gaudi.c @@ -4908,7 +4908,7 @@ static int gaudi_pin_memory_before_cs(struct hl_device *hdev, list_add_tail(&userptr->job_node, parser->job_userptr_list); - rc = hdev->asic_funcs->asic_dma_map_sgtable(hdev, userptr->sgt, dir); + rc = hl_dma_map_sgtable(hdev, userptr->sgt, dir); if (rc) { dev_err(hdev->dev, "failed to map sgt with DMA region\n"); goto unpin_memory; @@ -9144,9 +9144,9 @@ static const struct hl_asic_funcs gaudi_funcs = { .asic_dma_pool_free = gaudi_dma_pool_free, .cpu_accessible_dma_pool_alloc = gaudi_cpu_accessible_dma_pool_alloc, .cpu_accessible_dma_pool_free = gaudi_cpu_accessible_dma_pool_free, - .hl_dma_unmap_sgtable = hl_dma_unmap_sgtable, + .dma_unmap_sgtable = hl_asic_dma_unmap_sgtable, .cs_parser = gaudi_cs_parser, - .asic_dma_map_sgtable = hl_dma_map_sgtable, + .dma_map_sgtable = hl_asic_dma_map_sgtable, .add_end_of_cb_packets = gaudi_add_end_of_cb_packets, .update_eq_ci = gaudi_update_eq_ci, .context_switch = gaudi_context_switch, diff --git a/drivers/accel/habanalabs/gaudi2/gaudi2.c b/drivers/accel/habanalabs/gaudi2/gaudi2.c index be4a745f8548..819660c684cf 100644 --- a/drivers/accel/habanalabs/gaudi2/gaudi2.c +++ b/drivers/accel/habanalabs/gaudi2/gaudi2.c @@ -11497,9 +11497,9 @@ static const struct hl_asic_funcs gaudi2_funcs = { .asic_dma_pool_free = gaudi2_dma_pool_free, .cpu_accessible_dma_pool_alloc = gaudi2_cpu_accessible_dma_pool_alloc, .cpu_accessible_dma_pool_free = gaudi2_cpu_accessible_dma_pool_free, - .hl_dma_unmap_sgtable = hl_dma_unmap_sgtable, + .dma_unmap_sgtable = hl_asic_dma_unmap_sgtable, .cs_parser = gaudi2_cs_parser, - .asic_dma_map_sgtable = hl_dma_map_sgtable, + .dma_map_sgtable = hl_asic_dma_map_sgtable, .add_end_of_cb_packets = NULL, .update_eq_ci = gaudi2_update_eq_ci, .context_switch = gaudi2_context_switch, diff --git a/drivers/accel/habanalabs/goya/goya.c b/drivers/accel/habanalabs/goya/goya.c index 024ccf2e159b..1322cb330c57 100644 --- a/drivers/accel/habanalabs/goya/goya.c +++ b/drivers/accel/habanalabs/goya/goya.c @@ -3358,7 +3358,7 @@ static int goya_pin_memory_before_cs(struct hl_device *hdev, list_add_tail(&userptr->job_node, parser->job_userptr_list); - rc = hdev->asic_funcs->asic_dma_map_sgtable(hdev, userptr->sgt, dir); + rc = hl_dma_map_sgtable(hdev, userptr->sgt, dir); if (rc) { dev_err(hdev->dev, "failed to map sgt with DMA region\n"); goto unpin_memory; @@ -5465,9 +5465,9 @@ static const struct hl_asic_funcs goya_funcs = { .asic_dma_pool_free = goya_dma_pool_free, .cpu_accessible_dma_pool_alloc = goya_cpu_accessible_dma_pool_alloc, .cpu_accessible_dma_pool_free = goya_cpu_accessible_dma_pool_free, - .hl_dma_unmap_sgtable = hl_dma_unmap_sgtable, + .dma_unmap_sgtable = hl_asic_dma_unmap_sgtable, .cs_parser = goya_cs_parser, - .asic_dma_map_sgtable = hl_dma_map_sgtable, + .dma_map_sgtable = hl_asic_dma_map_sgtable, .add_end_of_cb_packets = goya_add_end_of_cb_packets, .update_eq_ci = goya_update_eq_ci, .context_switch = goya_context_switch,