From patchwork Thu Feb 10 07:28:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 12741503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C9F2C433EF for ; Thu, 10 Feb 2022 07:29:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 355696B0096; Thu, 10 Feb 2022 02:29:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3051E6B0098; Thu, 10 Feb 2022 02:29:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12EDD6B0099; Thu, 10 Feb 2022 02:29:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id EDDEE6B0096 for ; Thu, 10 Feb 2022 02:29:34 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B7A31824C42F for ; Thu, 10 Feb 2022 07:29:34 +0000 (UTC) X-FDA: 79126045068.11.25872D5 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf04.hostedemail.com (Postfix) with ESMTP id 6F86C40006 for ; Thu, 10 Feb 2022 07:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=aMJ9qYaobNHVTX1OfqZ5Wr1+1E3WUjs5OUy37ghm71g=; b=NO3gTYBWJ1BZfRr9EU8Y/wxtdS u3Aul9AJ7lp8VQMpGmrpIRC6K6FoNk3/9FB9pWeiN7v5tl0bikLDv3kezRT6N7XGiLjDt2IezY+0c s53GZhWR4RBUzTEGRZ2wCF3DxPy1lY/zjc9wgNcHGz9LAeLkyCBw52LnOr1iPEOYoF9ZE+Ww0Pz2L kX8Z/nvrkmbWuvz6W+aD4w7q/8yfPhePPtuEZM2wuifDmWYDNuZiz+VjfcjQrChcBqKY4DaCYzrJf 6kp1ODst1H4mtTOQGU5qONIVyoGQ+vmgQnXmIme1ovuaPYAEEwFvEUcsj8xzN7FIebVtx2s9Sbr2t LVeJ7dmQ==; Received: from [2001:4bb8:188:3efc:8014:b2f2:fdfd:57ea] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nI3tK-002sAt-Ci; Thu, 10 Feb 2022 07:29:22 +0000 From: Christoph Hellwig To: Andrew Morton , Dan Williams Cc: Felix Kuehling , Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= , "Pan, Xinhui" , Ben Skeggs , Karol Herbst , Lyude Paul , Jason Gunthorpe , Alistair Popple , Logan Gunthorpe , Ralph Campbell , linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, nvdimm@lists.linux.dev, linux-mm@kvack.org, Alex Sierra Subject: [PATCH 18/27] drm/amdkfd: add SPM support for SVM Date: Thu, 10 Feb 2022 08:28:19 +0100 Message-Id: <20220210072828.2930359-19-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220210072828.2930359-1-hch@lst.de> References: <20220210072828.2930359-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Stat-Signature: tbbnnqai6ky9kmtij9ii65g976knpr9j Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=NO3gTYBW; spf=none (imf04.hostedemail.com: domain of BATV+4801fb5fa93fa27e8109+6745+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+4801fb5fa93fa27e8109+6745+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 6F86C40006 X-HE-Tag: 1644478174-549594 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Sierra When CPU is connected throug XGMI, it has coherent access to VRAM resource. In this case that resource is taken from a table in the device gmc aperture base. This resource is used along with the device type, which could be DEVICE_PRIVATE or DEVICE_COHERENT to create the device page map region. Signed-off-by: Alex Sierra Reviewed-by: Felix Kuehling Signed-off-by: Christoph Hellwig --- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 28 ++++++++++++++---------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index e27ca375876230..2c51f2ac3b46ac 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -933,7 +933,7 @@ int svm_migrate_init(struct amdgpu_device *adev) { struct kfd_dev *kfddev = adev->kfd.dev; struct dev_pagemap *pgmap; - struct resource *res; + struct resource *res = NULL; unsigned long size; void *r; @@ -948,28 +948,34 @@ int svm_migrate_init(struct amdgpu_device *adev) * should remove reserved size */ size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20); - res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); - if (IS_ERR(res)) - return -ENOMEM; + if (adev->gmc.xgmi.connected_to_cpu) { + pgmap->range.start = adev->gmc.aper_base; + pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1; + pgmap->type = MEMORY_DEVICE_COHERENT; + } else { + res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (IS_ERR(res)) + return -ENOMEM; + pgmap->range.start = res->start; + pgmap->range.end = res->end; + pgmap->type = MEMORY_DEVICE_PRIVATE; + } - pgmap->type = MEMORY_DEVICE_PRIVATE; pgmap->nr_range = 1; - pgmap->range.start = res->start; - pgmap->range.end = res->end; pgmap->ops = &svm_migrate_pgmap_ops; pgmap->owner = SVM_ADEV_PGMAP_OWNER(adev); - pgmap->flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE; - + pgmap->flags = 0; /* Device manager releases device-specific resources, memory region and * pgmap when driver disconnects from device. */ r = devm_memremap_pages(adev->dev, pgmap); if (IS_ERR(r)) { pr_err("failed to register HMM device memory\n"); - /* Disable SVM support capability */ pgmap->type = 0; - devm_release_mem_region(adev->dev, res->start, resource_size(res)); + if (pgmap->type == MEMORY_DEVICE_PRIVATE) + devm_release_mem_region(adev->dev, res->start, + res->end - res->start + 1); return PTR_ERR(r); }