From patchwork Tue May 9 22:15:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Deucher X-Patchwork-Id: 13236192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21200C7EE22 for ; Tue, 9 May 2023 22:15:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3C3CF10E41A; Tue, 9 May 2023 22:15:34 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2072.outbound.protection.outlook.com [40.107.220.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id 682F310E41A; Tue, 9 May 2023 22:15:32 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Sa82gcPWt6leez/CiTaDImREuNcNvO/zhfVg8KcCvDznm3nLhpXLUFAPao7tL1fwE5lncXaI7HlhkB/0gwSLimExNu4F2MqgsnJ//vsyxYX8Q7SdyVxqm4lrHiNUYYottaYqnmdTY4K69mzIWLwo9/IHdJK0JKt73Ex+7INN9ZsV9zXaFbmLPdsuEz5lkFr263KDSlm0xk2EYCAuyBY8hmc1V213KZUgSs9jkXJ2NKUs2LT1/4sf9ZM6SGdYmAJz8rQVvRAo1IU4GbfmgGryYUx1hjYirPF2KOaNmK/oWBtE56qrFKVoLpwxvS0iwO66qXFC39us20dBkvTDRWMfMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9sGacVdxtVfcZfVn4cGDXzfQKmh3YLDw8k4VkubKKIg=; b=cuR5dY3FEALdGtoqUKKz71WgOVMk7FMVfanCItLRuNl6mDVkM1f4yjuF+xNldE+6HGSSgKLDUU6EWkl6vTJ6wwWUmCcfADRwpeTMHC0zKpD4Uzp+IjQjIfRjUAkgvP8BhduDpjxel7YpahydmOXccf5jmEWiHT0APAj3wIME5l/uSSG/1AFGrV977PC7RwIPDCdFVrAShL9bbxJWnN167EAesBdHY7x9d0COaEnqiOZ6UE1uBGrbHkpQKkXFDRNeX4IMmrVxR8bU7sz1FBU4IicY1c7eFLRIaj6p1KDMVZuFmPhwKFStW/buf5avJroxEwi8kuFtTi8F4DQLUXOkEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=bestguesspass action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9sGacVdxtVfcZfVn4cGDXzfQKmh3YLDw8k4VkubKKIg=; b=5FtW9WSkVb14RjQmF6Lk1wJ17SyiNtayimisAd8ifbCKsw3I/WenSkfBBQfpH1cmMO8/8AwnUQM43IGFFpMCg6OkYCLJYQx0xoSZo3DsTAWyKAL/O20jCkvFDriSzE6pGHsVRX3TcHn8qh+0FPw4Z+TzXC+FeQgwcFp3CfJXPrc= Received: from BN8PR12CA0035.namprd12.prod.outlook.com (2603:10b6:408:60::48) by CH3PR12MB8545.namprd12.prod.outlook.com (2603:10b6:610:163::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.33; Tue, 9 May 2023 22:15:30 +0000 Received: from BN8NAM11FT113.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::ec) by BN8PR12CA0035.outlook.office365.com (2603:10b6:408:60::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.18 via Frontend Transport; Tue, 9 May 2023 22:15:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT113.mail.protection.outlook.com (10.13.176.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.18 via Frontend Transport; Tue, 9 May 2023 22:15:28 +0000 Received: from tr4.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 9 May 2023 17:15:28 -0500 From: Alex Deucher To: , Subject: [PATCH 1/3] drm/ttm: export ttm_pool_fini for cleanup Date: Tue, 9 May 2023 18:15:14 -0400 Message-ID: <20230509221516.477025-1-alexander.deucher@amd.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT113:EE_|CH3PR12MB8545:EE_ X-MS-Office365-Filtering-Correlation-Id: b0f63184-4be8-4118-156a-08db50dae5a9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: E9Z8hEXJVzP3M8go9yfxOO+ZBZHTJKMKvsfUzYX4hzQrO3L8EXxOH174oDz3oJelJW3LqxRgQ43/3LfpS3VaQs4+w+eL+j6oZ9Z1/FMB5mbF6VtZVJro7BYSuu6PS5CtXd+L6p2S47okC2YzQ0szz8PafxlV+NYQx0ZSxeIkeXpdNe7xn8LAPcSNxC4qLWbb4erk1ACLfzLWUQwMklvL5lW/GrOSVRBFoZhq+f33p84yfpV9XG63szt7plKN0ZXaXkEGGMa2m60bCL2ECkPKoYLJpFyQaj32FGruzrwA8T3MNbchx2W9net4itScsUO3OC17JNpSdU+ZFWC44A67Xm+MISsGSZ2PvCfy7MzJt/LukBzWixKlSuB3UgQ6YRPiHaaBx/jhNOrZRhxjVigPRWUD2QYkRgoJew3XNOuLtAK9oeradLN1VhhFTE3RmNKOuroaqvNmi1vg+UbLrg4diR9hbr7svcYQ4jqqCQu5MjXEFJfn9CoVNlHHwSggjuYnLtoBtQJqnKZHpXgEg6fxhPCxMTsmAXzRZ/qLcHaK21Q12BAlSd5a2Gsh4y1LjehcrE61tsCuGDW8BAdV2atFo+NkiDj1q7192Fi1mxdwjA0Ftr+zuBpITDR6iX8cMBCqnhn8X5Tsm9H/1XN086F/n/ezTu8zk+rdJVWlJeeSPYrWaQ18I/cXMhehQ0SAeyLEa0Wk1Ukp1aK1t5jDq6+uy8qORP8Y/kJbKmYrqCrcoWY= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(136003)(346002)(396003)(376002)(39860400002)(451199021)(46966006)(40470700004)(36840700001)(5660300002)(6666004)(4326008)(26005)(316002)(81166007)(41300700001)(82310400005)(356005)(82740400003)(86362001)(54906003)(1076003)(70586007)(110136005)(70206006)(2906002)(4744005)(450100002)(8936002)(8676002)(16526019)(186003)(7696005)(36756003)(478600001)(36860700001)(40460700003)(47076005)(2616005)(426003)(336012)(40480700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2023 22:15:28.8744 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b0f63184-4be8-4118-156a-08db50dae5a9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT113.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8545 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , Felix Kuehling , Rajneesh Bhardwaj Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Rajneesh Bhardwaj ttm_pool_init is exported and used outside of ttm subsystem with amdgpu_ttm interface, similarly export ttm_pool_fini for proper cleanup. Reviewed-by: Felix Kuehling Signed-off-by: Rajneesh Bhardwaj Signed-off-by: Alex Deucher --- drivers/gpu/drm/ttm/ttm_pool.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index 1068a41cded1..d4f6cc262e9a 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -573,6 +573,7 @@ void ttm_pool_fini(struct ttm_pool *pool) */ synchronize_shrinkers(); } +EXPORT_SYMBOL(ttm_pool_fini); /* As long as pages are available make sure to release at least one */ static unsigned long ttm_pool_shrinker_scan(struct shrinker *shrink, From patchwork Tue May 9 22:15:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alex Deucher X-Patchwork-Id: 13236194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D801BC7EE2A for ; Tue, 9 May 2023 22:15:39 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 019F310E41E; Tue, 9 May 2023 22:15:39 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2086.outbound.protection.outlook.com [40.107.102.86]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5724710E41B; Tue, 9 May 2023 22:15:34 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ampaMCd7xIAMbjuC5bfjgXbjnKSUYQEJ4Z1ngw0CQ6JL+P2qXcF816h2rsw0c0upAeC7eAvvz+uxLAdLQmnKpA2a5y9WrpTicktiJWDZ93MQ0r6ojjIb9E+n/sd7Ota0PTFnzgRlCepBKvOTM/5u/DpwAxlzAOJW65WUZkLug+ACC3ArE/WMOjyllZ3OSr91WFKI+24k/OdE54u4o2YoHyR6B+Hb1h2cX2E6roEYjaEh6gmGM7lo+ibohfQRXdBQXlO/ZwEgyLhXNMh/e97uMI3hf5VA+6Bxv1PHPAFIUD10yd+9gXizi5V3iFgvN+GlLP76UN8beO2FfXe47jzcgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NY217qADMG7VF9uUfKvWt3SvicNbGLYCg4EOhtbiP1E=; b=TEwTw0HZHnzsE3MzSaXnIW84U5wPu53j8qD/FQ38WKxP718XbBPXbVgDBhqAAFT3fb7OFWWEcj6yL7yohNG0cKcasfTCod27m4YhNdI32Y2hFBsYnMRMXu168Uh6tWnqm3Gg6zvhWmwz0qgSy9tOZuS/VSKn5qAh+HTGm9QS6Zn4hI7z0cboaFcBbN2QDI5ozSi+xRiA3cSuAYWM4TcU6WBwwIkay5FUDMkyaj96+BmOO55BCDtererT2vub5y53/CgVhM1SxlGneZcWcKdAJjGmT7hHr4cPW4p02oISuIxi5FFaAFf4tMqQ7P0Elzq+Fv16ZHTaVumKlRSagyzM3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NY217qADMG7VF9uUfKvWt3SvicNbGLYCg4EOhtbiP1E=; b=Qa7UB8Lp9O9YnnkZMDyVe57Bp/DoSXjkjWWENe3PLDinlrl5hZz/4Oj+K1EIfC7BWwz/HjDuf5VpQBFBut5HuuTCzwCbcCE+mOp6Vj/xKkCxnkmHBNOwN3li2JMcDBDELgYeZJ0gQdZGl3ndJv7RfYwZJz88xhXGWET1adygWmE= Received: from BN8PR12CA0029.namprd12.prod.outlook.com (2603:10b6:408:60::42) by CY5PR12MB6408.namprd12.prod.outlook.com (2603:10b6:930:3b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.33; Tue, 9 May 2023 22:15:31 +0000 Received: from BN8NAM11FT113.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::7b) by BN8PR12CA0029.outlook.office365.com (2603:10b6:408:60::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.18 via Frontend Transport; Tue, 9 May 2023 22:15:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT113.mail.protection.outlook.com (10.13.176.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.18 via Frontend Transport; Tue, 9 May 2023 22:15:31 +0000 Received: from tr4.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 9 May 2023 17:15:28 -0500 From: Alex Deucher To: , Subject: [PATCH 2/3] drm/amdgpu: Set TTM pools for memory partitions Date: Tue, 9 May 2023 18:15:15 -0400 Message-ID: <20230509221516.477025-2-alexander.deucher@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230509221516.477025-1-alexander.deucher@amd.com> References: <20230509221516.477025-1-alexander.deucher@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT113:EE_|CY5PR12MB6408:EE_ X-MS-Office365-Filtering-Correlation-Id: fa501250-ec23-47aa-b2ef-08db50dae6ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: anNVKc/BJ0AtkOy1msj7+SrZivGVTfvIXgMf9li02BzM1aDGEPWn30c7XXKoosK1z4RJk0lYM+gmlw+LpGiFJW4mbAOIisVbZeVREBJR10lUO4dsqSiIoFJPdNvXXcgjZx2/+XBNzikiyyx5JlL0eWeIZ4TRQBKJzplXs1LnzOUWRCxBcUBa4lBEQOlCObDhBG7h9aAG2Bh9gD/rcDo5IcD4tVwD6ua6RevkGBHbbshzgg8CydbXt0cl8jGrjDchLOO937EC2y1kqJTkjFWEZfasrXIkDI5Yhldmeq/tAjPZ5LR6BjsUsAMN6eeNGw16SHryhhYKiDk/bi90Mm3VIx4u1Fmp8zSWE29x9amDC3ffSuyYMFffyoEuH575AmDRLNoS/MVQm7vuV6o8LEpL9CBJvJeBfUYQ8ACkx8lMMgYBFJ1kd++qdl7+xU1FCEL12WPQn1+i312Dfei5xVu6Xk1SfYqUO4FKQN+hdmiTZbp5tI00YXC+0EVaRhjjyPu5kSu+S01mCOYRkGZj9FInY/vnYz0CeiLpvI6tW2ujrDZLyMnjOpEw+vo24HmElz1Yn+TU+lJVaQqpiiMFAdEZA/ozZF7Nq6RKBs8QB8caDOtDEsEtGBXOoJ6fZMX5ww4nfU8fCBzUG+f9nnM55BngyRRJue4CqgRG68/54vSHZdcP0QZzRO3HBBx8Xwrpfz1xjirr+M+jcUScYn04BkuARph9VRNKqgGmcL/YLDCElgg= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(136003)(396003)(39860400002)(346002)(451199021)(40470700004)(46966006)(36840700001)(2616005)(2906002)(186003)(36860700001)(336012)(86362001)(4326008)(41300700001)(36756003)(16526019)(450100002)(70206006)(70586007)(47076005)(1076003)(83380400001)(426003)(26005)(5660300002)(40460700003)(7696005)(82310400005)(6666004)(110136005)(82740400003)(81166007)(54906003)(40480700001)(356005)(316002)(8936002)(8676002)(478600001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2023 22:15:31.0149 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fa501250-ec23-47aa-b2ef-08db50dae6ef X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT113.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6408 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Philip Yang , Mukul Joshi , Felix Kuehling , Rajneesh Bhardwaj , Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Philip Yang For native mode only, create TTM pool for each memory partition to store the NUMA node id, then the TTM pool will be selected using memory partition id to allocate memory from the correct partition. Acked-by: Christian König (rajneesh: changed need_swiotlb and need_dma32 to false for pool init) Reviewed-by: Felix Kuehling Acked-and-tested-by: Mukul Joshi Signed-off-by: Philip Yang Signed-off-by: Rajneesh Bhardwaj Signed-off-by: Alex Deucher --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 61 ++++++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 1 + 2 files changed, 60 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 1582ef092bf1..f4f92133f505 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -635,6 +635,7 @@ struct amdgpu_ttm_tt { struct task_struct *usertask; uint32_t userflags; bool bound; + int32_t pool_id; }; #define ttm_to_amdgpu_ttm_tt(ptr) container_of(ptr, struct amdgpu_ttm_tt, ttm) @@ -1063,6 +1064,7 @@ static struct ttm_tt *amdgpu_ttm_tt_create(struct ttm_buffer_object *bo, return NULL; } gtt->gobj = &bo->base; + gtt->pool_id = NUMA_NO_NODE; if (abo->flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC) caching = ttm_write_combined; @@ -1089,6 +1091,7 @@ static int amdgpu_ttm_tt_populate(struct ttm_device *bdev, { struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm); + struct ttm_pool *pool; pgoff_t i; int ret; @@ -1103,7 +1106,11 @@ static int amdgpu_ttm_tt_populate(struct ttm_device *bdev, if (ttm->page_flags & TTM_TT_FLAG_EXTERNAL) return 0; - ret = ttm_pool_alloc(&adev->mman.bdev.pool, ttm, ctx); + if (adev->mman.ttm_pools && gtt->pool_id >= 0) + pool = &adev->mman.ttm_pools[gtt->pool_id]; + else + pool = &adev->mman.bdev.pool; + ret = ttm_pool_alloc(pool, ttm, ctx); if (ret) return ret; @@ -1124,6 +1131,7 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, { struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm); struct amdgpu_device *adev; + struct ttm_pool *pool; pgoff_t i; amdgpu_ttm_backend_unbind(bdev, ttm); @@ -1142,7 +1150,13 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, ttm->pages[i]->mapping = NULL; adev = amdgpu_ttm_adev(bdev); - return ttm_pool_free(&adev->mman.bdev.pool, ttm); + + if (adev->mman.ttm_pools && gtt->pool_id >= 0) + pool = &adev->mman.ttm_pools[gtt->pool_id]; + else + pool = &adev->mman.bdev.pool; + + return ttm_pool_free(pool, ttm); } /** @@ -1732,6 +1746,41 @@ static int amdgpu_ttm_reserve_tmr(struct amdgpu_device *adev) return 0; } +static int amdgpu_ttm_pools_init(struct amdgpu_device *adev) +{ + int i; + + if (!adev->gmc.is_app_apu || !adev->gmc.num_mem_partitions) + return 0; + + adev->mman.ttm_pools = kcalloc(adev->gmc.num_mem_partitions, + sizeof(*adev->mman.ttm_pools), + GFP_KERNEL); + if (!adev->mman.ttm_pools) + return -ENOMEM; + + for (i = 0; i < adev->gmc.num_mem_partitions; i++) { + ttm_pool_init(&adev->mman.ttm_pools[i], adev->dev, + adev->gmc.mem_partitions[i].numa.node, + false, false); + } + return 0; +} + +static void amdgpu_ttm_pools_fini(struct amdgpu_device *adev) +{ + int i; + + if (!adev->gmc.is_app_apu || !adev->mman.ttm_pools) + return; + + for (i = 0; i < adev->gmc.num_mem_partitions; i++) + ttm_pool_fini(&adev->mman.ttm_pools[i]); + + kfree(adev->mman.ttm_pools); + adev->mman.ttm_pools = NULL; +} + /* * amdgpu_ttm_init - Init the memory management (ttm) as well as various * gtt/vram related fields. @@ -1758,6 +1807,12 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) DRM_ERROR("failed initializing buffer object driver(%d).\n", r); return r; } + + r = amdgpu_ttm_pools_init(adev); + if (r) { + DRM_ERROR("failed to init ttm pools(%d).\n", r); + return r; + } adev->mman.initialized = true; /* Initialize VRAM pool with all of VRAM divided into pages */ @@ -1905,6 +1960,8 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) if (!adev->mman.initialized) return; + amdgpu_ttm_pools_fini(adev); + amdgpu_ttm_training_reserve_vram_fini(adev); /* return the stolen vga memory back to VRAM */ if (!adev->gmc.is_app_apu) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h index da6544fdc8dd..8ef048a0a33e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h @@ -49,6 +49,7 @@ struct amdgpu_gtt_mgr { struct amdgpu_mman { struct ttm_device bdev; + struct ttm_pool *ttm_pools; bool initialized; void __iomem *aper_base_kaddr; From patchwork Tue May 9 22:15:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Deucher X-Patchwork-Id: 13236193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B5DAC7EE23 for ; Tue, 9 May 2023 22:15:36 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4F79110E422; Tue, 9 May 2023 22:15:35 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2082.outbound.protection.outlook.com [40.107.220.82]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7579A10E41A; Tue, 9 May 2023 22:15:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Pc663TfYN4e//18Nub/E4vXUAjzMssPMxYd9MDSKHzrNlPOMEQwk3sF1FC8frMyG1V5aOpLt90xJbyMP2xil6Xu3K+GAmC10mqXBkg3BHM2XWUWD1bOachVPmAXwuvX77wasYIEr22U6RhVSF0aJ1s3ea6I58JW1B/BMaQy4GWpQGGR0aPYV8jRiAn8tZc4jBjtG93IMAdaxc+ZiPmvvFiEKcDaO2esZrWcO4qqN8fbadxTDt4HbCrgQPuA/xhSHM9TBRhZImdiHgMyzSdOZGT9wD2zkp+q0nEZMcq7tswgRxfO/rOSAQJHVzxuq13MHNZtp+xI3gjyz2AOStsOaGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y8MfPrhoFdoRzioq+o4vqk8J0BFSaUapjDMGGavZNDk=; b=mcGAcj+RliDAV6tKTDGHS4BcyFo9iEDv59CI0FsqwCDYJahUFFTXGQBcmmvQqNy4azXZAUEwy5qLIC9pEDyfGy2KAeNwSpUV4GnNT0Yf5NcIGFuXwjBs0ilvzF1ipJiV5YVMRCsyOpkeV1lrHJoUc3FSWpkpPb+bIMPiAeM1XHHxHbM4SLBekhdHdLrfOcd06ytw7qPy0Accef4LsrWkIwbujpH4KlTuXlxcuun1th5DABjYJgrk5th6eMWVhjJMpbW6J1i/zpHTW0ab56amkiHi5BK5FWEOAZDn3QQIBLJllMLdmCFBDhW3NMqsdJL0lbs0+0muq+QwN9+/MrzGFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y8MfPrhoFdoRzioq+o4vqk8J0BFSaUapjDMGGavZNDk=; b=wZExzkmuztl3a9yLgEKofBycdK1LTuoZgUFCG9p/Mmw5z1e8/Of/wNs9OSlv31lGa6W+YOf8UfG3fgnegMOCnN9hDwOHuicQpO5szClbc3Xo1eSHWLZEanSE0F7wFObbDXsQbAMOlyYzzxozUZaPb9y+hJFu/VlXEz+rhwT1tes= Received: from BN8PR12CA0025.namprd12.prod.outlook.com (2603:10b6:408:60::38) by SA0PR12MB4590.namprd12.prod.outlook.com (2603:10b6:806:93::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.31; Tue, 9 May 2023 22:15:31 +0000 Received: from BN8NAM11FT113.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::40) by BN8PR12CA0025.outlook.office365.com (2603:10b6:408:60::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.18 via Frontend Transport; Tue, 9 May 2023 22:15:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT113.mail.protection.outlook.com (10.13.176.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6387.18 via Frontend Transport; Tue, 9 May 2023 22:15:31 +0000 Received: from tr4.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 9 May 2023 17:15:29 -0500 From: Alex Deucher To: , Subject: [PATCH 3/3] drm/amdkfd: Native mode memory partition support Date: Tue, 9 May 2023 18:15:16 -0400 Message-ID: <20230509221516.477025-3-alexander.deucher@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230509221516.477025-1-alexander.deucher@amd.com> References: <20230509221516.477025-1-alexander.deucher@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT113:EE_|SA0PR12MB4590:EE_ X-MS-Office365-Filtering-Correlation-Id: e8b43520-c75e-4adf-4ce4-08db50dae72b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yVzPnpIgR8F/oA5mx3haVlHHM13E9O4m7jPiimf5fjaantKK+cjiHliJILaBt8n158zld+6aOzv8fJ4iXAArZBfBgYZ60wOEmt9KwqQzY1hItyevgm8LbVWSdDb1hDxHcTg9aMDSBQk5qnqhJiQ8Ou803cxAMG/JNm2iiudaVbfq8wPKxG/2d5k1hVNjnpZkyXo8wo2wmdDDSf8+398w/kkOVmHYHuyJlVN9L4g/G+6gJkDHuRDqsskqqOvAasqN1QvvJenOMxn5/vo0tnP1WfqfrLWOBr6kXW/UnMjv+eIilCdmFUzNdDP9d9868Y4LNWkt8ywe2Axryi4W8gSvM8nkPlZHbXe/8JY2+/gWyOYO7q/NSILmf8jRFqzzZU/PktVlBpeHgeXbZ0TXj6841Lud7LtI0pgPmVxxDaxjQFftupcR3REROISzf2KWQaUXING3ZIE+XqHvm+tGAxgicUzHHuFFRg8WO+EoygjQl4BuBfs/LJKbcfNbsFHXgfO/SVbi8u1445+0EIdjqRribAupaykIf2Hod4TdeNr+6Ar2Xvs1ePDnvNlSX09VBnKctzve26YF6V0eAg0Re4rkayc1+dI4EHH08WugnqI6ZBL53oWiSULZ+TUHIwYUFPMmrF09/yriagvWmNwox2cUYOPjNyA2j/1c9qeN3UqE9lQUz2asP9ecCqFrTSEHcrlWzuh7pUjmH2x1L9O3UAhMLR+EOfS5OgIP7ourEwdW2J4= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(136003)(396003)(346002)(39860400002)(451199021)(46966006)(36840700001)(40470700004)(82310400005)(8676002)(8936002)(5660300002)(41300700001)(316002)(4326008)(16526019)(40460700003)(36860700001)(83380400001)(450100002)(70206006)(70586007)(1076003)(36756003)(186003)(478600001)(26005)(7696005)(2906002)(40480700001)(82740400003)(47076005)(2616005)(426003)(336012)(86362001)(110136005)(356005)(81166007)(54906003)(6666004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2023 22:15:31.4055 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e8b43520-c75e-4adf-4ce4-08db50dae72b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT113.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4590 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , Mukul Joshi , Felix Kuehling , Rajneesh Bhardwaj , Philip Yang Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Rajneesh Bhardwaj For native mode, after amdgpu_bo is created on CPU domain, then call amdgpu_ttm_tt_set_mem_pool to select the TTM pool using bo->mem_id. ttm_bo_validate will allocate the memory to the correct memory partition before mapping to GPUs. Reviewed-by: Felix Kuehling Acked-and-tested-by: Mukul Joshi Signed-off-by: Philip Yang Signed-off-by: Rajneesh Bhardwaj Signed-off-by: Alex Deucher --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 18 ++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 1 + 3 files changed, 26 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 0ebd39a41e74..c6bf66c9377f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -1642,6 +1642,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( uint64_t aligned_size; u64 alloc_flags; int ret; + int mem_id = 0; /* Fixme : to be changed when mem_id support patch lands, until then NPS1, SPX only */ /* * Check on which domain to allocate BO @@ -1749,6 +1750,11 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( ((*mem)->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM)) { bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT; bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT; + ret = amdgpu_ttm_tt_set_mem_pool(&bo->tbo, mem_id); + if (ret) { + pr_debug("failed to set ttm mem pool %d\n", ret); + goto err_set_mem_partition; + } } add_kgd_mem_to_kfd_bo_list(*mem, avm->process_info, user_addr); @@ -1777,6 +1783,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( allocate_init_user_pages_failed: err_pin_bo: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); +err_set_mem_partition: drm_vma_node_revoke(&gobj->vma_node, drm_priv); err_node_allow: /* Don't unreserve system mem limit twice */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index f4f92133f505..7885162b7368 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1159,6 +1159,24 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, return ttm_pool_free(pool, ttm); } +/** + * amdgpu_ttm_tt_set_mem_pool - Set the TTM memory pool for the TTM BO + * @tbo: The ttm_buffer_object that backs the VRAM bo + * @mem_id: to select the initialized ttm pool corresponding to the memory partition + */ +int amdgpu_ttm_tt_set_mem_pool(struct ttm_buffer_object *tbo, int mem_id) +{ + struct ttm_tt *ttm = tbo->ttm; + struct amdgpu_ttm_tt *gtt; + + if (!ttm && !ttm_tt_is_populated(ttm)) + return -EINVAL; + + gtt = ttm_to_amdgpu_ttm_tt(ttm); + gtt->pool_id = mem_id; + return 0; +} + /** * amdgpu_ttm_tt_get_userptr - Return the userptr GTT ttm_tt for the current * task diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h index 8ef048a0a33e..fe32de1bf4d5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h @@ -192,6 +192,7 @@ bool amdgpu_ttm_tt_has_userptr(struct ttm_tt *ttm); struct mm_struct *amdgpu_ttm_tt_get_usermm(struct ttm_tt *ttm); bool amdgpu_ttm_tt_affect_userptr(struct ttm_tt *ttm, unsigned long start, unsigned long end, unsigned long *userptr); +int amdgpu_ttm_tt_set_mem_pool(struct ttm_buffer_object *tbo, int mem_id); bool amdgpu_ttm_tt_userptr_invalidated(struct ttm_tt *ttm, int *last_invalidated); bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm);