From patchwork Fri Aug 5 06:29:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 12936865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AD83C00140 for ; Fri, 5 Aug 2022 06:30:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 656A98E0003; Fri, 5 Aug 2022 02:30:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6073C8E0001; Fri, 5 Aug 2022 02:30:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4327A8E0003; Fri, 5 Aug 2022 02:30:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 30E6D8E0001 for ; Fri, 5 Aug 2022 02:30:13 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EA0711217DF for ; Fri, 5 Aug 2022 06:30:12 +0000 (UTC) X-FDA: 79764564264.05.02636DD Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2066.outbound.protection.outlook.com [40.107.237.66]) by imf11.hostedemail.com (Postfix) with ESMTP id 50C554014A for ; Fri, 5 Aug 2022 06:30:12 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FfnQKy4k91ro0c+WH9tDZkAWHbxXbqu/KMaz0T0v2eC5lsfjuKatDlPkr1fA0gySceeb7WmUsfu+BrfeVFJjlNNwWG3AlKVjIK4sZTmnuNyX8Wf/Q/qC3pez9QWFg0xzlrUi7leomBw0Euiw8IpaDSdbGvAQs6ZSpqTG90iXItIxTru2wd4JR56IamYSef5D3HxOWdHON/n4bg7TQsDjS3OXyQuXNshS51huH19u95fr+gHw6xnXzH9fNdC/7BqZj0cYvYIjSJyzz3M4gRpL8FzH5KEn3WQ0IS2R0YTxTpnTU7IBzvegsY1e+0Y9n3QQWLEZVV9qzqdl/W5Aq23B0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y9YP5CB/KLf/ZQSCWaYUwoTyOvoICu+A6z2ecRVYt2E=; b=UDOGNt/3ke9IYnNw9vMvHTcerShlTLjUIzMU4W0XD+joPtsVGHR3+0JN3BQyXeWNvCNpcW5/yxuvmts2nD507Vm1NydmVASe8qkBAPabxBxwMWeueWWnOhfXxPqTFyfMOClcBtSB17DcKNB6N6SPLVAWlTCC0gAOxaq03l43jNcZZAG4FNjQm040biFAWbKMdttaoq7xVekQKzlPS50McTOieGrFyhjlwmiuMdR2CB75zD3FOxTbW2To8LIBv6xP/2F+ZcrmvtEyDD+9pKpLb1M3OWRgdqSzg+RjcpNGctCmqkRMy+hzCKiPV8n2WiBNba6gbCHi7LAoDudzHmGy8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y9YP5CB/KLf/ZQSCWaYUwoTyOvoICu+A6z2ecRVYt2E=; b=JUY2ZQ64MEu56l240LyVmWinhpi8EpEhVNBr69lXRjaV93GgP10RVyU9wVxfLXtdp2WFU7MDhR9AT9qi/i8Ob4LwKHk8mF4X/P2EblyZI1jrJCC30eSAKzlvHVVbP/ihAJUAXLznlDtDXbankwSyjTRITMeBabancxUjly4egYCU15DFeceZ3Zn819PQMFy2NdPw0WefpE+W2bWM5rsg9IFBbE1bQdITrg+SBACW5inf1RcvmdjUcMhht9Oeyv6Tan9lxoJuYyACD7nVQoZIr3DhfxThNo1Cyg1bUox8IEyOzGimXpWZIE0ruOdrbMaSNHcYhZ6mEBXlObY4Ifi+Cg== Received: from DM6PR12MB3179.namprd12.prod.outlook.com (2603:10b6:5:183::18) by SA1PR12MB5613.namprd12.prod.outlook.com (2603:10b6:806:23e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.15; Fri, 5 Aug 2022 06:30:10 +0000 Received: from DM6PR12MB3179.namprd12.prod.outlook.com ([fe80::d0c7:5814:170e:b016]) by DM6PR12MB3179.namprd12.prod.outlook.com ([fe80::d0c7:5814:170e:b016%4]) with mapi id 15.20.5504.015; Fri, 5 Aug 2022 06:30:09 +0000 From: Alistair Popple To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Sierra Guiza, Alejandro (Alex)" , Chaitanya Kulkarni , Dan Williams , Felix Kuehling , Jason Gunthorpe , John Hubbard , Logan Gunthorpe , Miaohe Lin , Muchun Song , Ralph Campbell , David Hildenbrand , Alistair Popple Subject: [PATCH v2 1/2] mm/gup.c: Don't pass gup_flags to check_and_migrate_movable_pages() Date: Fri, 5 Aug 2022 16:29:52 +1000 Message-Id: <3e20a542e756bbc0f66435c9344ff674f3ff7ac7.1659680600.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.35.1 X-ClientProxiedBy: SJ0PR03CA0271.namprd03.prod.outlook.com (2603:10b6:a03:39e::6) To DM6PR12MB3179.namprd12.prod.outlook.com (2603:10b6:5:183::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 81594e28-aa32-4358-4016-08da76abf1e5 X-MS-TrafficTypeDiagnostic: SA1PR12MB5613:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ESspMRI40DlnbJVzB2kC0Cu4QoYLbPLylcf+Yv6SmPuwK864blvOMqH6fOk2m+Vjp5otcsUQtafu7j+x7xGJzhk9+vQAjMn85Ma+V3QnLGig6++3ioTdjaB4Xiq8TNrT7tLJsOgWYICmdAuToFSlfODY8rJZTcKprdmRMmXIU1vPzF1YBevQOwBCAq6ZZ2jF0T3++Z2tf7od3dYituN+gFpV3xcS/iyvzN/NRNdpl5GTLXPVD/apmrA6HTMqb+2V9uMxKbKV3WTCGAxCUzScLZ2GBjmwo8Ja9eUjUncIJeUcSllsNl2KkWiljz+/Qy5PCzf+n+HNJPIKlAL36hxsXni5qWvyZQtiAl7whAoXdZpNF81bDaADysaCg0SZJE9hXobJIFlxh8scekl0MH3RHykEWkw6ldgA83P45aRve1W0MvEijDrLFrSIYWvqrWNGMK1HADRMEE+3BeRwuu+cCp4KZxToVbcEoELqvrIDMIq9gdBG6N4YEyVUcBFO8T06cstTX1DIWuhJoY7qWAnrARrT8NyzYc3tMbj3bsIKB2ABC+7X6Ia5FU+kMLFpzU2S1cudQ73Hqrxd+HW7ItUnPzYhNCOJ4B2Nj/Jzfn+5zqA10kXU5AF1voJ6eaULevPOxgMe8F5H+FM1YpqVg18LbEgm0ZY0/3V3nrjJGtXtE4spWtnNN5cCyOMLJRPzA7SfedSQA2XTef/t/+YN4IrJnox+ROzi4RtTinnbEglWrot9UsCPIFs1nw+kGgLyZA5L X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3179.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(39860400002)(366004)(396003)(136003)(346002)(83380400001)(38100700002)(2906002)(54906003)(2616005)(26005)(6666004)(186003)(107886003)(6506007)(6512007)(86362001)(66946007)(4326008)(36756003)(8676002)(5660300002)(66476007)(66556008)(6916009)(8936002)(478600001)(6486002)(316002)(41300700001)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: lgQXvoRdmwQ9fNak11bfMLx6DWvnD8repEz+BGmcNDRJzeLa1Dmyj9iwdUgZomJ0bjmC//w2eW3+h6AfiFD4DTba5/DY68+5d8LB/zhl/lygoOwogYa07m5uN+pHZf11CGI8owg6DE0jViVDps9QoxgeFE9B7TNcRlb/o2pmiIIiBCdv+wvQaB/wd1GvsGkoZC+UgHcqPImjVLDykaUhKkO7c+P2nefIhO0Ts+/8bSwxsf+/baL4e8QwQqf8aNwV0w+YHp9IEZDtSdOzGChK6ayjAlQ7dwG9yTxhS9W9fMHFotUnMOae2GxxKNsZS8cCcxHh5K5KsDURhjslZkUc56rnE502XBPAOwCoHhTDiC6MUUzvuk16Y2bb2s8TtcnXRaK6rceufwrFuKFe9cYzFgs+63BQLPEvvOM3OqvX0lwWUCm7LSFp2KFGe++UPlTF/NPV/EA0asZHDXtxkxujDiS06GCmqcBgNeBHDohzYYd6UNOE/QEp2WxwsoE9P6cFYuuIqjaSnxu7KT0FsNhi9V5tiFhidtZKHx/enLCe6t497f6APZ6frE1zFR+8Zv+QksF+l4IOFainbY33F5hoOZ2XMgYSqYhEL1r3htUg+WjmV0mclBUUhpXiBRmD/Xc61QvC8te/CPxoOCQe2cJcojd7YgirrwPNSSmGcvtmKR01BtsX0kYpArnLfZaDYj4aUpkR2iHwEYQFu20h2N1gktmQMqGPGbs6EO1zY5oK44ooKsehsyKFhg1UTowxIKAH1il+v1WdV0LrpmkN2i0vZW0mskiRgodtMlagT27cXUYbMp0uNKbBJYN/iVfwq//66qbAcEwWOl97weJbl4LqknWDWB55bgJNos7ZUuffwFCHr9extOR0nT6J7iJFgJJjRsLoDz9H5nbOCsPv+d+Zhx/N0RoiRhYLcw+VmfakUGWqui5pqsnfKEcMucfWyLnLbFRmtkbqxExfAMWqbkzwN4e+oIuS2GXv1TfYTopegdGsRJLj/Lehsjy5UaqymOs5hOVWJm6h8uN+ntZBWtzb8Qt7xmGiIhPsAB0cKpAikHk4D47mMvKLgMPuRLjiq63N9pH+snjchfuuLH6LgZtata56b7FBleAhHjFNw4l7KVSuO7uexPsreyO5hV+nfoUlyKkp9K9UhscKenJ5UtuZA2hBBA2jB3kWjzTn4R29kBasVKNqraK5rdfpaxB83xzhd3OqtqLrbosqshZD+7AM9theuA6AUszXOsaLEgHNEastlq6Xb+DawjZv9NjJuWReLXtUSDeyDOdvZ59JjFYUVyjmdKH3UUB0qrrXffBUrWlvWcpP2CKCFwxPApKke171m1O9n8V+Z+GVgWAhtNVL3ooAUPGG0+CzyEAS0gExwXRogSPO0j6JuBWbLZgCi8nw0Ba32xDl5fmw6EoPVgEIqQ37+Se6R5pJ65ppoyjZoHWFW7lU/23SSlBk9G9yqO9hu/JRxaQdW0aEJYByDoHehbmvIJS8uminEJ2OfGbtvVwsNLfm6DQecLO2gBDMbeUXtPKM+sJ0HLvxFIVRmdF5ztKB54Loj0RClh0pgPNATD1bI1GuqaPK++ckM6UjlJyC X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81594e28-aa32-4358-4016-08da76abf1e5 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3179.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2022 06:30:09.8039 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CqpGJi+z0cuVNc7BGjyF1tBn9AUWV5gRbNttY3hoReRSyt+QgOgK4bJNMbacwVHHxGlJxxqruH7WKgTlSN6LNA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB5613 ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659681012; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Y9YP5CB/KLf/ZQSCWaYUwoTyOvoICu+A6z2ecRVYt2E=; b=3528HxfIgcB9v/e6h5oM06HCPAqbkaUCAy1m0SiqPMEfg3RNGRKR+fnx/JYvsvQw5m3o5m jTirMtexG7GCUjIuTbg77dfSNeDc+IsE0BZbFgw9G6mDhy9ukhH6NIn5Ns6i7wY1Mo/RCg ah1U1jBLi/DdL9gT4uPRnA5am9NPGkM= ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=JUY2ZQ64; spf=pass (imf11.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.66 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1659681012; a=rsa-sha256; cv=pass; b=eH9QezmOVaj5wXxeCiT/8zFjA6Yut3Y3h2+Khcn0GOd+uvBYpXQkRdF5p7bUae2jBNZbA9 3Mrb1CoALHaulzacCy+W440n70GjKYbD9RH1uPUB8IF22NJ8DfE9vYUYUEtXWFIvY6jynA y7Weynb5sKJr7PAF8hVscCNbvEQk0sg= X-Stat-Signature: f5fruqpzedzujb8fzhbhbbgw337uumdn X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 50C554014A Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=JUY2ZQ64; spf=pass (imf11.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.66 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com X-HE-Tag: 1659681012-657173 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: gup_flags is passed to check_and_migrate_movable_pages() so that it can call either put_page() or unpin_user_page() to drop the page reference. However check_and_migrate_movable_pages() is only called for FOLL_LONGTERM, which implies FOLL_PIN so there is no need to pass gup_flags. Signed-off-by: Alistair Popple Reviewed-by: John Hubbard Reviewed-by: David Hildenbrand --- mm/gup.c | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) base-commit: 360614c01f81f48a89d8b13f8fa69c3ae0a1f5c7 diff --git a/mm/gup.c b/mm/gup.c index c6d060d..e26ccc0 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1907,8 +1907,7 @@ struct page *get_dump_page(unsigned long addr) * Return negative error if migration fails. */ static long check_and_migrate_movable_pages(unsigned long nr_pages, - struct page **pages, - unsigned int gup_flags) + struct page **pages) { unsigned long isolation_error_count = 0, i; struct folio *prev_folio = NULL; @@ -1941,10 +1940,8 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages, * Migration will fail if the page is pinned, so convert * the pin on the source page to a normal reference. */ - if (gup_flags & FOLL_PIN) { - get_page(&folio->page); - unpin_user_page(&folio->page); - } + get_page(&folio->page); + unpin_user_page(&folio->page); ret = migrate_device_coherent_page(&folio->page); if (ret) @@ -1998,10 +1995,7 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages, if (!pages[i]) continue; - if (gup_flags & FOLL_PIN) - unpin_user_page(pages[i]); - else - put_page(pages[i]); + unpin_user_page(pages[i]); } if (!list_empty(&movable_page_list)) { @@ -2023,8 +2017,7 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages, } #else static long check_and_migrate_movable_pages(unsigned long nr_pages, - struct page **pages, - unsigned int gup_flags) + struct page **pages) { return nr_pages; } @@ -2053,7 +2046,9 @@ static long __gup_longterm_locked(struct mm_struct *mm, NULL, gup_flags); if (rc <= 0) break; - rc = check_and_migrate_movable_pages(rc, pages, gup_flags); + + WARN_ON_ONCE(!(gup_flags & FOLL_PIN)); + rc = check_and_migrate_movable_pages(rc, pages); } while (!rc); memalloc_pin_restore(flags); From patchwork Fri Aug 5 06:29:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 12936866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA726C00140 for ; Fri, 5 Aug 2022 06:30:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2869A8E0005; Fri, 5 Aug 2022 02:30:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2102B8E0001; Fri, 5 Aug 2022 02:30:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0147E8E0005; Fri, 5 Aug 2022 02:30:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DA2738E0001 for ; Fri, 5 Aug 2022 02:30:16 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AA3E71417C2 for ; Fri, 5 Aug 2022 06:30:16 +0000 (UTC) X-FDA: 79764564432.13.4617769 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2071.outbound.protection.outlook.com [40.107.95.71]) by imf30.hostedemail.com (Postfix) with ESMTP id 233AB80132 for ; Fri, 5 Aug 2022 06:30:16 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QcvnJcIhxbdvRmXujmR2gPE9oVy3JITPD3lHne4xDUW0iAnF2p8yzv1QjciQDTMWCD8Nt/KFK8YnuPbFLEb6yI0unUQFR/rpNavNEReBnvE7lvsBs4qDTlb+Klfk2Bvn7CBEipdG1rJCRH5siImgfTbuA9lEFFYScovk5bbC+0zS+wiPMPwkR00Gr3Zpr6/epaPFD+Ul3Lf6/Mp+85ICoCcEJMw+rPdLPwoO9V3VHm5iy5UMNh30gt0vBWtLbEAlF8FgYEVqkTa6CIy6nKgaLz+TPER3QXENZCbo/TSuWtY9Jo3uA89/mo2BxMIdm+jqg2HL1vDJSTTVdUEOn1hxAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Rmm7Lmiwgjz3sy/7/4287JSulfBY6cPf23Vy1NwWlb4=; b=L8JsDvHzUTXT5QsEdqm/r2OcXhEONJtWZW2SOvZKqB6RH0mxDSk6C+T9yRvdgnsH0WgAI3NgTKlA80v+V4HBy+iY0kTuWb97xneaZMGSI6g8z9vifVXLwrfsyLSKEq5sYgrqjTtDFz3OMndVvjYpEfDr7oGhy8BUVI2pNOCL9g2BRZsCsk73H6pDgnc1q2T65V8B2gkmZhq86BkDbobDmA5nThH7aXmHDXU/3O+n3GC/lWwyYFW9uRBEJhoSHsKWSo0ptBGgFF118Kq6oS/XiX/dQyO9I5DiSSZXRtzxqE9DN1ceDU/kPvxYdCUU4K9swLYvOPjmx+fexQYndWq1sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Rmm7Lmiwgjz3sy/7/4287JSulfBY6cPf23Vy1NwWlb4=; b=FgmjD0U6g8Uoyw5WUON+1m04Zav5rn+lfVi0XT568hF9wtxUg0BNgcbdr5nBTi6fTVjB6RMLF3djJ6u33s/65ExwkyHUviRVWjAqOAPEdvt+rr4jwp4DNT59j51xNAxV7EsWiQdGiPgR6D4LmCd8x4cUnE3YHuftgRKMRX/bGe5k+mePD5riRy08JhOvfYgiDd7NaJ1S9MsDLKrtIAFpep3cIxXaRX6bhb14Hfw0m7A8Xn5yDAeUzEactxU1YQVSQ7rreoC1x58HTCQ7I96526K8FcLk2k8vfvAmlQbZ4/hrpWZjVZt1n+rRtvVyqDzet62Yo2E1PuZzjjTyicNB2g== Received: from DM6PR12MB3179.namprd12.prod.outlook.com (2603:10b6:5:183::18) by SA1PR12MB5613.namprd12.prod.outlook.com (2603:10b6:806:23e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5504.15; Fri, 5 Aug 2022 06:30:14 +0000 Received: from DM6PR12MB3179.namprd12.prod.outlook.com ([fe80::d0c7:5814:170e:b016]) by DM6PR12MB3179.namprd12.prod.outlook.com ([fe80::d0c7:5814:170e:b016%4]) with mapi id 15.20.5504.015; Fri, 5 Aug 2022 06:30:14 +0000 From: Alistair Popple To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Sierra Guiza, Alejandro (Alex)" , Chaitanya Kulkarni , Dan Williams , Felix Kuehling , Jason Gunthorpe , John Hubbard , Logan Gunthorpe , Miaohe Lin , Muchun Song , Ralph Campbell , David Hildenbrand , Alistair Popple Subject: [PATCH v2 2/2] mm/gup.c: Refactor check_and_migrate_movable_pages() Date: Fri, 5 Aug 2022 16:29:53 +1000 Message-Id: <3f6d04570907ee80f030ffaa85e8f506b3d31fc1.1659680600.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <3e20a542e756bbc0f66435c9344ff674f3ff7ac7.1659680600.git-series.apopple@nvidia.com> References: <3e20a542e756bbc0f66435c9344ff674f3ff7ac7.1659680600.git-series.apopple@nvidia.com> X-ClientProxiedBy: BYAPR02CA0042.namprd02.prod.outlook.com (2603:10b6:a03:54::19) To DM6PR12MB3179.namprd12.prod.outlook.com (2603:10b6:5:183::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a2954f86-daee-448d-a208-08da76abf45a X-MS-TrafficTypeDiagnostic: SA1PR12MB5613:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Fglv3BFkCwvb7lgUeTeshZ2ymIF3gUStnAa+aXMueRwmrPTjmvj5l/OKEwv3xt3OvSRjvLSMwlxWpzMoyTDATHvjWOiRZvDxNFbxrxdm9Lv7MUEokOuH+MqQ4yN7NrFkQAlbV94u7nQZeVbsfbhVKZIa/DpVVXLnse2da/WpIIz9bfqd382mE5lMcJLjtsRi89q2oCnMOuihu8SKuArbOaxVTKrzPujudCb3XdXcgfJpvhJyx1zYazRIs/RwtoRCj/RPl9TNICwZa2+j04ppeIu9vAEeIb0CUxMjVgIQ1hLBjNcU1QztjeTJ75v3U0q7LvSpDTmYM8f8sUyVSRsV0KIH0JquM+FN3VAVf2/fmcnC5il9F/vEvO0Ohx0bCkd9baGw7Q0a02saFJrTBwUsPusVhCviC4s38Xr2gEUlDV6MyCLJorqse7kHYccnaEy/X1w9T6zAJSkNqbIpCrw1SmvWMYsITzsF3U2WMkPpZoiZMmgngc6CEAPEpfm2b5y13dCo0raok8Iyyc3RjmEhJV4pccXjJMnoORWF7gelyyyIMBaIt83U4NEAR7x01YGOZfS+0gEGrf9vygLfD6GUXNhGVLlEDoO3PtEpOBqMP7TRexj8Ej0N/GBV2VWApwMHo7wIQNowOmbS9Col2/2hKviGAzw9U8FJhyovGEAv33JELEHT6+8XPsMpB4jsSFsctjQzvNNO3peUlCmS/Q4/RsofN+OyUO9fNaY8mYIm50SChVejO3xeYkttkDvsnOdDpp3W8HZZekK7xK8ERfQeSxVmMLdDFuwBMhhvGSCKq+InNLwucT94gjeMtF2KDwvdLrx815y7J/wm3ZTCCmuVGg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3179.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(39860400002)(366004)(396003)(136003)(346002)(83380400001)(38100700002)(2906002)(54906003)(2616005)(26005)(6666004)(186003)(107886003)(6506007)(6512007)(86362001)(66946007)(4326008)(36756003)(8676002)(5660300002)(66476007)(66556008)(6916009)(8936002)(478600001)(6486002)(966005)(316002)(41300700001)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: c2JbYF3VxdHHbUgIZs+bCWs2oxsMIERzfb+ZVIgRb1z6XMVi+lPTOB36Ksjwt5HXaEQmSKtBADNu5P3Ksh7xaLUcnZozZwi0ll5OXBrtuOyHiTOKQEU8qjkPy0Ex9HlHB/ToTN8ph66/wup1GhIHXXM1j6TnH8tutpGwqBWzV24XJrrl0FKxCtmEjjV3LuVVf2nF4UFM030zeNx5Jo7K7Klkgdp9yjkaVUvo6CS8f2GKH/pl5Db/ycVS/PMNMHXrycX6F53MOwH2dWLDXkr/PrGmbYUt+r5BCyP97dy/AAzi4mgJgyW9AJc84MNLL5UU9CDGi9naS1MlxrEu5BW7adCABhKo9u2LSm2qhHCxA7B97LMMnqKuFpOqnp05Q6ks9yLH74fSSS01JHiGXqJZUkbDoQ2Y+JdSDwbawg6XvzanB9ny7NAxeJrdHQ7A+Df7GDDTsTHKBXMOUZE8gvCHT/Ivj3zkyZfM/rX8lxfzZf89/0YgqNED3Pn7g2NfAzCCYZfeV8zL4dhzzNeiSpbq7l86s8J2SmaYxaOSoQnWWpPv1NBy9C5EToyrLhHqI/1c8r8sfWSk5JE8t92WSj6POIYBZMMeMzYIiFayLKLMvATkw86zTZ255dyADiSm/b5LcwNs4sohRXfNPjjUjYD1Mn11X72mRQvgwqerY14afmlh57g6WCyqimLYt+cHPSy5U482oGw621eRFaeBczJPn+HXvmxwzo+sskdwLQwnZsyKf77MAQp7CraBOXDQrtUn22upqDxr3bcW0HbN1wNrJq4By8yfFtskqtrxlWHZnmsgThYEwkpWyxI+717hlBm7yn4/UR17K4wTE0MngkiP3S5L4k9J9MlLYqECRocKMQszkILUaJb0kQewaIHzSrHMuLOUe5S/xTjoRg5XLxOzow3UukjnMIGadIE+Qp0PXZx+fT1wtYgIGawYHjd6L75K4gELHtlxplUC9MCNJnGfMkcHE/PbqSsB5ks6sG8NmQHvc/0hfmaS1nA1NWsmelJDVNx5egjY4s+gTWfHAyMenVph99aW8r2LCKsrL/IvicQPNsTKnLrhy1UZVfGJQQ4YfxzkyBU2+ev8OE/dSmkdoub6raaC4k2LbEE1IxqQnN+PkdeRmoJAiei/hEO4LEcU/zjerPFaYDk49qdu6Qh+GwG6DjQocJ3wNR/BZYFpWanxlqoRNV+y35ZdmJ09+BuzjWmYs9rl7KLnt22UkSGMfJ5/iLBkSDJsj+FiX/LJvmS8fH6ujpy+ShBoqEZFX84YEANeXE5aJiLJ03do28Nit5I4TLAUEd+A2ki13DGdIruEOpp328OSts2J1z+NlXMhoF9OtJNBJxlE429ohn62A4iRcsPkjKlI9l2BzNHWjP1eCd1fMKOe9ebzIzXWit0NYE9uq4Ntx0d0wUo5csuDe5n/gYF70eOWbhSUo46A1NvVFFRIO4/VnPTATfnQdm6BXEjRdNk1lHQaA9QMbd86blMWILyqzD4ve2Buhl5xlAxFUD5nBTd9qzOERs469kfRQmKjVV2Kk3K3dKVwExLcfoXd3KbyNXqz4tH5Q6mLnz05vhP3pTyda5YTZG+ULa/V X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2954f86-daee-448d-a208-08da76abf45a X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3179.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2022 06:30:13.8974 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2XjNXgMxKWfDw0yqxW6VS+2UWQpsl3OVRi5ztpLJtLANzbP7NyGXC8JEHxDPDrGImBW0Tufe4OdWJTvQlBk7cA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB5613 ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659681016; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Rmm7Lmiwgjz3sy/7/4287JSulfBY6cPf23Vy1NwWlb4=; b=YqNk+WOOVgmWYv8gaNCR7nCtlH5ElNGUovl+/fAWYA5JG1qRcefXDUx/Z7RVjO8ZbzaDs9 ldhNl8TbgduV7ksxzGtDwGnVufYznweJBPntxSgmAgs9LISB/qcMURqiCEa61BbfzGZ4aH eAHJVmkTg5qKcs6vEiR0aEWI+jOdBZo= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=FgmjD0U6; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf30.hostedemail.com: domain of apopple@nvidia.com designates 40.107.95.71 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1659681016; a=rsa-sha256; cv=pass; b=yVIlOBel9VyEeqYI7/2n0D2egxMCNvclT1QSXy2dq44gN/u42CSKr5PX4mkSye366ISJ+9 f5soSwe2HjyyaIad91LqvJF6VijdyJK2yQeNgeIJg46TTYtz5QuK3B72SgSawZu12C9rPZ 9nx5CAE9GgZqJnE2IowIlsgadmSVdU0= X-Stat-Signature: w3oic671pauig5w8no7ef5gyhx8kyqxw X-Rspamd-Queue-Id: 233AB80132 X-Rspamd-Server: rspam01 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=FgmjD0U6; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf30.hostedemail.com: domain of apopple@nvidia.com designates 40.107.95.71 as permitted sender) smtp.mailfrom=apopple@nvidia.com X-Rspam-User: X-HE-Tag: 1659681016-380307 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When pinning pages with FOLL_LONGTERM check_and_migrate_movable_pages() is called to migrate pages out of zones which should not contain any longterm pinned pages. When migration succeeds all pages will have been unpinned so pinning needs to be retried. Migration can also fail, in which case the pages will also have been unpinned but the operation should not be retried. If all pages are in the correct zone nothing will be unpinned and no retry is required. The logic in check_and_migrate_movable_pages() tracks unnecessary state and the return codes for each case are difficult to follow. Refactor the code to clean this up. No behaviour change is intended. Signed-off-by: Alistair Popple --- Changes for v2: - Split into different functions as suggested by John. - Made error handling more conventional as requested by Jason. Personally I'm not entirely convinced the conventional approach to error handling is easier to follow here but have left it in for feedback as I might be getting too familiar with the code. Originally posted as "mm/gup.c: Simplify and fix check_and_migrate_movable_pages() return codes"[1]. Changes from that version: - Restore the original isolation failure behaviour and don't fail the pup. Instead retry indefinitely. - Unpin all pages on retry or failure rather than just failure. [1] https://lore.kernel.org/linux-mm/814dee5d3aadd38c3370eaaf438ba7eee9bf9d2b.1659399696.git-series.apopple@nvidia.com/ --- mm/gup.c | 159 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 88 insertions(+), 71 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e26ccc0..60cb30a 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1900,20 +1900,14 @@ struct page *get_dump_page(unsigned long addr) #endif /* CONFIG_ELF_CORE */ #ifdef CONFIG_MIGRATION -/* - * Check whether all pages are pinnable, if so return number of pages. If some - * pages are not pinnable, migrate them, and unpin all pages. Return zero if - * pages were migrated, or if some pages were not successfully isolated. - * Return negative error if migration fails. - */ -static long check_and_migrate_movable_pages(unsigned long nr_pages, - struct page **pages) +static int collect_unpinnable_pages(struct list_head *movable_page_list, + unsigned long nr_pages, + struct page **pages) { - unsigned long isolation_error_count = 0, i; + unsigned long i; + int collected = 0; struct folio *prev_folio = NULL; - LIST_HEAD(movable_page_list); - bool drain_allow = true, coherent_pages = false; - int ret = 0; + bool drain_allow = true; for (i = 0; i < nr_pages; i++) { struct folio *folio = page_folio(pages[i]); @@ -1922,43 +1916,16 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages, continue; prev_folio = folio; - /* - * Device coherent pages are managed by a driver and should not - * be pinned indefinitely as it prevents the driver moving the - * page. So when trying to pin with FOLL_LONGTERM instead try - * to migrate the page out of device memory. - */ - if (folio_is_device_coherent(folio)) { - /* - * We always want a new GUP lookup with device coherent - * pages. - */ - pages[i] = 0; - coherent_pages = true; - - /* - * Migration will fail if the page is pinned, so convert - * the pin on the source page to a normal reference. - */ - get_page(&folio->page); - unpin_user_page(&folio->page); + if (folio_is_longterm_pinnable(folio)) + continue; - ret = migrate_device_coherent_page(&folio->page); - if (ret) - goto unpin_pages; + collected++; + if (folio_is_device_coherent(folio)) continue; - } - if (folio_is_longterm_pinnable(folio)) - continue; - /* - * Try to move out any movable page before pinning the range. - */ if (folio_test_hugetlb(folio)) { - if (isolate_hugetlb(&folio->page, - &movable_page_list)) - isolation_error_count++; + isolate_hugetlb(&folio->page, movable_page_list); continue; } @@ -1967,59 +1934,109 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages, drain_allow = false; } - if (folio_isolate_lru(folio)) { - isolation_error_count++; + if (!folio_isolate_lru(folio)) continue; - } - list_add_tail(&folio->lru, &movable_page_list); + + list_add_tail(&folio->lru, movable_page_list); node_stat_mod_folio(folio, NR_ISOLATED_ANON + folio_is_file_lru(folio), folio_nr_pages(folio)); } - if (!list_empty(&movable_page_list) || isolation_error_count || - coherent_pages) - goto unpin_pages; + return collected; +} - /* - * If list is empty, and no isolation errors, means that all pages are - * in the correct zone. - */ - return nr_pages; +static int migrate_unpinnable_pages(struct list_head *movable_page_list, + unsigned long nr_pages, + struct page **pages) +{ + int ret, i; -unpin_pages: - /* - * pages[i] might be NULL if any device coherent pages were found. - */ for (i = 0; i < nr_pages; i++) { - if (!pages[i]) + struct folio *folio = page_folio(pages[i]); + + if (folio_is_device_coherent(folio)) { + /* + * Migration will fail if the page is pinned, so convert + * the pin on the source page to a normal reference. + */ + pages[i] = NULL; + get_page(&folio->page); + unpin_user_page(&folio->page); + + if (migrate_device_coherent_page(&folio->page)) { + ret = -EBUSY; + goto err; + } + continue; + } + /* + * We can't migrate pages with unexpected references, so drop + * the reference obtained by get_user_pages(). + * folio_isolate_lru() takes a reference so the page won't be + * freed. + */ unpin_user_page(pages[i]); + pages[i] = NULL; } - if (!list_empty(&movable_page_list)) { + if (!list_empty(movable_page_list)) { struct migration_target_control mtc = { .nid = NUMA_NO_NODE, .gfp_mask = GFP_USER | __GFP_NOWARN, }; - ret = migrate_pages(&movable_page_list, alloc_migration_target, - NULL, (unsigned long)&mtc, MIGRATE_SYNC, - MR_LONGTERM_PIN, NULL); - if (ret > 0) /* number of pages not migrated */ + if (migrate_pages(movable_page_list, alloc_migration_target, + NULL, (unsigned long)&mtc, MIGRATE_SYNC, + MR_LONGTERM_PIN, NULL)) { ret = -ENOMEM; + goto err; + } } - if (ret && !list_empty(&movable_page_list)) - putback_movable_pages(&movable_page_list); + putback_movable_pages(movable_page_list); + + return 0; + +err: + for (i = 0; i < nr_pages; i++) + if (pages[i]) + unpin_user_page(pages[i]); + putback_movable_pages(movable_page_list); + return ret; } + +/* + * Check whether all pages are pinnable. If some pages are not pinnable migrate + * them and unpin all the pages. Returns -EAGAIN if pages were unpinned or zero + * if all pages are pinnable and in the right zone. Other errors indicate + * migration failure. + */ +static long check_and_migrate_movable_pages(unsigned long nr_pages, + struct page **pages) +{ + int ret; + long collected; + LIST_HEAD(movable_page_list); + + collected = collect_unpinnable_pages(&movable_page_list, nr_pages, pages); + if (!collected) + return 0; + + ret = migrate_unpinnable_pages(&movable_page_list, nr_pages, pages); + if (!ret) + return -EAGAIN; + else + return ret; +} #else static long check_and_migrate_movable_pages(unsigned long nr_pages, struct page **pages) { - return nr_pages; + return 0; } #endif /* CONFIG_MIGRATION */ @@ -2049,10 +2066,10 @@ static long __gup_longterm_locked(struct mm_struct *mm, WARN_ON_ONCE(!(gup_flags & FOLL_PIN)); rc = check_and_migrate_movable_pages(rc, pages); - } while (!rc); + } while (rc == -EAGAIN); memalloc_pin_restore(flags); - return rc; + return rc ? rc : nr_pages; } static bool is_valid_gup_flags(unsigned int gup_flags)