From patchwork Fri Oct 18 22:34:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 13842474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04CA1D3E18D for ; Fri, 18 Oct 2024 22:34:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E2226B00AD; Fri, 18 Oct 2024 18:34:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58F5E6B00AE; Fri, 18 Oct 2024 18:34:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E21A6B00B4; Fri, 18 Oct 2024 18:34:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1DA926B00AD for ; Fri, 18 Oct 2024 18:34:22 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CAA5A801BC for ; Fri, 18 Oct 2024 22:34:10 +0000 (UTC) X-FDA: 82688177586.24.B02347B Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2072.outbound.protection.outlook.com [40.107.236.72]) by imf15.hostedemail.com (Postfix) with ESMTP id 3AB14A000A for ; Fri, 18 Oct 2024 22:34:07 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=IC86ihv6; spf=pass (imf15.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.236.72 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729290784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=fVzzE9PJ88+5Jt6bTYpFS4uzjPfO9T0eul4pMPYXUSY=; b=qF8zuCIAtIMSCyXrDcV6gQ/ERM7llD2btw7bDeKyBxqBDcdgcgcUxoU9BRZREdIY0P9s6V 4tV3+cRga8AYiY+xciDvl9YzYdfI0DnyZW+ItfS6C94WpFGRgy01k5RYRKGVxfd9UxPm7t C3sTX9t6FWBnAawOFGxivxDoCKMtFrI= ARC-Authentication-Results: i=2; imf15.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=IC86ihv6; spf=pass (imf15.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.236.72 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729290784; a=rsa-sha256; cv=pass; b=Fa6as+CIcR/JFRkaoYBeOkqo51ufaw14hvOq/rCqxNZWmNBmF5nIos8TN/0PUGX8ka8Rzp zoNpW8EEBkFsofIwCJWjRefScr8iVyhECLu8ZI4bD4B72n3HezqSb+7o9bN1eQ+VnAjLOd Rh1DQZI3+f4so1uuWEhy0GycXx5js0g= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vergSXURYi0+fNr+eiHLW4NVcJOZ9Kom+sdm2GtvsAzLwo9hoDI9eXxdjj3Iw6ta0Im78TDfR1Cl8c4YLFWLRdFDma+96zXYAKYuQ81gw/jLBOSa1tXCSoAk0JSTHyx2ZqglbqvGsbxmufuN8fRZFFhwDlkgl4CQSl2leoRsy5UItLMicXFS0aPa2aJ5xggxAwsZVRNy/X9GtBhxJk0aIIFQfZhrEdwjI+KRc2aPpcdCPAUeWvA1xKCrxSPw28PBcQnu0QL5ABfpYp38pNa3Dy7bMaDZD7swYaGtxMYeWSX83eLFEy7JOogUBimXq8L8q7MlgiQStOGGUMpXj03sRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fVzzE9PJ88+5Jt6bTYpFS4uzjPfO9T0eul4pMPYXUSY=; b=rOIiuBAjBJbPhw/AdfK271QzeyuZOb+Y2gwdz8GYx0WLmBKYLgfeLH+5GumzLTlgUlhVUw+va1p4F6ARoaJIWFKqmBxxdKAW/wxMMigCzq+DV3XGiCKDNEqLx/DFinkQdg675lsTcZCHXR+C42Y1p9j9w2xaB95iV5pgPZ+uk6NpWnB68KzkjGCbJAWunpyBMpeW+b/zanNTgklmbnxBDZUU6YlwRlzlSWV8WTYYT23pviuRNnGxtjU9r8NXf+95frt1OdGOVlu5c9C9rmHq54sydVOwZEYfH/O7c1Rx5c++8nLkzucpOIA+d8OEywb+IgfWGv9IKB2Mlek1GSvGUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fVzzE9PJ88+5Jt6bTYpFS4uzjPfO9T0eul4pMPYXUSY=; b=IC86ihv6uN6GYSY0LFMW07bmAlcz8/UGnvGzHilZPt55q99GrBa+fTPvfX9xc5I3VtKZ1YjimWkayPAaSiD48+p805UQBLzk96LK0ODPyD7yd4rgzISI1C2mIZdKzmnXFRWyyGOtfyLvo5H/YbUIUCJUSip3N+aNhb8+Cl1pr+nRqSqUEZocgBg7eYqlSZRbtABi+9zT4lk5S6rHodWsWJJqUYSOCRrJ9tUaVxtYijq4yutxaXXwPohvdLpuolIj8OwGQSvpd0WbVB79lMRUFyRXFX7GWq1g1d42pgiaGPQ7DA0SwuroryjZRtmymuO0qn16MYR/URoFH76c3R7hVw== Received: from SJ0PR12MB5469.namprd12.prod.outlook.com (2603:10b6:a03:37f::16) by MW6PR12MB8897.namprd12.prod.outlook.com (2603:10b6:303:24a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.20; Fri, 18 Oct 2024 22:34:15 +0000 Received: from SJ0PR12MB5469.namprd12.prod.outlook.com ([fe80::ff21:d180:55f2:d0c0]) by SJ0PR12MB5469.namprd12.prod.outlook.com ([fe80::ff21:d180:55f2:d0c0%6]) with mapi id 15.20.8069.020; Fri, 18 Oct 2024 22:34:15 +0000 From: John Hubbard To: Andrew Morton Cc: LKML , linux-mm@kvack.org, John Hubbard , David Hildenbrand , Alistair Popple , Shigeru Yoshida , Jason Gunthorpe , Minchan Kim , Pasha Tatashin Subject: [PATCH v3] mm/gup: stop leaking pinned pages in low memory conditions Date: Fri, 18 Oct 2024 15:34:11 -0700 Message-ID: <20241018223411.310331-1-jhubbard@nvidia.com> X-Mailer: git-send-email 2.47.0 X-NVConfidentiality: public X-ClientProxiedBy: SJ0PR03CA0140.namprd03.prod.outlook.com (2603:10b6:a03:33c::25) To SJ0PR12MB5469.namprd12.prod.outlook.com (2603:10b6:a03:37f::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR12MB5469:EE_|MW6PR12MB8897:EE_ X-MS-Office365-Filtering-Correlation-Id: eb06045d-24de-4625-0fa3-08dcefc4ff02 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: xAAf/4FjVdU3ZLO/kOFT7pkddnWMMRjt/MeH5a9jJz1vkZdmkLr8BbGAi/udFoo32jow/S+7++0vsRsLbwVY9iTe99mPjine8JsLDjk/YE4bxpE+6saU/9NzwPUMHcWthiozOyU0quYr7P4zkvx1/MPcz/W2ZUqx2vXqbsuCzXm9KA5oYtMu5X5rkh2rqDve6S4eLznJmASz1EJIjO1oIBAVaxrrX92vtmOd3NaLO0WYboSyRtKDrqWKvgrEmakmoB61PqtZ2W+pjLZjf+brLhuf8ZEvjp2zVzmD3jqdNAVSWFe1Z/rvhIFk5pZhJSrrcxFPX0rQfQ7K91/+iTxRyOFtVE6VTLSyfpR87HbhGBml2kCRfwGuJeoVvOvGcxz8kl1fC4+3c9MPFhsgMNvvfa5ar/4K3Uaa5xdQJ4Ag0cYPSbq+s2dtte6ZRkX2U+9E9/JNMXTqrxj2a7OvB6gGY28mzTQv8Yollb/h/1FGgKcEFYbRGvZS3PeYtvPeDAteNHq5hCziG3cBJ3jC8tbYogNkydC35sX7c43f5VQERl9TFYNjeToUcqH4hyOCGNsE7hYS/ixLDs/oEUcUJ/xVCsY5/gUUEqK5MZEhh0HBTfugk71T1Axqfd74fn4pkwCmDMy5RHHUi5EC74tIqjAtwYryABQAsLpbtr1wIEgJGG4Z4rBsDN9Ts0USm4rOj6ZRnoO6pSgs4QZt722dH29f5F4mhocgXTg9WUtkbL8SLJx4dcdtBAptPDhF4GI3WEYLSDMjTSbxlq9IXMTx03PCCeDAlRwIWG09bjEWwFqtOgG0k/qyl3JCAlWwE1SY1lOmnT0TFXFgNCad/SxbAok9rRfKECKpcpig3dOywxXnIXtgFTqk51/O6Rvyu3nRWPNiMB9LFDQVEa3QROiSuK5Np0P0Z1Lod6W2pnQ0WpT+f8kuEvBEiB2fbN63fSlJSuOkWTOakD4QmFDqO6W0FtRvGOff8U7TpOgYsKG3cP6HYsT+3n36FvTkGjZ0p9AGfi6hMChj/MEtcq005NIUUO/sk6lFU7Uwnp7FEkhjBp7ePz48iwE3iqVPUBwgWvYQFpEGHilxP71ShmJo/KCe/p4ZSoD07gz2oKiVUeDW8SPL8TEg0EDuzEDC2Je5bUeXAXHofzYkiax+V+LRprwTHGsigV41DCLIFIOXTZ8AtvEyA1CKcFx3LWXs7SdFhPEW2JzU4SHoUWMNn2NuafSZh8mp2Aze4Q37eZnYtZdl7yHI2PAYjg2aIRRyz7mt6OAdqrLKDBaQAUq8e8Sx/FZK781siTLb/hc4NMoDri6qo57/bAWiYrKm26vRPj7SUUwHMiQI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR12MB5469.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7Q7bOU44HIqjXXdPye7JuEaft2Nv6jLOjNCRxMuc7rDz1FGvmD7P9ob54BQwoJ0ouPibazPOloAvWDM/QKPGWqcCrJ7je2gQLbsBuQK2QN5Pwdl0Pen9m9wp/1odATLeiBVx4G4Tro9V2FUlxOY9cQ2ds3fKK84W6/qTkaQyqIp2yyND872i4lM6v9joSowwrXEZTlRCjoLeCXvZnb4cxt0Y3OEneGgC3uaRLU7mwl2HMAhcVfrBp7UwF1o4QVw4l6q0KsIFqmxo/5HiBdveHq01GzyOnPlGGdBbvkP/h97/4hDr95r5l7ZaBNEiw5056HFm0LZZroTbvBtSvY0OoIHnuvzytVvQvNQS2LfhV6YChAe9fKh5UyTw/rzgBcjEzhjh/hRbMPEy1sijEgQePslGoNDjMW8MXzxYzErjybuP2HIDNvCUyLeB5qctp7dmsTXqf8yx92ai5Pz+tz+fEBGEXBcSu/Ln1GHGpqwgGL0k35oDdag7JK2ihsgd17RhtNOjeWcKzsTm+FSPcycEwgT05NX+SBZKlFok41cQfcGyT70Dau+KTP//SsFNKSmAz66sHroFoxmIbcLJ7J0+cdRqT8gXEeHl8c/GBXU4u8+E5Idwvx1QVoTL7Xq1pdcxQqFyeCSY6/B53fIbzkMNeyGMe1oSKKWyhUvCPBJnKFwjeoNpNUHXZgb11LkWa5txWrGjB70wDtvw1H00ICY0CH/onzhpNDuydLrQToOIUz1FGVMizA/ae75YFSB/Nk0KHI+UwnIx6JFOCv/x2IzAPai03WFxgfhvWJigGIySCfjvGnXjLeOFEJ04NtQbALCWYz0tWt+21+W1jtp3MBBlqa4DfYZ4mWGqvmF3CVxDZbH89FsOTaQ1c+/IiUwb1r08Eo+ZnwZer1HSKAF3rS01t7FH9wK6WBdUoA5++FhQCd2rtFP67fCxi7XAPsPXbBwTzzWJpb3YFgQJGpMNM+aPDVRk93sYgkq+Vx6hQPjbW9FCQNMd/NGJTC2lNHRi8zUaJNu9av6PiWK1RzqJbVlLCR/gUs5S8gtp9h92RcL+Hd5uiG+tPcqhqzsUt0pE/Niu6LRygDtHDNdL9SCQh2ULaLO2wrmrtV867LEOwgg7PEAayWAAdAFVtQhniDIKGldGtHnsRn6+0lr9MfVNl/cHlSYzAeJ5bavJXaXnBNgeQS+76zpKE1dLYRZEcwGw5V7kTadwmaYUK+8YHIEhJbZI4OxAa06Hl+3SRk5A95mcxxmtgV+miskbw6K5bpRhQtd+byaMRyzhbrkB4evPyqU/vA33i6y+JGHb17NV4eNS7ttfxNxvemdfWZ05OTra4CqCTZibpgd/rTBpu4q40AdcAoCLk5VDZ0vrmniqb+IpbYp2pOpSYtlhZzj20ONZc6unrMXpztFyANFHsnkDkF/AOG9X6+Qs20l6JbhfcYeygyYoOg61un4RnE8G3s8R3gXYthYya2KqM8P2WOXPakR3T0vPPI9Nk0aGdn29gIV7xJc/Qv+HQpuLA6oJp+ThJ3R7RA90pS+9CvZ5H2xUbafHUWrsVWzTJ8yJKrB9dOZBBluCIPd6Bg62obIZ/IXtkPhx X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: eb06045d-24de-4625-0fa3-08dcefc4ff02 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR12MB5469.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2024 22:34:15.1933 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: q6eTCkVlsJRNYw74y2oXl7UbgzjuXU7k6IbZ8p0RBB1oHJdyaIvp6rKolyFYqux8fKw5oZ11nxBxuxCQ7zKbDA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8897 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 3AB14A000A X-Stat-Signature: c7e9p9mmbyfyuaynhq3fi6pep74u7hy4 X-HE-Tag: 1729290847-595905 X-HE-Meta: U2FsdGVkX1/72rPJYWpt7nrvhE29/UTXhCOHK6AUy0JbEHseIQdy7ldyKHqvcS6FBdjNbJopPKxHeuyDZSzPLJOIWc90FdgxjQ2myaHwSIAkDUCFFMF7DnJe0PdjEFRyIlJFV9Vn8PsdArHTCPnW37jHNtYAGGHZu7Jf9ILo+EZpV1rn0g48PPna7ikrpvdzpg8Nc2+En2znxKupDE4fKQ5KMeP63RNOqaCwr3ktCyDxBmi16TtFdkkvwskRwRZz9lUxDXjUjIsCHilOpAI2uSy13UCY2SP8j85jaJZcPUw5YfZsQFCQAsQHL8vEnihL52n9gr9VSLoLqtuBloQptOuFOLJpnUMFt+gtucFxLLq2JL2A/R2rB26pm3negTT87IOF6lBQvdF6FYJSTSesASyXIgMaNX7e9lqlf3UpgEtpw2kpjNat/ao9cqzwEw05y8pE76WmY13JjuLsZLXlvnm4AEZ2jTxmP4tdtqaWeHSNnEwMmjDltPM1k5IUwvuIuaoW7hG69TGs9hleZ0rMFm9DzeR/shopTcP6xxps1YJM7SJkB5l2zkAFKr2n5t/ATAHs5urFGI2ZUGzcFm+nnXDfasPnqPKtHOKihhHTd5ub33o4TfMtc8Koj6gT5PPbdf6UG7tC6d4DM2XfYHQr4RMY344kcB8vyhaeLWTJW1DC5wPeQgkXY/+FCqudwpwB+r1ixU5TY9MQ1GZnTyvXO905Fxu1sYxiGO9ksY31Nr+LRXpmNtEsMOy58fGTUUKi5/o5p+qigcELf6ijsuOdKuRU8Xrv/Z02QbPp2Do9MW8wFeGE/DiiGeDm9Ex03VXupJuu3iuoss1LkFChgf4kRvM7e4pwRpYzmIZShheoiMkON8mW6SvJApSLtV9qurteAqBuyvc9x2EcoIOzZRTclRmBYUoUak2pCluQtrGoOvSUkpqZjxuNSRDTcShEHSsDOC8wfvf0g7ZJcJ7EK33 KcjdWA2t TdvrklywlkANNAScyNnw185YAK4aWFFvUVdgcIs/1txZMikTC6Z5qyzl1ZsGh4kjJbJyo9ke4tN/o5CuodlgzDIbQFBFGH23ZXPjikKRz42z7SiM4CicX+B0MA6719W4YVJrodSEhxm3K4H+98vi+DLZ1h/bobW3gAfqEyTaFS2LDwPhAzferOEte7+RDiySgRRTxb2+Z4cLIpchD17TmnZGmnS96/BbDw08iKxbixSCmUD63XiYEE2WBYNcC/lUXlC1mWUd6wugcEFm6eACcPV5Oa1/k13vGZ9T5pRK176NWnXpCRJdtCePxIknN1v+tfwzDs+SvQpwB9CHRtX6Euc0h1hPArduXmJyW2W4JbTbnSWM26Qn/wUi1AcNfWGU7+QWpTG2+5EzVoVZW7fR4GvkP1w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a driver tries to call any of the pin_user_pages*(FOLL_LONGTERM) family of functions, and requests "too many" pages, then the call will erroneously leave pages pinned. This is visible in user space as an actual memory leak. Repro is trivial: just make enough pin_user_pages(FOLL_LONGTERM) calls to exhaust memory. The root cause of the problem is this sequence, within __gup_longterm_locked(): __get_user_pages_locked() rc = check_and_migrate_movable_pages() ...which gets retried in a loop. The loop error handling is incomplete, clearly due to a somewhat unusual and complicated tri-state error API. But anyway, if -ENOMEM, or in fact, any unexpected error is returned from check_and_migrate_movable_pages(), then __gup_longterm_locked() happily returns the error, while leaving the pages pinned. In the failed case, which is an app that requests (via a device driver) 30720000000 bytes to be pinned, and then exits, I see this: $ grep foll /proc/vmstat nr_foll_pin_acquired 7502048 nr_foll_pin_released 2048 And after applying this patch, it returns to balanced pins: $ grep foll /proc/vmstat nr_foll_pin_acquired 7502048 nr_foll_pin_released 7502048 Note that the child routine, check_and_migrate_movable_folios(), avoids this problem, by unpinning any folios in the **folios argument, before returning an error. Fix this by making check_and_migrate_movable_pages() behave in exactly the same way as check_and_migrate_movable_folios(): unpin all pages in **pages, before returning an error. Also, documentation was an aggravating factor, so: 1) Consolidate the documentation for these two routines, now that they have identical external behavior. 2) Rewrite the consolidated documentation: a) Clearly list the three return code cases, and what happens in each case. b) Mention that one of the cases unpins the pages or folios, before returning an error code. Fixes: 24a95998e9ba ("mm/gup.c: simplify and fix check_and_migrate_movable_pages() return codes") Suggested-by: David Hildenbrand Cc: Alistair Popple Cc: Shigeru Yoshida Cc: Jason Gunthorpe Cc: Minchan Kim Cc: Pasha Tatashin Signed-off-by: John Hubbard Reviewed-by: Alistair Popple --- Changes since v2: 1) Applied the fix to check_and_migrate_movable_pages(), instead of its caller, as per David Hildenbrand's suggestion (thanks!). 2) Dropped Alistair's reviewed-by tag, because the fix has changed. 3) Reworked the relevant documentation, and commit description, as described above. thanks, John Hubbard mm/gup.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) base-commit: b04ae0f45168973edb658ac2385045ac13c5aca7 diff --git a/mm/gup.c b/mm/gup.c index a82890b46a36..4637dab7b54f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2394,20 +2394,25 @@ static int migrate_longterm_unpinnable_folios( } /* - * Check whether all folios are *allowed* to be pinned indefinitely (longterm). + * Check whether all folios are *allowed* to be pinned indefinitely (long term). * Rather confusingly, all folios in the range are required to be pinned via * FOLL_PIN, before calling this routine. * - * If any folios in the range are not allowed to be pinned, then this routine - * will migrate those folios away, unpin all the folios in the range and return - * -EAGAIN. The caller should re-pin the entire range with FOLL_PIN and then - * call this routine again. + * Return values: * - * If an error other than -EAGAIN occurs, this indicates a migration failure. - * The caller should give up, and propagate the error back up the call stack. - * - * If everything is OK and all folios in the range are allowed to be pinned, + * 0: if everything is OK and all folios in the range are allowed to be pinned, * then this routine leaves all folios pinned and returns zero for success. + * + * -EAGAIN: if any folios in the range are not allowed to be pinned, then this + * routine will migrate those folios away, unpin all the folios in the range. If + * migration of the entire set of folios succeeds, then -EAGAIN is returned. The + * caller should re-pin the entire range with FOLL_PIN and then call this + * routine again. + * + * -ENOMEM, or any other -errno: if an error *other* than -EAGAIN occurs, this + * indicates a migration failure. The caller should give up, and propagate the + * error back up the call stack. The caller does not need to unpin any folios in + * that case, because this routine will do the unpinning. */ static long check_and_migrate_movable_folios(unsigned long nr_folios, struct folio **folios) @@ -2425,10 +2430,8 @@ static long check_and_migrate_movable_folios(unsigned long nr_folios, } /* - * This routine just converts all the pages in the @pages array to folios and - * calls check_and_migrate_movable_folios() to do the heavy lifting. - * - * Please see the check_and_migrate_movable_folios() documentation for details. + * Return values and behavior are the same as those for + * check_and_migrate_movable_folios(). */ static long check_and_migrate_movable_pages(unsigned long nr_pages, struct page **pages) @@ -2437,8 +2440,10 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages, long i, ret; folios = kmalloc_array(nr_pages, sizeof(*folios), GFP_KERNEL); - if (!folios) + if (!folios) { + unpin_user_pages(pages, nr_pages); return -ENOMEM; + } for (i = 0; i < nr_pages; i++) folios[i] = page_folio(pages[i]);