From patchwork Mon Feb 6 07:47:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F1BCC61DA4 for ; Mon, 6 Feb 2023 07:48:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D41DD6B0078; Mon, 6 Feb 2023 02:48:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CC9C06B007B; Mon, 6 Feb 2023 02:48:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF5996B007D; Mon, 6 Feb 2023 02:48:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9CDC46B0078 for ; Mon, 6 Feb 2023 02:48:47 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 64A8EA0B6A for ; Mon, 6 Feb 2023 07:48:47 +0000 (UTC) X-FDA: 80436090294.13.8BCE345 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf27.hostedemail.com (Postfix) with ESMTP id 820534000B for ; Mon, 6 Feb 2023 07:48:44 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Y4HQgHQb; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf27.hostedemail.com: domain of apopple@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675669724; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ggww8xGqVkEn1m6fFSqIeGeKfBiyQH3gSOfKyWdB+MU=; b=hC5sbA3ztYiidUnffLsfmxsCKXuw4F9ZG/yMw99iFxRUrvBsZxeNIvTJD67m/bQeE/r81G qnGeETDL/CIxLNKf544M21FYcIK/5HSelGzSF33zhczL5Z3Ukcy8OVWzL6V0S1zaEOKb7B ncYKaZXldZlHkJZlZvpdcrQlgqyTABc= ARC-Authentication-Results: i=2; imf27.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Y4HQgHQb; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf27.hostedemail.com: domain of apopple@nvidia.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1675669724; a=rsa-sha256; cv=pass; b=ww8t4nbXA4mF/pUjSHgs6xCWlbwQmbs7MHXxKnr/41IsA0CMInbxLBBXZdxoVLl3pTvuI8 I2yUyQqzG7bgUqYu89rngedwANw0CYpumLhFegDSJJUHTPjgHqrAlhDTnUwjdsUIFT0Y81 mo5ZiQW9deNOAZVaYcGPStk0A4TfVgk= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E3zRX50/Lp7tD2TtC29nYwzq6A4eIeezs2ezZLJbj5liFzIktI7OS/65Hdh0xdp/PbjYV4dFv+Sfu+uAVbcPXM8eyXcRocoD2zYb2z/CjuVbgm8wovpcT3YDm8Xc6nM8tbGiKCvwSE23Wky+fOWMCzGDoPVmQRD3fA1WLL3UwycdPIC5/p+uybOlCh1MzBl+B75jF/G774DNZqcRUJ+0Op60U7lfa4JnFF0616GDdrUQxuI69isqjPKJaTg25VkAYWAM4tk6zwg9aJvrbGYMRcMXK9TLfMTmDixIohIc98CIWnJmGhPE8kG3ZO3sYzL8rIXc/P5PRKjgvYqAmfNTMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ggww8xGqVkEn1m6fFSqIeGeKfBiyQH3gSOfKyWdB+MU=; b=nRqcU8SlI5ozKvuRTjwIW7R1YRuy+MsMXgCydX2k1NVuRPIGWcYO8v5Xv4jnYToHxkvADsrPTDLwrUvNDhxWy8WF0yPBNQPUG+jlpNQTy7572cdqNHgV+oP/R2sRd05uPR3VcyUY/5LKmnMIkDEz0ta3etjkfVH4U8FOVAI/uD8OqMf6iaK8edHggSwZSUXatrSS6TxXu3XNiH8Z/RTEtSkKO7FZBpkm/7kMghp8mEqm0sECRqs5QO4kYWsHR3s1zM61ZQCwbZKmhjgXiYWJm2i6jVGaV4H+Gyxw2V9KsoEGgTDt1E4tWgvsWNXfQ5u9CVTPHm9aVfyXpwIVtnLWpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ggww8xGqVkEn1m6fFSqIeGeKfBiyQH3gSOfKyWdB+MU=; b=Y4HQgHQbpd91DuQaDucY1BFD5ChpJLapbErCJK2J+Qdg5Zx3stAKxZfg6VuRoL8s4vyXjQXCzFra6r6nA+gOs06zJbufQur7eGvFXmHJbzoSFxkGL6DF+KhHE3q3v1ciT5JfcQtrpuAa7n1zHzaXkTUZEZfvuMMfQKE71BIrdvzHusNUsFlE0TrMLO/PV0c0lXrVjw4CBw7P+B5+SVnn04YuhMPSKptsOCYFvaEaexRvZykf3inWlxlrpw8GKCfR1+2MVlVy2sYAd831w0X9oO2VtIruCF1GfWwqfQyAItcM+2e0HwP31U+iWy2oh1Y/G8H42/B1RHbxRGtzlWnuuA== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by CY8PR12MB7097.namprd12.prod.outlook.com (2603:10b6:930:63::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.34; Mon, 6 Feb 2023 07:48:42 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:48:42 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH 04/19] infiniband/umem: Convert to use vm_account Date: Mon, 6 Feb 2023 18:47:41 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SYXPR01CA0103.ausprd01.prod.outlook.com (2603:10c6:0:2d::12) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|CY8PR12MB7097:EE_ X-MS-Office365-Filtering-Correlation-Id: 5a73dbe3-3319-4ce2-5ac6-08db08169148 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F/B80LQTVpL3jSLjBEjJ9QzNuXDWASskW1lHet9GroRZuF0YWlRLQJ2l6+dKcvBNT4uIdj3tgA50O6uDks7H3QyeQQ2mmNBfrQwGddxplhOX+lzizG0xwlfnhw733kRTbNW5m9EF7VmCDQU/ksswQ0Xvfq8Zq5s/1UC4s79qXLLvWzyRqJah4PaY4VukpAf2mK2sTLhRsGaV6p8fdYqI3lR4POWM93c/dKkAxlQRI1TzhldgsdwWxlXfew/fjG+QalrH9eRuTwPNLzgF/9s4+R239SOVYyzPdOq//F8y2RIxBal2+R78/9ZQkj7F/8dFZ0z8ET88cIyClSZIqNkOYA8/1YeyVWPGUG84L8qdhCssAy54zvJBGXlpzMkl4XjBp0yo9xqHAqwmu84/gql6ORMSdWpkFbleqf0pxocB31fOAliRLE26qLtbqQ5/1Lca0tKOMITqd7aBQJe8IrKMOSwTmj4A+z8CJ1yt7E9cQo19MW0LjJsqd5DhISCVs0UdExjs44vaMeUT/xLHO1DHOzY/Sbk/juEiillJDTHT6z++t33Sss25mBfe3AwjY4SPgZ2HzujYCccPx1VNfUfWbHNgowcQCR+Y6K0iLAeGv3EktxQpf8nahk4jzTTFBrN2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(39860400002)(396003)(136003)(346002)(376002)(451199018)(38100700002)(36756003)(6506007)(6512007)(26005)(186003)(478600001)(86362001)(6486002)(8676002)(66476007)(83380400001)(66946007)(2616005)(66556008)(4326008)(316002)(54906003)(7416002)(2906002)(8936002)(5660300002)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ubW3xrU0eGbFRq+G6RJcwlsfu6Ao+kgomSilgOyEGWZ8GnBsXzXYNDlWM32S9LpWG3caLY+45OVebDiKsG9RjYQeSCNLseBRz9Q/nsngnSZ44QPcejoiJJjHzbe+9ASDL3KZ8Vd0FbNgelO3asfiSNs0ykO57Hd+OOqppUqmwpg7qGk4FHDR1rZAQZHVxbyBswPbqdDLmXVPJRxyCYz4g/dIWPObS6YibcM90G1kPud6Gd5blcwePU/DQnD4ZbnfqTi9NAxTLz826/zRE4QvOemkr6fxeGr3PGVoSADxhqbu3CVMSY0Gq+0slPx+k9H6FuCEXqCS+wCVlcLsb3rNn4KJzc9RR6qwUx1IAXOLDhhxzkHWeRWm7RpWkM353mwqX5pqx7c2zklTbKFIvUzos4hva0ZzNjGOZJKkXRguAFsTKMyK8Tt26D/Ns1IxMBNu0ogQD9zsGTvBcvoq2mdnjJkH+x8SU8+r2xdUC2IC48OwBPfd9O68iZybjp5hi0m69EU3qrtcrReabHCrO8UdlOL7243ACaNmlyABkfU7nkpKepp9R/Y+ScXtI7UMtyNAuVB5UluoNjLx5SATikTtw3PtXPj7herx2KK1/o4QHS4ytiLWQRauR0Ws5IsuY34yrO46U72rdx1OhHXfLQ5FfPdDIYm+ygPb5/6WSDtwnf6pYl/41nKB46r8CqJpsGKRxl+fXlc6aLTc0HUfX++4VycAwoCRmWeM3cQXQehFj22YVnMpSUZ72bvYKsBKZvBwAS2e6VpDieFx3qm2e7di/Q7qfeNmeY3uMX91421Vq9H+jPqS6i2NbI84Z8PkJPevUu+1jyhZF6+MZ2fee+SoP0ej9TZOPhjjP6k0bUJHvHRgNXNrfe5doszBcgA8QxcXajOIPoQDnzbS/aADmmNO0lmPTRh3sfBtZo+Y/9OywcgJS2X/JKVA+KWID+sg/Mj8JQT6Mf0B+7Yc9IrP76s7IdRSyVbwCBhqFbSnSRy6NbEGh8EPkIQs/coWPsjzqdN77uueeJ19G3sj6OZTe4kcqMm/bmGB4ckjmBJ919mPyo55zKyQWnDcjsPxCzWlkzdNhB3K5qPbi7G3kAaz8qYId16bwGZPrtOwpf5EelwU35o2B7ndMRLqlEyGSpWH7fNS6fW9hlNlG+jcFpO6rRe79QlXH8I9m4iEbxiySI3St6d0lzO+cUX8H+0kcAMswl5HYwzMJpYKvRsQMEnzR255R2ocS4daaQwzJsP1RBaLJ52cus8zYbOpvYvQSjy2a5LDeu73N4hMIbrEhUQT5ChLS1NLaVdjr8gbt1akfdL6bpbbMnnWV8bv0aC54TcvNKS+Ea8b5udNZfZWD1J4odCyKsX2QXxwIs4UP5+CwYxbY4AGVtxB5nc1i9UQRCLplKteBtBcmzKOF6ugM99Dyo8d1JzOejv5dkdXCneCfDsamuTe/PKBihj2UHA0YSZdSzOY3vjXsyfib9hdwrNQmS+uL+u6RxerWAXuDRWa7M8M0C/HudBP80dHFEgxiVEn7NnRTfXODB3v3WRylzD02OaypRCs1sYhWDuc649+9Wfhf4wF6+wFKVpWik+fdBDInjfZ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5a73dbe3-3319-4ce2-5ac6-08db08169148 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:48:42.3891 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7xkdaR23n6b0+bhaWr+Avl1uc+gTnm7SJDHdHr2AR6iQKfTKWkLr3YjTvCIVM1p3eNklwHEvskiS3YS7BrKIEA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7097 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 820534000B X-Rspam-User: X-Stat-Signature: fh1eq6hkad1y8weqhpfhaotxxs87hg8n X-HE-Tag: 1675669724-366253 X-HE-Meta: U2FsdGVkX1/pM6yi1EmvbtyNxYwM4MA576jfxPN8dw6YlSfM8rnKkremN+/YxSTPT12Z100Iejp6h1F3BFzMf3Uc3/co5PfHZRy1J7Dafmy160qnJlaapwvTHCxjZfISSmKN5329aPNs+JCYdZ3RV87509MeNTnHZUm5fFjl9SkYoOqD6c017JxWtjcWr+a5SnTRICLUlnRVXfTqfdKHsxBjaML/eDyUqKdRObHDIz1zxAzui4WXM4tmoYVby+T7ONDYBb2brDt8Th7dR7su8Ck90hHienoqmjirhQJNJbWNTa6Ov8CTEHCCE4aZoKznDr0VBhYqihzQdsjeEX4KeYHzKFK1lnUuZQGb2GEvtR30sNWjH2XNOfY6Sykf3bpw0/LRyjds5/Ipf0xciCZbXfqgzTtZPwvV88Mt+/3I/0/4zKNEfUXa/0xXKpQiRnzgwzR7vLjiva3HmrE2eAFR7SrTWHKEyZNYvDIZjKnfA8JBPA9+SzoqlpwWp61ogdQTBzeX47Q4HTJODVLAxf1C+qSYIAyG4a9xYrDWRMF+oqypAvrW4ePQo9G2dlxGbOr0iQc72fxmea5O+ZUqWUllFo30Ni/pESHCtJBzzbGTSHIndTZxFht9yoKG+21MfA8+NRqI0GQZ8b2mbeIy+7R+rxHR9GZIEGdPOcRqb564oBa7TC+5TWGst9wliZyUrjuazCI/jqQe65iGqrCbl7gKc2j52cMzMQFR7s4nzXtj99usMlp1qH/mNpR+T6aQMnzx8v22DcDSozAZDkFLbalyTicySilkcMc+19DNMQdfYYFnSPjCrkM7iH918YKWHHs9QNc3IV6JVU29jEtJYRDn+JdlSqPWXVlLR3G+ChQzpLrbWuKBUXNWDIVo19u57nEINqosrbxBmf7tE5bwTxiUI3pglevv0lSzCvR5YUjNz1I1EVzREQ0wke+iLwP+z4iDBx0ArK/21tT2GLJl/Xt C8/i2Hlq 5IAmUdxccEiL8cz5s4kPuFjps/OdIJm/2ATxFX5DbCqcEP8KE8r1rCiCjGFRhxXzWSeVZ+IWMzMNpqULyvWdR89h6cokOlbGakD7qFknlwn5nnVnFQq569KNqKmKgAZSAX5gM2JkG84RvfJ48534trPS7Fwt9pzL1PvKsgc3LyzZvUGBqRcBt3aGFFJ4lUypEA0aupJ0U+fxR81Go/KN7vZ4m2Er1F2svOLmnXyZtQ4brcBz5uYWn+tAc6cCG8xuMceZkXfW38NFzPpkLBd+kei7xgsHoPfoQJrAMbpjwHRjhY0YrpGKN0yXN8NYdH/+6xDDdyG7MGD4DtnTxtdCvz/J6iIUh/cfGh5UZ6PAmfLOsj05WO5fP3CXNg3o1DfRMMS4DOCQsRd+wd/JwmhGghTWD4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Converts the infiniband core umem code to use the vm_account structure so that pinned pages can be charged to the correct cgroup with account_pinned_vm(). Signed-off-by: Alistair Popple Cc: Jason Gunthorpe Cc: Leon Romanovsky Cc: linux-rdma@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- drivers/infiniband/core/umem.c | 16 ++++++---------- drivers/infiniband/core/umem_odp.c | 6 ++++++ include/rdma/ib_umem.h | 2 ++ 3 files changed, 14 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 755a9c5..479b7f0 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -149,8 +149,6 @@ struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr, { struct ib_umem *umem; struct page **page_list; - unsigned long lock_limit; - unsigned long new_pinned; unsigned long cur_base; unsigned long dma_attr = 0; struct mm_struct *mm; @@ -186,6 +184,7 @@ struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr, umem->writable = ib_access_writable(access); umem->owning_mm = mm = current->mm; mmgrab(mm); + vm_account_init_current(&umem->vm_account); page_list = (struct page **) __get_free_page(GFP_KERNEL); if (!page_list) { @@ -199,11 +198,7 @@ struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr, goto out; } - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - - new_pinned = atomic64_add_return(npages, &mm->pinned_vm); - if (new_pinned > lock_limit && !capable(CAP_IPC_LOCK)) { - atomic64_sub(npages, &mm->pinned_vm); + if (vm_account_pinned(&umem->vm_account, npages)) { ret = -ENOMEM; goto out; } @@ -248,12 +243,13 @@ struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr, umem_release: __ib_umem_release(device, umem, 0); - atomic64_sub(ib_umem_num_pages(umem), &mm->pinned_vm); + vm_unaccount_pinned(&umem->vm_account, ib_umem_num_pages(umem)); out: free_page((unsigned long) page_list); umem_kfree: if (ret) { mmdrop(umem->owning_mm); + vm_account_release(&umem->vm_account); kfree(umem); } return ret ? ERR_PTR(ret) : umem; @@ -275,8 +271,8 @@ void ib_umem_release(struct ib_umem *umem) __ib_umem_release(umem->ibdev, umem, 1); - atomic64_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); - mmdrop(umem->owning_mm); + vm_unaccount_pinned(&umem->vm_account, ib_umem_num_pages(umem)); + vm_account_release(&umem->vm_account); kfree(umem); } EXPORT_SYMBOL(ib_umem_release); diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index e9fa22d..4fbca3e 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -130,6 +130,7 @@ struct ib_umem_odp *ib_umem_odp_alloc_implicit(struct ib_device *device, umem->ibdev = device; umem->writable = ib_access_writable(access); umem->owning_mm = current->mm; + vm_account_init_current(&umem->vm_account); umem_odp->is_implicit_odp = 1; umem_odp->page_shift = PAGE_SHIFT; @@ -137,6 +138,7 @@ struct ib_umem_odp *ib_umem_odp_alloc_implicit(struct ib_device *device, ret = ib_init_umem_odp(umem_odp, NULL); if (ret) { put_pid(umem_odp->tgid); + vm_account_release(&umem->vm_account); kfree(umem_odp); return ERR_PTR(ret); } @@ -179,6 +181,7 @@ ib_umem_odp_alloc_child(struct ib_umem_odp *root, unsigned long addr, umem->address = addr; umem->writable = root->umem.writable; umem->owning_mm = root->umem.owning_mm; + umem->vm_account = root->umem.vm_account; odp_data->page_shift = PAGE_SHIFT; odp_data->notifier.ops = ops; @@ -239,6 +242,7 @@ struct ib_umem_odp *ib_umem_odp_get(struct ib_device *device, umem_odp->umem.address = addr; umem_odp->umem.writable = ib_access_writable(access); umem_odp->umem.owning_mm = current->mm; + vm_account_init_current(&umem_odp->umem.vm_account); umem_odp->notifier.ops = ops; umem_odp->page_shift = PAGE_SHIFT; @@ -255,6 +259,7 @@ struct ib_umem_odp *ib_umem_odp_get(struct ib_device *device, err_put_pid: put_pid(umem_odp->tgid); + vm_account_release(&umem_odp->umem.vm_account); kfree(umem_odp); return ERR_PTR(ret); } @@ -278,6 +283,7 @@ void ib_umem_odp_release(struct ib_umem_odp *umem_odp) kvfree(umem_odp->pfn_list); } put_pid(umem_odp->tgid); + vm_account_release(&umem_odp->umem.vm_account); kfree(umem_odp); } EXPORT_SYMBOL(ib_umem_odp_release); diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h index 92a673c..065cd2c 100644 --- a/include/rdma/ib_umem.h +++ b/include/rdma/ib_umem.h @@ -10,6 +10,7 @@ #include #include #include +#include #include struct ib_ucontext; @@ -19,6 +20,7 @@ struct dma_buf_attach_ops; struct ib_umem { struct ib_device *ibdev; struct mm_struct *owning_mm; + struct vm_account vm_account; u64 iova; size_t length; unsigned long address;