From patchwork Mon Feb 6 07:47:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9854FC05027 for ; Mon, 6 Feb 2023 07:50:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A4716B0087; Mon, 6 Feb 2023 02:50:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3535B6B0088; Mon, 6 Feb 2023 02:50:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A62D6B0089; Mon, 6 Feb 2023 02:50:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0715B6B0087 for ; Mon, 6 Feb 2023 02:50:18 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DD53C160BB7 for ; Mon, 6 Feb 2023 07:50:17 +0000 (UTC) X-FDA: 80436094074.23.0246D21 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2044.outbound.protection.outlook.com [40.107.223.44]) by imf04.hostedemail.com (Postfix) with ESMTP id 253F640004 for ; Mon, 6 Feb 2023 07:50:14 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ZtlV3xqk; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf04.hostedemail.com: domain of apopple@nvidia.com designates 40.107.223.44 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675669815; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kJx4d6L32xsb7lM+TLJ9IAT+Zep1O/ydPt8JKIy3HJs=; b=ZMgM7OXjehtsqaGrNyOxwq1Kk5rYbtKPgNCVmaZnT3A2knj4ZOQZkiPob+hidp/5RCV/tE tOlahVJ0RK+8Lm94TrWgYATt+NvDMMi2ckWXDkNl4+UV7khnVQw5v5EZ1K7n/4NkKMO9Tu LQ00KJfH/TCta7P7sC3x3Z8vQLg2VMA= ARC-Authentication-Results: i=2; imf04.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=ZtlV3xqk; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf04.hostedemail.com: domain of apopple@nvidia.com designates 40.107.223.44 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1675669815; a=rsa-sha256; cv=pass; b=Di95MiYM+m27/9t4FppIcookkx9ydq3fdMh0Ao3EEnXQ+TTcbIztW5rkxp8JALBnbdlbSw 8i/FgTQsA3c5KzXY7Rin/1b4RBeSqI7V2YCYJreI8kMKCIZi4Vg+px/34NXQmDxifD3FGh f4jSg60NmvnDZ8xQLU8fwDNZXfcDHKQ= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RyP2XQnz0N73PcQxjcUT4uAOGaufarRgV3t4vLzSzzaurNS9YOcieNt9HHUpRyhwuM0ST/JTfrnniUiAz2oiucF4XvGxWynkY75nZ+/jKnYF5jyJfTFHJLzLl+inSQwhMUp8KF/KTV6R0uHjgfrLSiQzdcb1w/PvSdL/whUjfUrVgoT0XHwiTaYRCpIpkuHalxJIBNxvQkJZ+M34gmal3k7aTaYlxnf1FZs2FlRsifhjhSiiUTTYgpp7TyZLepR00cZWTRFO5Vl65rxRkGfwPYmaCuzRjYw17K5klZ03gT+QESmo5i4AZg+VzA4BP9dJ3jf92mFMPCEYcQtJMJBfzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kJx4d6L32xsb7lM+TLJ9IAT+Zep1O/ydPt8JKIy3HJs=; b=ffHCQDtLu++R1E5ReQ7NDzjd2FnKcI31hET478cSznvJlv8G0gIdRGmRi5ovquC71bWDlL3D6+QbMGZacXA/lCwmCpeTFM13yhgSlUzhUUj9KSftTbmdDfMNVcPrh4lGp8gYIzzVm9MlilIb7XqLKIR+WCii0mBFs9TLYxqUTlGDY5ZlMzx3rW6jbN5nbnTOgzU78KBoqWZVQijbW2YIpB1a2PAihRChWoHiG6qBbl8fZ7Gtjnz7uWkLhJBs3AucxGtk52dUSy3xz+Kbyvj/mGn4YsqIkOTRx+jmFqkX+P8PUHkR5ft9LW0YSUvvdsqfYgbZBBhii8UPBuVSo4TIXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kJx4d6L32xsb7lM+TLJ9IAT+Zep1O/ydPt8JKIy3HJs=; b=ZtlV3xqkVCdEoJMfxTySEMtXYlBcu3RxzlI8GNdhznj8a3/UJ6829hjPsh6TZDRu2tpHt00m6JeVT9mO19xTSFl14OLTKMl2qMVxWKCQ08L7ww//5xTfGaLbzbp2s/s+uLONIvlAYGNugDKhqxzwxxpHpX1Q05Ec8ZZUZEusqbBXkWlLLK1Eb7YUEv1W9yawV01c44sPu3WlKPNxoO1W2hfj+pC9GmCitT/gDCXV23DrMjJW056Q12HoqU6w/6LxfLme95bxIla/oE9b5Tf8DZO97bHQwNjl3phF0Ws5x7pCH4BQc+7zmpLqeADSEpnxfCVrYTxZfdoWv2as/21C2A== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DS0PR12MB8573.namprd12.prod.outlook.com (2603:10b6:8:162::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.32; Mon, 6 Feb 2023 07:50:13 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:50:13 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple Subject: [PATCH 18/19] mm/mmap: Charge locked memory to pins cgroup Date: Mon, 6 Feb 2023 18:47:55 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SYBPR01CA0114.ausprd01.prod.outlook.com (2603:10c6:10:1::30) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|DS0PR12MB8573:EE_ X-MS-Office365-Filtering-Correlation-Id: 427e0896-7913-4612-52a1-08db0816c76f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N8UuAVBxTWt5/pJf6aD+fYRN9CGKIzqzua+ZekDKIw+u7Cw+ngRBnvwoB9EW06W5OR0HXypCbToXB/cTcd5hiyTJ4YX4oO9HDgWQ64BG1KiuYLkpOYZfB9slmftj8NkWdQMDICe4WBXEAudCmcOSQpCOVpdMi/UhZdvuH1pZtbZrSg0jh32quXs3bIsZh/I+WAET/3XXCpMr4+ZnA1P14qXZR1GGx+UvgPTLunxC0V3bx9EMasETCjCThtsfLbPoQlyi/xXMFqF4N5AMD6+Ggcx/PYVi46aL5nnpIQS1piY4WVYUz6r/yGju67Hw2klWH6/UgXbVn6p2oPQOrUVL/DkCe/WilMc60kJsFbu/ASymE+K8BUoQBoqefccmV2J5IQab1Nm5oQ6n6h4dYvPMCzFxe/uZ6DscTTIgCKgE3W/bhaglxgsPWqQG5+aCypZ4hqCrRcjKkIJ5hu2vhB1yUq4iQsDz1zDM6T/dgbV26YW+Jrusm+AVcjKbr59XR4/09BclrJ0OjBoo7tyF8XARfOT4an4oPbYDJvUzT34OHJaTUSauIUkw6aBmCNAH5iLUWR5hsrxp3vf3lAG59oP4GKI5Z6z2vAHJDaOLw+36QOOi0Ayqvl0QKAnlu1SkoOoCznQ1YH5U4TKxwNkqESttoA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(396003)(136003)(39860400002)(366004)(346002)(451199018)(54906003)(316002)(86362001)(6666004)(107886003)(6486002)(38100700002)(186003)(2616005)(6506007)(26005)(478600001)(6512007)(7416002)(5660300002)(83380400001)(2906002)(36756003)(66946007)(8936002)(4326008)(41300700001)(66476007)(8676002)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8l7kC3OaK8BVTwVGt5goAI8wanQmAamsLKwN4NQpQIf4Omil0phmccbSqFeG1Zj6ueoJx6xMDXquKDcQe5Er2cYvwVL4XX+2VHao5jZOwFkVnR28N1Sn4fn0tSRX1+dzFIipP5IPwtwykErOR0e3SkceMaAVzAQtlErlGG3oyjrFLk8oqRyfSp6gJpZZw+8PQWV9V1LO6JuAIUrR/CTzXD79JZnPtEBmhASDkMQHjJoUQ5TOxO5ONGXsnwI+n6lyNQm5W9G/fVdR3JBwGneg3qghzZf8j50sqW5ZOOiTAa4kxg8orZXUEuTgdmYGeAgWNpCtE2HQSKk1vvxJEJYsHzROorPqUshOjTR6tdE+QrGL5cuytzC+/2qr+X1Pit8TjfmyvELisUPWATl7QwbXwk4yOOBRCnCbW2FNSIzcYCDmt39TTs4w6RPni5BgIjHahkFqf5q4NFxlmctuxuHZ5eFe4T11pSkBRpqdK1cVGKu0J9ulk2cBYqJ9BNm8uW8eLEYWTXUG+C8fZ89hUQxsappHro1KbasRoaW5vum5BgCfWUBHNcU5aMcGJVQX8rrPQ5MwtgFllVkVtAj7RWLpfe0F60pGmjJyN6g+97czYDhflfKCZW7gbDXBJP8l8MFpLc6fNTJNNtm6jhsK9ZYyyS3KRgKwRF+gRJk2BBcjoi0Djtpx2dOJlX+F4Ioo8q4GvL0cKkLrXbH5g6Q5KNBtmmNjf3YpDhxpekbGluthLLUc2v5CCpKr9/ArAtsUb089j0p6582e7zMikLUgOQhOJ79dy9LNIY3hqH1wKTO5ieCBb/VsE5LpJpP4+WmZ0bKchrCLR2lEQShlUg4cSvKBX36Zo33V0/f1MtH/lr8xbj+BrwvlrpHbSHHelW3vZf9S7bVlqXuCzjKH/ulZdQDcAw+MvUsTD4vddgzeEBIE5+BoB4NxoR+FJbzLNPBGF1xv/OC2WXsNaOXRWGO24ASTDFKdpJBRWDSAs51qCJ2LqdhVJqwQZ6HOr5zz8VG3PTKVvQUKbflh2hf0LzJ0UDeGJ/+5xt81MysKO6VIcEElzItaP+lrrwX9ooJ+sTQWdI8Htq9Lxus9hnccpTFqmaVqQfxIhDIfSyqxv2qiaAUmWYENztI8W/pdTYAvSX+6Dmj4U3QQguzMoHuY/A9xU0ZXsadHFEhGh0KF2W8A5LPt4WSB6UolJ8T1uaNfmqZHKrMjZ9P004V0HSzhoV3uo6ZD0Pa8I73QgtzYTsKMjTacKcbXZgbPhGioZvLA4EZTdjWrS2lzZ5HJPAT/FO8Gm2C5UWy/2Y4ndsjX+Haq9k5c07U8wL0Le846YqGhqna+JnRUPv+xoJNaKPqpiutkX9qoF7mRn31zGwTrz8YYXtQL+by/WxJZOa6XPJSR9TAIG0g9xFEIVNuPlXyCkNJv2CptGfhmkYmsH+3Gr4cF1eOWtFDCPDAkkQPXJV+ubu4+1Q4mQMhH6v5DN3cF2YZ6EV5MphqpPQOlr9u3GhFBBwt0x4RCVlUeFqdEoU88XsrsQxekmUkMCAP54hc8+2jswdSuEFxkvPyie5g73YQgJQXm/7CKzyMT0q01G3Bv2NS8CUew X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 427e0896-7913-4612-52a1-08db0816c76f X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:50:13.2727 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tD8S4PSSN2NXQP7cLnr9ez07RFo1dGSONbQ2/RD76uWwk7J/TTGeB0H+KUhyxz4kuaksMFWXf+J2eqKvEedMsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8573 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 253F640004 X-Stat-Signature: ggfnaprj48z6xenofu93oxjihuzi763n X-Rspam-User: X-HE-Tag: 1675669814-97177 X-HE-Meta: U2FsdGVkX19npgWGC+ddr7L7M+Q0Z4CwdpYmQutNN1Hz46pXk30iA7CtDCFQfI/XAOQtmyPzGtdwotZ1MhMVPy3EyV9LmbBlMr2+kIBHGDaDkqSkRTfv606I7CkeI2GTO21TsSSUvTSkZHrCACulHbelRPMTAEh5D5cgujpVfLC4QiTA9VLjP4lR0K0AAJcKMVZu+EWxEd0gaJIWXu6tlJ2FroZlVwbVr7oxouUGhXOTZPYcosH1ROGiNq6Ku94k/J5jtdvfcoboDmG7RAB4avhbZA5CXI7p1sRD6NgjWRY5uVE7sCwB9hTLN1r34/QVWGsFrxMwWXf9tRYri6vB9gt+/yw4RK/48Y6vvZWNK+vJTxhkAgnAT4C7rKmxQf1Et9UHk+IZieKp6rRwG45TwLUOxl+7smmQS/Fgu1rNEEo6gdkJFoz9x+U5mbAkBr7hechcpdR9xrR7rste0YO65xb+7udOyWqRkVypNCVdh8Z5GIFlnD+pxYzw7x8sej333oid91/lDpbY6acbrQFwcLq/830WHql/NwWkiir27btqiWWOmGzw4mbKQ7kMOE+TA3B75Vo6g4rriItYA1beFkxp3BQ1TIrU7E61v5nAz0DMo6pYD8toe6W7r0zmwTRYdVkt8VhN1WT5+yizzs0wz7loneQUeD4rGhqxgwjTkjb8n/DIN9JSoHKEcFCySrE3qK/NEI01eBLhxUki7grjMKGfV+fsa3Yd2KgSz37Y11hMzu/gsjZLQchBIC1QPhnvp8/nZKdt6Vnr4Qo0UkhYkq4Il7o/E9+X2p0U9EYyJ7T6/pZG/OyVHEBlLmaNCGx70A2hz1qZo5GqE7ekq8LBK01LVw0tNXP+PRFzL4WL8byqWiV7xabcsdROPV6YCKPzDgwPnBvwfpo/OONtJ+wHqrXI2qQPXyu+zcLEGzOIG1KrQwhbrQTqlO6aKIkFNkrtfgd6Bq8ykYnZQ3fYr2s J2KwJ+dO /SZXoxb4j2FAOXR1PDEDS6AwCQgBPqDlRCRduf2icDB04xj5IXbEjUpkZrWEkhUQqhdlAjQf363gVQbqVxrBrgbWjCG87eYYtlY0qhwBWz0QzucA2xQgCbkYWN9xAADGwoZqnSf24s4FgMhJ9v6OGCzDisjG1gXne2pKn6gJnPIw7dJWXMH3rQ/Dmlh/FB9yImarFjb2e68n8rJPnZHGQPtrKDgTgY2AKkIXBQEmkyljcgg29zDdd10Hf7su3lQd22HQe2df8VMR+6osX/+4T2Vh8g3bULOyQcxDI2K4oohrxg02c45VR+vyze525QE9IngDQxLxddyaB6Kv6I05jKbLtq29J5XiCGKnJ2eGag5i4lMI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000244, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: account_locked_vm() is used to account memory to mm->locked_vm. This adds accounting to the pins cgorup as it behaves similarly and should be accounted against the same global limit if set. This means memory must now be unaccounted for correctly, as the cgroup typically outlives both the mm and the task. It is assumed that callers of account_locked_vm() only do accounting against the current task. Callers that need to do accounting against remote tasks should use account_pinned_vm() and associated struct vm_account to hold references to the cgroup. Signed-off-by: Alistair Popple Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- mm/util.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/mm/util.c b/mm/util.c index 1ca0dfe..755bada 100644 --- a/mm/util.c +++ b/mm/util.c @@ -589,15 +589,21 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages, struct task_struct *task, bool bypass_rlim) { unsigned long locked_vm, limit; + struct pins_cgroup *pins_cg = get_pins_cg(task); int ret = 0; mmap_assert_write_locked(mm); + if (pins_cg && !pins_try_charge(pins_cg, pages)) + return -ENOMEM; + locked_vm = mm->locked_vm; if (!bypass_rlim) { limit = task_rlimit(task, RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (locked_vm + pages > limit) + if (locked_vm + pages > limit) { + pins_uncharge(pins_cg, pages); ret = -ENOMEM; + } } if (!ret) @@ -607,6 +613,12 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages, (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT, task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); + pr_debug("%s: [%d] caller %ps %lu %lu/%lu%s\n", __func__, task->pid, + (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT, + task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); + + if (pins_cg) + put_pins_cg(pins_cg); return ret; } EXPORT_SYMBOL_GPL(__account_locked_vm); @@ -622,8 +634,18 @@ void __unaccount_locked_vm(struct mm_struct *mm, unsigned long pages) { unsigned long locked_vm = mm->locked_vm; + /* + * TODO: Convert book3s vio to use pinned vm to ensure + * unaccounting happens to the correct cgroup. + */ + struct pins_cgroup *pins_cg = get_pins_cg(current); + mmap_assert_write_locked(mm); WARN_ON_ONCE(pages > locked_vm); + if (pins_cg) { + pins_uncharge(pins_cg, pages); + put_pins_cg(pins_cg); + } mm->locked_vm = locked_vm - pages; } EXPORT_SYMBOL_GPL(__unaccount_locked_vm);