From patchwork Mon Feb 6 07:47:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13129439 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAE00C05027 for ; Mon, 6 Feb 2023 07:50:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229956AbjBFHuH (ORCPT ); Mon, 6 Feb 2023 02:50:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229873AbjBFHuB (ORCPT ); Mon, 6 Feb 2023 02:50:01 -0500 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on20603.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eaa::603]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7ED691DB8C; Sun, 5 Feb 2023 23:49:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fl0haf409z2GVuUVMTZhi74sVRNd/3eb3P+XKkXnOrz/OZay6AtJ+tlwc1aZhLVpArLa9aEv6XGmL2kHSNcMwyt+TG95tPddfsQYKjHPpki8hvX5nJHMI0SRWQYk0HYIDl5tVq2yKNTTzFGyKmXlNODVjuVz3+ObltL36ZyQhmXQczcaOjDCKjQVydnMHlzEj2nvm3uI5RnIBPEt2D0K+h5xC2bentbNI4CCphhWtQ7112FIsdxQ46MSgEEm3o0iWmd686aof8ngHzfeS8AZGhrqRY1O6c6/a3xfqhNSFJmqJc0mDM9wOw7LxQl+tiuVNqm+r+LdC4dWCQvdxm2fgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ekaPX/00S+h70lFxojvYEzzxngZqMCIdq9dGbDFXgv0=; b=BK3OS8gOFEMksJ5VUhjDVC+0VX9SjWKxeKdwBquaqec5ZUx+y7/efgn3BS28Mi6oLOk2KIaEIVrUPXZAhhHJfgOfVbX/hD/pRcWU2zYSIJ5QPgy5Xxd/z1cErKhbtrsLj1glJD+iXd3A0qzkdg9O3XP6yuwMSXCjG7aex+pvBBmYoN8uNFy7e7E0uiOOiUXyoXF08vTnxNOOOopxH5JjpmMk/iHeAdUjuMOW59ncl57U7wTY488D0sGHJx/I5CiPjTWyV1tdIg16qzENQOyS0HVAGHSZS10ISckpLfVCWUQrNDMIs9OXYIJ2htnCq4xqmpC9Axlm8bs93hRI5gblsQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ekaPX/00S+h70lFxojvYEzzxngZqMCIdq9dGbDFXgv0=; b=dNyGq21XS2lvzTJSJQWV2Dx2hCd50LZlN6ntem5wh+6FJ/wf25TDeGen5qzV/Ru+sSrrRfFfbT2vX0LXlq9OCiuFpBI7KMGEUOyWR6fOXlmu1oVxm+MKYfKxOSs17oG9LlGjoGhF9kxJWb4vSXo/L+2bzdrveRZ7QGe/9BJ6D1ZOXD71dpYeFa9kyJfAUNPBXvBBiuKAoTkCB9TNSundH10vtOke9XV5FGVFHq9TfBBSkuNyXy595aiL5Go9zAK6DAFviaAhBZGTVTWhHUbjkI3iI5WmEX/SU1onOp1pJvG/7mW5f5AdV9KcXE4gbAW5baCMauVktp/tKI5dQecLUQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DS0PR12MB8573.namprd12.prod.outlook.com (2603:10b6:8:162::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.32; Mon, 6 Feb 2023 07:49:27 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::4bd4:de67:b676:67df%6]) with mapi id 15.20.6064.032; Mon, 6 Feb 2023 07:49:27 +0000 From: Alistair Popple To: linux-mm@kvack.org, cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch, "Daniel P . Berrange" , Alex Williamson , Alistair Popple , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 11/19] xdp: convert to use vm_account Date: Mon, 6 Feb 2023 18:47:48 +1100 Message-Id: X-Mailer: git-send-email 2.39.0 In-Reply-To: References: X-ClientProxiedBy: SYAPR01CA0006.ausprd01.prod.outlook.com (2603:10c6:1::18) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|DS0PR12MB8573:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e499f2e-8b0b-4fff-206e-08db0816ac13 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: U/Hb5bUvPgq/8Bo/zhdDbA4PUQM4ddPIcEFNo7CVj8XX4kbiyc/wkH3Kp1WtHKJMiUG5tR1bYnYSI76gmFHStENAOhqNWW3CXT7bQp6RTqgIEtBpq2hsclqS4KNds1lKhU9c+UnZfOlsJMqDte4FbavG2LRznoGOMIVDtpg9TUojH0btY4kRvyAKcl8enew/vHUNWuEPfUx700I1hHO6teXAPLBXayByqXmuvl1AfcSNaVhLiqbOM+vREwqQEFDAEkqQfTZ7/7U/i6CzV/zDWLKrSZOnahP/PmyinY/kap4pNBnXyi+BCkzGfDfIdDW0XlFcqhSrSubdBDMw5kwQoOhB57LDCDevnsz1+B/xtgk56PDdP4Mxs6Q/o4Zkodw6uThm6TaB6wsN06uBi68F3vUs3Drq40ujckInxTegARQVlbHQ/LegvaWh43lulz+YE7Wy9n9lT6FvIWL7OoYadIzAPPnORUJJRRo1IGAcmzWVfYOQsLyELfGfdSlToN6X7uM46Z9LFTtLL9PFLYnYg2nvm9VJ2p0M/q69vpOc0O6qq4KhQxyvT71NJIC6t9VAQi4aFIfaqIkwVxqiOLYEEhRBpVDR3TkDb7hEJLkVzruSCx69/o0TIZzAbciD9oMN39mkfXEOyrH7vdOChdS+hQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(396003)(136003)(39860400002)(366004)(346002)(451199018)(54906003)(316002)(86362001)(6666004)(6486002)(38100700002)(186003)(2616005)(6506007)(26005)(478600001)(6512007)(7416002)(5660300002)(83380400001)(66574015)(2906002)(36756003)(66946007)(8936002)(4326008)(41300700001)(66476007)(8676002)(66556008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?D7uxsrNSd4gdjxjXR39Ev+fBUv88?= =?utf-8?q?xIdG+/vw6hQF4DdiL5i0N1SWaPStyyHjrljxkxxN5PhVwiarIhwn1PYfhNsk4Es6v?= =?utf-8?q?xb9qvz08NBRAibPAgoH5oYtNLg5/U8Dpwq6Mqj/9cP3Ati6K2KwUB19x8eCfINY9u?= =?utf-8?q?DDJ5Qj79Tcq3UTdV3V0Y0yi/wGH+Q02O38d6UywtNb7qXaH0K2/iZ83su6hwOWiFa?= =?utf-8?q?Nl8FHRjAr/i6GocDXnTO1c2z3PHz4H6mcaNp73rlFjcVsbDjTqaeqI/tvHcL1EO4m?= =?utf-8?q?2/QyncpA/4m5/+tzxtEoooMVNrOSbT+V+8iUGq7P/+ssw3Z5DOPxApgyI5vuwTRnC?= =?utf-8?q?AZhOIDlJfHocP2vlKTm8EKkUtpQoTRAmcLOb2/1mZeQ4c3mW6ueeS0S0Yzjr7ER1F?= =?utf-8?q?wFD4cpQPzB+0F/23gaSjxukg2SP5tGG11rv952yHZEG88YMdMvzNPvL+HDXKUDK26?= =?utf-8?q?unpp6Bo/ZbFnmYF+GWNkhVtQRT0EOZtfC6ojqpmwDDc2fX1aQXIwPZTd1VegU0ivd?= =?utf-8?q?HydwqINnjl3aTI+vUQZefxdViuY9IGKKRAfYncHdrKXcMTTbkrFQOnOZ5b7/2G2oy?= =?utf-8?q?Herzo6E+EFFUm7S6fm0F6HNLct1ffAWbOOBWbRyPh/JtXgIk3SXRtArIzq4f9jsK5?= =?utf-8?q?GCZNhAYJ3BkB/s0XWQeWcKD25vNiyL91PuBUViJqQWtmX5kFR0yJUpUZAWNw+kaKR?= =?utf-8?q?1Aq06fp+SYF2l2H9GRzj39maOasQRKnwRODzRJln+VzIzEqTMdp/p7CwG6HSVljFu?= =?utf-8?q?1CI9SlloJg8zZmUyAbhJxskhbQb5sa4Mtar2u6G1q7Or/6TANgx9rJWix59QfsvEX?= =?utf-8?q?mCHCmlwsWaxzOfexwzJq6nl+cEEHJ1bAmVcax2FJs8HPgXg3r9sktS1NVBX3L6Mk7?= =?utf-8?q?FNEbRa73ZSHWL8eQOAEUNmMYMr7Iok4Zm1ja9RMj0UoRfkPV0P43DGRS+VUYe0ud2?= =?utf-8?q?82/tT+EO0UyIEpg8OTGXP5ak9x5DIDh5qO7l2R0z5Z1wo3TzMqPtHX7uJ6P3ke2K0?= =?utf-8?q?GU9dx5mea66YDJKb0WC2cWEvVlHD2xSB3e192N/9DX1ML8bIW4EtmC7+fb93KNynY?= =?utf-8?q?RWc2XRRJnbyt33zedFYSy4euYcYADBrNpIqreRt584gsL9fVhxAYmAk2F3If38qH8?= =?utf-8?q?36wvvWGn6XhldBAGBofFCRHJSaMLzTDhKMyaJw8g4pZhNa7xz4JAxRmysaqoRgVLl?= =?utf-8?q?JdZHH0J/Oc3hwl2Aw2oHlo8WYgqtzGJ8oPSbDsoEOTzDR99RIVrg0SbVa5R6UdWJf?= =?utf-8?q?dvSnPjaptlALjHkJach30IF504m3yWq7AzI6tPxXoi4OvoWfFWkqSTHtsno43erN/?= =?utf-8?q?/VSulbHq+vz/Sohs3EtVMoFTJFtJDYU2qnOFgp0jbpcLWfN27qRBMLWkDOnSlAPkc?= =?utf-8?q?6S9DM5NaSy60NrWixOH67guHnmAFHARfVfApqD73DCFmO7OOoaDHVbGKxdbB4e/qY?= =?utf-8?q?4z4HkhdyzNWoR/xpWkxHC7m9fpPqFFqerQnhPUeZw80g/hf6wgGI6cCAygo0RZv6i?= =?utf-8?q?cGB6AIAhNoyY?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e499f2e-8b0b-4fff-206e-08db0816ac13 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:49:27.3700 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XJm4eb5u09y5J03JJHgvgQg5AfLScdpIxX+UezKBxSPaIYMXVvTqX7hlUQm5cMjyyZJZmClUqbaQCKoadT1diw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8573 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Switch to using the new vm_account struct to charge pinned pages and enforce the rlimit. This will allow a future change to also charge a cgroup for limiting the number of pinned pages. Signed-off-by: Alistair Popple Cc: "Björn Töpel" Cc: Magnus Karlsson Cc: Maciej Fijalkowski Cc: Jonathan Lemon Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Jesper Dangaard Brouer Cc: John Fastabend Cc: netdev@vger.kernel.org Cc: bpf@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- include/net/xdp_sock.h | 3 ++- net/xdp/xdp_umem.c | 38 +++++++++++++------------------------- 2 files changed, 15 insertions(+), 26 deletions(-) diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index 3057e1a..9a21054 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -12,6 +12,7 @@ #include #include #include +#include #include struct net_device; @@ -25,7 +26,7 @@ struct xdp_umem { u32 chunk_size; u32 chunks; u32 npgs; - struct user_struct *user; + struct vm_account vm_account; refcount_t users; u8 flags; bool zc; diff --git a/net/xdp/xdp_umem.c b/net/xdp/xdp_umem.c index 4681e8e..4b5fb2f 100644 --- a/net/xdp/xdp_umem.c +++ b/net/xdp/xdp_umem.c @@ -29,12 +29,10 @@ static void xdp_umem_unpin_pages(struct xdp_umem *umem) umem->pgs = NULL; } -static void xdp_umem_unaccount_pages(struct xdp_umem *umem) +static void xdp_umem_unaccount_pages(struct xdp_umem *umem, u32 npgs) { - if (umem->user) { - atomic_long_sub(umem->npgs, &umem->user->locked_vm); - free_uid(umem->user); - } + vm_unaccount_pinned(&umem->vm_account, npgs); + vm_account_release(&umem->vm_account); } static void xdp_umem_addr_unmap(struct xdp_umem *umem) @@ -54,13 +52,15 @@ static int xdp_umem_addr_map(struct xdp_umem *umem, struct page **pages, static void xdp_umem_release(struct xdp_umem *umem) { + u32 npgs = umem->npgs; + umem->zc = false; ida_free(&umem_ida, umem->id); xdp_umem_addr_unmap(umem); xdp_umem_unpin_pages(umem); - xdp_umem_unaccount_pages(umem); + xdp_umem_unaccount_pages(umem, npgs); kfree(umem); } @@ -127,24 +127,13 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem, unsigned long address) static int xdp_umem_account_pages(struct xdp_umem *umem) { - unsigned long lock_limit, new_npgs, old_npgs; - - if (capable(CAP_IPC_LOCK)) - return 0; - - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - umem->user = get_uid(current_user()); + vm_account_init(&umem->vm_account, current, + current_user(), VM_ACCOUNT_USER); + if (vm_account_pinned(&umem->vm_account, umem->npgs)) { + vm_account_release(&umem->vm_account); + return -ENOBUFS; + } - do { - old_npgs = atomic_long_read(&umem->user->locked_vm); - new_npgs = old_npgs + umem->npgs; - if (new_npgs > lock_limit) { - free_uid(umem->user); - umem->user = NULL; - return -ENOBUFS; - } - } while (atomic_long_cmpxchg(&umem->user->locked_vm, old_npgs, - new_npgs) != old_npgs); return 0; } @@ -204,7 +193,6 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) umem->chunks = chunks; umem->npgs = (u32)npgs; umem->pgs = NULL; - umem->user = NULL; umem->flags = mr->flags; INIT_LIST_HEAD(&umem->xsk_dma_list); @@ -227,7 +215,7 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) out_unpin: xdp_umem_unpin_pages(umem); out_account: - xdp_umem_unaccount_pages(umem); + xdp_umem_unaccount_pages(umem, npgs); return err; }