From patchwork Fri Nov 11 19:31:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040730 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3E7FC43217 for ; Fri, 11 Nov 2022 19:32:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233772AbiKKTck (ORCPT ); Fri, 11 Nov 2022 14:32:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231261AbiKKTcj (ORCPT ); Fri, 11 Nov 2022 14:32:39 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A011776FB2 for ; Fri, 11 Nov 2022 11:32:38 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id y4so4984807plb.2 for ; Fri, 11 Nov 2022 11:32:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bffBMRisH/b+vAP1Qh1wMiOBQlQqteRIJ6zyoQtDJas=; b=ecYd+DOsarDN9nGYeDUQjp32U50kSYUHLt/+HgmV5f7639ZPIcP8m+eFxH/qSl5PIO /s+0dhBS+Ec49nujDK6xDIDhKQlWELzxMT1h3A/UMtPqSC7E8czjLgzoQuOIuzdmv78j /RID9Y+f+fIGnSnmKHjbq4ILxf7EkcmOHYmXDy8j0toxvqpdstYSgMGVx5jbBwPlyJqQ If4d5V+Ei1sJhEk2HCwFALldiMCY6muyyFaUHG94wlFWRDJmhJ4QoqADJfzKWy5NJAKj jXz7QC1AMq9PbVqXmzhTydHdbaAWqjYRhgCUqqzhvNmiwTH+5lLJe10R2E/bZ54tl2TK DPHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bffBMRisH/b+vAP1Qh1wMiOBQlQqteRIJ6zyoQtDJas=; b=FjbMU+5wLQZ2An3BUt9Ut2qPA8lixnOqvHYI41nWf/9pTqV1D+s0RVIBauD4T6N2V2 Or5N1LKt3S3obUln1yd3AQd29XEAvD05MHjyzbbo4RMO7GgVJ9v8ARDOvvvqC7SjbT8Y wYgRCihXhq8yjKPLCLvsgZp2rIVtrIXDAjHBjVFFaCZ/uTdRHULAod9pVPv05KDkVX5s DFSlZqnuYSDjaMy3hj9NusNryYVXWjv6Wl//efhqy0oQoRJE3mVjXnk0yQ42TtPWvpCn JYx6Fl3XtiGJJtcNhTCGs9LZQUWpmdysYQ9CoLg/ySrZpe0IRZ0idbVCAcoaBexEWpxk Ps9A== X-Gm-Message-State: ANoB5pm6uXKkdDhMYj7ghp4Yc3VKS73PRTM1CvCjU8Rka+azwVjGq0BI chIUyfsAGrIqflJotnNmyewbQtLh9PQv/A== X-Google-Smtp-Source: AA0mqf6PdF3/ZCYdb9vMhChFTAIDXYzS+tWDI8DhwiC1DQ3x0tEZEr0rZ5pYK+bmocyIXClUPTtzNg== X-Received: by 2002:a17:902:c193:b0:17f:3da:f19d with SMTP id d19-20020a170902c19300b0017f03daf19dmr3811189pld.147.1668195157938; Fri, 11 Nov 2022 11:32:37 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id c65-20020a621c44000000b005624e2e0508sm1909132pfc.207.2022.11.11.11.32.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:32:37 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 01/26] bpf: Remove local kptr references in documentation Date: Sat, 12 Nov 2022 01:01:59 +0530 Message-Id: <20221111193224.876706-2-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1743; i=memxor@gmail.com; h=from:subject; bh=TQdLtozHcFAX41FmGoXIb11rizNdf5YDECl5nuh1YTs=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqInYkpbjgYlHHXeNDaXnxMAMbY+CWQTumnO8K0c JvRxkNiJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8Ryl3OD/ 0dSZ2FJu0WMinq1u/x8gS5ZHWP1GG4JC0plreBy24v5XcPVx4W0LfoAm2g3uZLXVK1OMaik3BYtzxN dUOuIknUuOWmkyrHLfl4b6EHYE/iXlKkYm6EBRlhFVT6+u8cgEjkLaDxzjcy0MaECsYgxXiSOPUpTL p7xOEwBrFLFd8gys+OOAXhPid+kFVnrSb2paPRKjj7rpBqYxzBnokaUL6npLoLpKpteDhpNQAe5rBQ rpt0Yzvj2PooKtai2dFNQheLq8F8uhjUEb7UKSwieyfy0RnX/dj8xgQxmF0xui6GURdOSOEhSJjFE6 RGrbY2xaIgEiBUyTSuorGJ6YYj7niFlV50l9WsXv4QdUHnsQxHC+Nim1M5WQD49wjvj2gM7lo9OypQ /qaT9AzoGwPZp23Z3MRdS2DybTvgPGgLsc02GEhRw9v6ZVKIbZRIXXwV+Isrv2bqxjwLh49Bg7/ljk +qob6VAMnYfbw9XqEpyiXaVVanPv0TFXNWtxvsbilpLHBJ1KWx0IGaDIcoVDE0RKsGPylIEZTDuWh9 xniAtl7vRpxVbzV8uTTo/YrN3I89zrikPNTAhffY7lemTZoivrkZwB4iXmwBjBWSyPTF3sjvQy6VYJ peGFOVGcsl2PHz/vcAMX5K/+u9qT2/vHR5P13+3x27lLN1jw8/PAgf3Wji1A== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net We don't want to commit to a specific name for these. Simply call them allocated objects coming from bpf_obj_new, which is completely clear in itself. Signed-off-by: Kumar Kartikeya Dwivedi --- Documentation/bpf/bpf_design_QA.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/Documentation/bpf/bpf_design_QA.rst b/Documentation/bpf/bpf_design_QA.rst index 17e774d96c5e..cec2371173d7 100644 --- a/Documentation/bpf/bpf_design_QA.rst +++ b/Documentation/bpf/bpf_design_QA.rst @@ -332,13 +332,14 @@ avoid defining types with 'bpf\_' prefix to not be broken in future releases. In other words, no backwards compatibility is guaranteed if one using a type in BTF with 'bpf\_' prefix. -Q: What is the compatibility story for special BPF types in local kptrs? ------------------------------------------------------------------------- -Q: Same as above, but for local kptrs (i.e. pointers to objects allocated using -bpf_obj_new for user defined structures). Will the kernel preserve backwards +Q: What is the compatibility story for special BPF types in allocated objects? +------------------------------------------------------------------------------ +Q: Same as above, but for allocated objects (i.e. objects allocated using +bpf_obj_new for user defined types). Will the kernel preserve backwards compatibility for these features? A: NO. Unlike map value types, there are no stability guarantees for this case. The -whole local kptr API itself is unstable (since it is exposed through kfuncs). +whole API to work with allocated objects and any support for special fields +inside them is unstable (since it is exposed through kfuncs). From patchwork Fri Nov 11 19:32:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040731 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDC7EC433FE for ; Fri, 11 Nov 2022 19:33:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233690AbiKKTdI (ORCPT ); Fri, 11 Nov 2022 14:33:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231261AbiKKTdH (ORCPT ); Fri, 11 Nov 2022 14:33:07 -0500 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 283D576FB2 for ; Fri, 11 Nov 2022 11:33:07 -0800 (PST) Received: by mail-pf1-x444.google.com with SMTP id k22so5703735pfd.3 for ; Fri, 11 Nov 2022 11:33:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RnrYmyFxoGEM6OlddgT2gVaTsDC543yW1/JtZQJqlOM=; b=m2Nv0IPc6x707CE04/zSUXq9xwtM8EpbXyPz3t+7nL5+evVEraxHLJ11lRhcgRu3A3 MKv5n2AhXYVT6Agd8YIhVJ1bu3J9D+Of5mO2efKrbhK8tmLwF0wqJo1D+T1ZOjQfsSnD xerEhyyXq/04KAUjWuB237cvaYxYm9KZrm4qQ1W/6y1Z4yApWUHfJlR4T/eAEFvn4qqR 8jp7dSz+SpmXpGbM3061vNR9qL/DgIX26+uhC8Lxh51I4ajPvYgBp5ytBCcNFkw9DQeW WGNiQWDcI5J5IcijqSkmvDLFiRvyN3y6ZDLWbFCjSfWQPSVf1TGg86R3Ht+tz8H/dEAr d/KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnrYmyFxoGEM6OlddgT2gVaTsDC543yW1/JtZQJqlOM=; b=UAG1/baSxihJWVtM39le32IN132uPp2s8yd+O1niq91Z2cvVU0V5JmUIBYPoufdrlW S6nhMv4P70xUq4aFGHreKL6U62XImwf1xAEH4pmOt4z83vkz0Pk/cFmkkucxHnmeZsHA MCTKN2I80AuvzFC8NIwP8r3aZQQU3jMpZUjbDtvWhoB04TJfvg8BTb3KIatXzVmbsyEu GtHhWuNj7+uw/fOpocmf4Hj49s1684k2CF9Kv0hDcVYwRkt5wve0lVhWWJ1w0b7qlKTE 3N7zc9WX/bPVCOn9eqqL8Cvz1c2MtJSBREVnIhuooPgP09mRZVURK7uz043WC3AIYFF1 5DvA== X-Gm-Message-State: ANoB5pm93+RWqvcDCy9K2oJwR0GItLm8IQhfXR6kFzCuoZmhnOokfaOx SCmAER+8AbITM+hNPxrBNxQDqshjlxjMIg== X-Google-Smtp-Source: AA0mqf7Bg8B9uVgnQbdERbcYfnpIBVkMv4kNuoIMG2psSBaO1hEFMkgnjwNrF+ObdoA0nJ/k3lOMew== X-Received: by 2002:a63:5f15:0:b0:46f:7e1c:aa2e with SMTP id t21-20020a635f15000000b0046f7e1caa2emr2939742pgb.145.1668195186442; Fri, 11 Nov 2022 11:33:06 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id w9-20020a17090a780900b00208c58d5a0esm5213875pjk.40.2022.11.11.11.33.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:06 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 02/26] bpf: Remove BPF_MAP_OFF_ARR_MAX Date: Sat, 12 Nov 2022 01:02:00 +0530 Message-Id: <20221111193224.876706-3-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1926; i=memxor@gmail.com; h=from:subject; bh=ZTLueoxLQrjQVPDjxfufu3gtMYHhhFj4zDRgWCv34xw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqInKwPAD2afMKABKxB3GU1fQw0uIn5UXRBoG2Y8 3eCPBMaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8RymkzD/ 9pShJ6gZc8JUSqnAtlGbDErdEmvHfqDne3mrCD8k6rv2Y0imZiaAvxtade4tumCbQdvntbfz3VxLAl vHQb4qBE2W0LLUi0TN7OU+Jt1H7wHuk4+W8sRyDBNXsbfCgaMEjT20mxsRG3uEF3q3/Ia5NNVLZyCK 7jkKgU0IekrNusx9A/vg5emLxr8cIVVneWaDw+eM0HlyiAbF9cKqkWOpm97LIbnHY0x9S1PVgfGlE8 15aPF3bcWd6c0NMrem6xhaSpZ/OXKWuqB5jOGVdWEuuFPYFq0C+0GqAPAUU/bWI3E3ZlUpBXC/0h1F BwjMqrQPcRRunKmyMY6ZJSNIdwNISQqQqS2zqhx4TsVbMaOav5/2Yg2mB2JYbAoe+J596WRSPOiun0 Wu5auMr9pX6PwK2jem4BAkl3AVGQhjRr6imaBq0HwIDyYEs0U4sahziffYWU4TN0xl/sEbbrJ88WXo WzoqHcqII9jCPVUFleE2S5gYWyLI3CMX1TjZb0SiqOou/RkuTzNVDZit/0HvhRSfKWlNsRgZsuiIgx bH0uTy5CbXrmHHqshbjKi7BzYQLRnwQx1AhP+Dry0Yz9FIHaII4BZ9rrOD5/6bTjFKOYHy2CrwkeD7 8V7iXboAdqn+jLCw3Hcv3w2Ut0gFx3BBRPulqwj/DJWdPiDPSqI78DQ7HeZw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net In f71b2f64177a ("bpf: Refactor map->off_arr handling"), map->off_arr was refactored to be btf_field_offs. The number of field offsets is equal to maximum possible fields limited by BTF_FIELDS_MAX. Hence, reuse BTF_FIELDS_MAX as spin_lock and timer no longer are to be handled specially for offset sorting, fix the comment, and remove incorrect WARN_ON as its rec->cnt can never exceed this value. The reason to keep separate constant was the it was always more 2 more than total kptrs. This is no longer the case. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 9 ++++----- kernel/bpf/btf.c | 2 +- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 798aec816970..1a66a1df1af1 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -165,9 +165,8 @@ struct bpf_map_ops { }; enum { - /* Support at most 8 pointers in a BTF type */ - BTF_FIELDS_MAX = 10, - BPF_MAP_OFF_ARR_MAX = BTF_FIELDS_MAX, + /* Support at most 10 fields in a BTF type */ + BTF_FIELDS_MAX = 10, }; enum btf_field_type { @@ -203,8 +202,8 @@ struct btf_record { struct btf_field_offs { u32 cnt; - u32 field_off[BPF_MAP_OFF_ARR_MAX]; - u8 field_sz[BPF_MAP_OFF_ARR_MAX]; + u32 field_off[BTF_FIELDS_MAX]; + u8 field_sz[BTF_FIELDS_MAX]; }; struct bpf_map { diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 5579ff3a5b54..12361d7b2498 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3584,7 +3584,7 @@ struct btf_field_offs *btf_parse_field_offs(struct btf_record *rec) u8 *sz; BUILD_BUG_ON(ARRAY_SIZE(foffs->field_off) != ARRAY_SIZE(foffs->field_sz)); - if (IS_ERR_OR_NULL(rec) || WARN_ON_ONCE(rec->cnt > sizeof(foffs->field_off))) + if (IS_ERR_OR_NULL(rec)) return NULL; foffs = kzalloc(sizeof(*foffs), GFP_KERNEL | __GFP_NOWARN); From patchwork Fri Nov 11 19:32:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040732 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FDEDC433FE for ; Fri, 11 Nov 2022 19:33:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231261AbiKKTdO (ORCPT ); Fri, 11 Nov 2022 14:33:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233727AbiKKTdN (ORCPT ); Fri, 11 Nov 2022 14:33:13 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8C8E78327 for ; Fri, 11 Nov 2022 11:33:11 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id m14-20020a17090a3f8e00b00212dab39bcdso8612904pjc.0 for ; Fri, 11 Nov 2022 11:33:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TNUXGrsIBZA47dfMoUXlsA9DJrJBOE3oKFjyxYnaenU=; b=U0j2FX9rWCKh9lkUjYJPY+TcvQMIGMnMViC03lSHTmf0fhsFq52xJVrZQJ5lcWqxE9 eUBIkNok8EZ/8breqQyzfx961AsbuE+gDAeUTVRTmNieNLISMZQvTClKoe++zIGBAxMM XthsueDL1LZPkn5bb3nuOA+MnqcKSGudAZVEcq3vutRsHvECXWMrV9vm5CUzJgiLXq9W RL4OAn4uFUnuu7I4DMCJfal9htrdGTRHtxO78N58CmqQ/ZNwqxqpcvhGaw4bDIFHK3my JB16Dfd5lUvwzSGBaRCMsb8LoeXMJYSeAAY+Z00qm+5UTN8w//5sDRcyWsjFUOhbd/cj JNag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TNUXGrsIBZA47dfMoUXlsA9DJrJBOE3oKFjyxYnaenU=; b=3J+XVNymWHJ8EvKgDJM5BtGz1Zbgd16uxvQzoCtD5RgaZo+heVSNxy+/RX6gbv8Fx9 UBNSyIUM3OxaNIqdxrE1P2uEXwu2hTFpioQXc3LLHNPwCdgTkqZbXcnelP8CFOKVkAn+ /kavpRMkeJVMElzHvUxQbU5G1wOpuFgvAm1eZhv0iIYEeL0nZxVuz3a3g/GnzhebZ49J tnEuxvJxvMFERrKdmPiFg1/pu3K1TepBRwuuielgW1FuTzZJ1Wi42MgILJynNX03zfA5 HPFy7qSWJvntW5OjK6zF3jaCEXFysV7ekkeWdDez2bWgu10JD7MABf+nfntOGDBPCCgt uCjg== X-Gm-Message-State: ANoB5pnSdybeD2d+15tjeS24ezzTPyniG3ZySy90LzAbkuYufsy5KULN VQa2tM7gEU6eSbKsm54Oj5PxBCYBwhPusg== X-Google-Smtp-Source: AA0mqf6uRnzLyo9dICzT9GeeX+dMt5EDjesaWlPbClxP5VLoGdk1bLzL4z+dGg8HSw/qkX9amGKvIQ== X-Received: by 2002:a17:902:a413:b0:186:8c19:d472 with SMTP id p19-20020a170902a41300b001868c19d472mr4073508plq.12.1668195190864; Fri, 11 Nov 2022 11:33:10 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id 77-20020a621750000000b0056baebf23e7sm1997392pfx.141.2022.11.11.11.33.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:10 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 03/26] bpf: Fix copy_map_value, zero_map_value Date: Sat, 12 Nov 2022 01:02:01 +0530 Message-Id: <20221111193224.876706-4-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1622; i=memxor@gmail.com; h=from:subject; bh=6RvsvjttmdEULzDCNignBfVVDaAhe2Uld3KegARQNKU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqInNLiBPyhOAGswnMZ++VCv2z7GCXT00P33PG8B xurXpGSJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8Rys87D/ 9GctXmhqpuNtgRtUlWBLwU9GtcQqF+WdfNVTR7j+NmXVsRw+J/yVJfMSWbpfTZdcKGRiKYd3oJ+jEM Cv9SM5N6NRfVS4AsiO6vL46jOtRchtQWP2XXAxD/LYSMdu+BsBXK8rLcFhyJoeSU2b97rCkHLlIE9E kNK/N/C1XdfN4H+V28w60Mbq1H/0uAF+hgXM4n+tTCu//iDwWz4a88P3TRBryAhy4PAznAh39JHk+D pGqmHlFPCBD31eWkQ2eqBpvZCLEJIvu8vbJ04+ElNYfOwSxhnUQY3XYE9W5WN5qQHxQD5VI7gSbfl2 cR0am3l1gAQlgGF1Cp8Sx+saH0UXw6lQ7d5GBVMXAUvFiusZZAeZeNn0/BXaduAcjyr9ogkmyJJIM9 ZtEbM2fM/z07HEbNFrPiwZYD36UuekwmINHQlIXCl7wnPie1Tfbkc5CFXhvOjdb0X0Vr7A5pIXo7kM STL9+d/cIn0gESpP6j6LVE19XROUFk8kDmKWjfczTMk9yq68TFySyHgDKPyaly+evseE1w+nv2iRIP I817lALLYDwSaK48jVq7JRhniYBJ2QWDhKamPAJk56YFgm+imvuZFCBJ2nsziTSL1YANfjoy7X0nuu RxoQ4t6Mlp3Ct6ZrV6S62YGyg4+9RbuzDuEzfk8jMOAQD5Wwn2iwLVCkXnDg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The current offset needs to also skip over the already copied region in addition to the size of the next field. This case manifests where there are gaps between adjacent special fields. It was observed that for a map value with size 48, having fields at: off: 0, 16, 32 size: 4, 16, 16 The current code does: memcpy(dst + 0, src + 0, 0) memcpy(dst + 4, src + 4, 12) memcpy(dst + 20, src + 20, 12) memcpy(dst + 36, src + 36, 12) With the fix, it is done correctly as: memcpy(dst + 0, src + 0, 0) memcpy(dst + 4, src + 4, 12) memcpy(dst + 32, src + 32, 0) memcpy(dst + 48, src + 48, 0) Fixes: 4d7d7f69f4b1 ("bpf: Adapt copy_map_value for multiple offset case") Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 1a66a1df1af1..f08eb2d27de0 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -360,7 +360,7 @@ static inline void bpf_obj_memcpy(struct btf_field_offs *foffs, u32 sz = next_off - curr_off; memcpy(dst + curr_off, src + curr_off, sz); - curr_off += foffs->field_sz[i]; + curr_off += foffs->field_sz[i] + sz; } memcpy(dst + curr_off, src + curr_off, size - curr_off); } @@ -390,7 +390,7 @@ static inline void bpf_obj_memzero(struct btf_field_offs *foffs, void *dst, u32 u32 sz = next_off - curr_off; memset(dst + curr_off, 0, sz); - curr_off += foffs->field_sz[i]; + curr_off += foffs->field_sz[i] + sz; } memset(dst + curr_off, 0, size - curr_off); } From patchwork Fri Nov 11 19:32:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040733 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8716C433FE for ; Fri, 11 Nov 2022 19:33:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233758AbiKKTdU (ORCPT ); Fri, 11 Nov 2022 14:33:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233727AbiKKTdT (ORCPT ); Fri, 11 Nov 2022 14:33:19 -0500 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D3A779D09 for ; Fri, 11 Nov 2022 11:33:17 -0800 (PST) Received: by mail-pg1-x544.google.com with SMTP id o13so5118605pgu.7 for ; Fri, 11 Nov 2022 11:33:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UFePjJAHPjAYwZJgMSRWyERIm/Ls7dFNVc7DC7N9KCo=; b=XHwjYOLUIsHB4uO2afJmShl+3dZt3D743dzOi3hIcTC8SHtbtOIErXpdI2V50x/7fi 3PYBSre6kenySSks7SGTiyQJe0botJdV1U7138nyHzmchD0vemKVZP12DkXCmQipfsGm M7Xg77MEHCn/ZSeIIfkSmSq4EviZ8DH6zOXiK95kJ9cp84rFBs5XBcs/edhLRLHrpy1l DZRnP0RQf7+Ij0SWN3DA0HwbFDJ3ssa1qNvfRazXi8IWIWHssGK9DABb0c3ktmIM8J9x ZBAPZQ3l52SoqViHGVplZWXnWcSnT4WR9P7e/eOy3uz9GrSdzUomK3bDopqXcZ6bmD9r YhqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UFePjJAHPjAYwZJgMSRWyERIm/Ls7dFNVc7DC7N9KCo=; b=BXHKeHMMX1JR/5qfDk2GUlHr6PJtEUCrP/VxgXgkUZ29RFLjNxFfu9R7RtLUuQ/vab SdJJv2wKpx7Jh0QtzLvAvHyayJYDEOZoPJab6CleHl8IV2unhV420fLh7bCz6RTHb5p+ PhzbZ6Q9kDMVTzs2+aaygI6o4048CtfcKbZdXkoCs0NDjLDOAsug2zi3QdFLRejsLGSm 8yXVHZ7BTG3avlZ8BJERbEjosUcxP6z/4gocxxG+wO4Ik0jGZp7BwbquX6I6mrKHmXMr EYGiBsah5/kDVutznJ64ezsPZEFcfXB1iuFuthg0fmynY5MX2dD/PB1GNmOkxADLCct9 cjhw== X-Gm-Message-State: ANoB5pma93GeE85Wp/QaHVNv9aL//rTzZ49eI7nixq49Ap03cPW4PNwj pR8egZgF3IGMUYzb7599dqeGB1eIFWu8RA== X-Google-Smtp-Source: AA0mqf4A7RWq0CMmSWvrbIhIMT7+s9HJYOqGEzlX6EPD9JR8kMy4IukBVuMmhTvfm9KYqoYQqp4lsg== X-Received: by 2002:a63:5b5b:0:b0:46e:dc0a:ba96 with SMTP id l27-20020a635b5b000000b0046edc0aba96mr2995275pgm.419.1668195196738; Fri, 11 Nov 2022 11:33:16 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id i13-20020a056a00004d00b0056286c552ecsm1932333pfk.184.2022.11.11.11.33.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:16 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 04/26] bpf: Support bpf_list_head in map values Date: Sat, 12 Nov 2022 01:02:02 +0530 Message-Id: <20221111193224.876706-5-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=16631; i=memxor@gmail.com; h=from:subject; bh=hjbmIBlRmEo7a2ts8DdFZUrI4CFe+Q1/4QATKUQTqZQ=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqInXouMquh+40o9Xor5anaRd5Wm+cIeVRsbjfhY I36ntuaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8Ryo2oD/ 0XvyAA5LytcBuyrD/RRxlynKE22onV1CgieRqUuuTRzR2FYIDPU1U8j8dcLBVNX/bipin3UVCgHHS6 e4bcVL78OS9sQawUh79hWpvrFRm2tCt1CqH+clSZ7jkfbDpz6ccB7L388g3idgz/0R4IVKxSahclRJ nuWfM4UkHct2ySqHN+iN+tK6Rql+HxaFx9sxuUfouNvOA0abhDq8o/DyjnacoesSfjrw9AA8DiiK0x WFPaY8+quIWWIUkdxsw7CIKB/9a1JTKwK+lArOan2TUu4BSnaSnLc2EI023qNE+XKUcwlGnDXBSaEK f1CJTJBX4jCijPOQI7YJx/q9oeFAkyDZlww2cCmp5pvGMTqzQoEuyeU3pyi2ZXs1yKCSF60UCEQ8Lv Bjk7HcKpb8NGeqizdXka0Q1TiyTAhxmq+rFCvXpLcHadG2TjCac1d9rJYspw0OTbVA/cgb9o1Y9Yg1 6HstT7cK9fQIGTNQh0qlbNnAEXLEJpuK4QI1UscsiVM5F/W2NVuuN0nGOQ0qbM7B6U2EOk2yBsRuWe Z0mFFdmK19wm1lfy2h0XvmlrwxUkOjPAeSGcQeq5ZlwOmzUA3qxPXAlb+JEiw/KiK4V7W/Uo8C3N6T yqKd/jDbUxy6qRUJ/dXJUWcbL6GQtbHAAmIrE6qWcpxBYZwdIoFj2PVl0JyA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add the support on the map side to parse, recognize, verify, and build metadata table for a new special field of the type struct bpf_list_head. To parameterize the bpf_list_head for a certain value type and the list_node member it will accept in that value type, we use BTF declaration tags. The definition of bpf_list_head in a map value will be done as follows: struct foo { struct bpf_list_node node; int data; }; struct map_value { struct bpf_list_head head __contains(foo, node); }; Then, the bpf_list_head only allows adding to the list 'head' using the bpf_list_node 'node' for the type struct foo. The 'contains' annotation is a BTF declaration tag composed of four parts, "contains:name:node" where the name is then used to look up the type in the map BTF, with its kind hardcoded to BTF_KIND_STRUCT during the lookup. The node defines name of the member in this type that has the type struct bpf_list_node, which is actually used for linking into the linked list. For now, 'kind' part is hardcoded as struct. This allows building intrusive linked lists in BPF, using container_of to obtain pointer to entry, while being completely type safe from the perspective of the verifier. The verifier knows exactly the type of the nodes, and knows that list helpers return that type at some fixed offset where the bpf_list_node member used for this list exists. The verifier also uses this information to disallow adding types that are not accepted by a certain list. For now, no elements can be added to such lists. Support for that is coming in future patches, hence draining and freeing items is done with a TODO that will be resolved in a future patch. Note that the bpf_list_head_free function moves the list out to a local variable under the lock and releases it, doing the actual draining of the list items outside the lock. While this helps with not holding the lock for too long pessimizing other concurrent list operations, it is also necessary for deadlock prevention: unless every function called in the critical section would be notrace, a fentry/fexit program could attach and call bpf_map_update_elem again on the map, leading to the same lock being acquired if the key matches and lead to a deadlock. While this requires some special effort on part of the BPF programmer to trigger and is highly unlikely to occur in practice, it is always better if we can avoid such a condition. While notrace would prevent this, doing the draining outside the lock has advantages of its own, hence it is used to also fix the deadlock related problem. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 17 ++++ include/uapi/linux/bpf.h | 10 +++ kernel/bpf/btf.c | 145 ++++++++++++++++++++++++++++++++- kernel/bpf/helpers.c | 32 ++++++++ kernel/bpf/syscall.c | 22 ++++- kernel/bpf/verifier.c | 7 ++ tools/include/uapi/linux/bpf.h | 10 +++ 7 files changed, 239 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f08eb2d27de0..05f98e9e5c48 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -175,6 +175,7 @@ enum btf_field_type { BPF_KPTR_UNREF = (1 << 2), BPF_KPTR_REF = (1 << 3), BPF_KPTR = BPF_KPTR_UNREF | BPF_KPTR_REF, + BPF_LIST_HEAD = (1 << 4), }; struct btf_field_kptr { @@ -184,11 +185,18 @@ struct btf_field_kptr { u32 btf_id; }; +struct btf_field_list_head { + struct btf *btf; + u32 value_btf_id; + u32 node_offset; +}; + struct btf_field { u32 offset; enum btf_field_type type; union { struct btf_field_kptr kptr; + struct btf_field_list_head list_head; }; }; @@ -266,6 +274,8 @@ static inline const char *btf_field_type_name(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: return "kptr"; + case BPF_LIST_HEAD: + return "bpf_list_head"; default: WARN_ON_ONCE(1); return "unknown"; @@ -282,6 +292,8 @@ static inline u32 btf_field_type_size(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: return sizeof(u64); + case BPF_LIST_HEAD: + return sizeof(struct bpf_list_head); default: WARN_ON_ONCE(1); return 0; @@ -298,6 +310,8 @@ static inline u32 btf_field_type_align(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: return __alignof__(u64); + case BPF_LIST_HEAD: + return __alignof__(struct bpf_list_head); default: WARN_ON_ONCE(1); return 0; @@ -403,6 +417,9 @@ static inline void zero_map_value(struct bpf_map *map, void *dst) void copy_map_value_locked(struct bpf_map *map, void *dst, void *src, bool lock_src); void bpf_timer_cancel_and_free(void *timer); +void bpf_list_head_free(const struct btf_field *field, void *list_head, + struct bpf_spin_lock *spin_lock); + int bpf_obj_name_cpy(char *dst, const char *src, unsigned int size); struct bpf_offload_dev; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 94659f6b3395..dd381086bad9 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -6887,6 +6887,16 @@ struct bpf_dynptr { __u64 :64; } __attribute__((aligned(8))); +struct bpf_list_head { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + +struct bpf_list_node { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + struct bpf_sysctl { __u32 write; /* Sysctl is being read (= 0) or written (= 1). * Allows 1,2,4-byte read, but no write. diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 12361d7b2498..c0d73d71c539 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3205,9 +3205,15 @@ enum { struct btf_field_info { enum btf_field_type type; u32 off; - struct { - u32 type_id; - } kptr; + union { + struct { + u32 type_id; + } kptr; + struct { + const char *node_name; + u32 value_btf_id; + } list_head; + }; }; static int btf_find_struct(const struct btf *btf, const struct btf_type *t, @@ -3261,6 +3267,63 @@ static int btf_find_kptr(const struct btf *btf, const struct btf_type *t, return BTF_FIELD_FOUND; } +static const char *btf_find_decl_tag_value(const struct btf *btf, + const struct btf_type *pt, + int comp_idx, const char *tag_key) +{ + int i; + + for (i = 1; i < btf_nr_types(btf); i++) { + const struct btf_type *t = btf_type_by_id(btf, i); + int len = strlen(tag_key); + + if (!btf_type_is_decl_tag(t)) + continue; + if (pt != btf_type_by_id(btf, t->type) || + btf_type_decl_tag(t)->component_idx != comp_idx) + continue; + if (strncmp(__btf_name_by_offset(btf, t->name_off), tag_key, len)) + continue; + return __btf_name_by_offset(btf, t->name_off) + len; + } + return NULL; +} + +static int btf_find_list_head(const struct btf *btf, const struct btf_type *pt, + const struct btf_type *t, int comp_idx, + u32 off, int sz, struct btf_field_info *info) +{ + const char *value_type; + const char *list_node; + s32 id; + + if (!__btf_type_is_struct(t)) + return BTF_FIELD_IGNORE; + if (t->size != sz) + return BTF_FIELD_IGNORE; + value_type = btf_find_decl_tag_value(btf, pt, comp_idx, "contains:"); + if (!value_type) + return -EINVAL; + list_node = strstr(value_type, ":"); + if (!list_node) + return -EINVAL; + value_type = kstrndup(value_type, list_node - value_type, GFP_KERNEL | __GFP_NOWARN); + if (!value_type) + return -ENOMEM; + id = btf_find_by_name_kind(btf, value_type, BTF_KIND_STRUCT); + kfree(value_type); + if (id < 0) + return id; + list_node++; + if (str_is_empty(list_node)) + return -EINVAL; + info->type = BPF_LIST_HEAD; + info->off = off; + info->list_head.value_btf_id = id; + info->list_head.node_name = list_node; + return BTF_FIELD_FOUND; +} + static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, int *align, int *sz) { @@ -3284,6 +3347,12 @@ static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, goto end; } } + if (field_mask & BPF_LIST_HEAD) { + if (!strcmp(name, "bpf_list_head")) { + type = BPF_LIST_HEAD; + goto end; + } + } /* Only return BPF_KPTR when all other types with matchable names fail */ if (field_mask & BPF_KPTR) { type = BPF_KPTR_REF; @@ -3339,6 +3408,12 @@ static int btf_find_struct_field(const struct btf *btf, if (ret < 0) return ret; break; + case BPF_LIST_HEAD: + ret = btf_find_list_head(btf, t, member_type, i, off, sz, + idx < info_cnt ? &info[idx] : &tmp); + if (ret < 0) + return ret; + break; default: return -EFAULT; } @@ -3393,6 +3468,12 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, if (ret < 0) return ret; break; + case BPF_LIST_HEAD: + ret = btf_find_list_head(btf, var, var_type, -1, off, sz, + idx < info_cnt ? &info[idx] : &tmp); + if (ret < 0) + return ret; + break; default: return -EFAULT; } @@ -3491,11 +3572,52 @@ static int btf_parse_kptr(const struct btf *btf, struct btf_field *field, return ret; } +static int btf_parse_list_head(const struct btf *btf, struct btf_field *field, + struct btf_field_info *info) +{ + const struct btf_type *t, *n = NULL; + const struct btf_member *member; + u32 offset; + int i; + + t = btf_type_by_id(btf, info->list_head.value_btf_id); + /* We've already checked that value_btf_id is a struct type. We + * just need to figure out the offset of the list_node, and + * verify its type. + */ + for_each_member(i, t, member) { + if (strcmp(info->list_head.node_name, __btf_name_by_offset(btf, member->name_off))) + continue; + /* Invalid BTF, two members with same name */ + if (n) + return -EINVAL; + n = btf_type_by_id(btf, member->type); + if (!__btf_type_is_struct(n)) + return -EINVAL; + if (strcmp("bpf_list_node", __btf_name_by_offset(btf, n->name_off))) + return -EINVAL; + offset = __btf_member_bit_offset(n, member); + if (offset % 8) + return -EINVAL; + offset /= 8; + if (offset % __alignof__(struct bpf_list_node)) + return -EINVAL; + + field->list_head.btf = (struct btf *)btf; + field->list_head.value_btf_id = info->list_head.value_btf_id; + field->list_head.node_offset = offset; + } + if (!n) + return -ENOENT; + return 0; +} + struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t, u32 field_mask, u32 value_size) { struct btf_field_info info_arr[BTF_FIELDS_MAX]; struct btf_record *rec; + u32 next_off = 0; int ret, i, cnt; ret = btf_find_field(btf, t, field_mask, info_arr, ARRAY_SIZE(info_arr)); @@ -3517,6 +3639,11 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type ret = -EFAULT; goto end; } + if (info_arr[i].off < next_off) { + ret = -EEXIST; + goto end; + } + next_off = info_arr[i].off + btf_field_type_size(info_arr[i].type); rec->field_mask |= info_arr[i].type; rec->fields[i].offset = info_arr[i].off; @@ -3539,12 +3666,24 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type if (ret < 0) goto end; break; + case BPF_LIST_HEAD: + ret = btf_parse_list_head(btf, &rec->fields[i], &info_arr[i]); + if (ret < 0) + goto end; + break; default: ret = -EFAULT; goto end; } rec->cnt++; } + + /* bpf_list_head requires bpf_spin_lock */ + if (btf_record_has_field(rec, BPF_LIST_HEAD) && rec->spin_lock_off < 0) { + ret = -EINVAL; + goto end; + } + return rec; end: btf_record_free(rec); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 283f55bbeb70..7bc71995f17c 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1706,6 +1706,38 @@ bpf_base_func_proto(enum bpf_func_id func_id) } } +void bpf_list_head_free(const struct btf_field *field, void *list_head, + struct bpf_spin_lock *spin_lock) +{ + struct list_head *head = list_head, *orig_head = list_head; + + BUILD_BUG_ON(sizeof(struct list_head) > sizeof(struct bpf_list_head)); + BUILD_BUG_ON(__alignof__(struct list_head) > __alignof__(struct bpf_list_head)); + + /* Do the actual list draining outside the lock to not hold the lock for + * too long, and also prevent deadlocks if tracing programs end up + * executing on entry/exit of functions called inside the critical + * section, and end up doing map ops that call bpf_list_head_free for + * the same map value again. + */ + __bpf_spin_lock_irqsave(spin_lock); + if (!head->next || list_empty(head)) + goto unlock; + head = head->next; +unlock: + INIT_LIST_HEAD(orig_head); + __bpf_spin_unlock_irqrestore(spin_lock); + + while (head != orig_head) { + void *obj = head; + + obj -= field->list_head.node_offset; + head = head->next; + /* TODO: Rework later */ + kfree(obj); + } +} + BTF_SET8_START(tracing_btf_ids) #ifdef CONFIG_KEXEC_CORE BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 85532d301124..fdbae52f463f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -536,6 +536,9 @@ void btf_record_free(struct btf_record *rec) module_put(rec->fields[i].kptr.module); btf_put(rec->fields[i].kptr.btf); break; + case BPF_LIST_HEAD: + /* Nothing to release for bpf_list_head */ + break; default: WARN_ON_ONCE(1); continue; @@ -578,6 +581,9 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) goto free; } break; + case BPF_LIST_HEAD: + /* Nothing to acquire for bpf_list_head */ + break; default: ret = -EFAULT; WARN_ON_ONCE(1); @@ -637,6 +643,11 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj) case BPF_KPTR_REF: field->kptr.dtor((void *)xchg((unsigned long *)field_ptr, 0)); break; + case BPF_LIST_HEAD: + if (WARN_ON_ONCE(rec->spin_lock_off < 0)) + continue; + bpf_list_head_free(field, field_ptr, obj + rec->spin_lock_off); + break; default: WARN_ON_ONCE(1); continue; @@ -965,7 +976,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, if (!value_type || value_size != map->value_size) return -EINVAL; - map->record = btf_parse_fields(btf, value_type, BPF_SPIN_LOCK | BPF_TIMER | BPF_KPTR, + map->record = btf_parse_fields(btf, value_type, + BPF_SPIN_LOCK | BPF_TIMER | BPF_KPTR | BPF_LIST_HEAD, map->value_size); if (!IS_ERR_OR_NULL(map->record)) { int i; @@ -1012,6 +1024,14 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, goto free_map_tab; } break; + case BPF_LIST_HEAD: + if (map->map_type != BPF_MAP_TYPE_HASH && + map->map_type != BPF_MAP_TYPE_LRU_HASH && + map->map_type != BPF_MAP_TYPE_ARRAY) { + ret = -EOPNOTSUPP; + goto free_map_tab; + } + break; default: /* Fail if map_type checks are missing for a field type */ ret = -EOPNOTSUPP; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d3b75aa0c54d..0374f03d1f56 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12805,6 +12805,13 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env, { enum bpf_prog_type prog_type = resolve_prog_type(prog); + if (btf_record_has_field(map->record, BPF_LIST_HEAD)) { + if (is_tracing_prog_type(prog_type)) { + verbose(env, "tracing progs cannot use bpf_list_head yet\n"); + return -EINVAL; + } + } + if (btf_record_has_field(map->record, BPF_SPIN_LOCK)) { if (prog_type == BPF_PROG_TYPE_SOCKET_FILTER) { verbose(env, "socket filter progs cannot use bpf_spin_lock yet\n"); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 94659f6b3395..dd381086bad9 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -6887,6 +6887,16 @@ struct bpf_dynptr { __u64 :64; } __attribute__((aligned(8))); +struct bpf_list_head { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + +struct bpf_list_node { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + struct bpf_sysctl { __u32 write; /* Sysctl is being read (= 0) or written (= 1). * Allows 1,2,4-byte read, but no write. From patchwork Fri Nov 11 19:32:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040734 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCEFDC4332F for ; Fri, 11 Nov 2022 19:33:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233727AbiKKTdX (ORCPT ); Fri, 11 Nov 2022 14:33:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233805AbiKKTdX (ORCPT ); Fri, 11 Nov 2022 14:33:23 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2890A78327 for ; Fri, 11 Nov 2022 11:33:22 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id b62so5160468pgc.0 for ; Fri, 11 Nov 2022 11:33:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X/wrTWjeoGJ5PcE2HVDewWAhvJm6XCc8VxMDXoI4aI8=; b=M3WyBvG5ePRNp2a+IfRSUn75h/jCLJMWEkqmDrYWf4YC1FKOKFHIqtj9RjUxC2plwO 47jzHhoTp55kBi1k495w00idySrv6UOMdYnaQcJm7UaQ/g++5VbRUHd0MANMVYo/kmLj PMvgzjQVIU9Q1aiGGb1TIxNm52Qkh9bDhYbHSKc3GcZgT2SyAay/PzDl+oZ7s8M1zSJX 06C9Nhi7joKGneWu28Jhh+41qOR8/kMuDprqzXc3XLe8sLzVfhOiybAeJh4zhZoywUtS WAhMP9S+TWqYcPRZmv4yi9LQbipHGNPGrqC++qlZWiQYcrnzNAoqTjQFQaJoWlFLRuRa mPnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X/wrTWjeoGJ5PcE2HVDewWAhvJm6XCc8VxMDXoI4aI8=; b=xj4alFVQd39b7VvT3JMv76vQ0i9IF0ynsrPeFj7OfT2MJqB8S2I3q2jxFReocQJlpq C3fBAAbKgT4scOAeW+vtMDUkr5Lcu1LdgrxJX89hqMsb+3eGQmyiOZN+08Kk+2XF0Lci 9udXTXYsalZknjN8EcAKaljkx6g0SZlZ5vBhmFLUc89/BKRtaQ38btTEvTIFRTgXshhy S7qF83m8WctVjPMPfomY+zC/N1VkBp2WIoDkrLBQgdzp6CEp6kqLQvyhQNbNHlJXumE7 A8HXuT43k5vaBciS75H2HzUUkV9COTxErUwejsi6RXwSgQ+fxvnTPzWNccazVdiFvW9s Mjjg== X-Gm-Message-State: ANoB5pm3d/4fyYwjLh7JmTeKA1btxOCXGN2hZZL1HuOmArcFzn8sUS5P MGlIpBC87nBzkgtASiRqVXaCnR/QLjkC5g== X-Google-Smtp-Source: AA0mqf7gEe2u9fvn5jnQSAJ8xOBSv9ycATrtV6ZoZejrUfqv8AYdlNijOb2gRcYBiq101+f0JBmTCA== X-Received: by 2002:a63:6482:0:b0:45f:88b2:1763 with SMTP id y124-20020a636482000000b0045f88b21763mr2817586pgb.469.1668195201470; Fri, 11 Nov 2022 11:33:21 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id ij15-20020a170902ab4f00b00176b3d7db49sm2126499plb.0.2022.11.11.11.33.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:21 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 05/26] bpf: Rename RET_PTR_TO_ALLOC_MEM Date: Sat, 12 Nov 2022 01:02:03 +0530 Message-Id: <20221111193224.876706-6-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2687; i=memxor@gmail.com; h=from:subject; bh=0z92ryWA6XsveKbJHr6S6jTfA+C5JIJe/ym7BPLG/iY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqInHRkIjZt8mnhFlc+mlBlx/wckOGz9VYukDvsY WaIsVzOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8RygLYD/ 907h9MF48dyj5DKVzSWd1lQcNQQCC5+g3Wx1+RyOP72ZDgEEjsRrwftNWxTMqa0zgPIDR1YRoNOR9y KS3+lCWVWLz1TMbUL3EqxT/A3hfUgh3ievkK7UaEmpi2pk6AnImsxsWuWR9+iPyd/boDSxqsjC5dnp Baik/5gOqmab7OVXndov6rpv8RqLD1mKesy1wkhAswUX7w29CcRAjVNdZR3juVPtAAqyblep1Gug5R Xc8bwZzpODAb0v4hB7I04y68nYp7GjRv6joxXHThpKvdWLVJyH/lfkwghBZ2xrBelZd0FVrNkIxPiD Vc9TZyK9HyggzcLj4UHeNEUxY582teCLdoQxQ+KY976brQnAkRkLQq0qk+8S3mPPT/nopzjyK/dyWF JXkWus/pZ8H+piw1eNE8v66ERalSsTGewrx04S4mfIXT6B+KEQVdroRXCUvvt+c9b8XCvqwf4lKxMc rsH+7jK/Fd7l3TGHEf+HY9iBW0+u+fJ250u52Gt7F5MaBpUtYv8ltpFWkFIBvGgPgw1kPNgNdkb6Eo QOJF83OMhHrgsM0/ujYWuWHi/OQpYNOX2AMNCpUryuKFfoicei16MaVeuGlt4B3Pn4mMbte4kqj9Uy 4wyVlOBBIlm4NzQt9U0PghSzbPbe+x7F7F02O91wDoVEB2KUK/1hfnUktSjA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, the verifier has two return types, RET_PTR_TO_ALLOC_MEM, and RET_PTR_TO_ALLOC_MEM_OR_NULL, however the former is confusingly named to imply that it carries MEM_ALLOC, while only the latter does. This causes confusion during code review leading to conclusions like that the return value of RET_PTR_TO_DYNPTR_MEM_OR_NULL (which is RET_PTR_TO_ALLOC_MEM | PTR_MAYBE_NULL) may be consumable by bpf_ringbuf_{submit,commit}. Rename it to make it clear MEM_ALLOC needs to be tacked on top of RET_PTR_TO_MEM. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 6 +++--- kernel/bpf/verifier.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 05f98e9e5c48..2fe3ec620d54 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -607,7 +607,7 @@ enum bpf_return_type { RET_PTR_TO_SOCKET, /* returns a pointer to a socket */ RET_PTR_TO_TCP_SOCK, /* returns a pointer to a tcp_sock */ RET_PTR_TO_SOCK_COMMON, /* returns a pointer to a sock_common */ - RET_PTR_TO_ALLOC_MEM, /* returns a pointer to dynamically allocated memory */ + RET_PTR_TO_MEM, /* returns a pointer to memory */ RET_PTR_TO_MEM_OR_BTF_ID, /* returns a pointer to a valid memory or a btf_id */ RET_PTR_TO_BTF_ID, /* returns a pointer to a btf_id */ __BPF_RET_TYPE_MAX, @@ -617,8 +617,8 @@ enum bpf_return_type { RET_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCKET, RET_PTR_TO_TCP_SOCK_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_TCP_SOCK, RET_PTR_TO_SOCK_COMMON_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCK_COMMON, - RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_ALLOC | RET_PTR_TO_ALLOC_MEM, - RET_PTR_TO_DYNPTR_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_ALLOC_MEM, + RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_ALLOC | RET_PTR_TO_MEM, + RET_PTR_TO_DYNPTR_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_MEM, RET_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_BTF_ID, /* This must be the last entry. Its purpose is to ensure the enum is diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 0374f03d1f56..2407e3b179ec 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7621,7 +7621,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn mark_reg_known_zero(env, regs, BPF_REG_0); regs[BPF_REG_0].type = PTR_TO_TCP_SOCK | ret_flag; break; - case RET_PTR_TO_ALLOC_MEM: + case RET_PTR_TO_MEM: mark_reg_known_zero(env, regs, BPF_REG_0); regs[BPF_REG_0].type = PTR_TO_MEM | ret_flag; regs[BPF_REG_0].mem_size = meta.mem_size; From patchwork Fri Nov 11 19:32:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040735 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7D71C4332F for ; Fri, 11 Nov 2022 19:33:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233802AbiKKTd3 (ORCPT ); Fri, 11 Nov 2022 14:33:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233828AbiKKTd1 (ORCPT ); Fri, 11 Nov 2022 14:33:27 -0500 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F17DA62EB for ; Fri, 11 Nov 2022 11:33:25 -0800 (PST) Received: by mail-pg1-x544.google.com with SMTP id s196so5141556pgs.3 for ; Fri, 11 Nov 2022 11:33:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Qn946/Dfi7ihlJL944GILOqS6Benn+3+4NY3sXoxuY=; b=Bd3aKnKOT/9np74KxEcC0tQPgcjuJ1bROkpwOxpKTtx+wNA2vQXMiC6Gnm/24ovXyU fGCbWG/gUS5DdoIYb8yRerwXf9eoq26hi5rhY895BoMHDmF5PscKIf3MUGIbI1TCM/I8 wB3fVfZZC7GSO8IVL7AZeyA4MVerN2TgJi+/eOKhd4ffgWb7cIl6ouSPd/3ITQ0JN/Af Yd+ysuZPnGFkCzJvMI7uGKnyxzsIk0gVLv47Z+213aJ4sr/u6lmwHDS1fHLk7nULhd0Y tMknmJAMrCcTxD/btsEzw7wpVFwbFBMD78ayBPL1n8q1Y/vP1F7SjnqCYEAMEgUqhu7U 0dhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Qn946/Dfi7ihlJL944GILOqS6Benn+3+4NY3sXoxuY=; b=1/EewteDVxfvgKYKWWFmtg7dDOtzsb46hlzsn2tMNvF8tLJAspAjW9B+jPnqwG/x0q n9gFj61Ls6sz74raosWJnv2ZPSdn4qgpCnv4XVvzHmLQ+rrvASVWZoUlM25phA0DFpqE t6Unuj1plTmHa8JeOZIvXkDCb4XMIJ1xGY3QoVBMvEga/6wxdZ10kGLu/s85Am7zMXMV NjVk+GkS3n2pvG7cNucaurgjwaOyyZgbEiu2mJpR6Zx8o41WTmTzlcRqDQXwFgh9F8cf uvX0NDgBX47prVXIlksy09GotRik7nUvXgKx7J5n6GrAeUsza8zT6K0WEMsKmbhBYN1N tVAA== X-Gm-Message-State: ANoB5pks9nQumB2bA2TmJP1pCJoSz7F56eGo7/N9F2FDcOki6ey1XwHF HYNR0pUhG5u3gtIa6cZwmIMJk0IhDqltsA== X-Google-Smtp-Source: AA0mqf5hOyjfbmQPJtK7ZDTPiU3ubdoTwQi77flbe4E9ZUpDtsLQB/hlOdcDoYo+q7PzOl1jwPKlVA== X-Received: by 2002:a63:e80f:0:b0:46b:4d6:e82b with SMTP id s15-20020a63e80f000000b0046b04d6e82bmr2888749pgh.88.1668195205179; Fri, 11 Nov 2022 11:33:25 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id p21-20020a631e55000000b004405c6eb962sm1707523pgm.4.2022.11.11.11.33.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:24 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 06/26] bpf: Rename MEM_ALLOC to MEM_RINGBUF Date: Sat, 12 Nov 2022 01:02:04 +0530 Message-Id: <20221111193224.876706-7-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9098; i=memxor@gmail.com; h=from:subject; bh=QiF5aiMWdeVPrX2X+RNmqO7yjZn/FhBvFcVkHpPSfBU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIn6ZPke4vV7BNRjXfeS4Pi5fu52IQiEddTUb6G YQ5duCiJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8RypF2EA ChO/4ly0Pzm7t4mLmn3cL0Xdq1ZEl2DZuvSdCk9XSlMDO4bn+WBMgO0VgamZEqG0HJe1Q9aMmCqcTN t+56IqynbsfrS74g7+95i5n/HpezZVlMdFfP+6PhoggPAyvvIrwQznXpXiH5E6irheJ9YsP2tWl/Oh oxQfTA4vnid4dHAdQeK5BFLjjsGxPV3+iC4u4oHb0x5cMoLj1jwnTDfIlEq69D43qxDrRGBZd7AgJl bwLZV7SfM+ys//fiT1dvOxYywkva0Nw2d5z6XvcQabZao1PyaH14s/b3lO//yiOH14UArHcd6ZHebe ylWPBuqB/ZNPgZMVRfTnN51rnMOLcn7LYWydu0T6pwK8Y6kWNN1cCP3TbLmGr6C4SWjRDpJ7D6Pon9 RHB3tJJQERUgVG37hCm5/YnP3haruPbNtK5oD1opWIZ/As8uO6K7hksXXFNK/NVlL4wtuufgV9JaOp t4/IyEEp2BAEvJvuX5wOjomriJhXqiuPS16rLA+Sqvvq834p3qfrhMzKavPdgS2WtLyrD4mRWXSIXy wuxO+NjVi2MrqjhiZ7bNaaXBYIpRDc4ALPekkYeq3gUNX/BZPbyNIyo8IuQEQT6pis9WYGx+M+SHlw LZWm1I+9H5F4yRfpuEjklur0uWQ3G3xrEB3daoKYzcbOtJvbtq9LdAc1eB1g== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, verifier uses MEM_ALLOC type tag to specially tag memory returned from bpf_ringbuf_reserve helper. However, this is currently only used for this purpose and there is an implicit assumption that it only refers to ringbuf memory (e.g. the check for ARG_PTR_TO_ALLOC_MEM in check_func_arg_reg_off). Hence, rename MEM_ALLOC to MEM_RINGBUF to indicate this special relationship and instead open the use of MEM_ALLOC for more generic allocations made for user types. Also, since ARG_PTR_TO_ALLOC_MEM_OR_NULL is unused, simply drop it. Finally, update selftests using 'alloc_' verifier string to 'ringbuf_'. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 11 ++++------- kernel/bpf/ringbuf.c | 6 +++--- kernel/bpf/verifier.c | 14 +++++++------- tools/testing/selftests/bpf/prog_tests/dynptr.c | 2 +- tools/testing/selftests/bpf/verifier/ringbuf.c | 2 +- tools/testing/selftests/bpf/verifier/spill_fill.c | 2 +- 6 files changed, 17 insertions(+), 20 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 2fe3ec620d54..afc1c51b59ff 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -488,10 +488,8 @@ enum bpf_type_flag { */ MEM_RDONLY = BIT(1 + BPF_BASE_TYPE_BITS), - /* MEM was "allocated" from a different helper, and cannot be mixed - * with regular non-MEM_ALLOC'ed MEM types. - */ - MEM_ALLOC = BIT(2 + BPF_BASE_TYPE_BITS), + /* MEM points to BPF ring buffer reservation. */ + MEM_RINGBUF = BIT(2 + BPF_BASE_TYPE_BITS), /* MEM is in user address space. */ MEM_USER = BIT(3 + BPF_BASE_TYPE_BITS), @@ -565,7 +563,7 @@ enum bpf_arg_type { ARG_PTR_TO_LONG, /* pointer to long */ ARG_PTR_TO_SOCKET, /* pointer to bpf_sock (fullsock) */ ARG_PTR_TO_BTF_ID, /* pointer to in-kernel struct */ - ARG_PTR_TO_ALLOC_MEM, /* pointer to dynamically allocated memory */ + ARG_PTR_TO_RINGBUF_MEM, /* pointer to dynamically reserved ringbuf memory */ ARG_CONST_ALLOC_SIZE_OR_ZERO, /* number of allocated bytes requested */ ARG_PTR_TO_BTF_ID_SOCK_COMMON, /* pointer to in-kernel sock_common or bpf-mirrored bpf_sock */ ARG_PTR_TO_PERCPU_BTF_ID, /* pointer to in-kernel percpu type */ @@ -582,7 +580,6 @@ enum bpf_arg_type { ARG_PTR_TO_MEM_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_MEM, ARG_PTR_TO_CTX_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_CTX, ARG_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_SOCKET, - ARG_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_ALLOC_MEM, ARG_PTR_TO_STACK_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_STACK, ARG_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_BTF_ID, /* pointer to memory does not need to be initialized, helper function must fill @@ -617,7 +614,7 @@ enum bpf_return_type { RET_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCKET, RET_PTR_TO_TCP_SOCK_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_TCP_SOCK, RET_PTR_TO_SOCK_COMMON_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCK_COMMON, - RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_ALLOC | RET_PTR_TO_MEM, + RET_PTR_TO_RINGBUF_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_RINGBUF | RET_PTR_TO_MEM, RET_PTR_TO_DYNPTR_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_MEM, RET_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_BTF_ID, diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 9e832acf4692..80f4b4d88aaf 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -447,7 +447,7 @@ BPF_CALL_3(bpf_ringbuf_reserve, struct bpf_map *, map, u64, size, u64, flags) const struct bpf_func_proto bpf_ringbuf_reserve_proto = { .func = bpf_ringbuf_reserve, - .ret_type = RET_PTR_TO_ALLOC_MEM_OR_NULL, + .ret_type = RET_PTR_TO_RINGBUF_MEM_OR_NULL, .arg1_type = ARG_CONST_MAP_PTR, .arg2_type = ARG_CONST_ALLOC_SIZE_OR_ZERO, .arg3_type = ARG_ANYTHING, @@ -490,7 +490,7 @@ BPF_CALL_2(bpf_ringbuf_submit, void *, sample, u64, flags) const struct bpf_func_proto bpf_ringbuf_submit_proto = { .func = bpf_ringbuf_submit, .ret_type = RET_VOID, - .arg1_type = ARG_PTR_TO_ALLOC_MEM | OBJ_RELEASE, + .arg1_type = ARG_PTR_TO_RINGBUF_MEM | OBJ_RELEASE, .arg2_type = ARG_ANYTHING, }; @@ -503,7 +503,7 @@ BPF_CALL_2(bpf_ringbuf_discard, void *, sample, u64, flags) const struct bpf_func_proto bpf_ringbuf_discard_proto = { .func = bpf_ringbuf_discard, .ret_type = RET_VOID, - .arg1_type = ARG_PTR_TO_ALLOC_MEM | OBJ_RELEASE, + .arg1_type = ARG_PTR_TO_RINGBUF_MEM | OBJ_RELEASE, .arg2_type = ARG_ANYTHING, }; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2407e3b179ec..5fca156eca43 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -577,8 +577,8 @@ static const char *reg_type_str(struct bpf_verifier_env *env, if (type & MEM_RDONLY) strncpy(prefix, "rdonly_", 32); - if (type & MEM_ALLOC) - strncpy(prefix, "alloc_", 32); + if (type & MEM_RINGBUF) + strncpy(prefix, "ringbuf_", 32); if (type & MEM_USER) strncpy(prefix, "user_", 32); if (type & MEM_PERCPU) @@ -5780,7 +5780,7 @@ static const struct bpf_reg_types mem_types = { PTR_TO_MAP_KEY, PTR_TO_MAP_VALUE, PTR_TO_MEM, - PTR_TO_MEM | MEM_ALLOC, + PTR_TO_MEM | MEM_RINGBUF, PTR_TO_BUF, }, }; @@ -5798,7 +5798,7 @@ static const struct bpf_reg_types int_ptr_types = { static const struct bpf_reg_types fullsock_types = { .types = { PTR_TO_SOCKET } }; static const struct bpf_reg_types scalar_types = { .types = { SCALAR_VALUE } }; static const struct bpf_reg_types context_types = { .types = { PTR_TO_CTX } }; -static const struct bpf_reg_types alloc_mem_types = { .types = { PTR_TO_MEM | MEM_ALLOC } }; +static const struct bpf_reg_types ringbuf_mem_types = { .types = { PTR_TO_MEM | MEM_RINGBUF } }; static const struct bpf_reg_types const_map_ptr_types = { .types = { CONST_PTR_TO_MAP } }; static const struct bpf_reg_types btf_ptr_types = { .types = { PTR_TO_BTF_ID } }; static const struct bpf_reg_types spin_lock_types = { .types = { PTR_TO_MAP_VALUE } }; @@ -5831,7 +5831,7 @@ static const struct bpf_reg_types *compatible_reg_types[__BPF_ARG_TYPE_MAX] = { [ARG_PTR_TO_BTF_ID] = &btf_ptr_types, [ARG_PTR_TO_SPIN_LOCK] = &spin_lock_types, [ARG_PTR_TO_MEM] = &mem_types, - [ARG_PTR_TO_ALLOC_MEM] = &alloc_mem_types, + [ARG_PTR_TO_RINGBUF_MEM] = &ringbuf_mem_types, [ARG_PTR_TO_INT] = &int_ptr_types, [ARG_PTR_TO_LONG] = &int_ptr_types, [ARG_PTR_TO_PERCPU_BTF_ID] = &percpu_btf_ptr_types, @@ -5952,14 +5952,14 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, case PTR_TO_MAP_VALUE: case PTR_TO_MEM: case PTR_TO_MEM | MEM_RDONLY: - case PTR_TO_MEM | MEM_ALLOC: + case PTR_TO_MEM | MEM_RINGBUF: case PTR_TO_BUF: case PTR_TO_BUF | MEM_RDONLY: case SCALAR_VALUE: /* Some of the argument types nevertheless require a * zero register offset. */ - if (base_type(arg_type) != ARG_PTR_TO_ALLOC_MEM) + if (base_type(arg_type) != ARG_PTR_TO_RINGBUF_MEM) return 0; break; /* All the rest must be rejected, except PTR_TO_BTF_ID which allows diff --git a/tools/testing/selftests/bpf/prog_tests/dynptr.c b/tools/testing/selftests/bpf/prog_tests/dynptr.c index 8fc4e6c02bfd..b0c06f821cb8 100644 --- a/tools/testing/selftests/bpf/prog_tests/dynptr.c +++ b/tools/testing/selftests/bpf/prog_tests/dynptr.c @@ -17,7 +17,7 @@ static struct { {"ringbuf_missing_release2", "Unreleased reference id=2"}, {"ringbuf_missing_release_callback", "Unreleased reference id"}, {"use_after_invalid", "Expected an initialized dynptr as arg #3"}, - {"ringbuf_invalid_api", "type=mem expected=alloc_mem"}, + {"ringbuf_invalid_api", "type=mem expected=ringbuf_mem"}, {"add_dynptr_to_map1", "invalid indirect read from stack"}, {"add_dynptr_to_map2", "invalid indirect read from stack"}, {"data_slice_out_of_bounds_ringbuf", "value is outside of the allowed memory range"}, diff --git a/tools/testing/selftests/bpf/verifier/ringbuf.c b/tools/testing/selftests/bpf/verifier/ringbuf.c index b64d33e4833c..84838feba47f 100644 --- a/tools/testing/selftests/bpf/verifier/ringbuf.c +++ b/tools/testing/selftests/bpf/verifier/ringbuf.c @@ -28,7 +28,7 @@ }, .fixup_map_ringbuf = { 1 }, .result = REJECT, - .errstr = "dereference of modified alloc_mem ptr R1", + .errstr = "dereference of modified ringbuf_mem ptr R1", }, { "ringbuf: invalid reservation offset 2", diff --git a/tools/testing/selftests/bpf/verifier/spill_fill.c b/tools/testing/selftests/bpf/verifier/spill_fill.c index e23f07175e1b..9bb302dade23 100644 --- a/tools/testing/selftests/bpf/verifier/spill_fill.c +++ b/tools/testing/selftests/bpf/verifier/spill_fill.c @@ -84,7 +84,7 @@ }, .fixup_map_ringbuf = { 1 }, .result = REJECT, - .errstr = "R0 pointer arithmetic on alloc_mem_or_null prohibited", + .errstr = "R0 pointer arithmetic on ringbuf_mem_or_null prohibited", }, { "check corrupted spill/fill", From patchwork Fri Nov 11 19:32:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040736 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3E66C433FE for ; Fri, 11 Nov 2022 19:33:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233805AbiKKTde (ORCPT ); Fri, 11 Nov 2022 14:33:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233865AbiKKTdc (ORCPT ); Fri, 11 Nov 2022 14:33:32 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B0C7B1E3 for ; Fri, 11 Nov 2022 11:33:30 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id m14-20020a17090a3f8e00b00212dab39bcdso8613558pjc.0 for ; Fri, 11 Nov 2022 11:33:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IDKjaXQrzw8xzAKuOLSOkHReeUI2bgGmwlrnNY+xHtA=; b=NhpRCOOVZdtqTH2jNaDSbQ96S1XZ+sdIUPAoEcXp1kHjvZShn+1NKcgUH0DtEGMh09 FViI4ClZKIYuDZ5Sc9/tcyWMJIXdC4bfGrzzeXPFQnzEnZIItFtnGESpme06sPgngzuE LCQGLg3noPY1K5MbHWoLpdyViFq87nF3otF7eCgsTxf1QkP1oUJIgZZ65b+LZauEBFu6 uc8ZRbXsWKOw5IhcZmK20OQm7STLiIDhNn/sFLchk3nk0UNU28MekSHAWFmnPzOwnLU0 78iti8qYQWi9kwfyEV7Xuzhz1jQ7shdPpfCppXZkz0t2k+E2gDEww1a0K6A6FZRJVyd9 CnKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IDKjaXQrzw8xzAKuOLSOkHReeUI2bgGmwlrnNY+xHtA=; b=kgoe88kkX7Nw7eSBdoG3ziEVxqBVmnQCZJxfpkTjLs3CqmNom8pJv3eKY/uzOEAvqx 4/lKUDfiCYawxJhQhkiHvaOx8yxWpkUzLvllwStYLoTEp7x/99tnXvEJl+0kxbELxaZi MrvPvqzJ0tdLM+NaQHe2y6XRbiQzeeRmNlhPMsNhlpd9NFTW+bhyx7of4UkcY8uYsUpN //E5PuERa+EcxC1L4Qir3hxTWJVpxX7Wx4QnjeEoWB8CxaFrCab95ZXMMCk9tkiMZm7R hfQLX6yUsN11oWFH8w2U2sLZJim8dBUHQtDvLn7BEfny9VDBOiCaHbmRVIxoa8GiD3Ex qEnw== X-Gm-Message-State: ANoB5pniLlmx6nhExmhzK7/KpFh2GYw5WbwpkWOvJKW7sSe3DwZbWNRg 9ilwS6hmeWvoojMKMiY1VwUI/kD/NVsekg== X-Google-Smtp-Source: AA0mqf7fqHoVBn1EKj/Bn8GibTQXjGaNnfOWESzI6+dqOCmZsACWk03WgnqoVwpYdP82mB/d3WYBVQ== X-Received: by 2002:a17:90b:3c44:b0:212:ec17:3fd0 with SMTP id pm4-20020a17090b3c4400b00212ec173fd0mr3423344pjb.241.1668195210052; Fri, 11 Nov 2022 11:33:30 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id m1-20020a17090ade0100b0020dc318a43esm1949847pjv.25.2022.11.11.11.33.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:29 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 07/26] bpf: Refactor btf_struct_access Date: Sat, 12 Nov 2022 01:02:05 +0530 Message-Id: <20221111193224.876706-8-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=12746; i=memxor@gmail.com; h=from:subject; bh=J59s9TVGQk1qIQ03OK9PLmTtdwn8dkmFxeIuacvaOIY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqInSTonag6VERR0d31rxPLsk5FtlPEhOYfOuBUo kWTQYQeJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iJwAKCRBM4MiGSL8Rypn4D/ 9mUUFTXN2NIrhQjI3WHoQqvV8O2gH16y/j9sJk+E9YxPU41oSKOCd+mrzvYuhdnfKS8j4JTv7ohozl 7/eTb6f0OUfZcSK0CciWJUZ/eSvrBzMuMFk/R13ARLXO5591KAjQ1xqncCtawwQUBYC39kS+9yeCEf s5mWkiYsqpShbgABqFLyZpAMYOkbMQ+TNGn6RthQT7deInX9bZfTRcPKpXkY/RsBrp8FjEndNAygem ghVJVHvY34GmZgkRRT2LUQUbygYryVwnNvC7/9i8+1oSdDbZzjikpebYQ8lELaaoifdo94hPbLEdQ3 u5ulRPS9QcpDu1ENBo6LEETWMHbGworB7VY2Lk9vTnK33Kf13jteuSboebGZMlMjt4DNZgUkQnrovi h8VLttcwmhNIhPh8eWQdPh6CH/RKI3japPNQisPnw64rmOG0z+5kml23Toj91bQWslVz82ZGmLnhZv mcxZdtSzcyHUUzATey9zGVD88eQrUS/st8C2FeWUJQqsbxnHZJV0zQmQKmW4xB5C7TTAtGJR2PjCKg NqhfrzlcaKuNAENmnCzSVPjI/3zJkT4SFSAife32LrQUvdrMMBwhVRAAzbbV+Ke3fc2Z6Z4fJi5ppu JU+7AEtNmmjybIeHtX1/S9RSoa5oF7MYVRriGl4xaQgoc5d2sYLFj4zSuTtw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Instead of having to pass multiple arguments that describe the register, pass the bpf_reg_state into the btf_struct_access callback. Currently, all call sites simply reuse the btf and btf_id of the reg they want to check the access of. The only exception to this pattern is the callsite in check_ptr_to_map_access, hence for that case create a dummy reg to simulate PTR_TO_BTF_ID access. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 17 ++++++++-------- include/linux/filter.h | 8 ++++---- kernel/bpf/btf.c | 11 +++++++---- kernel/bpf/verifier.c | 12 ++++++----- net/bpf/bpf_dummy_struct_ops.c | 14 ++++++------- net/core/filter.c | 34 +++++++++++++------------------- net/ipv4/bpf_tcp_ca.c | 13 ++++++------ net/netfilter/nf_conntrack_bpf.c | 17 +++++++--------- 8 files changed, 60 insertions(+), 66 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index afc1c51b59ff..49f9d2bec401 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -771,6 +771,7 @@ struct bpf_prog_ops { union bpf_attr __user *uattr); }; +struct bpf_reg_state; struct bpf_verifier_ops { /* return eBPF function prototype for verification */ const struct bpf_func_proto * @@ -792,9 +793,8 @@ struct bpf_verifier_ops { struct bpf_insn *dst, struct bpf_prog *prog, u32 *target_size); int (*btf_struct_access)(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag); }; @@ -2080,9 +2080,9 @@ static inline bool bpf_tracing_btf_ctx_access(int off, int size, return btf_ctx_access(off, size, type, prog, info); } -int btf_struct_access(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, +int btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag); bool btf_struct_ids_match(struct bpf_verifier_log *log, const struct btf *btf, u32 id, int off, @@ -2333,9 +2333,8 @@ static inline struct bpf_prog *bpf_prog_by_id(u32 id) } static inline int btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag) { return -EACCES; diff --git a/include/linux/filter.h b/include/linux/filter.h index efc42a6e3aed..787d35dbf5b0 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -568,10 +568,10 @@ struct sk_filter { DECLARE_STATIC_KEY_FALSE(bpf_stats_enabled_key); extern struct mutex nf_conn_btf_access_lock; -extern int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, u32 *next_btf_id, - enum bpf_type_flag *flag); +extern int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag); typedef unsigned int (*bpf_dispatcher_fn)(const void *ctx, const struct bpf_insn *insnsi, diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index c0d73d71c539..875355ff3718 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6017,15 +6017,18 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf, return -EINVAL; } -int btf_struct_access(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype __maybe_unused, +int btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype __maybe_unused, u32 *next_btf_id, enum bpf_type_flag *flag) { + const struct btf *btf = reg->btf; enum bpf_type_flag tmp_flag = 0; + const struct btf_type *t; + u32 id = reg->btf_id; int err; - u32 id; + t = btf_type_by_id(btf, id); do { err = btf_struct_walk(log, btf, t, off, size, &id, &tmp_flag); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5fca156eca43..bbbb0f27c7d1 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4683,16 +4683,14 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, } if (env->ops->btf_struct_access) { - ret = env->ops->btf_struct_access(&env->log, reg->btf, t, - off, size, atype, &btf_id, &flag); + ret = env->ops->btf_struct_access(&env->log, reg, off, size, atype, &btf_id, &flag); } else { if (atype != BPF_READ) { verbose(env, "only read is supported\n"); return -EACCES; } - ret = btf_struct_access(&env->log, reg->btf, t, off, size, - atype, &btf_id, &flag); + ret = btf_struct_access(&env->log, reg, off, size, atype, &btf_id, &flag); } if (ret < 0) @@ -4718,6 +4716,7 @@ static int check_ptr_to_map_access(struct bpf_verifier_env *env, { struct bpf_reg_state *reg = regs + regno; struct bpf_map *map = reg->map_ptr; + struct bpf_reg_state map_reg; enum bpf_type_flag flag = 0; const struct btf_type *t; const char *tname; @@ -4756,7 +4755,10 @@ static int check_ptr_to_map_access(struct bpf_verifier_env *env, return -EACCES; } - ret = btf_struct_access(&env->log, btf_vmlinux, t, off, size, atype, &btf_id, &flag); + /* Simulate access to a PTR_TO_BTF_ID */ + memset(&map_reg, 0, sizeof(map_reg)); + mark_btf_ld_reg(env, &map_reg, 0, PTR_TO_BTF_ID, btf_vmlinux, *map->ops->map_btf_id, 0); + ret = btf_struct_access(&env->log, &map_reg, off, size, atype, &btf_id, &flag); if (ret < 0) return ret; diff --git a/net/bpf/bpf_dummy_struct_ops.c b/net/bpf/bpf_dummy_struct_ops.c index e78dadfc5829..2d434c1f4617 100644 --- a/net/bpf/bpf_dummy_struct_ops.c +++ b/net/bpf/bpf_dummy_struct_ops.c @@ -156,29 +156,29 @@ static bool bpf_dummy_ops_is_valid_access(int off, int size, } static int bpf_dummy_ops_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag) { const struct btf_type *state; + const struct btf_type *t; s32 type_id; int err; - type_id = btf_find_by_name_kind(btf, "bpf_dummy_ops_state", + type_id = btf_find_by_name_kind(reg->btf, "bpf_dummy_ops_state", BTF_KIND_STRUCT); if (type_id < 0) return -EINVAL; - state = btf_type_by_id(btf, type_id); + t = btf_type_by_id(reg->btf, reg->btf_id); + state = btf_type_by_id(reg->btf, type_id); if (t != state) { bpf_log(log, "only access to bpf_dummy_ops_state is supported\n"); return -EACCES; } - err = btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + err = btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); if (err < 0) return err; diff --git a/net/core/filter.c b/net/core/filter.c index cb3b635e35be..199632e6a7cb 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -8651,28 +8651,25 @@ static bool tc_cls_act_is_valid_access(int off, int size, DEFINE_MUTEX(nf_conn_btf_access_lock); EXPORT_SYMBOL_GPL(nf_conn_btf_access_lock); -int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, u32 *next_btf_id, - enum bpf_type_flag *flag); +int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag); EXPORT_SYMBOL_GPL(nfct_btf_struct_access); static int tc_cls_act_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { int ret = -EACCES; if (atype == BPF_READ) - return btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + return btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_lock(&nf_conn_btf_access_lock); if (nfct_btf_struct_access) - ret = nfct_btf_struct_access(log, btf, t, off, size, atype, next_btf_id, flag); + ret = nfct_btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_unlock(&nf_conn_btf_access_lock); return ret; @@ -8738,21 +8735,18 @@ void bpf_warn_invalid_xdp_action(struct net_device *dev, struct bpf_prog *prog, EXPORT_SYMBOL_GPL(bpf_warn_invalid_xdp_action); static int xdp_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { int ret = -EACCES; if (atype == BPF_READ) - return btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + return btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_lock(&nf_conn_btf_access_lock); if (nfct_btf_struct_access) - ret = nfct_btf_struct_access(log, btf, t, off, size, atype, next_btf_id, flag); + ret = nfct_btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_unlock(&nf_conn_btf_access_lock); return ret; diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 6da16ae6a962..d15c91de995f 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -69,18 +69,17 @@ static bool bpf_tcp_ca_is_valid_access(int off, int size, } static int bpf_tcp_ca_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { + const struct btf_type *t; size_t end; if (atype == BPF_READ) - return btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + return btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); + t = btf_type_by_id(reg->btf, reg->btf_id); if (t != tcp_sock_type) { bpf_log(log, "only read is supported\n"); return -EACCES; diff --git a/net/netfilter/nf_conntrack_bpf.c b/net/netfilter/nf_conntrack_bpf.c index 8639e7efd0e2..24002bc61e07 100644 --- a/net/netfilter/nf_conntrack_bpf.c +++ b/net/netfilter/nf_conntrack_bpf.c @@ -191,19 +191,16 @@ BTF_ID(struct, nf_conn___init) /* Check writes into `struct nf_conn` */ static int _nf_conntrack_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { - const struct btf_type *ncit; - const struct btf_type *nct; + const struct btf_type *ncit, *nct, *t; size_t end; - ncit = btf_type_by_id(btf, btf_nf_conn_ids[1]); - nct = btf_type_by_id(btf, btf_nf_conn_ids[0]); - + ncit = btf_type_by_id(reg->btf, btf_nf_conn_ids[1]); + nct = btf_type_by_id(reg->btf, btf_nf_conn_ids[0]); + t = btf_type_by_id(reg->btf, reg->btf_id); if (t != nct && t != ncit) { bpf_log(log, "only read is supported\n"); return -EACCES; From patchwork Fri Nov 11 19:32:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040737 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA779C433FE for ; Fri, 11 Nov 2022 19:33:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233865AbiKKTdh (ORCPT ); Fri, 11 Nov 2022 14:33:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233828AbiKKTdg (ORCPT ); Fri, 11 Nov 2022 14:33:36 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1107712AD4 for ; Fri, 11 Nov 2022 11:33:35 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id b62so5160882pgc.0 for ; Fri, 11 Nov 2022 11:33:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EkCP4Dony/PsvTJnx7VXRtAY09Fp+elLOavyTVsL+mM=; b=O6EEv9Zn/QdYjXyJkambhe8De3lN2lBQmXNkhVlUtGVpGEPtGa0B42FatsfrrPub37 wjD1aqT63Oo4uZxK8+f8Ic3Fw9IIMFsvyiE572gkpMqLhZlg299ERvbROPvVVoiqQcXK POi2I4yY6gntkzYN5+Pgbb+vTzBe6XecXro/jcsmCre3JLlU9s0lTvajslftKZ89Uu+d LnWStP6z3r3Ne/bfpDA3OqKYcE9oEsibotezHOUa3ZiTm3ZlYW6joIkN9XuuTFullnxM GdFbj9gkj7KwR/s74b8KMwfhds5997fuwLKuuCHeJafnqupgglL7KvKHDdbdy1CL7i33 G1wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EkCP4Dony/PsvTJnx7VXRtAY09Fp+elLOavyTVsL+mM=; b=pgH3zfJgVI4i3DWUP1ehSssa/DlYu/lv7phNNmTsS0P/W8Z/HokjxPrfaIsslJXpto K3ZsggYB7NPqr/7+j3Qh7x7lw1YKYkI+mSAxLTH5MPk10OeeZKS2868QobWRKL7izuvP XHWjf0L3bIVVrpF+SetxZF1143iWGddFZA6h0CA3jk0YqPh+pY2bFO5Mt8eG+xhGCWCg ixYqvPdv466gmwwHj0FMSYWnaWTydtdPfjTEjYl7c9hXhZ7qG7z/fk05DmoTGDmB+AkM Lx6q2uqH9yQ0IfhK6mDDmfHBhHI0LaQD9Q0VXOJNkVdczADasa/cy9hzj83dSD4aj/gA tyOQ== X-Gm-Message-State: ANoB5plwebz8lG8zWkxID6lSF/Z1o6QxqfT1sB7DdEJ4y75qUVYajBv5 XUTUqX0i2cof/OKEyUSRm4kJ2UclK6mZFg== X-Google-Smtp-Source: AA0mqf6jF0yHfS4VbGr42Dg87w+qb37XiIdNf3nmDXbtWnuF6xjenR+C9M2/J6v7TOfEdEXHoYn1uw== X-Received: by 2002:a63:1665:0:b0:426:9c23:9f94 with SMTP id 37-20020a631665000000b004269c239f94mr2929036pgw.105.1668195214260; Fri, 11 Nov 2022 11:33:34 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id u3-20020a170902e80300b00172e19c5f8bsm2089877plg.168.2022.11.11.11.33.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:33 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 08/26] bpf: Introduce allocated objects support Date: Sat, 12 Nov 2022 01:02:06 +0530 Message-Id: <20221111193224.876706-9-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4871; i=memxor@gmail.com; h=from:subject; bh=meS7gQZTEGq/DUEJpmd9jyqWTUThhVE5S28cbBs7iUA=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIo0xyMZd3AtlXePT7oI/kJPUEhIBQ1Tl7N1l1f t0tz1JaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8Ryk17EA CZIAsithALdp/QdbU9kHEYOH1EgKJlh4WqdQ5Wecvz4eFTDOcfFtnXwCFwOzWKvYIsDaTuOGu60yXO NR5aDUaxVmFOyuhU/XCX7yMHKXl1qg4f7e7D28C1nl0uPiYN4Y8hgMpdNpPybQEbKm1aLlrJWm8QaX H+7IIZwewP844hw+sd8adNWMsDP0wKcKBrXzotf/aX6u+Qdgqke1jazmZdoJ2JlMPHm1aR3lEhINAe OPiz6ulKw0AtYzLGzLsWl7u0qqqu120KCvkIj9VeVvL1LBh9xwKHYwBawiA+04mlvkBJ2SZUaPfY9n XdAQf23GfT1kIWJOz4+O/5adfe8FKFtJGOc2+oDb4tTP37Ona3z0XH+KfeAISvrO3HuStEvtvQegaH sFeDadm+KXpYO4sMhgdOUi92D/sYWwm1WBXl+ofKu9PI/3WHfgqQVdTdJcDd810Xyp2ywZNoqNkFSr kyn4PBEJC750ltQi4XGtqdSzXzMTyGkNkK6X1OjPQxbvfwGbaXngNF1hStspqPovOZwWIpUB0sbK4B HHa2+UFNUywdpFuh7cOn/CVrYtXpgha1CPxCo6LmuDex0JBsbdAf5HghwBC3oPluFWCkPPo/8LhtWA YYPWmJNa+J/SNByItwugDIdOUz9I5/JyFY+HPlsD4zR1oYDqv0X4swBakfeQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce support for representing pointers to objects allocated by the BPF program, i.e. PTR_TO_BTF_ID that point to a type in program BTF. This is indicated by the presence of MEM_ALLOC type flag in reg->type to avoid having to check btf_is_kernel when trying to match argument types in helpers. Whenever walking such types, any pointers being walked will always yield a SCALAR instead of pointer. In the future we might permit kptr inside such allocated objects (either kernel or local), and it will then form a PTR_TO_BTF_ID of the respective type. For now, such allocated objects will always be referenced in verifier context, hence ref_obj_id == 0 for them is a bug. It is allowed to write to such objects, as long fields that are special are not touched (support for which will be added in subsequent patches). Note that once such a pointer is marked PTR_UNTRUSTED, it is no longer allowed to write to it. No PROBE_MEM handling is therefore done for loads into this type unless PTR_UNTRUSTED is part of the register type, since they can never be in an undefined state, and their lifetime will always be valid. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 11 +++++++++++ kernel/bpf/btf.c | 5 +++++ kernel/bpf/verifier.c | 25 +++++++++++++++++++++++-- 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 49f9d2bec401..3cab113b149e 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -524,6 +524,11 @@ enum bpf_type_flag { /* Size is known at compile time. */ MEM_FIXED_SIZE = BIT(10 + BPF_BASE_TYPE_BITS), + /* MEM is of a an allocated object of type from program BTF. This is + * used to tag PTR_TO_BTF_ID allocated using bpf_obj_new. + */ + MEM_ALLOC = BIT(11 + BPF_BASE_TYPE_BITS), + __BPF_TYPE_FLAG_MAX, __BPF_TYPE_LAST_FLAG = __BPF_TYPE_FLAG_MAX - 1, }; @@ -2791,4 +2796,10 @@ struct bpf_key { bool has_ref; }; #endif /* CONFIG_KEYS */ + +static inline bool type_is_alloc(u32 type) +{ + return type & MEM_ALLOC; +} + #endif /* _LINUX_BPF_H */ diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 875355ff3718..9a596f430558 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6034,6 +6034,11 @@ int btf_struct_access(struct bpf_verifier_log *log, switch (err) { case WALK_PTR: + /* For local types, the destination register cannot + * become a pointer again. + */ + if (type_is_alloc(reg->type)) + return SCALAR_VALUE; /* If we found the pointer or scalar on t+off, * we're done. */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index bbbb0f27c7d1..5cf14c1391a5 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4682,14 +4682,27 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, return -EACCES; } - if (env->ops->btf_struct_access) { + if (env->ops->btf_struct_access && !type_is_alloc(reg->type)) { + if (!btf_is_kernel(reg->btf)) { + verbose(env, "verifier internal error: reg->btf must be kernel btf\n"); + return -EFAULT; + } ret = env->ops->btf_struct_access(&env->log, reg, off, size, atype, &btf_id, &flag); } else { - if (atype != BPF_READ) { + /* Writes are permitted with default btf_struct_access for local + * kptrs (which always have ref_obj_id > 0), but not for + * untrusted PTR_TO_BTF_ID | MEM_ALLOC. + */ + if (atype != BPF_READ && reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { verbose(env, "only read is supported\n"); return -EACCES; } + if (type_is_alloc(reg->type) && !reg->ref_obj_id) { + verbose(env, "verifier internal error: ref_obj_id for allocated object must be non-zero\n"); + return -EFAULT; + } + ret = btf_struct_access(&env->log, reg, off, size, atype, &btf_id, &flag); } @@ -5968,6 +5981,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, * fixed offset. */ case PTR_TO_BTF_ID: + case PTR_TO_BTF_ID | MEM_ALLOC: /* When referenced PTR_TO_BTF_ID is passed to release function, * it's fixed offset must be 0. In the other cases, fixed offset * can be non-zero. @@ -13650,6 +13664,13 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) break; case PTR_TO_BTF_ID: case PTR_TO_BTF_ID | PTR_UNTRUSTED: + /* PTR_TO_BTF_ID | MEM_ALLOC always has a valid lifetime, unlike + * PTR_TO_BTF_ID, and an active ref_obj_id, but the same cannot + * be said once it is marked PTR_UNTRUSTED, hence we must handle + * any faults for loads into such types. BPF_WRITE is disallowed + * for this case. + */ + case PTR_TO_BTF_ID | MEM_ALLOC | PTR_UNTRUSTED: if (type == BPF_READ) { insn->code = BPF_LDX | BPF_PROBE_MEM | BPF_SIZE((insn)->code); From patchwork Fri Nov 11 19:32:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040738 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2036FC433FE for ; Fri, 11 Nov 2022 19:33:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233844AbiKKTdn (ORCPT ); Fri, 11 Nov 2022 14:33:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233898AbiKKTdl (ORCPT ); Fri, 11 Nov 2022 14:33:41 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38ACBFD1B for ; Fri, 11 Nov 2022 11:33:39 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id g24so4980446plq.3 for ; Fri, 11 Nov 2022 11:33:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fzUGPdmLNHyd54YMyzSVBkp458VV4a2sZ3x6/aexVDw=; b=F+cTiPJqnQ7TDecMVyeGUKJbBumFv8I4816VWpOgIjrPHj0z63b2JNuhnmsQ8aN8rk OjyOuils8GmOj2gNdZtTnaFnAqD0xqI9HmU3bsgpvUHuUiqKfkJNO+VJw4dRzxHauzMa Wv5uHqwzfynbrDsHCkrZ0aiLeoOXljcB6TsdeZ9obpNRVGqQH6unVXF+JRsnREeoRZyT FD/zQyhd/KPjK/IJUwpGH4eHYIrbeTb/K0O+A07LZ8pQJYZuQcqGympMuTltdAJTHIFN zE8BZlj96HoUvAMaqpEJkvUiWzTwRsA5rgq6c95SFiqTuo3SHmOPn1+mil7ei/ZhDW0O TyCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fzUGPdmLNHyd54YMyzSVBkp458VV4a2sZ3x6/aexVDw=; b=2zeExVqGOcFCXpi6fHGv8AdX1JRBTI/AkqjBf0BPfrnJ8KM8+Z60P/eqZ8EhkBiAEA JrdJpiywwECHbOjF/d2nnG6F1NX2b/esXd0Ng6J5h4F0xjqR4kHNU9yoHAeUR1YKAFAi aVyBkiTk0NTNwctPwK1oFKlMOUejeWBi8lbuTnFVVI5B5vEPdvhdZ8rnJTEFrtYFFwQ2 8cNf7CXQBbnqVCMVZlLgf2x8HKcnEGiCZzg4/LVPrQLODV1+K5eP7gkWP6cw5dS+fJpY w3QJd6ui2GoNSYIe4+6rPsvyi7D5vKFkibKGtdNLzvnC0pPDJyI0sAiLG8frZC6AiG9S TJWQ== X-Gm-Message-State: ANoB5plgxCp4cyK2wMTrT9Mrpvhl6rgzX6ye7S+cVdHeWXp9MN3aPTyH JWr/5+dcvjFZKdt5UdTJmzBQuMpzZ6N3ng== X-Google-Smtp-Source: AA0mqf6UqGexT4+PJKW6/YpZ2GamPn6XtIFKHT9/mP8cOtJczQu89+BD3tbhVj0/IhN0Y5QWDIC3sg== X-Received: by 2002:a17:90b:1908:b0:213:90f5:274 with SMTP id mp8-20020a17090b190800b0021390f50274mr3306128pjb.182.1668195218678; Fri, 11 Nov 2022 11:33:38 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id 12-20020a630c4c000000b0047063eb4098sm1685057pgm.37.2022.11.11.11.33.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:38 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 09/26] bpf: Recognize lock and list fields in allocated objects Date: Sat, 12 Nov 2022 01:02:07 +0530 Message-Id: <20221111193224.876706-10-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=14783; i=memxor@gmail.com; h=from:subject; bh=k3bTOlJBS1pj78pjWenDcXoiNplza9PbDH+DEd0TXpg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoI8TvmAmFLn+jTZqW6t6HjoxM6+PInOzqQWaM mUG8dtmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RyiLDEA CJVe8SkLRdzwSuXgm9s4n9VF6NtM9xzMl8mAJsVc0HkfWO2uQRGJD3yVhVYfckpLsddO8vdGizmOMh oOPDyPVBX5L5aa/NHsdOEpjtptnik4LgDST6VF4kxQEZu6+K5Sa3gyXRuHFkc9aIWdneAxy6yYpo3x Bz0IMWeYcXapXwbbyf94mBQ2pjdaxt6a8MNmqUJ5rt9ywZumzFgucNhK7AhpGJVhCVIfCie2pJDU58 mYDClP0mqrWS2/jMWwkHwAiesObt03291hsoR8owgf3SwdvSwAQmoI7icDNeLqa7wdUpxbNYPfrOcC O4lVahLmnkGehaq1F5iNP069mP9P47RMmeZ52oP681nPKdc/l7ii81e9KpuZKaOakVrNYm/p8RM6lx GmIA9dEwz1ZaZB3MJQ396xIE+m8st89XjsW2LyOjPPtRxHnnAd0dNSONiz188Uwox64iIQWJvLbnBb 5YaIMSDig7JV1a4PUHWKowGTMfdNQEtNyGerXv621kGsYlBtWuBGlTsvj5wG3RdkB+9e+1lODMJAvy AH837KQMS26uPnQTl8hpg2R8EssGLX6vBwLWN9kU3oyFIBmXExaPajf8jxPekXgbN3wHFBVhBS5u83 XMwFqmUihwRipdWlwdCo3iA8wUYqZH4gQLmrJRcN2nMkGVrWappBJlLx5tnA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow specifying bpf_spin_lock, bpf_list_head, bpf_list_node fields in a allocated object. Also update btf_struct_access to reject direct access to these special fields. A bpf_list_head allows implementing map-in-map style use cases, where an allocated object with bpf_list_head is linked into a list in a map value. This would require embedding a bpf_list_node, support for which is also included. The bpf_spin_lock is used to protect the bpf_list_head and other data. While we strictly don't require to hold a bpf_spin_lock while touching the bpf_list_head in such objects, as when have access to it, we have complete ownership of the object, the locking constraint is still kept and may be conditionally lifted in the future. Note that the specification of such types can be done just like map values, e.g.: struct bar { struct bpf_list_node node; }; struct foo { struct bpf_spin_lock lock; struct bpf_list_head head __contains(bar, node); struct bpf_list_node node; }; struct map_value { struct bpf_spin_lock lock; struct bpf_list_head head __contains(foo, node); }; To recognize such types in user BTF, we build a btf_struct_metas array of metadata items corresponding to each BTF ID. This is done once during the btf_parse stage to avoid having to do it each time during the verification process's requirement to inspect the metadata. Moreover, the computed metadata needs to be passed to some helpers in future patches which requires allocating them and storing them in the BTF that is pinned by the program itself, so that valid access can be assumed to such data during program runtime. A key thing to note is that once a btf_struct_meta is available for a type, both the btf_record and btf_field_offs should be available. It is critical that btf_field_offs is available in case special fields are present, as we extensively rely on special fields being zeroed out in map values and allocated objects in later patches. The code ensures that by bailing out in case of errors and ensuring both are available together. If the record is not available, the special fields won't be recognized, so not having both is also fine (in terms of being a verification error and not a runtime bug). Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 7 ++ include/linux/btf.h | 35 ++++++++ kernel/bpf/btf.c | 196 +++++++++++++++++++++++++++++++++++++++---- kernel/bpf/syscall.c | 4 + 4 files changed, 224 insertions(+), 18 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 3cab113b149e..4cd3c9e6f50b 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -176,6 +176,7 @@ enum btf_field_type { BPF_KPTR_REF = (1 << 3), BPF_KPTR = BPF_KPTR_UNREF | BPF_KPTR_REF, BPF_LIST_HEAD = (1 << 4), + BPF_LIST_NODE = (1 << 5), }; struct btf_field_kptr { @@ -276,6 +277,8 @@ static inline const char *btf_field_type_name(enum btf_field_type type) return "kptr"; case BPF_LIST_HEAD: return "bpf_list_head"; + case BPF_LIST_NODE: + return "bpf_list_node"; default: WARN_ON_ONCE(1); return "unknown"; @@ -294,6 +297,8 @@ static inline u32 btf_field_type_size(enum btf_field_type type) return sizeof(u64); case BPF_LIST_HEAD: return sizeof(struct bpf_list_head); + case BPF_LIST_NODE: + return sizeof(struct bpf_list_node); default: WARN_ON_ONCE(1); return 0; @@ -312,6 +317,8 @@ static inline u32 btf_field_type_align(enum btf_field_type type) return __alignof__(u64); case BPF_LIST_HEAD: return __alignof__(struct bpf_list_head); + case BPF_LIST_NODE: + return __alignof__(struct bpf_list_node); default: WARN_ON_ONCE(1); return 0; diff --git a/include/linux/btf.h b/include/linux/btf.h index d80345fa566b..a01a8da20021 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -6,6 +6,8 @@ #include #include +#include +#include #include #include @@ -78,6 +80,17 @@ struct btf_id_dtor_kfunc { u32 kfunc_btf_id; }; +struct btf_struct_meta { + u32 btf_id; + struct btf_record *record; + struct btf_field_offs *field_offs; +}; + +struct btf_struct_metas { + u32 cnt; + struct btf_struct_meta types[]; +}; + typedef void (*btf_dtor_kfunc_t)(void *); extern const struct file_operations btf_fops; @@ -408,6 +421,23 @@ static inline struct btf_param *btf_params(const struct btf_type *t) return (struct btf_param *)(t + 1); } +static inline int btf_id_cmp_func(const void *a, const void *b) +{ + const int *pa = a, *pb = b; + + return *pa - *pb; +} + +static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id) +{ + return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL; +} + +static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) +{ + return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func); +} + #ifdef CONFIG_BPF_SYSCALL struct bpf_prog; @@ -423,6 +453,7 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type, s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id); int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, struct module *owner); +struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id); #else static inline const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id) @@ -454,6 +485,10 @@ static inline int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dt { return 0; } +static inline struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id) +{ + return NULL; +} #endif static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type *t) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 9a596f430558..fbcb846188e2 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -237,6 +237,7 @@ struct btf { struct rcu_head rcu; struct btf_kfunc_set_tab *kfunc_set_tab; struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab; + struct btf_struct_metas *struct_meta_tab; /* split BTF support */ struct btf *base_btf; @@ -1642,8 +1643,30 @@ static void btf_free_dtor_kfunc_tab(struct btf *btf) btf->dtor_kfunc_tab = NULL; } +static void btf_struct_metas_free(struct btf_struct_metas *tab) +{ + int i; + + if (!tab) + return; + for (i = 0; i < tab->cnt; i++) { + btf_record_free(tab->types[i].record); + kfree(tab->types[i].field_offs); + } + kfree(tab); +} + +static void btf_free_struct_meta_tab(struct btf *btf) +{ + struct btf_struct_metas *tab = btf->struct_meta_tab; + + btf_struct_metas_free(tab); + btf->struct_meta_tab = NULL; +} + static void btf_free(struct btf *btf) { + btf_free_struct_meta_tab(btf); btf_free_dtor_kfunc_tab(btf); btf_free_kfunc_set_tab(btf); kvfree(btf->types); @@ -3353,6 +3376,12 @@ static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, goto end; } } + if (field_mask & BPF_LIST_NODE) { + if (!strcmp(name, "bpf_list_node")) { + type = BPF_LIST_NODE; + goto end; + } + } /* Only return BPF_KPTR when all other types with matchable names fail */ if (field_mask & BPF_KPTR) { type = BPF_KPTR_REF; @@ -3396,6 +3425,7 @@ static int btf_find_struct_field(const struct btf *btf, switch (field_type) { case BPF_SPIN_LOCK: case BPF_TIMER: + case BPF_LIST_NODE: ret = btf_find_struct(btf, member_type, off, sz, field_type, idx < info_cnt ? &info[idx] : &tmp); if (ret < 0) @@ -3456,6 +3486,7 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, switch (field_type) { case BPF_SPIN_LOCK: case BPF_TIMER: + case BPF_LIST_NODE: ret = btf_find_struct(btf, var_type, off, sz, field_type, idx < info_cnt ? &info[idx] : &tmp); if (ret < 0) @@ -3671,6 +3702,8 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type if (ret < 0) goto end; break; + case BPF_LIST_NODE: + break; default: ret = -EFAULT; goto end; @@ -5141,6 +5174,118 @@ static int btf_parse_hdr(struct btf_verifier_env *env) return btf_check_sec_info(env, btf_data_size); } +static const char *local_kptr_fields[] = { + "bpf_spin_lock", + "bpf_list_head", + "bpf_list_node", +}; + +static struct btf_struct_metas * +btf_parse_struct_metas(struct bpf_verifier_log *log, struct btf *btf) +{ + union { + struct btf_id_set set; + struct { + u32 _cnt; + u32 _ids[ARRAY_SIZE(local_kptr_fields)]; + } _arr; + } lkf; + struct btf_struct_metas *tab = NULL; + int i, n, id, ret; + + memset(&lkf, 0, sizeof(lkf)); + + for (i = 0; i < ARRAY_SIZE(local_kptr_fields); i++) { + /* Try to find whether this special type exists in user BTF, and + * if so remember its ID so we can easily find it among members + * of structs that we iterate in the next loop. + */ + id = btf_find_by_name_kind(btf, local_kptr_fields[i], BTF_KIND_STRUCT); + if (id < 0) + continue; + lkf.set.ids[lkf.set.cnt++] = id; + } + + if (!lkf.set.cnt) + return NULL; + sort(&lkf.set.ids, lkf.set.cnt, sizeof(lkf.set.ids[0]), btf_id_cmp_func, NULL); + + n = btf_nr_types(btf); + for (i = 1; i < n; i++) { + const struct btf_member *member; + struct btf_field_offs *foffs; + struct btf_struct_meta *type; + struct btf_record *record; + const struct btf_type *t; + int j; + + t = btf_type_by_id(btf, i); + if (!t) { + ret = -EINVAL; + goto free; + } + if (!__btf_type_is_struct(t)) + continue; + + cond_resched(); + + for_each_member(j, t, member) { + if (btf_id_set_contains(&lkf.set, member->type)) + goto parse; + } + continue; + parse: + if (!tab) { + tab = kzalloc(offsetof(struct btf_struct_metas, types[1]), + GFP_KERNEL | __GFP_NOWARN); + if (!tab) + return ERR_PTR(-ENOMEM); + } else { + struct btf_struct_metas *new_tab; + + new_tab = krealloc(tab, offsetof(struct btf_struct_metas, types[tab->cnt + 1]), + GFP_KERNEL | __GFP_NOWARN); + if (!new_tab) { + ret = -ENOMEM; + goto free; + } + tab = new_tab; + } + type = &tab->types[tab->cnt]; + + type->btf_id = i; + record = btf_parse_fields(btf, t, BPF_SPIN_LOCK | BPF_LIST_HEAD | BPF_LIST_NODE, t->size); + if (IS_ERR_OR_NULL(record)) { + ret = PTR_ERR_OR_ZERO(record) ?: -EFAULT; + goto free; + } + foffs = btf_parse_field_offs(record); + if (WARN_ON_ONCE(IS_ERR_OR_NULL(foffs))) { + btf_record_free(record); + ret = -EFAULT; + goto free; + } + type->record = record; + type->field_offs = foffs; + tab->cnt++; + } + return tab; +free: + btf_struct_metas_free(tab); + return ERR_PTR(ret); +} + +struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id) +{ + struct btf_struct_metas *tab; + + BUILD_BUG_ON(offsetof(struct btf_struct_meta, btf_id) != 0); + tab = btf->struct_meta_tab; + if (!tab) + return NULL; + return bsearch(&btf_id, tab->types, tab->cnt, sizeof(tab->types[0]), btf_id_cmp_func); +} + static int btf_check_type_tags(struct btf_verifier_env *env, struct btf *btf, int start_id) { @@ -5191,6 +5336,7 @@ static int btf_check_type_tags(struct btf_verifier_env *env, static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size, u32 log_level, char __user *log_ubuf, u32 log_size) { + struct btf_struct_metas *struct_meta_tab; struct btf_verifier_env *env = NULL; struct bpf_verifier_log *log; struct btf *btf = NULL; @@ -5259,15 +5405,24 @@ static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size, if (err) goto errout; + struct_meta_tab = btf_parse_struct_metas(log, btf); + if (IS_ERR(struct_meta_tab)) { + err = PTR_ERR(struct_meta_tab); + goto errout; + } + btf->struct_meta_tab = struct_meta_tab; + if (log->level && bpf_verifier_log_full(log)) { err = -ENOSPC; - goto errout; + goto errout_meta; } btf_verifier_env_free(env); refcount_set(&btf->refcnt, 1); return btf; +errout_meta: + btf_free_struct_meta_tab(btf); errout: btf_verifier_env_free(env); if (btf) @@ -6028,6 +6183,28 @@ int btf_struct_access(struct bpf_verifier_log *log, u32 id = reg->btf_id; int err; + while (type_is_alloc(reg->type)) { + struct btf_struct_meta *meta; + struct btf_record *rec; + int i; + + meta = btf_find_struct_meta(btf, id); + if (!meta) + break; + rec = meta->record; + for (i = 0; i < rec->cnt; i++) { + struct btf_field *field = &rec->fields[i]; + u32 offset = field->offset; + if (off < offset + btf_field_type_size(field->type) && offset < off + size) { + bpf_log(log, + "direct access to %s is disallowed\n", + btf_field_type_name(field->type)); + return -EACCES; + } + } + break; + } + t = btf_type_by_id(btf, id); do { err = btf_struct_walk(log, btf, t, off, size, &id, &tmp_flag); @@ -7269,23 +7446,6 @@ bool btf_is_module(const struct btf *btf) return btf->kernel_btf && strcmp(btf->name, "vmlinux") != 0; } -static int btf_id_cmp_func(const void *a, const void *b) -{ - const int *pa = a, *pb = b; - - return *pa - *pb; -} - -bool btf_id_set_contains(const struct btf_id_set *set, u32 id) -{ - return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL; -} - -static void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) -{ - return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func); -} - enum { BTF_MODULE_F_LIVE = (1 << 0), }; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index fdbae52f463f..c96039a4e57f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -537,6 +537,7 @@ void btf_record_free(struct btf_record *rec) btf_put(rec->fields[i].kptr.btf); break; case BPF_LIST_HEAD: + case BPF_LIST_NODE: /* Nothing to release for bpf_list_head */ break; default: @@ -582,6 +583,7 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) } break; case BPF_LIST_HEAD: + case BPF_LIST_NODE: /* Nothing to acquire for bpf_list_head */ break; default: @@ -648,6 +650,8 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj) continue; bpf_list_head_free(field, field_ptr, obj + rec->spin_lock_off); break; + case BPF_LIST_NODE: + break; default: WARN_ON_ONCE(1); continue; From patchwork Fri Nov 11 19:32:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040739 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C2EEC4332F for ; Fri, 11 Nov 2022 19:33:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233870AbiKKTdp (ORCPT ); Fri, 11 Nov 2022 14:33:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233828AbiKKTdo (ORCPT ); Fri, 11 Nov 2022 14:33:44 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9373FCD0 for ; Fri, 11 Nov 2022 11:33:43 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id y13so5693119pfp.7 for ; Fri, 11 Nov 2022 11:33:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HJoDj6xrXZktYRxgvC1zzU8QhhM+z5Xg3gZS3dvgLB4=; b=enPQg/Zx6GAniJGD4re64/imcXgUX3pKLjBjOK8YotWPe65MKWjBtF+KLwp9jO45Lf SC2luCJTFU3BXq/DjkaNpevfCstfcl6gvzsdKvJtcja0Z9y6L82o0PzWx4HZ9SVz5/Z7 tGSivYpzerSAJ9T3tSFOW4HlPKMRUkeBbb7nCOFufb931Z3eaIRIHqzW7KauKspfou82 nNv4fMz3LusjSDxknYpyvbLYqHetwlXhV7UzEv7HLnA4wL5Ww872VWd2k2R2d5y2IpW1 LMcfeyLj2uZ+Z2auoJX8LI7Cl6QKZYSTyv6pfnWp0CFa4DpN/kG8hEMd7cnCSaX9+2G5 hOhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HJoDj6xrXZktYRxgvC1zzU8QhhM+z5Xg3gZS3dvgLB4=; b=Fh4fbC2bf664JMdKEM3/OsAzA18dQG+fuFUgntDdwGp8SA0tMP3FKg4BCKSnYkOxFu gXy0HlrXrVZWqMjFK6lUZ7yCOGG5REDUe12G+/gIPPw2UiS9e4uDl/kYTyaJYHsJ8WMO NayCIhVsKzJ2auTrPnutsZGjRzk/TJjfgp0hpDkILOLdx4FQULRXMu/sWB66pxRpyaJu KHsOFa4FqGyTLhHtJ+bd+aS1UZpJraZd3iiyoXq4iZ4CvLu9tlQ+QdIDOqwnx2w010X6 2frEwlZXQr2LT3MS0+TSpd8wZMJwA9rd8bwzKrei492xJGFDazFWyMEOYwncLcEz7AL4 WRxw== X-Gm-Message-State: ANoB5pl8+UwwNjsdqXTzNSYyzIvymPSk8Dx3We+9gutn3QTto2/r31n9 03yor3eU+xNVirULUxzBYAmhLX8aXrHt5Q== X-Google-Smtp-Source: AA0mqf5zZWtrJzgT0Iyn6JZC+TPeuSYtNsOKHs+Jh6X+yUap44YjsA2ZGuStkCuKKZ34dOFQquDBWA== X-Received: by 2002:a05:6a00:2487:b0:56c:afe:e8bf with SMTP id c7-20020a056a00248700b0056c0afee8bfmr4149122pfv.51.1668195222900; Fri, 11 Nov 2022 11:33:42 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id x12-20020aa79acc000000b0056ddd2b5e9bsm1941489pfp.41.2022.11.11.11.33.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:42 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 10/26] bpf: Verify ownership relationships for user BTF types Date: Sat, 12 Nov 2022 01:02:08 +0530 Message-Id: <20221111193224.876706-11-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5571; i=memxor@gmail.com; h=from:subject; bh=ixmYvmiPkP0VLegCTd0JqLgG+KhMlzR4IX850VHvYng=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoVisz7aF7uHg3UxnNyEgFFhl7I1JfrCtmfRj+ zl7HHfKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8Ryu6KD/ 4rW+dxhYQYDMVOFHGn3aBHRDFeBJ+SXJKDJRoY0gGVIBb9UDa89BYxLKu8yQlLGPejQTzKEKDSr3e1 Dxg9N5iI4GMr3qEUH9uDdNJbAXgcAjzEybOFk+dHCAV0Y86wN+h8aUTI+UuFMQoggakmDEqH11i0kJ lg83GXm8MzcUhQD8WlMEZn/O1Yf4JxUCZVr8vO8wL+BJ60viHNknkjci7xsBYmLFtmFlgr0HplJJat SMg1DJnMfEE62uKIpbc5fCez1L59g4tn4DF0Cxe+Vxsnjtcap3gxA/qUKYZktypdaBJ0Y6IFTadKpc wHFPROre67jzcvboWiyS1+LHZhBCV6MPajERYwcNZnEeUa0I0Vl8Ixyjjsh+gEX3TGF7Oag4Yg5maD TZ1jABjWotmhtRoCWG011jSprZHVkDLXft/BLm4rdDL1Pr2XYkGQ/BAVgPfy/D5kzL0uJpTM1DneTH NyPA6RxHTtXmOGH/COpI/CVfUC/xVmjb2fHzPBVrSULp/3FF4lPOgr250J55kCWEnblh4+VWGxaUbj cIGxi0DeV38jqeAJ7uGV0SOBYhrDEnwOWKF+wDFysCwLWgSrKCSlOcP/b8N2/pCz1HQIgAvYX1GJH3 8hQjl5Q2NaTpGBpwfECVisOOrRYuUjbJ7zaqOAk25rCvPKkBySmW78o12k8Q== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Ensure that there can be no ownership cycles among different types by way of having owning objects that can hold some other type as their element. For instance, a map value can only hold allocated objects, but these are allowed to have another bpf_list_head. To prevent unbounded recursion while freeing resources, elements of bpf_list_head in local kptrs can never have a bpf_list_head which are part of list in a map value. Later patches will verify this by having dedicated BTF selftests. Also, to make runtime destruction easier, once btf_struct_metas is fully populated, we can stash the metadata of the value type directly in the metadata of the list_head fields, as that allows easier access to the value type's layout to destruct it at runtime from the btf_field entry of the list head itself. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 1 + include/linux/btf.h | 1 + kernel/bpf/btf.c | 71 ++++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 4 +++ 4 files changed, 77 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 4cd3c9e6f50b..c88f75a68893 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -190,6 +190,7 @@ struct btf_field_list_head { struct btf *btf; u32 value_btf_id; u32 node_offset; + struct btf_record *value_rec; }; struct btf_field { diff --git a/include/linux/btf.h b/include/linux/btf.h index a01a8da20021..42d8f3730a8d 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -178,6 +178,7 @@ int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t); int btf_find_timer(const struct btf *btf, const struct btf_type *t); struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t, u32 field_mask, u32 value_size); +int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec); struct btf_field_offs *btf_parse_field_offs(struct btf_record *rec); bool btf_type_is_void(const struct btf_type *t); s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind); diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index fbcb846188e2..ef8710fab997 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3723,6 +3723,67 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type return ERR_PTR(ret); } +int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec) +{ + int i; + + /* There are two owning types, kptr_ref and bpf_list_head. The former + * only supports storing kernel types, which can never store references + * to program allocated local types, atleast not yet. Hence we only need + * to ensure that bpf_list_head ownership does not form cycles. + */ + if (IS_ERR_OR_NULL(rec) || !(rec->field_mask & BPF_LIST_HEAD)) + return 0; + for (i = 0; i < rec->cnt; i++) { + struct btf_struct_meta *meta; + u32 btf_id; + + if (!(rec->fields[i].type & BPF_LIST_HEAD)) + continue; + btf_id = rec->fields[i].list_head.value_btf_id; + meta = btf_find_struct_meta(btf, btf_id); + if (!meta) + return -EFAULT; + rec->fields[i].list_head.value_rec = meta->record; + + if (!(rec->field_mask & BPF_LIST_NODE)) + continue; + + /* We need to ensure ownership acyclicity among all types. The + * proper way to do it would be to topologically sort all BTF + * IDs based on the ownership edges, since there can be multiple + * bpf_list_head in a type. Instead, we use the following + * reasoning: + * + * - A type can only be owned by another type in user BTF if it + * has a bpf_list_node. + * - A type can only _own_ another type in user BTF if it has a + * bpf_list_head. + * + * We ensure that if a type has both bpf_list_head and + * bpf_list_node, its element types cannot be owning types. + * + * To ensure acyclicity: + * + * When A only has bpf_list_head, ownership chain can be: + * A -> B -> C + * Where: + * - B has both bpf_list_head and bpf_list_node. + * - C only has bpf_list_node. + * + * When A has both bpf_list_head and bpf_list_node, some other + * type already owns it in the BTF domain, hence it can not own + * another owning type through any of the bpf_list_head edges. + * A -> B + * Where: + * - B only has bpf_list_node. + */ + if (meta->record->field_mask & BPF_LIST_HEAD) + return -ELOOP; + } + return 0; +} + static int btf_field_offs_cmp(const void *_a, const void *_b, const void *priv) { const u32 a = *(const u32 *)_a; @@ -5412,6 +5473,16 @@ static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size, } btf->struct_meta_tab = struct_meta_tab; + if (struct_meta_tab) { + int i; + + for (i = 0; i < struct_meta_tab->cnt; i++) { + err = btf_check_and_fixup_fields(btf, struct_meta_tab->types[i].record); + if (err < 0) + goto errout_meta; + } + } + if (log->level && bpf_verifier_log_full(log)) { err = -ENOSPC; goto errout_meta; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index c96039a4e57f..4669020bb47d 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1044,6 +1044,10 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, } } + ret = btf_check_and_fixup_fields(btf, map->record); + if (ret < 0) + goto free_map_tab; + if (map->ops->map_check_btf) { ret = map->ops->map_check_btf(map, btf, key_type, value_type); if (ret < 0) From patchwork Fri Nov 11 19:32:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040740 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BA66C4332F for ; Fri, 11 Nov 2022 19:33:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233096AbiKKTdu (ORCPT ); Fri, 11 Nov 2022 14:33:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233878AbiKKTdt (ORCPT ); Fri, 11 Nov 2022 14:33:49 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EDCAFCD0 for ; Fri, 11 Nov 2022 11:33:48 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id p21so4969899plr.7 for ; Fri, 11 Nov 2022 11:33:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/EGc1pHleAs/SoS9r6ncZf8V6a43PizugzUT5QRzGdA=; b=M2fIwbt/z7fOdSEykeQXw9PwhD6zK0O884LGWvSR7l5Iry5XVljlQGV44Jag4tYE7o sOdKUTQ82ABXI9dBffRsUOgQhxg7DkL1avm8+oh8BSVddRr+rQP+8JfzBu1BzbaMVm0U NxKQhp0YMwafsrTgkXc/d293vKcXByfv1cLF91FdlCCUWStBGwHZE/287z+L82gDKDMz 0z+AVBWXQBGCoTYM+c8/oc+unYA6wtD/7xefXU6wgPaXxSWIjx8zimZLl6ftuW6PSnbp OATu5nhE84Z/I++Sxr1RvjW38rQpbpeUfQCO2bxxbU8zfSuo71haO55efC5dbtqZChsD fy7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/EGc1pHleAs/SoS9r6ncZf8V6a43PizugzUT5QRzGdA=; b=QqNCAf0/SPNwBgVmtVtAtsqjIxd+kt8buh/8urFB2bDJp9Fa+gzZ5xt9xMmsInSUNd egUCL1+EjB/vUWoyarO8Mv8yvy12i3bpV5iL/Ilpd7ik4ombpfwC3IkGmOSpUxSRmD/8 FhSvuqOVlozJD865gCpzS1/OMVdSnvyQ9CgTkZizoCdKujYMu/QdEbohJRhMrPTLEzXR LhQRsbnmFzLjMF9Icn8DmaaXd6dnDGpzc9PKlUwLOBYxdy5ETak3aShxaTZQVlmZM+JO whkEhh2/3P5jPFaBwaF4zGx7vp+KGeYOGscRWNKdpBeJXNu3wbz5T5RxW0OCU3c1PSEI lnDg== X-Gm-Message-State: ANoB5plMehCSm1D41mUQsR4GXc2/lySft+FACOPj+pouRoC6maoXkCmt /PbhGeb9H/j26TNbDQ0SJ8fD9Ch3QA7e8g== X-Google-Smtp-Source: AA0mqf63eyzXCOuAmemYVYy8DWm4rGxvTPgJfbBmpikLpA/N+HUqyJRCr/7n9tRhk0TUDgpOd4ghkQ== X-Received: by 2002:a17:902:848f:b0:180:a0e6:f81b with SMTP id c15-20020a170902848f00b00180a0e6f81bmr3859855plo.78.1668195227762; Fri, 11 Nov 2022 11:33:47 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id y18-20020a17090a8b1200b0020d39ffe987sm1932697pjn.50.2022.11.11.11.33.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:47 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 11/26] bpf: Allow locking bpf_spin_lock in allocated objects Date: Sat, 12 Nov 2022 01:02:09 +0530 Message-Id: <20221111193224.876706-12-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6759; i=memxor@gmail.com; h=from:subject; bh=D5yGHY+U0HGnEPj45nqZS0kfv31XewpVRUnqLpSLWo4=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoJ78ZLBua/KMV37yLSTn7miT8kAC4xE+7NMeL jdtwOoCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RyoqpD/ 0Q/uS4HCksgGaJzHyC7+1znzKgFsSiD+RWYBylfCJqt1ThQuoPhESHVerW72Wm8OJaCMC31Lgk9apY 4escegc083IsBSg7Iu0z9tyQ1cAPPvlQMANtUl6fu/WjH0dmrZR+G3UGNuGTJdICklX1qHCehBSGdN Rzf7hlNQ5m85JQzU6OegzrHqGvxfdplnuAOMaYnfzdq481Lyt3j9dgJNGugTvEbErP+m3iadNYxpDk K2h8JhqmpxryWXRRdO3uQoZAtOUjejkiK7bRBTF9g15KCTdjsdjC3k0j5ZkT7YdklCze3MqrWYylBg BoAkVeETy1R6TVIu80yq9vUAi9EAyaBuN5aupBlPNluhuttfT53Dg/7T3Ae0dD8/hCf3gpQSAlfqpw eR1S09+OXPYzazOTTnrqV/hf2rnOH7n2gKwaJXh5g1xUA21G0tqJvm8qWlLWC+2QCzyO6KZDPltrZS Q2TH+IJTGNwpoD1+pwbWqkp343EZ/oVaqrwQlAM5Pi824SeGhJkLyEKLk18lEkCkUu1MN0lXNUFBJf mWYi4kYmd/+wE24/EDmbEXr9gPkekHymiIOwHmgg8hWHIZwBcpsCa6izNC1w/A/kbNvWm2n3gNWfz6 MRj0MHi5WB+HSzvxGh0ecpv2Nj2L7bjkTTGihX98eYJ+ICqQUwuRAz2Lfu3A== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow locking a bpf_spin_lock in an allocated object, in addition to already support map value pointers. The handling is similar to that of map values, by just preserving the reg->id of PTR_TO_BTF_ID | MEM_ALLOC as well, and adjusting process_spin_lock to work with them and remember the id in verifier state. Refactor the existing process_spin_lock to work with PTR_TO_BTF_ID | MEM_ALLOC in addition to PTR_TO_MAP_VALUE. We need to update the reg_may_point_to_spin_lock which is used in mark_ptr_or_null_reg to preserve reg->id, that will be used in env->cur_state->active_spin_lock to remember the currently held spin lock. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/helpers.c | 2 ++ kernel/bpf/verifier.c | 70 ++++++++++++++++++++++++++++++++----------- 2 files changed, 55 insertions(+), 17 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 7bc71995f17c..5bc0b9f0f306 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -336,6 +336,7 @@ const struct bpf_func_proto bpf_spin_lock_proto = { .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_SPIN_LOCK, + .arg1_btf_id = BPF_PTR_POISON, }; static inline void __bpf_spin_unlock_irqrestore(struct bpf_spin_lock *lock) @@ -358,6 +359,7 @@ const struct bpf_func_proto bpf_spin_unlock_proto = { .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_SPIN_LOCK, + .arg1_btf_id = BPF_PTR_POISON, }; void copy_map_value_locked(struct bpf_map *map, void *dst, void *src, diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5cf14c1391a5..3831364af1ce 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -453,8 +453,16 @@ static bool reg_type_not_null(enum bpf_reg_type type) static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg) { - return reg->type == PTR_TO_MAP_VALUE && - btf_record_has_field(reg->map_ptr->record, BPF_SPIN_LOCK); + struct btf_record *rec = NULL; + + if (reg->type == PTR_TO_MAP_VALUE) { + rec = reg->map_ptr->record; + } else if (reg->type == (PTR_TO_BTF_ID | MEM_ALLOC)) { + struct btf_struct_meta *meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (meta) + rec = meta->record; + } + return btf_record_has_field(rec, BPF_SPIN_LOCK); } static bool type_is_rdonly_mem(u32 type) @@ -5583,8 +5591,10 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno]; struct bpf_verifier_state *cur = env->cur_state; bool is_const = tnum_is_const(reg->var_off); - struct bpf_map *map = reg->map_ptr; u64 val = reg->var_off.value; + struct bpf_map *map = NULL; + struct btf_record *rec; + struct btf *btf = NULL; if (!is_const) { verbose(env, @@ -5592,19 +5602,32 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, regno); return -EINVAL; } - if (!map->btf) { - verbose(env, - "map '%s' has to have BTF in order to use bpf_spin_lock\n", - map->name); - return -EINVAL; + if (reg->type == PTR_TO_MAP_VALUE) { + map = reg->map_ptr; + if (!map->btf) { + verbose(env, + "map '%s' has to have BTF in order to use bpf_spin_lock\n", + map->name); + return -EINVAL; + } + rec = map->record; + } else { + struct btf_struct_meta *meta; + + btf = reg->btf; + meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (meta) + rec = meta->record; } - if (!btf_record_has_field(map->record, BPF_SPIN_LOCK)) { - verbose(env, "map '%s' has no valid bpf_spin_lock\n", map->name); + + if (!btf_record_has_field(rec, BPF_SPIN_LOCK)) { + verbose(env, "%s '%s' has no valid bpf_spin_lock\n", map ? "map" : "local", + map ? map->name : "kptr"); return -EINVAL; } - if (map->record->spin_lock_off != val + reg->off) { + if (rec->spin_lock_off != val + reg->off) { verbose(env, "off %lld doesn't point to 'struct bpf_spin_lock' that is at %d\n", - val + reg->off, map->record->spin_lock_off); + val + reg->off, rec->spin_lock_off); return -EINVAL; } if (is_lock) { @@ -5810,13 +5833,19 @@ static const struct bpf_reg_types int_ptr_types = { }, }; +static const struct bpf_reg_types spin_lock_types = { + .types = { + PTR_TO_MAP_VALUE, + PTR_TO_BTF_ID | MEM_ALLOC, + } +}; + static const struct bpf_reg_types fullsock_types = { .types = { PTR_TO_SOCKET } }; static const struct bpf_reg_types scalar_types = { .types = { SCALAR_VALUE } }; static const struct bpf_reg_types context_types = { .types = { PTR_TO_CTX } }; static const struct bpf_reg_types ringbuf_mem_types = { .types = { PTR_TO_MEM | MEM_RINGBUF } }; static const struct bpf_reg_types const_map_ptr_types = { .types = { CONST_PTR_TO_MAP } }; static const struct bpf_reg_types btf_ptr_types = { .types = { PTR_TO_BTF_ID } }; -static const struct bpf_reg_types spin_lock_types = { .types = { PTR_TO_MAP_VALUE } }; static const struct bpf_reg_types percpu_btf_ptr_types = { .types = { PTR_TO_BTF_ID | MEM_PERCPU } }; static const struct bpf_reg_types func_ptr_types = { .types = { PTR_TO_FUNC } }; static const struct bpf_reg_types stack_ptr_types = { .types = { PTR_TO_STACK } }; @@ -5941,6 +5970,11 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, return -EACCES; } } + } else if (type_is_alloc(reg->type)) { + if (meta->func_id != BPF_FUNC_spin_lock && meta->func_id != BPF_FUNC_spin_unlock) { + verbose(env, "verifier internal error: unimplemented handling of MEM_ALLOC\n"); + return -EFAULT; + } } return 0; @@ -6057,7 +6091,8 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, goto skip_type_check; /* arg_btf_id and arg_size are in a union. */ - if (base_type(arg_type) == ARG_PTR_TO_BTF_ID) + if (base_type(arg_type) == ARG_PTR_TO_BTF_ID || + base_type(arg_type) == ARG_PTR_TO_SPIN_LOCK) arg_btf_id = fn->arg_btf_id[arg]; err = check_reg_type(env, regno, arg_type, arg_btf_id, meta); @@ -6675,9 +6710,10 @@ static bool check_btf_id_ok(const struct bpf_func_proto *fn) int i; for (i = 0; i < ARRAY_SIZE(fn->arg_type); i++) { - if (base_type(fn->arg_type[i]) == ARG_PTR_TO_BTF_ID && !fn->arg_btf_id[i]) - return false; - + if (base_type(fn->arg_type[i]) == ARG_PTR_TO_BTF_ID) + return !!fn->arg_btf_id[i]; + if (base_type(fn->arg_type[i]) == ARG_PTR_TO_SPIN_LOCK) + return fn->arg_btf_id[i] == BPF_PTR_POISON; if (base_type(fn->arg_type[i]) != ARG_PTR_TO_BTF_ID && fn->arg_btf_id[i] && /* arg_btf_id and arg_size are in a union. */ (base_type(fn->arg_type[i]) != ARG_PTR_TO_MEM || From patchwork Fri Nov 11 19:32:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040741 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FFE1C433FE for ; Fri, 11 Nov 2022 19:33:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233828AbiKKTdy (ORCPT ); Fri, 11 Nov 2022 14:33:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233898AbiKKTdx (ORCPT ); Fri, 11 Nov 2022 14:33:53 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A39EFD1B for ; Fri, 11 Nov 2022 11:33:52 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id j12so4973042plj.5 for ; Fri, 11 Nov 2022 11:33:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ruv1j+cZdobWg2qL9jWAKCo/MTt4/bsWMBmKgD+pRUI=; b=AxG4jxUjB+Ed9plYgaBG0b3OEDq+1B7E6tPLMv9OPYVDCno9tUIJE+akGJsrsb/meq N6NEBd1J/hsj1rWPZ7cD2jqg4xD8b5kSjY4mzk/Ba9cdnzgvNZGFOM4mk6deXUEW5XeN htx8aGne7Fzr0X/+zZizvspwHhWXu6uwneq2OIU1DczrtQW2gl34882WilLXEjuovz6q NFGBnArlc4ILzZo5ePwbEtrCAfhKmlPq6uqYX7s3sy5z1Q8ece+8DfdO0DTBaXRyEDec 2RUi0uJMEIzY3JcpoNQpF/AHW2wRiVf24ckje+gC+uEmWI991bsyZ8x2pxG1IRTSn/yf nDSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ruv1j+cZdobWg2qL9jWAKCo/MTt4/bsWMBmKgD+pRUI=; b=aMHrhub+kSqYbd4RksSBxBRFF2ypVygLifaq7rwaE7ogrbyO/bn3gJdzyA8BJ5f3en sTnGOpWW86u1Dv4Fy8md30Sani+v8N0EYrhZgfyP5St610F0sjF+PPI6YGpOhMYq+oH7 vUXxfjrfjLubKMDr8YNT+2SnWWeOOZZ7+ymGVRSEnUddGtMXC2DclR3TkegPTl1Ek43O duw6mIaVvH9IuAQ4sjOKTV1qkl+pq2KzYzBq/BPXqYgop04bF0vxgP/5Tr5fp2fgyU4O CvXGG3YkKN4k92BKUYxIHQvR6VmCHwg1un2bDcK5U0wCJReORgFcmkAvI4CkwN69ebOB hvYg== X-Gm-Message-State: ANoB5pnmi2YqcqMEXn0zVPCndtDqLMMEh8xbNbDC2KmoZ56FRCM5JmMO JhrQeKx9lz/yi/wCurHPaPWekSuLx7aW6Q== X-Google-Smtp-Source: AA0mqf6XygZDbvgqzx8wHb7otizhDh/3BiDe5r0CHyqV0bP6n6QY1WEDsoE4M2pbSx1PxKj7OFl4Lg== X-Received: by 2002:a17:902:9a84:b0:186:dd96:ce45 with SMTP id w4-20020a1709029a8400b00186dd96ce45mr3876133plp.73.1668195231349; Fri, 11 Nov 2022 11:33:51 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id q8-20020a170902dac800b0018725c2fc46sm2053778plx.303.2022.11.11.11.33.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:51 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 12/26] bpf: Allow locking bpf_spin_lock global variables Date: Sat, 12 Nov 2022 01:02:10 +0530 Message-Id: <20221111193224.876706-13-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6954; i=memxor@gmail.com; h=from:subject; bh=Tbs9M3TN39iyfYHv7AxXD1oQurmlF6bK44azbWHrVsw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoIFIyshcZo+KB+O/XZEVYBEvH2TmA3BZEv985 FpmVvBGJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8Ryg9lD/ 9FhRw6RGpuqMVHa61/3IzGFdVL0uVqanJ8aUUH7mmK4pJ/p9iNeNlZZRSMayMKOH4M7mlWzVf3ebol EUDoMMmbVL28PF62Ri8yDToIVJt8zCWQgdiRoIXaa0SAPy/lIO7fqlbr6IC8jZWZoxzsMEyKOrtTjl zvG1JnyujsMELbIORDXp/ujBYPXTnt5pWy9blzRBUftcJIlp6CydNSCkpmMh+gP42NOEt7tKIom+Xb xN9HyZucbnmncv+YvBHxqeFP6wfTDKlMebDase2vLG7DuYd2pgv9eG2KA5vWOSw1Rs3ts4Ztlmb5De 19baOkrMbNbCBcVZkdKTGJnA9UQD0bzBNrL2vh33TcKV8JlhVJatztFsI6f9TiGCJOQDxaiwArRR7A dux0l2+pSkkfolBsvJrxI2Ab1u38qwedvuMZ8EAFuOYTS2ySlEWpRuLLfmPQ6cPCbbH1QXJlqP0dF9 Xq66GxueF4qP1UzNjprieZDTK5k+M5tvlCqXzYalvM4PTw8efTphdeSWcWEqPj5abaEKvl2HLo1vBw HBuqKOJ3QHjG62HnC/mla+zy4OX/1/S7BYnVNJKfvBGZXhGP5jNks15Q9U/R16ds0bC6mqQ9vaqtvh 834j+PRwSix/mHtJdYTSqqLJujPiYUAbWdRyk4QAPStHA4zr+hGGgmf8Kclg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Global variables reside in maps accessible using direct_value_addr callbacks, so giving each load instruction's rewrite a unique reg->id disallows us from holding locks which are global. The reason for preserving reg->id as a unique value for registers that may point to spin lock is that two separate lookups are treated as two separate memory regions, and any possible aliasing is ignored for the purposes of spin lock correctness. This is not great especially for the global variable case, which are served from maps that have max_entries == 1, i.e. they always lead to map values pointing into the same map value. So refactor the active_spin_lock into a 'active_lock' structure which represents the lock identity, and instead of the reg->id, remember two fields, a pointer and the reg->id. The pointer will store reg->map_ptr or reg->btf. It's only necessary to distinguish for the id == 0 case of global variables, but always setting the pointer to a non-NULL value and using the pointer to check whether the lock is held simplifies code in the verifier. This is generic enough to allow it for global variables, map lookups, and allocated objects at the same time. Note that while whether a lock is held can be answered by just comparing active_lock.ptr to NULL, to determine whether the register is pointing to the same held lock requires comparing _both_ ptr and id. Finally, as a result of this refactoring, pseudo load instructions are not given a unique reg->id, as they are doing lookup for the same map value (max_entries is never greater than 1). Essentially, we consider that the tuple of (ptr, id) will always be unique for any kind of argument to bpf_spin_{lock,unlock}. Note that this can be extended in the future to also remember offset used for locking, so that we can introduce multiple bpf_spin_lock fields in the same allocation. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf_verifier.h | 10 ++++++++- kernel/bpf/verifier.c | 41 ++++++++++++++++++++++++------------ 2 files changed, 37 insertions(+), 14 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 1a32baa78ce2..fa738abea267 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -323,7 +323,15 @@ struct bpf_verifier_state { u32 branches; u32 insn_idx; u32 curframe; - u32 active_spin_lock; + struct { + /* This can either be reg->map_ptr or reg->btf, but it is only + * used to check whether the lock is held or not by comparing to + * NULL. + */ + void *ptr; + /* This will be reg->id */ + u32 id; + } active_lock; bool speculative; /* first and last insn idx of this verifier state */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 3831364af1ce..bd13104b018a 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1210,7 +1210,8 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state, } dst_state->speculative = src->speculative; dst_state->curframe = src->curframe; - dst_state->active_spin_lock = src->active_spin_lock; + dst_state->active_lock.ptr = src->active_lock.ptr; + dst_state->active_lock.id = src->active_lock.id; dst_state->branches = src->branches; dst_state->parent = src->parent; dst_state->first_insn_idx = src->first_insn_idx; @@ -5582,7 +5583,7 @@ int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state * Since only one bpf_spin_lock is allowed the checks are simpler than * reg_is_refcounted() logic. The verifier needs to remember only * one spin_lock instead of array of acquired_refs. - * cur_state->active_spin_lock remembers which map value element got locked + * cur_state->active_lock remembers which map value element got locked * and clears it after bpf_spin_unlock. */ static int process_spin_lock(struct bpf_verifier_env *env, int regno, @@ -5631,22 +5632,35 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, return -EINVAL; } if (is_lock) { - if (cur->active_spin_lock) { + if (cur->active_lock.ptr) { verbose(env, "Locking two bpf_spin_locks are not allowed\n"); return -EINVAL; } - cur->active_spin_lock = reg->id; + if (map) + cur->active_lock.ptr = map; + else + cur->active_lock.ptr = btf; + cur->active_lock.id = reg->id; } else { - if (!cur->active_spin_lock) { + void *ptr; + + if (map) + ptr = map; + else + ptr = btf; + + if (!cur->active_lock.ptr) { verbose(env, "bpf_spin_unlock without taking a lock\n"); return -EINVAL; } - if (cur->active_spin_lock != reg->id) { + if (cur->active_lock.ptr != ptr || + cur->active_lock.id != reg->id) { verbose(env, "bpf_spin_unlock of different lock\n"); return -EINVAL; } - cur->active_spin_lock = 0; + cur->active_lock.ptr = NULL; + cur->active_lock.id = 0; } return 0; } @@ -10573,8 +10587,8 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn) insn->src_reg == BPF_PSEUDO_MAP_IDX_VALUE) { dst_reg->type = PTR_TO_MAP_VALUE; dst_reg->off = aux->map_off; - if (btf_record_has_field(map->record, BPF_SPIN_LOCK)) - dst_reg->id = ++env->id_gen; + WARN_ON_ONCE(map->max_entries != 1); + /* We want reg->id to be same (0) as map_value is not distinct */ } else if (insn->src_reg == BPF_PSEUDO_MAP_FD || insn->src_reg == BPF_PSEUDO_MAP_IDX) { dst_reg->type = CONST_PTR_TO_MAP; @@ -10652,7 +10666,7 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn) return err; } - if (env->cur_state->active_spin_lock) { + if (env->cur_state->active_lock.ptr) { verbose(env, "BPF_LD_[ABS|IND] cannot be used inside bpf_spin_lock-ed region\n"); return -EINVAL; } @@ -11918,7 +11932,8 @@ static bool states_equal(struct bpf_verifier_env *env, if (old->speculative && !cur->speculative) return false; - if (old->active_spin_lock != cur->active_spin_lock) + if (old->active_lock.ptr != cur->active_lock.ptr || + old->active_lock.id != cur->active_lock.id) return false; /* for states to be equal callsites have to be the same @@ -12563,7 +12578,7 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } - if (env->cur_state->active_spin_lock && + if (env->cur_state->active_lock.ptr && (insn->src_reg == BPF_PSEUDO_CALL || insn->imm != BPF_FUNC_spin_unlock)) { verbose(env, "function calls are not allowed while holding a lock\n"); @@ -12600,7 +12615,7 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } - if (env->cur_state->active_spin_lock) { + if (env->cur_state->active_lock.ptr) { verbose(env, "bpf_spin_unlock is missing\n"); return -EINVAL; } From patchwork Fri Nov 11 19:32:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040742 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C4EEC433FE for ; Fri, 11 Nov 2022 19:34:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233938AbiKKTd6 (ORCPT ); Fri, 11 Nov 2022 14:33:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233878AbiKKTd6 (ORCPT ); Fri, 11 Nov 2022 14:33:58 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E3C91D66E for ; Fri, 11 Nov 2022 11:33:57 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id y4so4987344plb.2 for ; Fri, 11 Nov 2022 11:33:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hYvHeXAuVygr4ZcQHncrz4UIBOo5UsmMrsEx+4QLuhc=; b=JYbPeRSHU/80WEiCO32CR3dSg8Ez470M5OHfO68M3G4w51ogjFbpyLXyVQFAdeswwH YIWBRAg4ZK7EYx1PFaXBrAwop1aAOoG3CMuC4ixH6MSrf0SHdbxh8uSY2w3HIWm8onjX CYdL1VZoJ7LbUmDOdlwWiM1cG70UwpYa4Tw9pthyh3BcjlhcDV0V4RE9R2TXJ1w3ENU8 fw3dZ6fnk/f+3DoRNUvb4SFoOf56w8FypJi/glAW6bY3FlKqbafel1x5BiAY6wCc+TKf zcttDPCPOp7rt7isoptfQgCXvvf2ZzIuv7bA9bqREy8QWdGqJCrN6aqqhO6ljk7cK3rn BHaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hYvHeXAuVygr4ZcQHncrz4UIBOo5UsmMrsEx+4QLuhc=; b=WJw5OTQazweV/oxcw7CdT2EyntkFBw6C9gn8T9UNv1Ai9he3P2ZeH2UCCyjC5rqFP+ LsWbblsED0OP9+Al9sGZ5EjimNVMWoas+THLILQGf74am5ZkqHyKdBQEZNorpZ1QWw/n YeKcsAD4FxndWs6Y7XbfO4AV8VLS0agZmvgfYesJRY2W5skS0ui3Iqdd2xZqfQajWA5E faXI2CDDDjeRtaQGBvIYI86eFMwJ1xyF1I4sHcCkZnjF+KjqPElOvp4XDB2ZJEb6VSCc HxarnM1VXGCoyPBDt0rFgFlTCCjOteBjV/1i+UaX2CIDpwGhjl5H/KWlbVBiV45NPmH/ IXMw== X-Gm-Message-State: ANoB5pmibsUz7u4mkQ3hYAtlexeunWVtnWaaD68e5O0d69gUTvan32j4 0m/W2n3zuw6e4TOuiCBziWjE50q7lLTfuQ== X-Google-Smtp-Source: AA0mqf764e53X/4OxLqDS0jvroyN6qzuwZZuHVrDAkgN0ixFZB0KJarF2XAVxkod5YTOeIT4g9i4aQ== X-Received: by 2002:a17:903:25cd:b0:188:4f86:e4d5 with SMTP id jc13-20020a17090325cd00b001884f86e4d5mr3793831plb.16.1668195236684; Fri, 11 Nov 2022 11:33:56 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id t13-20020a1709027fcd00b001871f532cf9sm2045692plb.286.2022.11.11.11.33.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:33:56 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 13/26] bpf: Allow locking bpf_spin_lock in inner map values Date: Sat, 12 Nov 2022 01:02:11 +0530 Message-Id: <20221111193224.876706-14-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1237; i=memxor@gmail.com; h=from:subject; bh=vEeT2/iSWZHC3KMjxfZP18u7ktP/IseF8QZTy1MJn24=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoHwor9e+vDGcaQsSs/hJmvUZYJzEVob7xslb7 HsZAtuOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RyvS+D/ 9fZJv4CFdTzpCSSZj8RgMhIkuNs6Yifw54whIwBJg6sLHesXesBm/lWQ1/yx1bsGFdXsMQtxWFet/d wHQoyO6ZzfLye4YV1fFJvZFyyx+w194n0LY3Rkqiut7+NL/UBQw2LufhpLwPxI7KRNqfNBLmdTe49D iDNHgtRR9hUPO61l5zjsg0U4pXIjtldVFd2AXJJHkO6Gcos48l14wRuQ9Ei5BmTaFUqhoyA1MubAgx SXRZB7vCum63NV/Cf8KbmP1wUgF6oQa2wzdwKdj/Olng3/CRTjXqHFKrdYoL8QKgwFLxQN9okJwo4B G6IdeO+6AUQlqPYRkqfkbtmQtSFk0Xth0YbKyZAgyoacZA1mRzr00atRiX7K1bO48HGELHfn6ZYcRb UElXRlFE6SxFEcyKUlaSzoYR/j8RRuGXUBl62dNXDHLCXab0HLPoW1lfsuRyVSGBoVTjwlLvqwMtSR EJYtA78wzGmCAE91OYtNCHLS/sQJok4ozeQhT1gqu28cDjRmJ2Sge0f+QYCWE+97682hN+Ec1cpDsY 5qXsUqs2y/NyfGueb+RcaC93l5CBpRQ6ovqjC7vAf1/qHhhohSxLpkMS80PudG/fEO/G91nG4kmEuC SbPiVYZCE8/zm3/wMpXK7kb2271PHMY2CviMwHRTxEkS1ynA0tRu4HQ/LUFg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net There is no need to restrict users from locking bpf_spin_lock in map values of inner maps. Each inner map lookup gets a unique reg->id assigned to the returned PTR_TO_MAP_VALUE which will be preserved after the NULL check. Distinct lookups into different inner map get unique IDs, and distinct lookups into same inner map also get unique IDs. Hence, lift the restriction by removing the check return -ENOTSUPP in map_in_map.c. Later commits will add comprehensive test cases to ensure that invalid cases are rejected. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/map_in_map.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/kernel/bpf/map_in_map.c b/kernel/bpf/map_in_map.c index 8ca0cca39d49..f31893a123a2 100644 --- a/kernel/bpf/map_in_map.c +++ b/kernel/bpf/map_in_map.c @@ -29,11 +29,6 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) return ERR_PTR(-ENOTSUPP); } - if (btf_record_has_field(inner_map->record, BPF_SPIN_LOCK)) { - fdput(f); - return ERR_PTR(-ENOTSUPP); - } - inner_map_meta_size = sizeof(*inner_map_meta); /* In some cases verifier needs to access beyond just base map. */ if (inner_map->ops == &array_map_ops) From patchwork Fri Nov 11 19:32:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040743 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 461CFC433FE for ; Fri, 11 Nov 2022 19:34:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233907AbiKKTeF (ORCPT ); Fri, 11 Nov 2022 14:34:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233878AbiKKTeD (ORCPT ); Fri, 11 Nov 2022 14:34:03 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D66002ED77 for ; Fri, 11 Nov 2022 11:34:01 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id gw22so5226535pjb.3 for ; Fri, 11 Nov 2022 11:34:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e7/XOu0Wx76DhsKmvgQemeJvNpKs35b4i8dv/j+6qWQ=; b=Y22vY+ZUbWV3aMvI3KEu32HVsSrlUKWFEHe6WzBbdzkFBMSgJgyO9ppIOoXs2nMSqk 35hzONkUtJFqFEh9uzNtRQ7/2rSd2jo7TwAXBNsfZR40NmvjeKCoI/vbVD85jwrL+ZYJ S/SNEMBWgr7estklGM4p6h8sCV/nxJ38QMpA4PvRM0rtdtEMD7SqD1EA+w5hXVTJa11P 2sDn6lwrbpJLUxHDp8KxITKaR5lCnya00wYYwahxj20EGvWOeX9QMmTNLADoy+2YqlYV mSOmN9iBFumFevQaGIdX65/6G+rGoy0WCqzae7UyAKHxwS470rqdESd6UYz+eiTWIAj+ NebQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e7/XOu0Wx76DhsKmvgQemeJvNpKs35b4i8dv/j+6qWQ=; b=YqzzqARk6idmQ+AdRCd5wMyl7rIN62D/z/lRDmbl/MDs/6AFRKSj6gsXdqhZavx+dP LF/MvZceu9Ul4KySKCaOUCpRcDdq+uRtA+vrXDo7eSYW1VkoVql3Fbphquv+kdiOiRUU NHPQhB/s8wu699ldshe0zQCSij5y7RQkdSJGkQxsALEbM6aJJr5uIcSxLDnqb2vwq9zq wjyCzBC7Dq11kLnDoYu5puYoreQ8LFsqWxKDIXtX83enHo9oXOdVde6l+p8+kOqJTsuy ujTpfbwReIL13NQlJGADWN1I3LaKnk7m1s+BgoS1oHncDfIwzCzhZakT5iqVAkxLH1Wx MuWw== X-Gm-Message-State: ANoB5pkGyWDDvjWho2x/41SPG04dhW9zhOpIUVhIQanibdZUFp545DWc h6N63zmEcsdAVIQYdGFMUQ5Xre9WlkuMKA== X-Google-Smtp-Source: AA0mqf7XxVP+3yP+L+M0742LQ805V4ZsWNQTRvtrN3q/NdEsvcndRXQLX6E/TZgSDF+2Sad3BC/mkw== X-Received: by 2002:a17:902:ab09:b0:179:b756:5b60 with SMTP id ik9-20020a170902ab0900b00179b7565b60mr4056794plb.22.1668195241114; Fri, 11 Nov 2022 11:34:01 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id jm5-20020a17090304c500b00186b758c9fasm2084831plb.33.2022.11.11.11.33.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:00 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 14/26] bpf: Rewrite kfunc argument handling Date: Sat, 12 Nov 2022 01:02:12 +0530 Message-Id: <20221111193224.876706-15-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=27126; i=memxor@gmail.com; h=from:subject; bh=+YK6mSQg9T3uUV93iK70SMNKLh+h9v0Ex9GE1XkHM8s=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoQVrNoH+adfg+I6aWGUHrzu60DbUYj9OxH5ia ttLpqPKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RysQYD/ 9avOlbBNCIbM1WSdGAfISz3lwRazKk8byjSX+IuClsUsXMrwUBJvHaKT5eiCrAsAXtgvUU85z5oTPP 4bcleCEzMB8MfaorIRSg3Igo9zYBGHUYdzyRmBEhqzvB9RTmyrVdQDpQRsXNBcuFc8jz6Y5U3te79c /lUF+W5kH13fTSXEQGWP1lThY5pzIE94ckY454E9R//dDqQppUviFcjOuFInjTytJQ+lh71w3R0JGH kIPSAECO6Ke1JHC+buuTRZ2L6H1N9DqtObkeDXGIBMoEyenwNXNhtOgxXDIBrp3eNCdJxo2abSLLRH h2OmLFqq8dLucznvBaYjtaAlyvRl7JgmvEUqKm6VLKtMUpG/jPhCR1D4S5nDszjuPONAtYR+TYnqhr +yg3rzEhFv31jd+Lp3va32dtUDahGablp7xQ+ig1bxbjERn+XwX/GJi0xqNond0gsEDCdf6LnOvlSN aMr2DUqshuQ6rHxsCQ0X4Ev0pW2PbCEQsrNjVN+6lBAvgBSx3OIRTA7UItdUw43JOtCYXyNH3iz3C4 T9vtRUoDQF2VM/kRkh9U5HXOZgATbA/UYYw1dw0Yz1EddUg6cfl/oG4dnJxdTW6+Dj+iVnnlysQ/1n KG2fJOs9KTmBkmYINNgGi+LqkSTR0mIGhuHEZ4sEJpztgSfyqVqgX50FR3OQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net As we continue to add more features, argument types, kfunc flags, and different extensions to kfuncs, the code to verify the correctness of the kfunc prototype wrt the passed in registers has become ad-hoc and ugly to read. To make life easier, and make a very clear split between different stages of argument processing, move all the code into verifier.c and refactor into easier to read helpers and functions. This also makes sharing code within the verifier easier with kfunc argument processing. This will be more and more useful in later patches as we are now moving to implement very core BPF helpers as kfuncs, to keep them experimental before baking into UAPI. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 31 +- kernel/bpf/btf.c | 16 +- kernel/bpf/verifier.c | 547 +++++++++++++++++- .../bpf/prog_tests/kfunc_dynptr_param.c | 2 +- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../selftests/bpf/verifier/ref_tracking.c | 4 +- 6 files changed, 569 insertions(+), 33 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index 42d8f3730a8d..d5b26380a60f 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -338,6 +338,16 @@ static inline bool btf_type_is_struct(const struct btf_type *t) return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION; } +static inline bool __btf_type_is_struct(const struct btf_type *t) +{ + return BTF_INFO_KIND(t->info) == BTF_KIND_STRUCT; +} + +static inline bool btf_type_is_array(const struct btf_type *t) +{ + return BTF_INFO_KIND(t->info) == BTF_KIND_ARRAY; +} + static inline u16 btf_type_vlen(const struct btf_type *t) { return BTF_INFO_VLEN(t->info); @@ -439,9 +449,10 @@ static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func); } -#ifdef CONFIG_BPF_SYSCALL struct bpf_prog; +struct bpf_verifier_log; +#ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); const char *btf_name_by_offset(const struct btf *btf, u32 offset); struct btf *btf_parse_vmlinux(void); @@ -455,6 +466,12 @@ s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id); int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, struct module *owner); struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id); +const struct btf_member * +btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, + const struct btf_type *t, enum bpf_prog_type prog_type, + int arg); +bool btf_types_are_same(const struct btf *btf1, u32 id1, + const struct btf *btf2, u32 id2); #else static inline const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id) @@ -490,6 +507,18 @@ static inline struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf { return NULL; } +static inline const struct btf_member * +btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, + const struct btf_type *t, enum bpf_prog_type prog_type, + int arg) +{ + return NULL; +} +static inline bool btf_types_are_same(const struct btf *btf1, u32 id1, + const struct btf *btf2, u32 id2) +{ + return false; +} #endif static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type *t) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index ef8710fab997..c99c254c3e85 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -478,16 +478,6 @@ static bool btf_type_nosize_or_null(const struct btf_type *t) return !t || btf_type_nosize(t); } -static bool __btf_type_is_struct(const struct btf_type *t) -{ - return BTF_INFO_KIND(t->info) == BTF_KIND_STRUCT; -} - -static bool btf_type_is_array(const struct btf_type *t) -{ - return BTF_INFO_KIND(t->info) == BTF_KIND_ARRAY; -} - static bool btf_type_is_datasec(const struct btf_type *t) { return BTF_INFO_KIND(t->info) == BTF_KIND_DATASEC; @@ -5535,7 +5525,7 @@ static u8 bpf_ctx_convert_map[] = { #undef BPF_MAP_TYPE #undef BPF_LINK_TYPE -static const struct btf_member * +const struct btf_member * btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, const struct btf_type *t, enum bpf_prog_type prog_type, int arg) @@ -6321,8 +6311,8 @@ int btf_struct_access(struct bpf_verifier_log *log, * end up with two different module BTFs, but IDs point to the common type in * vmlinux BTF. */ -static bool btf_types_are_same(const struct btf *btf1, u32 id1, - const struct btf *btf2, u32 id2) +bool btf_types_are_same(const struct btf *btf1, u32 id1, + const struct btf *btf2, u32 id2) { if (id1 != id2) return false; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index bd13104b018a..075f0d9207c7 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7850,19 +7850,523 @@ static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, } } +struct bpf_kfunc_call_arg_meta { + /* In parameters */ + struct btf *btf; + u32 func_id; + u32 kfunc_flags; + const struct btf_type *func_proto; + const char *func_name; + /* Out parameters */ + u32 ref_obj_id; + u8 release_regno; + bool r0_rdonly; + u64 r0_size; +}; + +static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_ACQUIRE; +} + +static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_RET_NULL; +} + +static bool is_kfunc_release(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_RELEASE; +} + +static bool is_kfunc_trusted_args(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_TRUSTED_ARGS; +} + +static bool is_kfunc_sleepable(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_SLEEPABLE; +} + +static bool is_kfunc_destructive(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_DESTRUCTIVE; +} + +static bool is_kfunc_arg_kptr_get(struct bpf_kfunc_call_arg_meta *meta, int arg) +{ + return arg == 0 && (meta->kfunc_flags & KF_KPTR_GET); +} + +static bool is_kfunc_arg_mem_size(const struct btf *btf, + const struct btf_param *arg, + const struct bpf_reg_state *reg) +{ + int len, sfx_len = sizeof("__sz") - 1; + const struct btf_type *t; + const char *param_name; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) + return false; + + /* In the future, this can be ported to use BTF tagging */ + param_name = btf_name_by_offset(btf, arg->name_off); + if (str_is_empty(param_name)) + return false; + len = strlen(param_name); + if (len < sfx_len) + return false; + param_name += len - sfx_len; + if (strncmp(param_name, "__sz", sfx_len)) + return false; + + return true; +} + +static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, + const struct btf_param *arg, + const struct bpf_reg_state *reg, + const char *name) +{ + int len, target_len = strlen(name); + const struct btf_type *t; + const char *param_name; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) + return false; + + param_name = btf_name_by_offset(btf, arg->name_off); + if (str_is_empty(param_name)) + return false; + len = strlen(param_name); + if (len != target_len) + return false; + if (strcmp(param_name, name)) + return false; + + return true; +} + +enum { + KF_ARG_DYNPTR_ID, +}; + +BTF_ID_LIST(kf_arg_btf_ids) +BTF_ID(struct, bpf_dynptr_kern) + +static bool is_kfunc_arg_dynptr(const struct btf *btf, + const struct btf_param *arg) +{ + const struct btf_type *t; + u32 res_id; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!t) + return false; + if (!btf_type_is_ptr(t)) + return false; + t = btf_type_skip_modifiers(btf, t->type, &res_id); + if (!t) + return false; + return btf_types_are_same(btf, res_id, btf_vmlinux, kf_arg_btf_ids[KF_ARG_DYNPTR_ID]); +} + +/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ +static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, + const struct btf *btf, + const struct btf_type *t, int rec) +{ + const struct btf_type *member_type; + const struct btf_member *member; + u32 i; + + if (!btf_type_is_struct(t)) + return false; + + for_each_member(i, t, member) { + const struct btf_array *array; + + member_type = btf_type_skip_modifiers(btf, member->type, NULL); + if (btf_type_is_struct(member_type)) { + if (rec >= 3) { + verbose(env, "max struct nesting depth exceeded\n"); + return false; + } + if (!__btf_type_is_scalar_struct(env, btf, member_type, rec + 1)) + return false; + continue; + } + if (btf_type_is_array(member_type)) { + array = btf_array(member_type); + if (!array->nelems) + return false; + member_type = btf_type_skip_modifiers(btf, array->type, NULL); + if (!btf_type_is_scalar(member_type)) + return false; + continue; + } + if (!btf_type_is_scalar(member_type)) + return false; + } + return true; +} + + +static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { +#ifdef CONFIG_NET + [PTR_TO_SOCKET] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK], + [PTR_TO_SOCK_COMMON] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK_COMMON], + [PTR_TO_TCP_SOCK] = &btf_sock_ids[BTF_SOCK_TYPE_TCP], +#endif +}; + +enum kfunc_ptr_arg_type { + KF_ARG_PTR_TO_CTX, + KF_ARG_PTR_TO_KPTR_STRONG, /* PTR_TO_KPTR but type specific */ + KF_ARG_PTR_TO_DYNPTR, + KF_ARG_PTR_TO_BTF_ID, /* Also covers reg2btf_ids conversions */ + KF_ARG_PTR_TO_MEM, + KF_ARG_PTR_TO_MEM_SIZE, /* Size derived from next argument, skip it */ +}; + +static enum kfunc_ptr_arg_type +get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, + const struct btf_type *t, const struct btf_type *ref_t, + const char *ref_tname, const struct btf_param *args, + int argno, int nargs) +{ + u32 regno = argno + 1; + struct bpf_reg_state *regs = cur_regs(env); + struct bpf_reg_state *reg = ®s[regno]; + bool arg_mem_size = false; + + /* In this function, we verify the kfunc's BTF as per the argument type, + * leaving the rest of the verification with respect to the register + * type to our caller. When a set of conditions hold in the BTF type of + * arguments, we resolve it to a known kfunc_ptr_arg_type. + */ + if (btf_get_prog_ctx_type(&env->log, meta->btf, t, resolve_prog_type(env->prog), argno)) + return KF_ARG_PTR_TO_CTX; + + if (is_kfunc_arg_kptr_get(meta, argno)) { + if (!btf_type_is_ptr(ref_t)) { + verbose(env, "arg#0 BTF type must be a double pointer for kptr_get kfunc\n"); + return -EINVAL; + } + ref_t = btf_type_by_id(meta->btf, ref_t->type); + ref_tname = btf_name_by_offset(meta->btf, ref_t->name_off); + if (!btf_type_is_struct(ref_t)) { + verbose(env, "kernel function %s args#0 pointer type %s %s is not supported\n", + meta->func_name, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + return KF_ARG_PTR_TO_KPTR_STRONG; + } + + if (is_kfunc_arg_dynptr(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_DYNPTR; + + if ((base_type(reg->type) == PTR_TO_BTF_ID || reg2btf_ids[base_type(reg->type)])) { + if (!btf_type_is_struct(ref_t)) { + verbose(env, "kernel function %s args#%d pointer type %s %s is not supported\n", + meta->func_name, argno, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + return KF_ARG_PTR_TO_BTF_ID; + } + + if (argno + 1 < nargs && is_kfunc_arg_mem_size(meta->btf, &args[argno + 1], ®s[regno + 1])) + arg_mem_size = true; + + /* This is the catch all argument type of register types supported by + * check_helper_mem_access. However, we only allow when argument type is + * pointer to scalar, or struct composed (recursively) of scalars. When + * arg_mem_size is true, the pointer can be void *. + */ + if (!btf_type_is_scalar(ref_t) && !__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0) && + (arg_mem_size ? !btf_type_is_void(ref_t) : 1)) { + verbose(env, "arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n", + argno, btf_type_str(ref_t), ref_tname, arg_mem_size ? "void, " : ""); + return -EINVAL; + } + return arg_mem_size ? KF_ARG_PTR_TO_MEM_SIZE : KF_ARG_PTR_TO_MEM; +} + +static int process_kf_arg_ptr_to_btf_id(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, + const struct btf_type *ref_t, + const char *ref_tname, u32 ref_id, + struct bpf_kfunc_call_arg_meta *meta, + int argno) +{ + const struct btf_type *reg_ref_t; + bool strict_type_match = false; + const struct btf *reg_btf; + const char *reg_ref_tname; + u32 reg_ref_id; + + if (reg->type == PTR_TO_BTF_ID) { + reg_btf = reg->btf; + reg_ref_id = reg->btf_id; + } else { + reg_btf = btf_vmlinux; + reg_ref_id = *reg2btf_ids[base_type(reg->type)]; + } + + if (is_kfunc_trusted_args(meta) || (is_kfunc_release(meta) && reg->ref_obj_id)) + strict_type_match = true; + + reg_ref_t = btf_type_skip_modifiers(reg_btf, reg_ref_id, ®_ref_id); + reg_ref_tname = btf_name_by_offset(reg_btf, reg_ref_t->name_off); + if (!btf_struct_ids_match(&env->log, reg_btf, reg_ref_id, reg->off, meta->btf, ref_id, strict_type_match)) { + verbose(env, "kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n", + meta->func_name, argno, btf_type_str(ref_t), ref_tname, argno + 1, + btf_type_str(reg_ref_t), reg_ref_tname); + return -EINVAL; + } + return 0; +} + +static int process_kf_arg_ptr_to_kptr_strong(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, + const struct btf_type *ref_t, + const char *ref_tname, + struct bpf_kfunc_call_arg_meta *meta, + int argno) +{ + struct btf_field *kptr_field; + + /* check_func_arg_reg_off allows var_off for + * PTR_TO_MAP_VALUE, but we need fixed offset to find + * off_desc. + */ + if (!tnum_is_const(reg->var_off)) { + verbose(env, "arg#0 must have constant offset\n"); + return -EINVAL; + } + + kptr_field = btf_record_find(reg->map_ptr->record, reg->off + reg->var_off.value, BPF_KPTR); + if (!kptr_field || kptr_field->type != BPF_KPTR_REF) { + verbose(env, "arg#0 no referenced kptr at map value offset=%llu\n", + reg->off + reg->var_off.value); + return -EINVAL; + } + + if (!btf_struct_ids_match(&env->log, meta->btf, ref_t->type, 0, kptr_field->kptr.btf, + kptr_field->kptr.btf_id, true)) { + verbose(env, "kernel function %s args#%d expected pointer to %s %s\n", + meta->func_name, argno, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + return 0; +} + +static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta) +{ + const char *func_name = meta->func_name, *ref_tname; + const struct btf *btf = meta->btf; + const struct btf_param *args; + u32 i, nargs; + int ret; + + args = (const struct btf_param *)(meta->func_proto + 1); + nargs = btf_type_vlen(meta->func_proto); + if (nargs > MAX_BPF_FUNC_REG_ARGS) { + verbose(env, "Function %s has %d > %d args\n", func_name, nargs, + MAX_BPF_FUNC_REG_ARGS); + return -EINVAL; + } + + /* Check that BTF function arguments match actual types that the + * verifier sees. + */ + for (i = 0; i < nargs; i++) { + struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[i + 1]; + const struct btf_type *t, *ref_t, *resolve_ret; + enum bpf_arg_type arg_type = ARG_DONTCARE; + u32 regno = i + 1, ref_id, type_size; + bool is_ret_buf_sz = false; + int kf_arg_type; + + t = btf_type_skip_modifiers(btf, args[i].type, NULL); + if (btf_type_is_scalar(t)) { + if (reg->type != SCALAR_VALUE) { + verbose(env, "R%d is not a scalar\n", regno); + return -EINVAL; + } + if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdonly_buf_size")) { + meta->r0_rdonly = true; + is_ret_buf_sz = true; + } else if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdwr_buf_size")) { + is_ret_buf_sz = true; + } + + if (is_ret_buf_sz) { + if (meta->r0_size) { + verbose(env, "2 or more rdonly/rdwr_buf_size parameters for kfunc"); + return -EINVAL; + } + + if (!tnum_is_const(reg->var_off)) { + verbose(env, "R%d is not a const\n", regno); + return -EINVAL; + } + + meta->r0_size = reg->var_off.value; + ret = mark_chain_precision(env, regno); + if (ret) + return ret; + } + continue; + } + + if (!btf_type_is_ptr(t)) { + verbose(env, "Unrecognized arg#%d type %s\n", i, btf_type_str(t)); + return -EINVAL; + } + + if (reg->ref_obj_id) { + if (is_kfunc_release(meta) && meta->ref_obj_id) { + verbose(env, "verifier internal error: more than one arg with ref_obj_id R%d %u %u\n", + regno, reg->ref_obj_id, + meta->ref_obj_id); + return -EFAULT; + } + meta->ref_obj_id = reg->ref_obj_id; + if (is_kfunc_release(meta)) + meta->release_regno = regno; + } + + ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); + ref_tname = btf_name_by_offset(btf, ref_t->name_off); + + kf_arg_type = get_kfunc_ptr_arg_type(env, meta, t, ref_t, ref_tname, args, i, nargs); + if (kf_arg_type < 0) + return kf_arg_type; + + switch (kf_arg_type) { + case KF_ARG_PTR_TO_BTF_ID: + if (!is_kfunc_trusted_args(meta)) + break; + if (!reg->ref_obj_id) { + verbose(env, "R%d must be referenced\n", regno); + return -EINVAL; + } + fallthrough; + case KF_ARG_PTR_TO_CTX: + /* Trusted arguments have the same offset checks as release arguments */ + arg_type |= OBJ_RELEASE; + break; + case KF_ARG_PTR_TO_KPTR_STRONG: + case KF_ARG_PTR_TO_DYNPTR: + case KF_ARG_PTR_TO_MEM: + case KF_ARG_PTR_TO_MEM_SIZE: + /* Trusted by default */ + break; + default: + WARN_ON_ONCE(1); + return -EFAULT; + } + + if (is_kfunc_release(meta) && reg->ref_obj_id) + arg_type |= OBJ_RELEASE; + ret = check_func_arg_reg_off(env, reg, regno, arg_type); + if (ret < 0) + return ret; + + switch (kf_arg_type) { + case KF_ARG_PTR_TO_CTX: + if (reg->type != PTR_TO_CTX) { + verbose(env, "arg#%d expected pointer to ctx, but got %s\n", i, btf_type_str(t)); + return -EINVAL; + } + break; + case KF_ARG_PTR_TO_KPTR_STRONG: + if (reg->type != PTR_TO_MAP_VALUE) { + verbose(env, "arg#0 expected pointer to map value\n"); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_kptr_strong(env, reg, ref_t, ref_tname, meta, i); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_DYNPTR: + if (reg->type != PTR_TO_STACK) { + verbose(env, "arg#%d expected pointer to stack\n", i); + return -EINVAL; + } + + if (!is_dynptr_reg_valid_init(env, reg)) { + verbose(env, "arg#%d pointer type %s %s must be valid and initialized\n", + i, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + + if (!is_dynptr_type_expected(env, reg, ARG_PTR_TO_DYNPTR | DYNPTR_TYPE_LOCAL)) { + verbose(env, "arg#%d pointer type %s %s points to unsupported dynamic pointer type\n", + i, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + break; + case KF_ARG_PTR_TO_BTF_ID: + /* Only base_type is checked, further checks are done here */ + if (reg->type != PTR_TO_BTF_ID && + (!reg2btf_ids[base_type(reg->type)] || type_flag(reg->type))) { + verbose(env, "arg#%d expected pointer to btf or socket\n", i); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_btf_id(env, reg, ref_t, ref_tname, ref_id, meta, i); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_MEM: + resolve_ret = btf_resolve_size(btf, ref_t, &type_size); + if (IS_ERR(resolve_ret)) { + verbose(env, "arg#%d reference type('%s %s') size cannot be determined: %ld\n", + i, btf_type_str(ref_t), ref_tname, PTR_ERR(resolve_ret)); + return -EINVAL; + } + ret = check_mem_reg(env, reg, regno, type_size); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_MEM_SIZE: + ret = check_kfunc_mem_size_reg(env, ®s[regno + 1], regno + 1); + if (ret < 0) { + verbose(env, "arg#%d arg#%d memory, len pair leads to invalid memory access\n", i, i + 1); + return ret; + } + /* Skip next '__sz' argument */ + i++; + break; + } + } + + if (is_kfunc_release(meta) && !meta->release_regno) { + verbose(env, "release kernel function %s expects refcounted PTR_TO_BTF_ID\n", + func_name); + return -EINVAL; + } + + return 0; +} + static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { const struct btf_type *t, *func, *func_proto, *ptr_type; struct bpf_reg_state *regs = cur_regs(env); - struct bpf_kfunc_arg_meta meta = { 0 }; const char *func_name, *ptr_type_name; + struct bpf_kfunc_call_arg_meta meta; u32 i, nargs, func_id, ptr_type_id; int err, insn_idx = *insn_idx_p; const struct btf_param *args; struct btf *desc_btf; u32 *kfunc_flags; - bool acq; /* skip for now, but return error when we find this in fixup_kfunc_call */ if (!insn->imm) @@ -7883,24 +8387,34 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, func_name); return -EACCES; } - if (*kfunc_flags & KF_DESTRUCTIVE && !capable(CAP_SYS_BOOT)) { - verbose(env, "destructive kfunc calls require CAP_SYS_BOOT capabilities\n"); + + /* Prepare kfunc call metadata */ + memset(&meta, 0, sizeof(meta)); + meta.btf = desc_btf; + meta.func_id = func_id; + meta.kfunc_flags = *kfunc_flags; + meta.func_proto = func_proto; + meta.func_name = func_name; + + if (is_kfunc_destructive(&meta) && !capable(CAP_SYS_BOOT)) { + verbose(env, "destructive kfunc calls require CAP_SYS_BOOT capability\n"); return -EACCES; } - acq = *kfunc_flags & KF_ACQUIRE; - - meta.flags = *kfunc_flags; + if (is_kfunc_sleepable(&meta) && !env->prog->aux->sleepable) { + verbose(env, "program must be sleepable to call sleepable kfunc %s\n", func_name); + return -EACCES; + } /* Check the arguments */ - err = btf_check_kfunc_arg_match(env, desc_btf, func_id, regs, &meta); + err = check_kfunc_args(env, &meta); if (err < 0) return err; /* In case of release function, we get register number of refcounted - * PTR_TO_BTF_ID back from btf_check_kfunc_arg_match, do the release now + * PTR_TO_BTF_ID in bpf_kfunc_arg_meta, do the release now. */ - if (err) { - err = release_reference(env, regs[err].ref_obj_id); + if (meta.release_regno) { + err = release_reference(env, regs[meta.release_regno].ref_obj_id); if (err) { verbose(env, "kfunc %s#%d reference has not been acquired before\n", func_name, func_id); @@ -7914,7 +8428,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, /* Check return type */ t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); - if (acq && !btf_type_is_struct_ptr(desc_btf, t)) { + if (is_kfunc_acquire(&meta) && !btf_type_is_struct_ptr(meta.btf, t)) { verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); return -EINVAL; } @@ -7953,20 +8467,23 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; } - if (*kfunc_flags & KF_RET_NULL) { + if (is_kfunc_ret_null(&meta)) { regs[BPF_REG_0].type |= PTR_MAYBE_NULL; /* For mark_ptr_or_null_reg, see 93c230e3f5bd6 */ regs[BPF_REG_0].id = ++env->id_gen; } mark_btf_func_reg_size(env, BPF_REG_0, sizeof(void *)); - if (acq) { + if (is_kfunc_acquire(&meta)) { int id = acquire_reference_state(env, insn_idx); if (id < 0) return id; - regs[BPF_REG_0].id = id; + if (is_kfunc_ret_null(&meta)) + regs[BPF_REG_0].id = id; regs[BPF_REG_0].ref_obj_id = id; } + if (reg_may_point_to_spin_lock(®s[BPF_REG_0]) && !regs[BPF_REG_0].id) + regs[BPF_REG_0].id = ++env->id_gen; } /* else { add_kfunc_call() ensures it is btf_type_is_void(t) } */ nargs = btf_type_vlen(func_proto); diff --git a/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c b/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c index c210657d4d0a..55d641c1f126 100644 --- a/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c +++ b/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c @@ -22,7 +22,7 @@ static struct { "arg#0 pointer type STRUCT bpf_dynptr_kern points to unsupported dynamic pointer type", 0}, {"not_valid_dynptr", "arg#0 pointer type STRUCT bpf_dynptr_kern must be valid and initialized", 0}, - {"not_ptr_to_stack", "arg#0 pointer type STRUCT bpf_dynptr_kern not to stack", 0}, + {"not_ptr_to_stack", "arg#0 expected pointer to stack", 0}, {"dynptr_data_null", NULL, -EBADMSG}, }; diff --git a/tools/testing/selftests/bpf/verifier/calls.c b/tools/testing/selftests/bpf/verifier/calls.c index e1a937277b54..86d6fef2e3b4 100644 --- a/tools/testing/selftests/bpf/verifier/calls.c +++ b/tools/testing/selftests/bpf/verifier/calls.c @@ -109,7 +109,7 @@ }, .prog_type = BPF_PROG_TYPE_SCHED_CLS, .result = REJECT, - .errstr = "arg#0 pointer type STRUCT prog_test_ref_kfunc must point", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_kfunc_call_test_acquire", 3 }, { "bpf_kfunc_call_test_release", 5 }, diff --git a/tools/testing/selftests/bpf/verifier/ref_tracking.c b/tools/testing/selftests/bpf/verifier/ref_tracking.c index f18ce867271f..4784471b0b7f 100644 --- a/tools/testing/selftests/bpf/verifier/ref_tracking.c +++ b/tools/testing/selftests/bpf/verifier/ref_tracking.c @@ -142,7 +142,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_user_key", 2 }, { "bpf_key_put", 4 }, @@ -163,7 +163,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_system_key", 1 }, { "bpf_key_put", 3 }, From patchwork Fri Nov 11 19:32:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040744 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99B71C4332F for ; Fri, 11 Nov 2022 19:34:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233980AbiKKTeI (ORCPT ); Fri, 11 Nov 2022 14:34:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233878AbiKKTeH (ORCPT ); Fri, 11 Nov 2022 14:34:07 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 713103E0B6 for ; Fri, 11 Nov 2022 11:34:05 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id c2so4952600plz.11 for ; Fri, 11 Nov 2022 11:34:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p4WOe+Oi2cOBX4Nsz00IluVF+r9yyqKIFQG+1fAw5G0=; b=PJCxDghmoQNtU7ZHOL1Q+TeUnksXkUi+oz8KQj5qKAh7l/oQKnL54vBtjXp3EIB7QV jRkVDcbyIn5G3xRx008WvqOBGNtGv3wlF+AK4dEk2jCn5cwC3JXBGrZnuUApIXRmNLUz zRwRPmWTo+QQDQ0e80PMRFDEPPvFdkll7awo87rUzJH+d9bKVOvJjigWLkyNS0jmF3bL H9bZt49itFVdebHiki1DMQSD1WkmI+Jx6rKwFv0KoVUmO/eI7oA0KC0M1kSnwCev1k0B ENPO9liM2rlFvcSELi188vq9xJ6wc6R1GeZkIKngf1uqk6Vh5MS3aY3T7xurjiDA1xXD enJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p4WOe+Oi2cOBX4Nsz00IluVF+r9yyqKIFQG+1fAw5G0=; b=qGh/dl6WF3V6vFr/ELcOIERBQH4J03LZEX2Wps4M4sVytnCEVtmy/jeeDEikRkYrCa KAjNz7gTmKG1j0A1ENMHS/A3gAshCcVDFDU4kEL7rA2hJ+LI1nQDaqmdmq8F++o+vvL5 CSnwP1ymjQi75h+gqjgkdC5vbTT3rBVZvNAal/IIyuTYOQZJiiboz3LNJENZZOebC3XV jTY21vobXNTofrADFeRPttpPWpY5NZkbXWCfLSRdrECg3LSxhC1Ftu6WdGJm9sXiSI+w WxXjBBEGVIBpIH/x4oZpFSGFK7t1hCzfQ7yaImHT8QSMKYCnagt0gIePRPIYLqhvJ2jx QqQg== X-Gm-Message-State: ANoB5plWM2pimziKpzzPV/+7M02jToq6UJjeXQkAs904YQoALqWKNKqO 1yh/X8jWldkFHcJAVNyffhzG97vywITiaw== X-Google-Smtp-Source: AA0mqf5ZwdkPqmHeWE5td9xVoWazq8JjWuhEGxWrQwdPhkonYqZ5pHIgqiOUTU2tOgFfp346aBJu8Q== X-Received: by 2002:a17:90a:5408:b0:212:ea91:77e5 with SMTP id z8-20020a17090a540800b00212ea9177e5mr3506256pjh.121.1668195244839; Fri, 11 Nov 2022 11:34:04 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id x10-20020a17090a46ca00b002131a9f8dcbsm5147723pjg.46.2022.11.11.11.34.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:04 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 15/26] bpf: Drop kfunc bits from btf_check_func_arg_match Date: Sat, 12 Nov 2022 01:02:13 +0530 Message-Id: <20221111193224.876706-16-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=18717; i=memxor@gmail.com; h=from:subject; bh=GeJ4KWmjj/EqpfkyT/gmtpntAB6QDf1NIR1fRa4O2TM=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIopRD1OGSSfqJ36i7AH8Dv6YQIXfc/GjbYbFG8 UqcIkmqJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RyhWGEA CGyNL7FIiUFAqDCKI/2Mmua2jtyB4Oh7pC5smQR559w716+N+YtcGbQDjN8utdzQeS62PnEpSD2hZl RdoSi05R4322ERd1aUuqJI8cxcvV5Tt2wsMmVu59DGUgm9RlOqkdl5ugsTRSFTTW771zfRofP9GlpY PTKIB7iS9j8ypqP8MRG6m1zr7m6VuomLl2jXcLsQGOeo/PWyKiImXHnv/t9IiG9ANRmXJ9QxRJp5tS qoPp4xhglmQYiLZkx550dwoKoWo/fonMaidOj21Y/FiUkeZdQsQ/CS+3sbWPrEiZiKpLYvtgkUQ9Oz S9bhRhhx8PAhbozNFaK44YU/TwD1VlYcAVhym7GYJ2zQ7HcVwnxlemF0Sa+9SUTel2baagn1t20C51 yhUF0tyHB4zijpeEFSTuAFrEwk7UWoNptgK/Y1AdbLlhLV1wBof9OjKDa+J9qz9SC21MlpcU3UUfza Nmbz1CNALPR355Ew1n+is6EhcNJ2J9VVNylPdMjIpr94MdgZCcnqOqANxpIjzE7S3N086D/Pe8UWit OkOQ7Uhi2nGLI+gTdgzHCrhnAgIIKXgHqFrzNHRmbuo8WQAvoAQFqFHplvpSrKeJ9X3K1TSn/Jsc5z cbBcA7x3339GrnGVPYrqhcHfO7yLalTzVWANGVa0t5uYj8lfCD7eNcLC8Oyg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Remove all kfunc related bits now from btf_check_func_arg_match, as users have been converted away to refactored kfunc argument handling. This is split into a separate commit to aid review, in order to compare what has been preserved from the removed bits easily instead of mixing removed hunks with previous patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 11 -- include/linux/bpf_verifier.h | 2 - kernel/bpf/btf.c | 364 +---------------------------------- kernel/bpf/verifier.c | 4 +- 4 files changed, 10 insertions(+), 371 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index c88f75a68893..62a16b699e71 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2108,22 +2108,11 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, const char *func_name, struct btf_func_model *m); -struct bpf_kfunc_arg_meta { - u64 r0_size; - bool r0_rdonly; - int ref_obj_id; - u32 flags; -}; - struct bpf_reg_state; int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *regs); int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *regs); -int btf_check_kfunc_arg_match(struct bpf_verifier_env *env, - const struct btf *btf, u32 func_id, - struct bpf_reg_state *regs, - struct bpf_kfunc_arg_meta *meta); int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *reg); int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prog, diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index fa738abea267..887fa4d922f6 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -597,8 +597,6 @@ int check_ptr_off_reg(struct bpf_verifier_env *env, int check_func_arg_reg_off(struct bpf_verifier_env *env, const struct bpf_reg_state *reg, int regno, enum bpf_arg_type arg_type); -int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, - u32 regno); int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, u32 regno, u32 mem_size); bool is_dynptr_reg_valid_init(struct bpf_verifier_env *env, diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index c99c254c3e85..6f268d279c19 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6594,122 +6594,19 @@ int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *pr return btf_check_func_type_match(log, btf1, t1, btf2, t2); } -static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { -#ifdef CONFIG_NET - [PTR_TO_SOCKET] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK], - [PTR_TO_SOCK_COMMON] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK_COMMON], - [PTR_TO_TCP_SOCK] = &btf_sock_ids[BTF_SOCK_TYPE_TCP], -#endif -}; - -/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ -static bool __btf_type_is_scalar_struct(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int rec) -{ - const struct btf_type *member_type; - const struct btf_member *member; - u32 i; - - if (!btf_type_is_struct(t)) - return false; - - for_each_member(i, t, member) { - const struct btf_array *array; - - member_type = btf_type_skip_modifiers(btf, member->type, NULL); - if (btf_type_is_struct(member_type)) { - if (rec >= 3) { - bpf_log(log, "max struct nesting depth exceeded\n"); - return false; - } - if (!__btf_type_is_scalar_struct(log, btf, member_type, rec + 1)) - return false; - continue; - } - if (btf_type_is_array(member_type)) { - array = btf_type_array(member_type); - if (!array->nelems) - return false; - member_type = btf_type_skip_modifiers(btf, array->type, NULL); - if (!btf_type_is_scalar(member_type)) - return false; - continue; - } - if (!btf_type_is_scalar(member_type)) - return false; - } - return true; -} - -static bool is_kfunc_arg_mem_size(const struct btf *btf, - const struct btf_param *arg, - const struct bpf_reg_state *reg) -{ - int len, sfx_len = sizeof("__sz") - 1; - const struct btf_type *t; - const char *param_name; - - t = btf_type_skip_modifiers(btf, arg->type, NULL); - if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) - return false; - - /* In the future, this can be ported to use BTF tagging */ - param_name = btf_name_by_offset(btf, arg->name_off); - if (str_is_empty(param_name)) - return false; - len = strlen(param_name); - if (len < sfx_len) - return false; - param_name += len - sfx_len; - if (strncmp(param_name, "__sz", sfx_len)) - return false; - - return true; -} - -static bool btf_is_kfunc_arg_mem_size(const struct btf *btf, - const struct btf_param *arg, - const struct bpf_reg_state *reg, - const char *name) -{ - int len, target_len = strlen(name); - const struct btf_type *t; - const char *param_name; - - t = btf_type_skip_modifiers(btf, arg->type, NULL); - if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) - return false; - - param_name = btf_name_by_offset(btf, arg->name_off); - if (str_is_empty(param_name)) - return false; - len = strlen(param_name); - if (len != target_len) - return false; - if (strcmp(param_name, name)) - return false; - - return true; -} - static int btf_check_func_arg_match(struct bpf_verifier_env *env, const struct btf *btf, u32 func_id, struct bpf_reg_state *regs, bool ptr_to_mem_ok, - struct bpf_kfunc_arg_meta *kfunc_meta, bool processing_call) { enum bpf_prog_type prog_type = resolve_prog_type(env->prog); - bool rel = false, kptr_get = false, trusted_args = false; - bool sleepable = false; struct bpf_verifier_log *log = &env->log; - u32 i, nargs, ref_id, ref_obj_id = 0; - bool is_kfunc = btf_is_kernel(btf); const char *func_name, *ref_tname; const struct btf_type *t, *ref_t; const struct btf_param *args; - int ref_regno = 0, ret; + u32 i, nargs, ref_id; + int ret; t = btf_type_by_id(btf, func_id); if (!t || !btf_type_is_func(t)) { @@ -6735,14 +6632,6 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, return -EINVAL; } - if (is_kfunc && kfunc_meta) { - /* Only kfunc can be release func */ - rel = kfunc_meta->flags & KF_RELEASE; - kptr_get = kfunc_meta->flags & KF_KPTR_GET; - trusted_args = kfunc_meta->flags & KF_TRUSTED_ARGS; - sleepable = kfunc_meta->flags & KF_SLEEPABLE; - } - /* check that BTF function arguments match actual types that the * verifier sees. */ @@ -6750,42 +6639,9 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, enum bpf_arg_type arg_type = ARG_DONTCARE; u32 regno = i + 1; struct bpf_reg_state *reg = ®s[regno]; - bool obj_ptr = false; t = btf_type_skip_modifiers(btf, args[i].type, NULL); if (btf_type_is_scalar(t)) { - if (is_kfunc && kfunc_meta) { - bool is_buf_size = false; - - /* check for any const scalar parameter of name "rdonly_buf_size" - * or "rdwr_buf_size" - */ - if (btf_is_kfunc_arg_mem_size(btf, &args[i], reg, - "rdonly_buf_size")) { - kfunc_meta->r0_rdonly = true; - is_buf_size = true; - } else if (btf_is_kfunc_arg_mem_size(btf, &args[i], reg, - "rdwr_buf_size")) - is_buf_size = true; - - if (is_buf_size) { - if (kfunc_meta->r0_size) { - bpf_log(log, "2 or more rdonly/rdwr_buf_size parameters for kfunc"); - return -EINVAL; - } - - if (!tnum_is_const(reg->var_off)) { - bpf_log(log, "R%d is not a const\n", regno); - return -EINVAL; - } - - kfunc_meta->r0_size = reg->var_off.value; - ret = mark_chain_precision(env, regno); - if (ret) - return ret; - } - } - if (reg->type == SCALAR_VALUE) continue; bpf_log(log, "R%d is not a scalar\n", regno); @@ -6798,88 +6654,14 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, return -EINVAL; } - /* These register types have special constraints wrt ref_obj_id - * and offset checks. The rest of trusted args don't. - */ - obj_ptr = reg->type == PTR_TO_CTX || reg->type == PTR_TO_BTF_ID || - reg2btf_ids[base_type(reg->type)]; - - /* Check if argument must be a referenced pointer, args + i has - * been verified to be a pointer (after skipping modifiers). - * PTR_TO_CTX is ok without having non-zero ref_obj_id. - */ - if (is_kfunc && trusted_args && (obj_ptr && reg->type != PTR_TO_CTX) && !reg->ref_obj_id) { - bpf_log(log, "R%d must be referenced\n", regno); - return -EINVAL; - } - ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); ref_tname = btf_name_by_offset(btf, ref_t->name_off); - /* Trusted args have the same offset checks as release arguments */ - if ((trusted_args && obj_ptr) || (rel && reg->ref_obj_id)) - arg_type |= OBJ_RELEASE; ret = check_func_arg_reg_off(env, reg, regno, arg_type); if (ret < 0) return ret; - if (is_kfunc && reg->ref_obj_id) { - /* Ensure only one argument is referenced PTR_TO_BTF_ID */ - if (ref_obj_id) { - bpf_log(log, "verifier internal error: more than one arg with ref_obj_id R%d %u %u\n", - regno, reg->ref_obj_id, ref_obj_id); - return -EFAULT; - } - ref_regno = regno; - ref_obj_id = reg->ref_obj_id; - } - - /* kptr_get is only true for kfunc */ - if (i == 0 && kptr_get) { - struct btf_field *kptr_field; - - if (reg->type != PTR_TO_MAP_VALUE) { - bpf_log(log, "arg#0 expected pointer to map value\n"); - return -EINVAL; - } - - /* check_func_arg_reg_off allows var_off for - * PTR_TO_MAP_VALUE, but we need fixed offset to find - * off_desc. - */ - if (!tnum_is_const(reg->var_off)) { - bpf_log(log, "arg#0 must have constant offset\n"); - return -EINVAL; - } - - kptr_field = btf_record_find(reg->map_ptr->record, reg->off + reg->var_off.value, BPF_KPTR); - if (!kptr_field || kptr_field->type != BPF_KPTR_REF) { - bpf_log(log, "arg#0 no referenced kptr at map value offset=%llu\n", - reg->off + reg->var_off.value); - return -EINVAL; - } - - if (!btf_type_is_ptr(ref_t)) { - bpf_log(log, "arg#0 BTF type must be a double pointer\n"); - return -EINVAL; - } - - ref_t = btf_type_skip_modifiers(btf, ref_t->type, &ref_id); - ref_tname = btf_name_by_offset(btf, ref_t->name_off); - - if (!btf_type_is_struct(ref_t)) { - bpf_log(log, "kernel function %s args#%d pointer type %s %s is not supported\n", - func_name, i, btf_type_str(ref_t), ref_tname); - return -EINVAL; - } - if (!btf_struct_ids_match(log, btf, ref_id, 0, kptr_field->kptr.btf, - kptr_field->kptr.btf_id, true)) { - bpf_log(log, "kernel function %s args#%d expected pointer to %s %s\n", - func_name, i, btf_type_str(ref_t), ref_tname); - return -EINVAL; - } - /* rest of the arguments can be anything, like normal kfunc */ - } else if (btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { + if (btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { /* If function expects ctx type in BTF check that caller * is passing PTR_TO_CTX. */ @@ -6889,109 +6671,10 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, i, btf_type_str(t)); return -EINVAL; } - } else if (is_kfunc && (reg->type == PTR_TO_BTF_ID || - (reg2btf_ids[base_type(reg->type)] && !type_flag(reg->type)))) { - const struct btf_type *reg_ref_t; - const struct btf *reg_btf; - const char *reg_ref_tname; - u32 reg_ref_id; - - if (!btf_type_is_struct(ref_t)) { - bpf_log(log, "kernel function %s args#%d pointer type %s %s is not supported\n", - func_name, i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - if (reg->type == PTR_TO_BTF_ID) { - reg_btf = reg->btf; - reg_ref_id = reg->btf_id; - } else { - reg_btf = btf_vmlinux; - reg_ref_id = *reg2btf_ids[base_type(reg->type)]; - } - - reg_ref_t = btf_type_skip_modifiers(reg_btf, reg_ref_id, - ®_ref_id); - reg_ref_tname = btf_name_by_offset(reg_btf, - reg_ref_t->name_off); - if (!btf_struct_ids_match(log, reg_btf, reg_ref_id, - reg->off, btf, ref_id, - trusted_args || (rel && reg->ref_obj_id))) { - bpf_log(log, "kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n", - func_name, i, - btf_type_str(ref_t), ref_tname, - regno, btf_type_str(reg_ref_t), - reg_ref_tname); - return -EINVAL; - } } else if (ptr_to_mem_ok && processing_call) { const struct btf_type *resolve_ret; u32 type_size; - if (is_kfunc) { - bool arg_mem_size = i + 1 < nargs && is_kfunc_arg_mem_size(btf, &args[i + 1], ®s[regno + 1]); - bool arg_dynptr = btf_type_is_struct(ref_t) && - !strcmp(ref_tname, - stringify_struct(bpf_dynptr_kern)); - - /* Permit pointer to mem, but only when argument - * type is pointer to scalar, or struct composed - * (recursively) of scalars. - * When arg_mem_size is true, the pointer can be - * void *. - * Also permit initialized local dynamic pointers. - */ - if (!btf_type_is_scalar(ref_t) && - !__btf_type_is_scalar_struct(log, btf, ref_t, 0) && - !arg_dynptr && - (arg_mem_size ? !btf_type_is_void(ref_t) : 1)) { - bpf_log(log, - "arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n", - i, btf_type_str(ref_t), ref_tname, arg_mem_size ? "void, " : ""); - return -EINVAL; - } - - if (arg_dynptr) { - if (reg->type != PTR_TO_STACK) { - bpf_log(log, "arg#%d pointer type %s %s not to stack\n", - i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - if (!is_dynptr_reg_valid_init(env, reg)) { - bpf_log(log, - "arg#%d pointer type %s %s must be valid and initialized\n", - i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - if (!is_dynptr_type_expected(env, reg, - ARG_PTR_TO_DYNPTR | DYNPTR_TYPE_LOCAL)) { - bpf_log(log, - "arg#%d pointer type %s %s points to unsupported dynamic pointer type\n", - i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - continue; - } - - /* Check for mem, len pair */ - if (arg_mem_size) { - if (check_kfunc_mem_size_reg(env, ®s[regno + 1], regno + 1)) { - bpf_log(log, "arg#%d arg#%d memory, len pair leads to invalid memory access\n", - i, i + 1); - return -EINVAL; - } - i++; - continue; - } - } - resolve_ret = btf_resolve_size(btf, ref_t, &type_size); if (IS_ERR(resolve_ret)) { bpf_log(log, @@ -7004,36 +6687,13 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, if (check_mem_reg(env, reg, regno, type_size)) return -EINVAL; } else { - bpf_log(log, "reg type unsupported for arg#%d %sfunction %s#%d\n", i, - is_kfunc ? "kernel " : "", func_name, func_id); + bpf_log(log, "reg type unsupported for arg#%d function %s#%d\n", i, + func_name, func_id); return -EINVAL; } } - /* Either both are set, or neither */ - WARN_ON_ONCE((ref_obj_id && !ref_regno) || (!ref_obj_id && ref_regno)); - /* We already made sure ref_obj_id is set only for one argument. We do - * allow (!rel && ref_obj_id), so that passing such referenced - * PTR_TO_BTF_ID to other kfuncs works. Note that rel is only true when - * is_kfunc is true. - */ - if (rel && !ref_obj_id) { - bpf_log(log, "release kernel function %s expects refcounted PTR_TO_BTF_ID\n", - func_name); - return -EINVAL; - } - - if (sleepable && !env->prog->aux->sleepable) { - bpf_log(log, "kernel function %s is sleepable but the program is not\n", - func_name); - return -EINVAL; - } - - if (kfunc_meta && ref_obj_id) - kfunc_meta->ref_obj_id = ref_obj_id; - - /* returns argument register number > 0 in case of reference release kfunc */ - return rel ? ref_regno : 0; + return 0; } /* Compare BTF of a function declaration with given bpf_reg_state. @@ -7063,7 +6723,7 @@ int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog, return -EINVAL; is_global = prog->aux->func_info_aux[subprog].linkage == BTF_FUNC_GLOBAL; - err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, NULL, false); + err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, false); /* Compiler optimizations can remove arguments from static functions * or mismatched type can be passed into a global function. @@ -7106,7 +6766,7 @@ int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, return -EINVAL; is_global = prog->aux->func_info_aux[subprog].linkage == BTF_FUNC_GLOBAL; - err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, NULL, true); + err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, true); /* Compiler optimizations can remove arguments from static functions * or mismatched type can be passed into a global function. @@ -7117,14 +6777,6 @@ int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, return err; } -int btf_check_kfunc_arg_match(struct bpf_verifier_env *env, - const struct btf *btf, u32 func_id, - struct bpf_reg_state *regs, - struct bpf_kfunc_arg_meta *meta) -{ - return btf_check_func_arg_match(env, btf, func_id, regs, true, meta, true); -} - /* Convert BTF of a function into bpf_reg_state if possible * Returns: * EFAULT - there is a verifier bug. Abort verification. diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 075f0d9207c7..dd0047ac53f9 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5539,8 +5539,8 @@ int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, return err; } -int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, - u32 regno) +static int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, + u32 regno) { struct bpf_reg_state *mem_reg = &cur_regs(env)[regno - 1]; bool may_be_null = type_may_be_null(mem_reg->type); From patchwork Fri Nov 11 19:32:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040745 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 787A7C433FE for ; Fri, 11 Nov 2022 19:34:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234004AbiKKTeL (ORCPT ); Fri, 11 Nov 2022 14:34:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234032AbiKKTeK (ORCPT ); Fri, 11 Nov 2022 14:34:10 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7847268296 for ; Fri, 11 Nov 2022 11:34:09 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id q71so5123602pgq.8 for ; Fri, 11 Nov 2022 11:34:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zRayPDTMNDdwJIvMlrtwqNk1Jz4XpAmdb2FhOmDxB74=; b=YYOfC+xGPhBmwJwRjpThfc7KVxqXzSHQJRsT2RBoyx6k+/pCHYi9ODYLK3i4h193mH +wlPCNlm405uuCPNQCDq6Hxr9KxlX+zHWM4JvD42RpbdidUTJR1crEfTjaUd/rCh//Vo oAJkWU0n31Yo+4iOQtGNpE6ZrrF8+4nMJ+V8Al5NpuEhcO/uxRAQ/34jDe9LDJWBJ5Os dFuXK5VihtcI4MDPXTp4NBkHhtMgYAfysTRRIraAOp7kSeHQn1Q58ZTteUPbDfDzHpod TIlI74hHs3LkXAslKdenh30tTeUFIQ27gLHXpMSuzA79KjPAjT9LPm6nFCMsRcWEynem JdTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zRayPDTMNDdwJIvMlrtwqNk1Jz4XpAmdb2FhOmDxB74=; b=miZw7bN2bOKdIaf62lpzzg7xmgRGOAXi1CVRZfmqtIAxdHUT0k6gK5iMAMULuwv9ui yHt8n+y9WrwbLJ5qesVK3U0gVuqcYnJe3Ho7hF4cixvLnNSVGunOtrpofcBDpmUlP+lk Ui98cihhqV32cMwc+GGtf+BfMcGlvDs+hI0udNcuR7m5nCyg/jac1iEitVn1iVoTEMES K5GuA/KjpP9OBfW9Y6CWjcORqW6GWHdL6koS06BU6BYx+Feda9fEOhX5JVryMkcjmB75 egWcC2ofWGY9ED+zO1V+NTP7Je6c1NcoDErnb8S5wpcWnyNrDpagXy0Ui4PoVz2hTp88 g+FQ== X-Gm-Message-State: ANoB5plU2Oz2G9XISjBeHYBUlWVfOIvWasBlAWo6JeKZCbiY7+8TMXTa Fexb3hiVja2wbiNwmLajFce3U5DG1BwyfA== X-Google-Smtp-Source: AA0mqf6UG2jDO+eyxGWYOETs4OHb/ebzBy63HInxgWMPhMmimDiHP31pAUG4k98sNimI16lTznJCkg== X-Received: by 2002:a63:1cf:0:b0:470:71df:a6c3 with SMTP id 198-20020a6301cf000000b0047071dfa6c3mr2890424pgb.447.1668195248721; Fri, 11 Nov 2022 11:34:08 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id s18-20020a170903215200b00186a6b6350esm2028952ple.268.2022.11.11.11.34.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:08 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 16/26] bpf: Support constant scalar arguments for kfuncs Date: Sat, 12 Nov 2022 01:02:14 +0530 Message-Id: <20221111193224.876706-17-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5263; i=memxor@gmail.com; h=from:subject; bh=bHdnbGRPK80DCis6rQXFsz8HZ2Ly3kkCygOqxu0OP08=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoc5QnrWiHuZEI2OBSPAwWl9WsOkhzTnjV5Up7 Z2rex4aJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RygMND/ 4vr0kw1ctsotc1PBrAl/r10+oqF9VXgg1d7w/lMcRSAi+7mn7S6hy57RHVTN6FKCmEwGTKgvUEiEyh LzKPcLorRqlDJ1gATB461hIOKcE/tHAFjoZPuj0ZMFKeffVPe978GuSbL4NGpslBvH3VUu6yIGJk0k y5Hfi1zE/hEYqyhe40R8zRpcC6DZtYr3cafqVTq6Lm9s9raXol11wGfP6cQxyEBFv3VAadp2Cou+VZ /jWDg7/7KtR06yjBqz30YXE4fwSMxPbJvMz6X+nbd3yhxsd/H+PV6Wu/AJhnKX90njorJ/uYsSUxMg NR92R5PIqXfLqdLtfohBb9ZI4mk8SxeY6xjDqWsF8b4nAbalP/51lQgyTMS+g9Kops3TqQXv6rfj33 vloDHiE7BS3eRuHEXyGEW0XopN+H+nAMaxnTpTmlkk4nK5YGkZwQiC6kz6679D2oJNpwx6SBMn5Ud2 wLWy6FwgYwEDZg5zHlclsA6cjKqD+vGuSCwA+tDWfSSj0SRTqcXtoGbkdAT7P/MLq4WCK0Idx4V0T4 hfUWzVoOPkOVLWwY53MBDPRc74OmEXHp6FV4I1rwbu3Ft1q7qC+ZFyhBe87f+OA3Z6bCk4A31CiRGE j8cDs3WnjOxy2yWZ7tztHhXAQYau6VPT9E8sGMrsh1lyXchDv1pt+zIF7nvg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow passing known constant scalars as arguments to kfuncs that do not represent a size parameter. This makes the search pruning optimization of verifier more conservative for such kfunc calls, and each non-distinct argument is considered unequivalent. We will use this support to then expose a bpf_obj_new function where it takes the local type ID of a type in program BTF, and returns a PTR_TO_BTF_ID | MEM_ALLOC to the local type, and allows programs to allocate their own objects. Signed-off-by: Kumar Kartikeya Dwivedi --- Documentation/bpf/kfuncs.rst | 22 ++++++++++++++ kernel/bpf/verifier.c | 59 +++++++++++++++++++++++++++--------- 2 files changed, 67 insertions(+), 14 deletions(-) diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst index 0f858156371d..8fa9c052417f 100644 --- a/Documentation/bpf/kfuncs.rst +++ b/Documentation/bpf/kfuncs.rst @@ -72,6 +72,28 @@ argument as its size. By default, without __sz annotation, the size of the type of the pointer is used. Without __sz annotation, a kfunc cannot accept a void pointer. +2.2.1 __k Annotation +-------------------- + +This annotation is only understood for scalar arguments, where it indicates that +the verifier must check the scalar argument to be a known constant, which does +not indicate a size parameter. + +An example is given below:: + + void *bpf_obj_new(u32 local_type_id__k, ...) + { + ... + } + +Here, bpf_obj_new uses local_type_id argument to find out the size of that type +ID in program's BTF and return a sized pointer to it. Each type ID will have a +distinct size, hence it is crucial to treat each such call as distinct when +values don't match. + +Hence, whenever a constant scalar argument is accepted by a kfunc which is not a +size parameter, __k suffix should be used. + .. _BPF_kfunc_nodef: 2.3 Using an existing kernel function diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index dd0047ac53f9..ad6d7531322c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7862,6 +7862,10 @@ struct bpf_kfunc_call_arg_meta { u8 release_regno; bool r0_rdonly; u64 r0_size; + struct { + u64 value; + bool found; + } arg_constant; }; static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) @@ -7899,30 +7903,40 @@ static bool is_kfunc_arg_kptr_get(struct bpf_kfunc_call_arg_meta *meta, int arg) return arg == 0 && (meta->kfunc_flags & KF_KPTR_GET); } -static bool is_kfunc_arg_mem_size(const struct btf *btf, - const struct btf_param *arg, - const struct bpf_reg_state *reg) +static bool __kfunc_param_match_suffix(const struct btf *btf, + const struct btf_param *arg, + const char *suffix) { - int len, sfx_len = sizeof("__sz") - 1; - const struct btf_type *t; + int suffix_len = strlen(suffix), len; const char *param_name; - t = btf_type_skip_modifiers(btf, arg->type, NULL); - if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) - return false; - /* In the future, this can be ported to use BTF tagging */ param_name = btf_name_by_offset(btf, arg->name_off); if (str_is_empty(param_name)) return false; len = strlen(param_name); - if (len < sfx_len) + if (len < suffix_len) return false; - param_name += len - sfx_len; - if (strncmp(param_name, "__sz", sfx_len)) + param_name += len - suffix_len; + return !strncmp(param_name, suffix, suffix_len); +} + +static bool is_kfunc_arg_mem_size(const struct btf *btf, + const struct btf_param *arg, + const struct bpf_reg_state *reg) +{ + const struct btf_type *t; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) return false; - return true; + return __kfunc_param_match_suffix(btf, arg, "__sz"); +} + +static bool is_kfunc_arg_sfx_constant(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__k"); } static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, @@ -8198,7 +8212,24 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ verbose(env, "R%d is not a scalar\n", regno); return -EINVAL; } - if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdonly_buf_size")) { + if (is_kfunc_arg_sfx_constant(meta->btf, &args[i])) { + /* kfunc is already bpf_capable() only, no need + * to check it here. + */ + if (meta->arg_constant.found) { + verbose(env, "verifier internal error: only one constant argument permitted\n"); + return -EFAULT; + } + if (!tnum_is_const(reg->var_off)) { + verbose(env, "R%d must be a known constant\n", regno); + return -EINVAL; + } + ret = mark_chain_precision(env, regno); + if (ret < 0) + return ret; + meta->arg_constant.found = true; + meta->arg_constant.value = reg->var_off.value; + } else if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdonly_buf_size")) { meta->r0_rdonly = true; is_ret_buf_sz = true; } else if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdwr_buf_size")) { From patchwork Fri Nov 11 19:32:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040747 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8450C433FE for ; Fri, 11 Nov 2022 19:34:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234045AbiKKTeR (ORCPT ); Fri, 11 Nov 2022 14:34:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234042AbiKKTeO (ORCPT ); Fri, 11 Nov 2022 14:34:14 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20E04787B6 for ; Fri, 11 Nov 2022 11:34:13 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id b29so5665379pfp.13 for ; Fri, 11 Nov 2022 11:34:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HGWnf0b9CSnMhFQPokhQQb0gfvxi5xidfwO3GZgYZNM=; b=COtc63719njWQlyKtOklcgyRj13CHhqNh02OeyFYcl6Gb6ikbtKS9eNHN2NRH7mdQ2 QDHaxPu9wbOLklSdA7ZvUkMpxjIH0s1+s46DaMpStFpVY14Y3ufanL9i1SavPbnm8kiT e33JRDRVzOrUZuyC4YaOb5MizCgGPB7OokqNaobuDZ/j9IMEv5it9+qlSCxQGHMiWMl8 K2ticD5mAJF8OBYeoOIfoP9oVTzSxJDFMC6M87iIR1vbWwU9KJCJB8jqq75AbYJgPILy of6T8AoE3XSdbDX7Orez8wqByDKZYzQ2cAqlaE3Hkm411a3WGR2tjV/AtIbgpQDUoL5p 2dnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HGWnf0b9CSnMhFQPokhQQb0gfvxi5xidfwO3GZgYZNM=; b=wIWOJqrBmIStV2KOmj0UFRV+VArr6c34XsCXvx5qOcqLGKLMWtHr5V8zfeSlhkIDDw Id6syp9AixF5wjvzSaA9vNQlaBS1U1wxK8/4Yd/N06ExJfP4WCHvOowXqRWzGVnw3u/W lKuRPYaMvJG7FZSgR3uY0RC/COZkQOVIWssNf7V6z7r270nfE3jbjyiBHeGYvbUyOzDc UEnG7p4CN8HIvypaCUdSxDXghJPkecMI2iuJuPD2esMnab/Q0Zgp7blUOVG8oUJ4cDJZ Nc4Kyo/mGYy27qoqnZ5ZgxnqGGeQmhXR8HVXumWVrLK9ntvel2Jz1fjg/I8jTPsTDooV 5UXw== X-Gm-Message-State: ANoB5pmxLT8CJ2OapKux0S/QZkiNzPYs51YWr39EpXI/zOv/Epq4c/8C mIrrzf25tscnw1gKUuex5k/6Sl04E0UvFQ== X-Google-Smtp-Source: AA0mqf4T78c+RVpaNwJDeV7+lPmyXEuDhcY/kcP7qbXcU1wNXFwONoyjDUyZqBC3bmfcr13suEnpiQ== X-Received: by 2002:a05:6a00:1d0f:b0:571:c567:bff1 with SMTP id a15-20020a056a001d0f00b00571c567bff1mr1869941pfx.52.1668195252423; Fri, 11 Nov 2022 11:34:12 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id m1-20020a1709026bc100b00186c9d17af2sm2084494plt.17.2022.11.11.11.34.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:12 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 17/26] bpf: Introduce bpf_obj_new Date: Sat, 12 Nov 2022 01:02:15 +0530 Message-Id: <20221111193224.876706-18-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=15820; i=memxor@gmail.com; h=from:subject; bh=4HijJ8xBWoL1WPj/vOgQD/rPW+mEUEvj/XrayTaNyH0=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIosTaauhW2AHwLcbqthNKxa9RQn1A9wkczrn1T VuKGmMWJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RykdOD/ 4/42JbDJHw5HBzdvIv+z5AJV2ko7RJtBWEesI75Og0of6cywZH0oGJShWLAZz/bW5pue/V2h+pfpqs YZ7g56fFqfwUib8Fy3FPchiGEAtHReUn8Jne7iMVT1iIu2WHlDc94zvvd3WnVP2wQMJvn3ZAhAPv2M 295GzKaKiLs8+V2ZTdCGum7CzS0vtGyPBaCTMHBb9sCnLHI4aO+kmctxcOYTQBfrtBj0tiAPTzR4QK z9gT9GVnfXLl9BQsTxQ1lSRuVpMFxd4awZWdBdpkE52ewlGMV9cwK+Sztj2VueQeayDSArVwGdNDA+ rtthbE8EEGicHS9521wY0wnr7OGe5hBImPhtN+ZzOi2XTYJTE8kbWHsBvhfxR4Ho7x928WlXcQt1qR i3Urps1GJzEr7fU5WZdrokJQfYFbn7sHoy/BXCFNICjH3o8sHTmJDb/C1eUdQUOgPC4/+ooO8WMh1O XhGg6KOBnGlAWSAMGfl9LvcAHyEuTFHiFQTS9cLzGOpr1s7JLsWxBcYGiDcjkMbzjNlnCeJW/rGiLu NB9sx6UPWLD3sfreYjrLIUKOeTfwBmZaf02jG/hzVByJ0FNhFmj1i5DeJzBaMcvszN4MZhZYLpncbx 54kBDWVzk1agvzgxP7lNq0jY3qVrmb8bGOuxmsZORKhCkERqWNDrgiP/Gh4w== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce type safe memory allocator bpf_obj_new for BPF programs. The kernel side kfunc is named bpf_obj_new_impl, as passing hidden arguments to kfuncs still requires having them in prototype, unlike BPF helpers which always take 5 arguments and have them checked using bpf_func_proto in verifier, ignoring unset argument types. Introduce __ign suffix to ignore a specific kfunc argument during type checks, then use this to introduce support for passing type metadata to the bpf_obj_new_impl kfunc. The user passes BTF ID of the type it wants to allocates in program BTF, the verifier then rewrites the first argument as the size of this type, after performing some sanity checks (to ensure it exists and it is a struct type). The second argument is also fixed up and passed by the verifier. This is the btf_struct_meta for the type being allocated. It would be needed mostly for the offset array which is required for zero initializing special fields while leaving the rest of storage in unitialized state. It would also be needed in the next patch to perform proper destruction of the object's special fields. Under the hood, bpf_obj_new will call bpf_mem_alloc and bpf_mem_free, using the any context BPF memory allocator introduced recently. To this end, a global instance of the BPF memory allocator is initialized on boot to be used for this purpose. This 'bpf_global_ma' serves all allocations for bpf_obj_new. In the future, bpf_obj_new variants will allow specifying a custom allocator. Note that now that bpf_obj_new can be used to allocate objects that can be linked to BPF linked list (when future linked list helpers are available), we need to also free the elements using bpf_mem_free. However, since the draining of elements is done outside the bpf_spin_lock, we need to do migrate_disable around the call since bpf_list_head_free can be called from map free path where migration is enabled. Otherwise, when called from BPF programs migration is already disabled. A convenience macro is included in the bpf_experimental.h header to hide over the ugly details of the implementation, leading to user code looking similar to a language level extension which allocates and constructs fields of a user type. struct bar { struct bpf_list_node node; }; struct foo { struct bpf_spin_lock lock; struct bpf_list_head head __contains(bar, node); }; void prog(void) { struct foo *f; f = bpf_obj_new(typeof(*f)); if (!f) return; ... } A key piece of this story is still missing, i.e. the free function, which will come in the next patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 21 ++-- include/linux/bpf_verifier.h | 2 + kernel/bpf/core.c | 16 +++ kernel/bpf/helpers.c | 47 ++++++-- kernel/bpf/verifier.c | 107 ++++++++++++++++-- .../testing/selftests/bpf/bpf_experimental.h | 25 ++++ 6 files changed, 195 insertions(+), 23 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpf_experimental.h diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 62a16b699e71..4635e31bd6fc 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -54,6 +54,8 @@ struct cgroup; extern struct idr btf_idr; extern spinlock_t btf_idr_lock; extern struct kobject *btf_kobj; +extern struct bpf_mem_alloc bpf_global_ma; +extern bool bpf_global_ma_set; typedef u64 (*bpf_callback_t)(u64, u64, u64, u64, u64); typedef int (*bpf_iter_init_seq_priv_t)(void *private_data, @@ -333,16 +335,19 @@ static inline bool btf_record_has_field(const struct btf_record *rec, enum btf_f return rec->field_mask & type; } -static inline void check_and_init_map_value(struct bpf_map *map, void *dst) +static inline void bpf_obj_init(const struct btf_field_offs *foffs, void *obj) { - if (!IS_ERR_OR_NULL(map->record)) { - struct btf_field *fields = map->record->fields; - u32 cnt = map->record->cnt; - int i; + int i; - for (i = 0; i < cnt; i++) - memset(dst + fields[i].offset, 0, btf_field_type_size(fields[i].type)); - } + if (!foffs) + return; + for (i = 0; i < foffs->cnt; i++) + memset(obj + foffs->field_off[i], 0, foffs->field_sz[i]); +} + +static inline void check_and_init_map_value(struct bpf_map *map, void *dst) +{ + bpf_obj_init(map->field_offs, dst); } /* memcpy that is used with 8-byte aligned pointers, power-of-8 size and diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 887fa4d922f6..306fc1d6cc4a 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -427,6 +427,8 @@ struct bpf_insn_aux_data { */ struct bpf_loop_inline_state loop_inline_state; }; + u64 obj_new_size; /* remember the size of type passed to bpf_obj_new to rewrite R1 */ + struct btf_struct_meta *kptr_struct_meta; u64 map_key_state; /* constant (32 bit) key tracking for maps */ int ctx_field_size; /* the ctx field size for load insn, maybe 0 */ u32 seen; /* this insn was processed by the verifier at env->pass_cnt */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 9c16338bcbe8..2e57fc839a5c 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -60,6 +61,9 @@ #define CTX regs[BPF_REG_CTX] #define IMM insn->imm +struct bpf_mem_alloc bpf_global_ma; +bool bpf_global_ma_set; + /* No hurry in this branch * * Exported for the bpf jit load helper. @@ -2746,6 +2750,18 @@ int __weak bpf_arch_text_invalidate(void *dst, size_t len) return -ENOTSUPP; } +#ifdef CONFIG_BPF_SYSCALL +static int __init bpf_global_ma_init(void) +{ + int ret; + + ret = bpf_mem_alloc_init(&bpf_global_ma, 0, false); + bpf_global_ma_set = !ret; + return ret; +} +late_initcall(bpf_global_ma_init); +#endif + DEFINE_STATIC_KEY_FALSE(bpf_stats_enabled_key); EXPORT_SYMBOL(bpf_stats_enabled_key); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 5bc0b9f0f306..c4f1c22cc44c 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "../../lib/kstrtox.h" @@ -1735,25 +1736,57 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head, obj -= field->list_head.node_offset; head = head->next; - /* TODO: Rework later */ - kfree(obj); + /* The contained type can also have resources, including a + * bpf_list_head which needs to be freed. + */ + bpf_obj_free_fields(field->list_head.value_rec, obj); + /* bpf_mem_free requires migrate_disable(), since we can be + * called from map free path as well apart from BPF program (as + * part of map ops doing bpf_obj_free_fields). + */ + migrate_disable(); + bpf_mem_free(&bpf_global_ma, obj); + migrate_enable(); } } -BTF_SET8_START(tracing_btf_ids) +__diag_push(); +__diag_ignore_all("-Wmissing-prototypes", + "Global functions as their definitions will be in vmlinux BTF"); + +void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign) +{ + struct btf_struct_meta *meta = meta__ign; + u64 size = local_type_id__k; + void *p; + + if (unlikely(!bpf_global_ma_set)) + return NULL; + p = bpf_mem_alloc(&bpf_global_ma, size); + if (!p) + return NULL; + if (meta) + bpf_obj_init(meta->field_offs, p); + return p; +} + +__diag_pop(); + +BTF_SET8_START(generic_btf_ids) #ifdef CONFIG_KEXEC_CORE BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) #endif -BTF_SET8_END(tracing_btf_ids) +BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_SET8_END(generic_btf_ids) -static const struct btf_kfunc_id_set tracing_kfunc_set = { +static const struct btf_kfunc_id_set generic_kfunc_set = { .owner = THIS_MODULE, - .set = &tracing_btf_ids, + .set = &generic_btf_ids, }; static int __init kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &tracing_kfunc_set); + return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set); } late_initcall(kfunc_init); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index ad6d7531322c..b7c663a2dbfa 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7939,6 +7939,11 @@ static bool is_kfunc_arg_sfx_constant(const struct btf *btf, const struct btf_pa return __kfunc_param_match_suffix(btf, arg, "__k"); } +static bool is_kfunc_arg_sfx_ignore(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__ign"); +} + static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, const struct btf_param *arg, const struct bpf_reg_state *reg, @@ -8207,6 +8212,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ int kf_arg_type; t = btf_type_skip_modifiers(btf, args[i].type, NULL); + + if (is_kfunc_arg_sfx_ignore(btf, &args[i])) + continue; + if (btf_type_is_scalar(t)) { if (reg->type != SCALAR_VALUE) { verbose(env, "R%d is not a scalar\n", regno); @@ -8386,6 +8395,17 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return 0; } +enum special_kfunc_type { + KF_bpf_obj_new_impl, +}; + +BTF_SET_START(special_kfunc_set) +BTF_ID(func, bpf_obj_new_impl) +BTF_SET_END(special_kfunc_set) + +BTF_ID_LIST(special_kfunc_list) +BTF_ID(func, bpf_obj_new_impl) + static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -8460,17 +8480,64 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); if (is_kfunc_acquire(&meta) && !btf_type_is_struct_ptr(meta.btf, t)) { - verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); - return -EINVAL; + /* Only exception is bpf_obj_new_impl */ + if (meta.btf != btf_vmlinux || meta.func_id != special_kfunc_list[KF_bpf_obj_new_impl]) { + verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); + return -EINVAL; + } } if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); mark_btf_func_reg_size(env, BPF_REG_0, t->size); } else if (btf_type_is_ptr(t)) { - ptr_type = btf_type_skip_modifiers(desc_btf, t->type, - &ptr_type_id); - if (!btf_type_is_struct(ptr_type)) { + ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); + + if (meta.btf == btf_vmlinux && btf_id_set_contains(&special_kfunc_set, meta.func_id)) { + if (!btf_type_is_void(ptr_type)) { + verbose(env, "kernel function %s must have void * return type\n", + meta.func_name); + return -EINVAL; + } + if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { + const struct btf_type *ret_t; + struct btf *ret_btf; + u32 ret_btf_id; + + if (((u64)(u32)meta.arg_constant.value) != meta.arg_constant.value) { + verbose(env, "local type ID argument must be in range [0, U32_MAX]\n"); + return -EINVAL; + } + + ret_btf = env->prog->aux->btf; + ret_btf_id = meta.arg_constant.value; + + /* This may be NULL due to user not supplying a BTF */ + if (!ret_btf) { + verbose(env, "bpf_obj_new requires prog BTF\n"); + return -EINVAL; + } + + ret_t = btf_type_by_id(ret_btf, ret_btf_id); + if (!ret_t || !__btf_type_is_struct(ret_t)) { + verbose(env, "bpf_obj_new type ID argument must be of a struct\n"); + return -EINVAL; + } + + mark_reg_known_zero(env, regs, BPF_REG_0); + regs[BPF_REG_0].type = PTR_TO_BTF_ID | MEM_ALLOC; + regs[BPF_REG_0].btf = ret_btf; + regs[BPF_REG_0].btf_id = ret_btf_id; + + env->insn_aux_data[insn_idx].obj_new_size = ret_t->size; + env->insn_aux_data[insn_idx].kptr_struct_meta = + btf_find_struct_meta(ret_btf, ret_btf_id); + } else { + verbose(env, "kernel function %s unhandled dynamic return type\n", + meta.func_name); + return -EFAULT; + } + } else if (!__btf_type_is_struct(ptr_type)) { if (!meta.r0_size) { ptr_type_name = btf_name_by_offset(desc_btf, ptr_type->name_off); @@ -8498,6 +8565,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; } + if (is_kfunc_ret_null(&meta)) { regs[BPF_REG_0].type |= PTR_MAYBE_NULL; /* For mark_ptr_or_null_reg, see 93c230e3f5bd6 */ @@ -14635,8 +14703,8 @@ static int fixup_call_args(struct bpf_verifier_env *env) return err; } -static int fixup_kfunc_call(struct bpf_verifier_env *env, - struct bpf_insn *insn) +static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, + struct bpf_insn *insn_buf, int insn_idx, int *cnt) { const struct bpf_kfunc_desc *desc; @@ -14655,8 +14723,21 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, return -EFAULT; } + *cnt = 0; insn->imm = desc->imm; + if (insn->off) + return 0; + if (desc->func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { + struct btf_struct_meta *kptr_struct_meta = env->insn_aux_data[insn_idx].kptr_struct_meta; + struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) }; + u64 obj_new_size = env->insn_aux_data[insn_idx].obj_new_size; + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_1, obj_new_size); + insn_buf[1] = addr[0]; + insn_buf[2] = addr[1]; + insn_buf[3] = *insn; + *cnt = 4; + } return 0; } @@ -14798,9 +14879,19 @@ static int do_misc_fixups(struct bpf_verifier_env *env) if (insn->src_reg == BPF_PSEUDO_CALL) continue; if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) { - ret = fixup_kfunc_call(env, insn); + ret = fixup_kfunc_call(env, insn, insn_buf, i + delta, &cnt); if (ret) return ret; + if (cnt == 0) + continue; + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta += cnt - 1; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; continue; } diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h new file mode 100644 index 000000000000..aeb6a7fcb7c4 --- /dev/null +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -0,0 +1,25 @@ +#ifndef __BPF_EXPERIMENTAL__ +#define __BPF_EXPERIMENTAL__ + +#include +#include +#include +#include + +/* Description + * Allocates an object of the type represented by 'local_type_id' in + * program BTF. User may use the bpf_core_type_id_local macro to pass the + * type ID of a struct in program BTF. + * + * The 'local_type_id' parameter must be a known constant. + * The 'meta' parameter is a hidden argument that is ignored. + * Returns + * A pointer to an object of the type corresponding to the passed in + * 'local_type_id', or NULL on failure. + */ +extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; + +/* Convenience macro to wrap over bpf_obj_new_impl */ +#define bpf_obj_new(type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) + +#endif From patchwork Fri Nov 11 19:32:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040746 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC4E5C4332F for ; Fri, 11 Nov 2022 19:34:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234031AbiKKTeS (ORCPT ); Fri, 11 Nov 2022 14:34:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234055AbiKKTeR (ORCPT ); Fri, 11 Nov 2022 14:34:17 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E029B7BE48 for ; Fri, 11 Nov 2022 11:34:15 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id g62so5683426pfb.10 for ; Fri, 11 Nov 2022 11:34:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Uvy4NXZKIbN/6I7uJYBvLP272GMrTz5GbhVR2rUVS0Y=; b=NOMPqQt/0Gnf1mlAJi7jT9xm88zluJ+3HrLBXgLfpaZDK55xuczrgdUkTP5ChJ8KmD 7SoCbFEveWP7MdtiR7vAc+bKqSbY2aZ5zTzmr1syYlCsVq8/NelKi6oxfmjigrFBGHC6 cmVDPiiJoUN+yAWKGOMtpx5zKzxxGk0aAC4mehrErQDUxZUZNV/Z+MUkUBMctl78Lw5h P8fJVCBfh3XiYdbyAg/U3TlRSl+qks8rLY6R9qqneb7C9YVRXNux5MUVsvJA+TkpNakR 9+aMCATyS3ibf4WX7ZKlZ9dsmRiMqTaJuDSMJqgJLxbt3hU9dG/fIUSYy1MkYtHvXY6D ksEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Uvy4NXZKIbN/6I7uJYBvLP272GMrTz5GbhVR2rUVS0Y=; b=38PEp8/xGltH/Dv5MWqLRchCxkCjhbVIiZEWmd7bfaXSCXKby96YQylDOhtAwUa2xH AkH2GJM77qQEp3riD2DrhS7vY39PLjulmBN76j68QNTy2pw0491uQbWOBlZk2qvSMYiD x9bRP2LlqwcxsddZJDqdbeMnAsD9Yfs0IwRhxOdCj9IfiOrlZQjOSzjaphPyUwlI9kN1 b/3h0iQi29bhIBYibvBXogADHfaFHjkNGWtgNZK9LIJpvY9itx2aWLYNV1XgDfWhdu+Y KJ/bDREM2BdA0a0vFwtcmcOmV9Q5MCyFymq1kL6nEXkZYOOOa1lHLiOU81wUJQtih8MQ QSVw== X-Gm-Message-State: ANoB5pntO9skqFE7vZHS4p3k2VfcfQcjclkwr8+nZQpIlbIwpLBsTYYx yZBzAZMwWDhamfzA+G5mvaegn8KGdE+Kiw== X-Google-Smtp-Source: AA0mqf5y0OAIkoq61OZarA99+k8Q/I9KW4ido6RGwdRVaVeRVHGXli99p35w2qNw5pRQbDlpcdfHtA== X-Received: by 2002:a62:6001:0:b0:52f:db84:81cf with SMTP id u1-20020a626001000000b0052fdb8481cfmr4236261pfb.26.1668195255333; Fri, 11 Nov 2022 11:34:15 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id k2-20020a628402000000b0056bd4ec964csm1941188pfd.194.2022.11.11.11.34.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:15 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 18/26] bpf: Introduce bpf_obj_drop Date: Sat, 12 Nov 2022 01:02:16 +0530 Message-Id: <20221111193224.876706-19-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7764; i=memxor@gmail.com; h=from:subject; bh=iFiLM3YROrBHvTmWIWtn4om7dtKuji+UnL/mVE5Lchk=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIovDlDjQISoeeodXnG6Rft4YOv0XyZu2R8Y9Oy uBnV4++JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8Ryv/2EA ClfHOYAZFnLnUYntH0N8LRtSW0kFolodOt6XoY0O+eC9jS8AiSLqKPus3BP7x8s0HoIeLlYLRJwAxS zwFaYNtBU31VSH4sFRazT0KKXBH7yd9o8TaMfZj1LfbrHA+Q5LKcUNRMR1cGZ3/QjKnt0xpaXcEKWk gd4gAS6iQGP3UM+JBU0ivEjQ35XCxW9R1h2HZY6OTZlIJEIYnK+TPP7OyKaoUgFEHjFjjLXVL/lTTS zNiYPNMgZ2wmIjSCVXk9RxjZBRc0vb8zLE3l5DYFcz7QG2Rr+CxMsN3dw8QI2gTkzKdPObYwnJ8Dg8 M19qvWQ4RGGCR5gCUvNRkmrdzOBCENZTkUBvEBb5AzQsrcte83jaPnSiSbc2jJ2iTpZahz7n5YWhOV 4FC8nnBvrJX2TSSNRSTr01kX7eVTs2LypYse2Mi7b7FNE5xMnmlOAiA0mM3ipbwKCe+bUL/9ogbnsk IzHux21IhFLNPKkeCqc9y1P0gjkPUh6ozFplzIJvbxYrJvZAdE/Y38KqIKItMehVO8Quqo3vx5/op3 5CyFrROMu8TCxosdiM4OtSNcS/IsuxphoB+MTRJ7f4UQjyrQ8tbEpaLQpSdkg2+oF6DMMqttVpsAqy E5sBe1wL3i5IxlsCElR2yVba31hzrLLZrFlFkn/6zhhPVbCGuUoFk23BcyqA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce bpf_obj_drop, which is the kfunc used to free allocated objects (allocated using bpf_obj_new). Pairing with bpf_obj_new, it implicitly destructs the fields part of object automatically without user intervention. Just like the previous patch, btf_struct_meta that is needed to free up the special fields is passed as a hidden argument to the kfunc. For the user, a convenience macro hides over the kernel side kfunc which is named bpf_obj_drop_impl. Continuing the previous example: void prog(void) { struct foo *f; f = bpf_obj_new(typeof(*f)); if (!f) return; bpf_obj_drop(f); } Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/helpers.c | 11 ++++ kernel/bpf/verifier.c | 66 +++++++++++++++---- .../testing/selftests/bpf/bpf_experimental.h | 13 ++++ 3 files changed, 79 insertions(+), 11 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index c4f1c22cc44c..71d803ca0c1d 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1770,6 +1770,16 @@ void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign) return p; } +void bpf_obj_drop_impl(void *p__alloc, void *meta__ign) +{ + struct btf_struct_meta *meta = meta__ign; + void *p = p__alloc; + + if (meta) + bpf_obj_free_fields(meta->record, p); + bpf_mem_free(&bpf_global_ma, p); +} + __diag_pop(); BTF_SET8_START(generic_btf_ids) @@ -1777,6 +1787,7 @@ BTF_SET8_START(generic_btf_ids) BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) #endif BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE) BTF_SET8_END(generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index b7c663a2dbfa..5f98aa4c9d7c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7866,6 +7866,10 @@ struct bpf_kfunc_call_arg_meta { u64 value; bool found; } arg_constant; + struct { + struct btf *btf; + u32 btf_id; + } arg_obj_drop; }; static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) @@ -7944,6 +7948,11 @@ static bool is_kfunc_arg_sfx_ignore(const struct btf *btf, const struct btf_para return __kfunc_param_match_suffix(btf, arg, "__ign"); } +static bool is_kfunc_arg_alloc_obj(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__alloc"); +} + static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, const struct btf_param *arg, const struct bpf_reg_state *reg, @@ -8044,6 +8053,7 @@ static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_CTX, + KF_ARG_PTR_TO_ALLOC_BTF_ID, /* Allocated object */ KF_ARG_PTR_TO_KPTR_STRONG, /* PTR_TO_KPTR but type specific */ KF_ARG_PTR_TO_DYNPTR, KF_ARG_PTR_TO_BTF_ID, /* Also covers reg2btf_ids conversions */ @@ -8051,6 +8061,20 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_MEM_SIZE, /* Size derived from next argument, skip it */ }; +enum special_kfunc_type { + KF_bpf_obj_new_impl, + KF_bpf_obj_drop_impl, +}; + +BTF_SET_START(special_kfunc_set) +BTF_ID(func, bpf_obj_new_impl) +BTF_ID(func, bpf_obj_drop_impl) +BTF_SET_END(special_kfunc_set) + +BTF_ID_LIST(special_kfunc_list) +BTF_ID(func, bpf_obj_new_impl) +BTF_ID(func, bpf_obj_drop_impl) + static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, @@ -8071,6 +8095,9 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (btf_get_prog_ctx_type(&env->log, meta->btf, t, resolve_prog_type(env->prog), argno)) return KF_ARG_PTR_TO_CTX; + if (is_kfunc_arg_alloc_obj(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_ALLOC_BTF_ID; + if (is_kfunc_arg_kptr_get(meta, argno)) { if (!btf_type_is_ptr(ref_t)) { verbose(env, "arg#0 BTF type must be a double pointer for kptr_get kfunc\n"); @@ -8289,6 +8316,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return kf_arg_type; switch (kf_arg_type) { + case KF_ARG_PTR_TO_ALLOC_BTF_ID: case KF_ARG_PTR_TO_BTF_ID: if (!is_kfunc_trusted_args(meta)) break; @@ -8325,6 +8353,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return -EINVAL; } break; + case KF_ARG_PTR_TO_ALLOC_BTF_ID: + if (reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + verbose(env, "arg#%d expected pointer to allocated object\n", i); + return -EINVAL; + } + if (!reg->ref_obj_id) { + verbose(env, "allocated object must be referenced\n"); + return -EINVAL; + } + if (meta->btf == btf_vmlinux && + meta->func_id == special_kfunc_list[KF_bpf_obj_drop_impl]) { + meta->arg_obj_drop.btf = reg->btf; + meta->arg_obj_drop.btf_id = reg->btf_id; + } + break; case KF_ARG_PTR_TO_KPTR_STRONG: if (reg->type != PTR_TO_MAP_VALUE) { verbose(env, "arg#0 expected pointer to map value\n"); @@ -8395,17 +8438,6 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return 0; } -enum special_kfunc_type { - KF_bpf_obj_new_impl, -}; - -BTF_SET_START(special_kfunc_set) -BTF_ID(func, bpf_obj_new_impl) -BTF_SET_END(special_kfunc_set) - -BTF_ID_LIST(special_kfunc_list) -BTF_ID(func, bpf_obj_new_impl) - static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -8532,6 +8564,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, env->insn_aux_data[insn_idx].obj_new_size = ret_t->size; env->insn_aux_data[insn_idx].kptr_struct_meta = btf_find_struct_meta(ret_btf, ret_btf_id); + } else if (meta.func_id == special_kfunc_list[KF_bpf_obj_drop_impl]) { + env->insn_aux_data[insn_idx].kptr_struct_meta = + btf_find_struct_meta(meta.arg_obj_drop.btf, + meta.arg_obj_drop.btf_id); } else { verbose(env, "kernel function %s unhandled dynamic return type\n", meta.func_name); @@ -14737,6 +14773,14 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, insn_buf[2] = addr[1]; insn_buf[3] = *insn; *cnt = 4; + } else if (desc->func_id == special_kfunc_list[KF_bpf_obj_drop_impl]) { + struct btf_struct_meta *kptr_struct_meta = env->insn_aux_data[insn_idx].kptr_struct_meta; + struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) }; + + insn_buf[0] = addr[0]; + insn_buf[1] = addr[1]; + insn_buf[2] = *insn; + *cnt = 3; } return 0; } diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index aeb6a7fcb7c4..8473395a11af 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -22,4 +22,17 @@ extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; /* Convenience macro to wrap over bpf_obj_new_impl */ #define bpf_obj_new(type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) +/* Description + * Free an allocated object. All fields of the object that require + * destruction will be destructed before the storage is freed. + * + * The 'meta' parameter is a hidden argument that is ignored. + * Returns + * Void. + */ +extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; + +/* Convenience macro to wrap over bpf_obj_drop_impl */ +#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) + #endif From patchwork Fri Nov 11 19:32:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040748 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B7AEC433FE for ; Fri, 11 Nov 2022 19:34:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234025AbiKKTeV (ORCPT ); Fri, 11 Nov 2022 14:34:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233898AbiKKTeU (ORCPT ); Fri, 11 Nov 2022 14:34:20 -0500 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5877576FB2 for ; Fri, 11 Nov 2022 11:34:19 -0800 (PST) Received: by mail-pj1-x1041.google.com with SMTP id c15-20020a17090a1d0f00b0021365864446so5510132pjd.4 for ; Fri, 11 Nov 2022 11:34:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JW6kTYVA+y8b43HdcehxOTQglQ0619cSpShurQ4DCNI=; b=VQ/++5mL2S2MoypmSgUmH8N9aeORZfHN3I7Vx9J846awQbfjrk+6JjtuMlHXvodZ9j H4f6tsP7HOwaqLUmZ0Nk6hfg3dlY4VYPD1lKPEgw1mNFLBQ7sC8UAxM9ngsjbQdrq0Av lP4YlkjmHlhyQ8xGgWhC6AsgObszTjBHIoevwl7zwuRM6pbUKGjT455oqhCuHcjZ79AJ /9s44YZNRUs6gt0FGyvVKsZ0ciV1LsSJTY8DIkA2/1JG0IrTqNRIpS+kiPW6OoSO2nfB 6Z3YEhVeWyXg8+K6R2rlmVlyzOtMITfRsOrohqOZbo3CVmmU2vzeu4/xtoYahvTMz2Be /yuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JW6kTYVA+y8b43HdcehxOTQglQ0619cSpShurQ4DCNI=; b=bUCQpBIg+BXEWJJpuEILG4xD+spCsS+Yyh96Rtf4Cg1MyTvXIrHk4FdfBCEibj5TLd vUY2enT0RmiAQXUsMcsC2pVLtdflrEHz6WqUSyAtgoqbhqmXBJvvB/qkTQs9i9VA8ep2 hth/OCjWRGqhN7UCvOGV6XpsNfeeaqlxmVPsx0HgBUDXN/rbQvNYuc36+9/12Z8vrWMb BDfXh3urr4EZ+ApNV4Pu8BYidfu3z5cB/RollnOlFwRg4ntcxC3/10GI9xnIAnSgD1NU WMRu+x6jQYPTKK/EI45yFXpT7yhVPoHK/OaLC76nRtAOg8kTkBrGh5U4g7YEJjWMD6Ee cp6A== X-Gm-Message-State: ANoB5pm9iIlfH3mWq7MwLIxvmYqYn0iTmNHtK26SsiET2/lIwu1AVdEc 02aMF/51dmvAXXQT4teO4u8aikIk+PaEYw== X-Google-Smtp-Source: AA0mqf4uiAKBfpHQt34rkbp6pZ8Y24BEK2LS6tq2fEuWbpfnsVerxjcqLG9nsn/ZpuYnbZGRBewWoA== X-Received: by 2002:a17:90a:9ce:b0:200:40a2:eaaa with SMTP id 72-20020a17090a09ce00b0020040a2eaaamr3483870pjo.68.1668195258621; Fri, 11 Nov 2022 11:34:18 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id d17-20020a170902ced100b001868ed86a95sm2090498plg.174.2022.11.11.11.34.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:18 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 19/26] bpf: Permit NULL checking pointer with non-zero fixed offset Date: Sat, 12 Nov 2022 01:02:17 +0530 Message-Id: <20221111193224.876706-20-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2227; i=memxor@gmail.com; h=from:subject; bh=QQzN5i+KVGSbJY/B6kG18+QcbhNJQ0FLx/9bPyIyj94=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIoyWIZpGQdJsqdKnCM9pbpjWfQelScT3yZZP6B sYxnWaOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKAAKCRBM4MiGSL8RylQwEA CTjXtLH73tr6rq8rQOltYG54cDIs/NYIiN/BqQoMlMJr+xjrjYW3IMb5qqbx9MU0RWFEYmgKEOnd5T txpaLRSNbnpkGAm0/Ken0E1hsxLc77IM4drbon5IMbhFrJoExpctmP2GTaQAXjOTQFSnRhWcqCqoyQ 6shobJNpfQwNdY4WX1FawP+nEAvM2UrpSMikvtzdYqBQkiAl3bLlZevfD0ILJTjZ7gbXI0RMGQ/oRr J71kUWiFfF6JfhlxsCBzPV25hZPZN8W0qweMkr34VflaHtKGTPz+FMFOnz9BGNl6sLzpdNMhZQnk5I hE5luMmmWMKKaAJTNHA0AtQEpJ4qj91TkqtxFKxEh+hT4/Ie2pHgN+4xYYRolntre2nD3dQIolmNRw gkpr2CaFGZbL/oRXKwF9/k/+N5titE8uI8NM/5Jwx0rVrB8rYzJgBjP9sU7aRIPH5gIAiZwt9HVSLg MRBT0XB1envsGDsOVhmE4sLuazMTbYYKjx1tJ9J/BGFShughLRO8EUEkSijpmDlnTC049H7kog+Zck +J17r48PbImWN1CVWKz2y74HXreh0UEacw+yiwWb8DdDvwoCtA9OyJhITyiKum1zG4r+OGc4sHrI0l yGOgcA19Aey3HUSkp8pIdGt+3sUCPuwH5Xh/UIDKiKYiSBE9brpHuCznK6Pw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Pointer increment on seeing PTR_MAYBE_NULL is already protected against, hence make an exception for PTR_TO_BTF_ID | MEM_ALLOC while still keeping the warning for other unintended cases that might creep in. bpf_list_pop_{front,_back} helpers planned to be introduced in next commit will return a MEM_ALLOC register with incremented offset pointing to bpf_list_node field. The user is supposed to then obtain the pointer to the entry using container_of after NULL checking it. The current restrictions trigger a warning when doing the NULL checking. Revisiting the reason, it is meant as an assertion which seems to actually work and catch the bad case. Hence, under no other circumstances can reg->off be non-zero for a register that has the PTR_MAYBE_NULL type flag set. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/verifier.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5f98aa4c9d7c..7f44885d04dc 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10791,15 +10791,20 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state, { if (type_may_be_null(reg->type) && reg->id == id && !WARN_ON_ONCE(!reg->id)) { - if (WARN_ON_ONCE(reg->smin_value || reg->smax_value || - !tnum_equals_const(reg->var_off, 0) || - reg->off)) { + if (reg->smin_value || reg->smax_value || !tnum_equals_const(reg->var_off, 0) || reg->off) { /* Old offset (both fixed and variable parts) should * have been known-zero, because we don't allow pointer * arithmetic on pointers that might be NULL. If we * see this happening, don't convert the register. + * + * But in some cases, some helpers that return local + * kptrs advance offset for the returned pointer. + * In those cases, it is fine to expect to see reg->off. */ - return; + if (WARN_ON_ONCE(reg->type != (PTR_TO_BTF_ID | MEM_ALLOC | PTR_MAYBE_NULL))) + return; + if (WARN_ON_ONCE(reg->smin_value || reg->smax_value || !tnum_equals_const(reg->var_off, 0))) + return; } if (is_null) { reg->type = SCALAR_VALUE; From patchwork Fri Nov 11 19:32:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040749 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAE42C4332F for ; Fri, 11 Nov 2022 19:34:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233898AbiKKTe3 (ORCPT ); Fri, 11 Nov 2022 14:34:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234090AbiKKTeZ (ORCPT ); Fri, 11 Nov 2022 14:34:25 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3ADEE787BD for ; Fri, 11 Nov 2022 11:34:23 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id e129so5117703pgc.9 for ; Fri, 11 Nov 2022 11:34:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1na8Keon9U5+0N7GdQ/P2q4OXwr+7tdDZS61apE46dU=; b=mH2PPkcwD/8wMr3GzT5eBAl13qMppec/UK/CsZ0jsOkfkqPWnzGoouKVpdCWqh7tn1 s6omKMdLRg1GbHoOLbfjKhjJAWB2AMgRqs82HNGko1vlNrb757KVu6/h7R5ms/qdK9FW 4/EjvqpS7J4jYcjwTq0kVTkpewM1k1LFhDZXnXepMvjJDAhMGa5+boH7Kgs5yfX1iCsT MG+CWM5QnTJZ4agjFxvcidpOnYpcomlgEFqt803UA7XHP4aHbFXMVk6V1fTKcdPhLjzg BqGiPM9yOY9b0Guea+4J3glPvrgbQLskgRsO/bJXhusrBxauDGJfMoxnVq/Go0QCKHLb WGVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1na8Keon9U5+0N7GdQ/P2q4OXwr+7tdDZS61apE46dU=; b=ZD4umG5R2YlJhGiVAsETtmUErojHTr1tRwRbNbjERPchfu/1X5fEpQzkY1rmbOrOGJ 3ZrWlk5c5OwAMWX9o9nOj9lkY4EvQjA2ajNzKF0pEceTYPBCE2v6xN+Dz0YZ3oOeCZtT L0GjI6EugTqacKtFVxwKy3gbFnREQglrbXF69qmBYRTRPTrMMZN6pA7jhCnSHTfFfKyD bMxKfkneW/QB7kg9he3DZ4p1FVH7Ml1BvDTlN+DoYqeAOBIGELEjZp/QTMVUhrQ6X8yH 7vuNvZzcgctUw2T/NE4MBTPmXmno8ebSuC41bk3ycZwbuy/fD21VztJp9IAcs8ukUVfV fv0w== X-Gm-Message-State: ANoB5pntA4gpY8PSyQe1D2ENCQ5ifw1zClLP84U+PLat0+OOx7mGsseL Jeo/L0tNGJLjbyYF9Og56E3ofuMuEMmT0A== X-Google-Smtp-Source: AA0mqf6yAXz+6srMEz2MZJbDXi42hxePnwZMGbBWnyvgBHSRWuDc7xcY08/VDpiUxzPqQWpfkyp2wg== X-Received: by 2002:aa7:8556:0:b0:563:b1bc:7f98 with SMTP id y22-20020aa78556000000b00563b1bc7f98mr4148092pfn.29.1668195262524; Fri, 11 Nov 2022 11:34:22 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id a27-20020a630b5b000000b00460ea630c1bsm1669176pgl.46.2022.11.11.11.34.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:22 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 20/26] bpf: Introduce single ownership BPF linked list API Date: Sat, 12 Nov 2022 01:02:18 +0530 Message-Id: <20221111193224.876706-21-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=20838; i=memxor@gmail.com; h=from:subject; bh=6gG4Oiy3q2ZQd1yhf0uTdLn6tho0O9NSCPb/KPvn67E=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIpBh8Vvtg4lTxU+861QD6qVUdFzPDRpI93xn+y 24/ejfyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8Ryk/UEA Cn7eeIhygg63Acq6sifLdLYJildThWIQhY4jUdFwoMGEgydv2Un33G/KH4K7TyuLSnIbYEQtpgeNdC LctD5D7TZ9CAnBqxXxTtutIk93GjWMjPJjvH4bfkje9SrkOZcxQQFv0ybQTgbdlib//Jaou1lhuW5e TGUesIUKcIZE+6jrtJls7a6Ja1I4LYivkuAvhjlkRvtaCoJT7O50atvM97r5xdeBd61fjo71QJ3EZ6 FyGqf1EbnU1ePcHWDOE0NKDoxWYBP2eyhCVrjYJtvPrf/Nam1paxEz5Y5eNr+ZfBJg24L+lPu6rCx6 jYSrzRBmhrhuBrLdgqPYjmZ9Kp5OGhMeYHItKYPZYMWMXmauvV5oaIDpuco0/PBYzPEBSuSqsaekkX 6Q8v6TMW/aoYLmq6M2IsbD7VhUYmY6uLSIWjcCzvvP02lnvxB/X7b7JBpTJX4BX7CWO8JU0jXCthke omef0XpQ7GNJ6RbiSC9rS0LX63HPC54MPaEGyvuSqsEoixJjSQeJgUlv6ybaBYBjqj/hCdRW1YIrvM +bkt5eTATGwAUxGC4ApE6fJhHthSjt+o3lWk03yHOyX5rhwsFkUJvzbepZjyBE5jIJ5jmqTuENjwNN qenQjRF6uz39meCKZ2iDv5GeTAw1oup13vP27l55syKLPcnleZZLjyJgUMWQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add a linked list API for use in BPF programs, where it expects protection from the bpf_spin_lock in the same allocation as the bpf_list_head. For now, only one bpf_spin_lock can be present hence that is assumed to be the one protecting the bpf_list_head. The following functions are added to kick things off: // Add node to beginning of list void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node); // Add node to end of list void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node); // Remove node at beginning of list and return it struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head); // Remove node at end of list and return it struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head); The lock protecting the bpf_list_head needs to be taken for all operations. The verifier ensures that the lock that needs to be taken is always held, and only the correct lock is taken for these operations. These checks are made statically by relying on the reg->id preserved for registers pointing into regions having both bpf_spin_lock and the objects protected by it. The comment over check_reg_allocation_locked in this change describes the logic in detail. Note that bpf_list_push_front and bpf_list_push_back are meant to consume the object containing the node in the 1st argument, however that specific mechanism is intended to not release the ref_obj_id directly until the bpf_spin_unlock is called. In this commit, nothing is done, but the next commit will be introducing logic to handle this case, so it has been left as is for now. bpf_list_pop_front and bpf_list_pop_back delete the first or last item of the list respectively, and return pointer to the element at the list_node offset. The user can then use container_of style macro to get the actual entry type. The verifier however statically knows the actual type, so the safety properties are still preserved. With these additions, programs can now manage their own linked lists and store their objects in them. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/helpers.c | 55 +++- kernel/bpf/verifier.c | 292 +++++++++++++++++- .../testing/selftests/bpf/bpf_experimental.h | 28 ++ 3 files changed, 361 insertions(+), 14 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 71d803ca0c1d..212e791d7452 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1780,6 +1780,50 @@ void bpf_obj_drop_impl(void *p__alloc, void *meta__ign) bpf_mem_free(&bpf_global_ma, p); } +static void __bpf_list_add(struct bpf_list_node *node, struct bpf_list_head *head, bool tail) +{ + struct list_head *n = (void *)node, *h = (void *)head; + + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + if (unlikely(!n->next)) + INIT_LIST_HEAD(n); + tail ? list_add_tail(n, h) : list_add(n, h); +} + +void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node) +{ + return __bpf_list_add(node, head, false); +} + +void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node) +{ + return __bpf_list_add(node, head, true); +} + +static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tail) +{ + struct list_head *n, *h = (void *)head; + + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + if (list_empty(h)) + return NULL; + n = tail ? h->prev : h->next; + list_del_init(n); + return (struct bpf_list_node *)n; +} + +struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head) +{ + return __bpf_list_del(head, false); +} + +struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) +{ + return __bpf_list_del(head, true); +} + __diag_pop(); BTF_SET8_START(generic_btf_ids) @@ -1788,6 +1832,10 @@ BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) #endif BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_list_push_front) +BTF_ID_FLAGS(func, bpf_list_push_back) +BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL) BTF_SET8_END(generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = { @@ -1797,7 +1845,12 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { static int __init kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set); + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set); + if (ret) + return ret; + return register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &generic_kfunc_set); } late_initcall(kfunc_init); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7f44885d04dc..7ad077d85e3b 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7870,6 +7870,9 @@ struct bpf_kfunc_call_arg_meta { struct btf *btf; u32 btf_id; } arg_obj_drop; + struct { + struct btf_field *field; + } arg_list_head; }; static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) @@ -7980,13 +7983,17 @@ static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, enum { KF_ARG_DYNPTR_ID, + KF_ARG_LIST_HEAD_ID, + KF_ARG_LIST_NODE_ID, }; BTF_ID_LIST(kf_arg_btf_ids) BTF_ID(struct, bpf_dynptr_kern) +BTF_ID(struct, bpf_list_head) +BTF_ID(struct, bpf_list_node) -static bool is_kfunc_arg_dynptr(const struct btf *btf, - const struct btf_param *arg) +static bool __is_kfunc_ptr_arg_type(const struct btf *btf, + const struct btf_param *arg, int type) { const struct btf_type *t; u32 res_id; @@ -7999,7 +8006,22 @@ static bool is_kfunc_arg_dynptr(const struct btf *btf, t = btf_type_skip_modifiers(btf, t->type, &res_id); if (!t) return false; - return btf_types_are_same(btf, res_id, btf_vmlinux, kf_arg_btf_ids[KF_ARG_DYNPTR_ID]); + return btf_types_are_same(btf, res_id, btf_vmlinux, kf_arg_btf_ids[type]); +} + +static bool is_kfunc_arg_dynptr(const struct btf *btf, const struct btf_param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_DYNPTR_ID); +} + +static bool is_kfunc_arg_list_head(const struct btf *btf, const struct btf_param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_LIST_HEAD_ID); +} + +static bool is_kfunc_arg_list_node(const struct btf *btf, const struct btf_param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_LIST_NODE_ID); } /* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ @@ -8056,6 +8078,8 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_ALLOC_BTF_ID, /* Allocated object */ KF_ARG_PTR_TO_KPTR_STRONG, /* PTR_TO_KPTR but type specific */ KF_ARG_PTR_TO_DYNPTR, + KF_ARG_PTR_TO_LIST_HEAD, + KF_ARG_PTR_TO_LIST_NODE, KF_ARG_PTR_TO_BTF_ID, /* Also covers reg2btf_ids conversions */ KF_ARG_PTR_TO_MEM, KF_ARG_PTR_TO_MEM_SIZE, /* Size derived from next argument, skip it */ @@ -8064,16 +8088,28 @@ enum kfunc_ptr_arg_type { enum special_kfunc_type { KF_bpf_obj_new_impl, KF_bpf_obj_drop_impl, + KF_bpf_list_push_front, + KF_bpf_list_push_back, + KF_bpf_list_pop_front, + KF_bpf_list_pop_back, }; BTF_SET_START(special_kfunc_set) BTF_ID(func, bpf_obj_new_impl) BTF_ID(func, bpf_obj_drop_impl) +BTF_ID(func, bpf_list_push_front) +BTF_ID(func, bpf_list_push_back) +BTF_ID(func, bpf_list_pop_front) +BTF_ID(func, bpf_list_pop_back) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) BTF_ID(func, bpf_obj_new_impl) BTF_ID(func, bpf_obj_drop_impl) +BTF_ID(func, bpf_list_push_front) +BTF_ID(func, bpf_list_push_back) +BTF_ID(func, bpf_list_pop_front) +BTF_ID(func, bpf_list_pop_back) static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, @@ -8116,6 +8152,12 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (is_kfunc_arg_dynptr(meta->btf, &args[argno])) return KF_ARG_PTR_TO_DYNPTR; + if (is_kfunc_arg_list_head(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_LIST_HEAD; + + if (is_kfunc_arg_list_node(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_LIST_NODE; + if ((base_type(reg->type) == PTR_TO_BTF_ID || reg2btf_ids[base_type(reg->type)])) { if (!btf_type_is_struct(ref_t)) { verbose(env, "kernel function %s args#%d pointer type %s %s is not supported\n", @@ -8211,6 +8253,194 @@ static int process_kf_arg_ptr_to_kptr_strong(struct bpf_verifier_env *env, return 0; } +/* Implementation details: + * + * Each register points to some region of memory, which we define as an + * allocation. Each allocation may embed a bpf_spin_lock which protects any + * special BPF objects (bpf_list_head, bpf_rb_root, etc.) part of the same + * allocation. The lock and the data it protects are colocated in the same + * memory region. + * + * Hence, everytime a register holds a pointer value pointing to such + * allocation, the verifier preserves a unique reg->id for it. + * + * The verifier remembers the lock 'ptr' and the lock 'id' whenever + * bpf_spin_lock is called. + * + * To enable this, lock state in the verifier captures two values: + * active_lock.ptr = Register's type specific pointer + * active_lock.id = A unique ID for each register pointer value + * + * Currently, PTR_TO_MAP_VALUE and PTR_TO_BTF_ID | MEM_ALLOC are the two + * supported register types. + * + * The active_lock.ptr in case of map values is the reg->map_ptr, and in case of + * allocated objects is the reg->btf pointer. + * + * The active_lock.id is non-unique for maps supporting direct_value_addr, as we + * can establish the provenance of the map value statically for each distinct + * lookup into such maps. They always contain a single map value hence unique + * IDs for each pseudo load pessimizes the algorithm and rejects valid programs. + * + * So, in case of global variables, they use array maps with max_entries = 1, + * hence their active_lock.ptr becomes map_ptr and id = 0 (since they all point + * into the same map value as max_entries is 1, as described above). + * + * In case of inner map lookups, the inner map pointer has same map_ptr as the + * outer map pointer (in verifier context), but each lookup into an inner map + * assigns a fresh reg->id to the lookup, so while lookups into distinct inner + * maps from the same outer map share the same map_ptr as active_lock.ptr, they + * will get different reg->id assigned to each lookup, hence different + * active_lock.id. + * + * In case of allocated objects, active_lock.ptr is the reg->btf, and the + * reg->id is a unique ID preserved after the NULL pointer check on the pointer + * returned from bpf_obj_new. Each allocation receives a new reg->id. + */ +static int check_reg_allocation_locked(struct bpf_verifier_env *env, struct bpf_reg_state *reg) +{ + void *ptr; + u32 id; + + switch ((int)reg->type) { + case PTR_TO_MAP_VALUE: + ptr = reg->map_ptr; + break; + case PTR_TO_BTF_ID | MEM_ALLOC: + ptr = reg->btf; + break; + default: + verbose(env, "verifier internal error: unknown reg type for lock check\n"); + return -EFAULT; + } + id = reg->id; + + if (!env->cur_state->active_lock.ptr) + return -EINVAL; + if (env->cur_state->active_lock.ptr != ptr || + env->cur_state->active_lock.id != id) { + verbose(env, "held lock and object are not in the same allocation\n"); + return -EINVAL; + } + return 0; +} + +static bool is_bpf_list_api_kfunc(u32 btf_id) +{ + return btf_id == special_kfunc_list[KF_bpf_list_push_front] || + btf_id == special_kfunc_list[KF_bpf_list_push_back] || + btf_id == special_kfunc_list[KF_bpf_list_pop_front] || + btf_id == special_kfunc_list[KF_bpf_list_pop_back]; +} + +static int process_kf_arg_ptr_to_list_head(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, u32 regno, + struct bpf_kfunc_call_arg_meta *meta) +{ + struct btf_record *rec = NULL; + struct btf_field *field; + u32 list_head_off; + + if (meta->btf != btf_vmlinux || !is_bpf_list_api_kfunc(meta->func_id)) { + verbose(env, "verifier internal error: bpf_list_head argument for unknown kfunc\n"); + return -EFAULT; + } + + if (reg->type == PTR_TO_MAP_VALUE) { + rec = reg->map_ptr->record; + } else /* PTR_TO_BTF_ID | MEM_ALLOC */ { + struct btf_struct_meta *meta; + + meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (!meta) { + verbose(env, "bpf_list_head not found for allocated object\n"); + return -EINVAL; + } + rec = meta->record; + } + + if (!tnum_is_const(reg->var_off)) { + verbose(env, + "R%d doesn't have constant offset. bpf_list_head has to be at the constant offset\n", + regno); + return -EINVAL; + } + + list_head_off = reg->off + reg->var_off.value; + field = btf_record_find(rec, list_head_off, BPF_LIST_HEAD); + if (!field) { + verbose(env, "bpf_list_head not found at offset=%u\n", list_head_off); + return -EINVAL; + } + + /* All functions require bpf_list_head to be protected using a bpf_spin_lock */ + if (check_reg_allocation_locked(env, reg)) { + verbose(env, "bpf_spin_lock at off=%d must be held for bpf_list_head\n", + rec->spin_lock_off); + return -EINVAL; + } + + if (meta->arg_list_head.field) { + verbose(env, "verifier internal error: repeating bpf_list_head arg\n"); + return -EFAULT; + } + meta->arg_list_head.field = field; + return 0; +} + +static int process_kf_arg_ptr_to_list_node(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, u32 regno, + struct bpf_kfunc_call_arg_meta *meta) +{ + struct btf_struct_meta *struct_meta; + struct btf_field *field; + struct btf_record *rec; + u32 list_node_off; + + if (meta->btf != btf_vmlinux || + (meta->func_id != special_kfunc_list[KF_bpf_list_push_front] && + meta->func_id != special_kfunc_list[KF_bpf_list_push_back])) { + verbose(env, "verifier internal error: bpf_list_head argument for unknown kfunc\n"); + return -EFAULT; + } + + if (!tnum_is_const(reg->var_off)) { + verbose(env, + "R%d doesn't have constant offset. bpf_list_head has to be at the constant offset\n", + regno); + return -EINVAL; + } + + struct_meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (!struct_meta) { + verbose(env, "bpf_list_node not found for allocated object\n"); + return -EINVAL; + } + rec = struct_meta->record; + + list_node_off = reg->off + reg->var_off.value; + field = btf_record_find(rec, list_node_off, BPF_LIST_NODE); + if (!field || field->offset != list_node_off) { + verbose(env, "bpf_list_node not found at offset=%u\n", list_node_off); + return -EINVAL; + } + + field = meta->arg_list_head.field; + + if (!btf_struct_ids_match(&env->log, reg->btf, reg->btf_id, 0, field->list_head.btf, + field->list_head.value_btf_id, true)) { + verbose(env, "bpf_list_head value type does not match arg#1\n"); + return -EINVAL; + } + + if (list_node_off != field->list_head.node_offset) { + verbose(env, "arg#1 offset must be for bpf_list_node at off=%d\n", + field->list_head.node_offset); + return -EINVAL; + } + return 0; +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta) { const char *func_name = meta->func_name, *ref_tname; @@ -8331,6 +8561,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; case KF_ARG_PTR_TO_KPTR_STRONG: case KF_ARG_PTR_TO_DYNPTR: + case KF_ARG_PTR_TO_LIST_HEAD: + case KF_ARG_PTR_TO_LIST_NODE: case KF_ARG_PTR_TO_MEM: case KF_ARG_PTR_TO_MEM_SIZE: /* Trusted by default */ @@ -8395,6 +8627,33 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return -EINVAL; } break; + case KF_ARG_PTR_TO_LIST_HEAD: + if (reg->type != PTR_TO_MAP_VALUE && + reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + verbose(env, "arg#%d expected pointer to map value or allocated object\n", i); + return -EINVAL; + } + if (reg->type == (PTR_TO_BTF_ID | MEM_ALLOC) && !reg->ref_obj_id) { + verbose(env, "allocated object must be referenced\n"); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_list_head(env, reg, regno, meta); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_LIST_NODE: + if (reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + verbose(env, "arg#%d expected pointer to allocated object\n", i); + return -EINVAL; + } + if (!reg->ref_obj_id) { + verbose(env, "allocated object must be referenced\n"); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_list_node(env, reg, regno, meta); + if (ret < 0) + return ret; + break; case KF_ARG_PTR_TO_BTF_ID: /* Only base_type is checked, further checks are done here */ if (reg->type != PTR_TO_BTF_ID && @@ -8526,11 +8785,6 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); if (meta.btf == btf_vmlinux && btf_id_set_contains(&special_kfunc_set, meta.func_id)) { - if (!btf_type_is_void(ptr_type)) { - verbose(env, "kernel function %s must have void * return type\n", - meta.func_name); - return -EINVAL; - } if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { const struct btf_type *ret_t; struct btf *ret_btf; @@ -8568,6 +8822,15 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, env->insn_aux_data[insn_idx].kptr_struct_meta = btf_find_struct_meta(meta.arg_obj_drop.btf, meta.arg_obj_drop.btf_id); + } else if (meta.func_id == special_kfunc_list[KF_bpf_list_pop_front] || + meta.func_id == special_kfunc_list[KF_bpf_list_pop_back]) { + struct btf_field *field = meta.arg_list_head.field; + + mark_reg_known_zero(env, regs, BPF_REG_0); + regs[BPF_REG_0].type = PTR_TO_BTF_ID | MEM_ALLOC; + regs[BPF_REG_0].btf = field->list_head.btf; + regs[BPF_REG_0].btf_id = field->list_head.value_btf_id; + regs[BPF_REG_0].off = field->list_head.node_offset; } else { verbose(env, "kernel function %s unhandled dynamic return type\n", meta.func_name); @@ -13235,11 +13498,14 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } - if (env->cur_state->active_lock.ptr && - (insn->src_reg == BPF_PSEUDO_CALL || - insn->imm != BPF_FUNC_spin_unlock)) { - verbose(env, "function calls are not allowed while holding a lock\n"); - return -EINVAL; + if (env->cur_state->active_lock.ptr) { + if ((insn->src_reg == BPF_REG_0 && insn->imm != BPF_FUNC_spin_unlock) || + (insn->src_reg == BPF_PSEUDO_CALL) || + (insn->src_reg == BPF_PSEUDO_KFUNC_CALL && + (insn->off != 0 || !is_bpf_list_api_kfunc(insn->imm)))) { + verbose(env, "function calls are not allowed while holding a lock\n"); + return -EINVAL; + } } if (insn->src_reg == BPF_PSEUDO_CALL) err = check_func_call(env, insn, &env->insn_idx); diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 8473395a11af..d6b143275e82 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -35,4 +35,32 @@ extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; /* Convenience macro to wrap over bpf_obj_drop_impl */ #define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) +/* Description + * Add a new entry to the beginning of the BPF linked list. + * Returns + * Void. + */ +extern void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node) __ksym; + +/* Description + * Add a new entry to the end of the BPF linked list. + * Returns + * Void. + */ +extern void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node) __ksym; + +/* Description + * Remove the entry at the beginning of the BPF linked list. + * Returns + * Pointer to bpf_list_node of deleted entry, or NULL if list is empty. + */ +extern struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head) __ksym; + +/* Description + * Remove the entry at the end of the BPF linked list. + * Returns + * Pointer to bpf_list_node of deleted entry, or NULL if list is empty. + */ +extern struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) __ksym; + #endif From patchwork Fri Nov 11 19:32:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040750 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36403C433FE for ; Fri, 11 Nov 2022 19:34:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234064AbiKKTeb (ORCPT ); Fri, 11 Nov 2022 14:34:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234107AbiKKTe1 (ORCPT ); Fri, 11 Nov 2022 14:34:27 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5615E79D09 for ; Fri, 11 Nov 2022 11:34:26 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id 130so5130313pgc.5 for ; Fri, 11 Nov 2022 11:34:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A4dsQ02V0zwu/uhCRg/SWqmuk5Rn6ce3adp8M1rfL/M=; b=oB2ATCRyjmYEfKui05qok7f9ngsrGsCz8dQ8ZdgCKUkxLgm2uhS+LSBiDP6B5WKEWl JHgCevmDTBpJHelapbtsuWgDfXamXXfoNYVollSE034JsVLCV2tT7jJH7gp7Iv+Fx8FA 7wBmhnsFR9kY8YSljqxYKWlbXHxQEBploSDF7Yw46On2YlFmuPDlDTFFyieycW080puO RYtfwkGdPwCjv+fFYg4RLFH2/C9Y4qc3S5aEMdyNjlkILqBrU7qhSy15WFSBy0rKilMc XPtw4CskB45pqzgI/GsvMr7jHtwtdxpABiNwPtLVmfZIPo75mKPnzxmB6kjFCVO1cspk 2QBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A4dsQ02V0zwu/uhCRg/SWqmuk5Rn6ce3adp8M1rfL/M=; b=Y6U/m7I48MBC48OcJPbY9ro1pstYSZxqNmI+e3+GA20A+iCrvORuNhvcpGB6juqlXb eA+QHhIjwUFCZw0SXFrQbQLBn9UWaRV3muq5t0lXoz+kHGcfIHO8ZOSGl2FZ+rYOeuo2 Y10pWJZ5Xj7ZZp6y3E1Kn5uTrhk3TRFfFcRJ4nncKCkae8Dj9fWXU+HFsKIxw7f0mbAN w/4fR/4o+MQxegzUjAPJScbJw3U2VkJQ8AqNH9j5VZjWmLVj+c8EbEhoPg5JhJlXMZYl DrjoESfPXb7gf1H+IIJC6pNmyq7yEcHt5IHwMDgoWPH9OleXLw8sR+JKAS4O5n6GwnS9 AWTA== X-Gm-Message-State: ANoB5pmO1PClyWIX641P9pwNsVNvEZjauO/L39vZcLrqvTWliplePOTF Ij6nxYJoUWOdx9Wj1NUY86pGw4UJhwLT3w== X-Google-Smtp-Source: AA0mqf49JSGR3OUpO6IaMQeCMsl6MR3CagF+m4Ww5HYyHL7p7mvvt2Y4/plaqF33RKvQyLZrOE3Spw== X-Received: by 2002:a63:5465:0:b0:46f:f2f6:b06 with SMTP id e37-20020a635465000000b0046ff2f60b06mr2948214pgm.376.1668195265690; Fri, 11 Nov 2022 11:34:25 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id y9-20020aa793c9000000b0056c349f5c70sm1947572pff.79.2022.11.11.11.34.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:25 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 21/26] bpf: Add 'release on unlock' logic for bpf_list_push_{front,back} Date: Sat, 12 Nov 2022 01:02:19 +0530 Message-Id: <20221111193224.876706-22-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5174; i=memxor@gmail.com; h=from:subject; bh=98NlxO9AjvvSWvC60uBCkXv9bNqgJ0s+pR+qr5k/Qtg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIp3VVR2rhKDAIsXd8qqP/8zFn4ivY2JZwM8X0S KHP2zm+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8RylCaD/ wJg76QdcxxTImPSBouF2SBeSnYTxiuCUdhzV4CPiuHYbrHlmmmcZW5JY9Mt/+SqOR8X4HXh/GPbX27 +KaA7EZf76YYuE76oEN1gvmQWi0QZG2VSsPmRfMwfdiKanGrnllQ8ZppgHh8ANJV510vj4p5o7fq1+ Q+Qq4L9rquJ69pugvC5Gj27Xcy+BNwZo9FiUI3cF3acNP9mLqidfElry4HLzaC32jGhKX8hylw0zxP lVmK/6Ew6NRAj2N5wgSHl/TjqdAJ3e475bB2SX+UgeJY8OtEa2tYlmhnKLOX25ETnzBz6jAWmRIw/w 7t4NWHDr1Em6Mbg0Z7ibin4zY9SRwzwOJZMMjwh4NmFEMZiJMwP72e3e23yQSG7nTS4w273s1uwl9c KscmN5Yqm3wIx0AKrcTWHLWuTXqhsXHoYTGp3Oab1LIHoafPVuhyQCOytH3tchr/hU+wqf2L3+aDD1 d7Aj7kw5h42iNy5UF5+E1+oqahtlUraORKeQLPTF4BInr92GAZIk0FmJWr8sHNAfKdXMeayK0FSMh8 JMwt9Yq51qQCGCPtQtSSURQKUVPI7wSdZfEjZ0n+Eh/KrCyot+0u4WQl48pEp4slV5LyZkXQOBR856 gED24FMTole+W/cT+Esp2vlfXSt+ur0JFNVc7Znad8BZMUHCZAq7WYNk8X8Q== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit implements the delayed release logic for bpf_list_push_front and bpf_list_push_back. Once a node has been added to the list, it's pointer changes to PTR_UNTRUSTED. However, it is only released once the lock protecting the list is unlocked. For such PTR_TO_BTF_ID | MEM_ALLOC with PTR_UNTRUSTED set but an active ref_obj_id, it is still permitted to read them as long as the lock is held. Writing to them is not allowed. This allows having read access to push items we no longer own until we release the lock guarding the list, allowing a little more flexibility when working with these APIs. Note that enabling write support has fairly tricky interactions with what happens inside the critical section. Just as an example, currently, bpf_obj_drop is not permitted, but if it were, being able to write to the PTR_UNTRUSTED pointer while the object gets released back to the memory allocator would violate safety properties we wish to guarantee (i.e. not crashing the kernel). The memory could be reused for a different type in the BPF program or even in the kernel as it gets eventually kfree'd. Not enabling bpf_obj_drop inside the critical section would appear to prevent all of the above, but that is more of an artifical limitation right now. Since the write support is tangled with how we handle potential aliasing of nodes inside the critical section that may or may not be part of the list anymore, it has been deferred to a future patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf_verifier.h | 5 ++++ kernel/bpf/verifier.c | 48 +++++++++++++++++++++++++++++++++++- 2 files changed, 52 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 306fc1d6cc4a..740e774e1c7a 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -223,6 +223,11 @@ struct bpf_reference_state { * exiting a callback function. */ int callback_ref; + /* Mark the reference state to release the registers sharing the same id + * on bpf_spin_unlock (for nodes that we will lose ownership to but are + * safe to access inside the critical section). + */ + bool release_on_unlock; }; /* state of the program: diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7ad077d85e3b..161b66689fdc 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5643,7 +5643,9 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, cur->active_lock.ptr = btf; cur->active_lock.id = reg->id; } else { + struct bpf_func_state *fstate = cur_func(env); void *ptr; + int i; if (map) ptr = map; @@ -5661,6 +5663,16 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, } cur->active_lock.ptr = NULL; cur->active_lock.id = 0; + + for (i = 0; i < fstate->acquired_refs; i++) { + /* WARN because this reference state cannot be freed + * before this point, as bpf_spin_lock critical section + * does not allow functions that release the allocated + * object immediately. + */ + if (fstate->refs[i].release_on_unlock) + WARN_ON_ONCE(release_reference(env, fstate->refs[i].id)); + } } return 0; } @@ -8253,6 +8265,39 @@ static int process_kf_arg_ptr_to_kptr_strong(struct bpf_verifier_env *env, return 0; } +static int ref_set_release_on_unlock(struct bpf_verifier_env *env, u32 ref_obj_id) +{ + struct bpf_func_state *state = cur_func(env); + struct bpf_reg_state *reg; + int i; + + /* bpf_spin_lock only allows calling list_push and list_pop, no BPF + * subprogs, no global functions. This means that the references would + * not be released inside the critical section but they may be added to + * the reference state, and the acquired_refs are never copied out for a + * different frame as BPF to BPF calls don't work in bpf_spin_lock + * critical sections. + */ + if (!ref_obj_id) { + verbose(env, "verifier internal error: ref_obj_id is zero for release_on_unlock\n"); + return -EFAULT; + } + for (i = 0; i < state->acquired_refs; i++) { + if (state->refs[i].id == ref_obj_id) { + WARN_ON_ONCE(state->refs[i].release_on_unlock); + state->refs[i].release_on_unlock = true; + /* Now mark everyone sharing same ref_obj_id as untrusted */ + bpf_for_each_reg_in_vstate(env->cur_state, state, reg, ({ + if (reg->ref_obj_id == ref_obj_id) + reg->type |= PTR_UNTRUSTED; + })); + return 0; + } + } + verbose(env, "verifier internal error: ref state missing for ref_obj_id\n"); + return -EFAULT; +} + /* Implementation details: * * Each register points to some region of memory, which we define as an @@ -8438,7 +8483,8 @@ static int process_kf_arg_ptr_to_list_node(struct bpf_verifier_env *env, field->list_head.node_offset); return -EINVAL; } - return 0; + /* Set arg#1 for expiration after unlock */ + return ref_set_release_on_unlock(env, reg->ref_obj_id); } static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta) From patchwork Fri Nov 11 19:32:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040751 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D9A5C433FE for ; Fri, 11 Nov 2022 19:34:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233943AbiKKTee (ORCPT ); Fri, 11 Nov 2022 14:34:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234042AbiKKTec (ORCPT ); Fri, 11 Nov 2022 14:34:32 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEEB376FB2 for ; Fri, 11 Nov 2022 11:34:29 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id 130so5687711pfu.8 for ; Fri, 11 Nov 2022 11:34:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MvMB46Vqr7eLH23u46H2EgXkWOTOPfK9QqKgaAYTLEg=; b=Y7OaUW2ALX2i/u4HzAxZQGSQrJp7VqGBQWiTyG1tqRCt+FychaNRkKXnqNOO63dAUE L/ohaV+qhQZaKoTS+agDJXrlBEjVbzLPX1AKO/p+/CG4JULPFJns5GMArcLjiPwWL083 SucIWoqYrjACC2ouf9ox2R0w6RKcDXGtGmZ3uAQo73DM+DwG/LLP8d+37XOGko0qHDys UxnkP+v8SejO8Ubn3Ig6Ch4GrFcqfOz8w12Vw3vc2HXNUxiuAmaH/HjMTZMYr7E8SV4r ua8ntv17m4u0yeOmZNzcBEXZilcQVpjXOXSjUnFVjSfxMd+U72Sis8TBnjeAaVjRslOc UmiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MvMB46Vqr7eLH23u46H2EgXkWOTOPfK9QqKgaAYTLEg=; b=nzR1pTL3V8yUr7fweRCHCUwpQ60AXXewOxZbEW468u2eP8xUtC/Rbrri1NRmzvnQNR /VztPLmvVGLi+bC3jOlfw98uz9zzkwiRaBb67zPzQN/yVWALT3Yo+uBBYqeYDRoS/sTO gmwZxzroDhI9IlRQAsm5CKGZOLz8UYmYMTaDNSOqnPkKpdO6jqrxzv0P3HB/mlatX+g8 86vKNmZVWxVXDjEV4VckVsOMqgOQIFS6SI9eNwSK5Ki9QSNEST3bRWdkRfqTm2WaH6rH R8qoP5v68Z0yunnf9SZ5BZ3PsdE0/p6EKCSzy3nMvYKhYIJ1QQxY86c7FTluFnwB8u2d YgOg== X-Gm-Message-State: ANoB5plQw3C+G0d/qo922mlR//Ekek4GCDj41nyR0mDLzEfKMBL5zQC0 SGiW/YqgLWUwDuKTabFEV45heZMH1wO6BQ== X-Google-Smtp-Source: AA0mqf6i4hDtvQ1qF24qkGv3+/Do6qo6k6bBfdWr4lzcmvyEVyZii8qVd6z9cRNs0RHTaXuZm7JK0Q== X-Received: by 2002:aa7:8dc2:0:b0:56b:b520:3751 with SMTP id j2-20020aa78dc2000000b0056bb5203751mr4090116pfr.29.1668195269164; Fri, 11 Nov 2022 11:34:29 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id q3-20020a170902a3c300b0017a018221e2sm2078108plb.70.2022.11.11.11.34.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:28 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 22/26] selftests/bpf: Add __contains macro to bpf_experimental.h Date: Sat, 12 Nov 2022 01:02:20 +0530 Message-Id: <20221111193224.876706-23-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=956; i=memxor@gmail.com; h=from:subject; bh=VcTPpE/Iv5Qqfs22C7MYEC67+XombxopywbBgVHOTXY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIpat2m++oaewJcZt2UDMupXoNxYDkyNqAT1xwv xR2sFL+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8Ryo5WD/ 9XAghm5/NCTIj2UYdQk2umfQQkc0it89IWJ+lNICxFF4jJNqNcZ5Fhgf0LnwcJQyI5C554S7WaUbi3 tKnnSUNh01Zt2og/Nej5IL2FjbQ4ZNNJRVWJxhzQ2bl/PEuhLgqRNA0sbCXOM0YLpRVwZsNte32d/Y QztKpJ9zoDzOUwr18zkHee9Pa2cA8D+JKN2+NH7W1TnRjf8KzVW3ayJy6dXFxPn4/iK/CYYPJFxSUf Z/gWkCLxtajJCaqvo0btHL7PA2xeeXU6YVtYXHLMWFi/CpKRX/MJE9x+ZCOjYIWhSmOXlkpzyurmwj U6joYmBAxHlXw28/xdCwToGVoALt0LIZmrV1cmVdxGC9Dkrl47u11jbm83rlG/ndmPwRIXhLfE2B1C yfcQE3FUP+Fg3/EFHd7Ps9moXt/OJU9hop0XKnSWMa0j4DkOVCTcy/TlLO17RhqgvKeBy4bYAVYM90 C8CcrtivfjH15ysfOYNJ80A9B0ZDL7yJxQC14W9bTGrAjqDPKRQAR5QmMhhHd+dyHoDr/kCIewKqNv 89CefMpFeaNUL/u/q1N8LPpFcufLM1JGNl4gME6uSUzTgbsRtwlys8QNUymzvyaG0iAclmvqX3hkpB PB/D66QSytniCrSbDR0BsbmnP6jX1UGxWNXgB5K2KJxBVHjhSrZVT65jDLBQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add user facing __contains macro which provides a convenient wrapper over the verbose kernel specific BTF declaration tag required to annotate BPF list head structs in user types. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/testing/selftests/bpf/bpf_experimental.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index d6b143275e82..424f7bbbfe9b 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -6,6 +6,8 @@ #include #include +#define __contains(name, node) __attribute__((btf_decl_tag("contains:" #name ":" #node))) + /* Description * Allocates an object of the type represented by 'local_type_id' in * program BTF. User may use the bpf_core_type_id_local macro to pass the From patchwork Fri Nov 11 19:32:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040752 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70142C4332F for ; Fri, 11 Nov 2022 19:34:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234032AbiKKTei (ORCPT ); Fri, 11 Nov 2022 14:34:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234072AbiKKTef (ORCPT ); Fri, 11 Nov 2022 14:34:35 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF3A47879A for ; Fri, 11 Nov 2022 11:34:33 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id z26so5718538pff.1 for ; Fri, 11 Nov 2022 11:34:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=orrU9bkwct7hSaHpeFwTQHPvPh3WfieLrB51Oysef3o=; b=UhrzKS3RempGl3UdBOI6x4RIvPzylyBqXcdKbeRodKSVgF9yd5V7PR8fe4S7z+v07/ LNPnuYza+aMv+jIgezsvzEdIdHLCZe4VQdns5UyUUFp74UjvqM2fWBe/kZz0xam8wnO5 32YDRtTcCyTXCO4arJjwYt5/CPJem7bTu6WmqRDknPnC/vqcdLkS5sSxhrNnsiY6oEYn bXCqjV+CtbcIDE/zjq1yv1cWEcfIpnLTAFQ19YEzvjd7ulGGILEIMD6Epj3mlvq6o3Ta ZbYDoOns49WiBRCl6mih/iEaYKhCaQmck0S/1zPzp5ik/CKBCe02MIHbuHNzkKBrFvUK l7Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=orrU9bkwct7hSaHpeFwTQHPvPh3WfieLrB51Oysef3o=; b=W3RFNDLYma8sfWaL7ZoTnC1djjdZHwxQGngrscX/3OWI0TOln2sVenyWckAir8P8Z8 K1xExAMbygZUNjQe/G7Z10k5k4rv0+oaU35LBJ8hKvZqmoPaOZwRb69jDPH0Hh0QG0MB LFb0J6D9ZyhsXLVQQf3nuS/YCgBF6EdA08svagKqApy+s8JwPyyne4y8hh0gIzdGqS88 fa3mmKlHC4O9Y7D6HEDiy99Li+NnnoWZ026tx/X79zVdvU7zIFC1LUIAGmm2xQ/JEC9+ IydNsWtveYBC+pjLQxKM5EBsq2MA9f8RCGza9Ie7KY6P3Va84MMkMx0QFd97Cc/PfYm7 X3Cw== X-Gm-Message-State: ANoB5plFgutMj5imrmqRK/poC+d2eE20yC1NBg0g2nZfYO11UplFN88F WskdBaTebB2dfYjXZUfJeNEzoFfvIaAkkw== X-Google-Smtp-Source: AA0mqf5lBS/o5j2eOxM6e315bR6/vRSuMiLfk2TeKXJQaaNzAR99e7KW00az8tJWzpX72Bt5XT2VnA== X-Received: by 2002:a63:4551:0:b0:435:7957:559d with SMTP id u17-20020a634551000000b004357957559dmr2858304pgk.122.1668195272978; Fri, 11 Nov 2022 11:34:32 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id q28-20020aa7983c000000b0056e8eb09d57sm1958897pfl.63.2022.11.11.11.34.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:32 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 23/26] selftests/bpf: Update spinlock selftest Date: Sat, 12 Nov 2022 01:02:21 +0530 Message-Id: <20221111193224.876706-24-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4240; i=memxor@gmail.com; h=from:subject; bh=7VEkgc+BO3uo7sABXUFzym09WvMjkfDkhbTvKZAA/+A=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIp4CA2EmruSzd0pkEr5lKkj3lWXgt8hXk7B9gB YUq2/fyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8RymVND/ 4o9zsxx3d0UQI/Z0ZnObXToZooc0BHXO8YyOD4umDcV6vKLn/+NBHBkljnZAG0rAE1gDjUakdAv59f EnleTqtxn6rC/d+737C5AtcoIqT/5Oe76wC+ugkBstQeNa3Z6OoM5jzDCStsBwTUxdwEMBAMKJyip8 /v+LwoJbLnS8gXz1YWhsNMcYBEIGOYTr+Q84lXn4uuin7Dtvqtue1/NFLZGAOEZbwotHFuyu+/3Vc1 acfXJ40ljYfuUm1wb1K3SXRy3UeEcwK6+zY+j1nApwPqfinPviK7plaYyk5nWW6PHxxde3aZrkdRU6 9h4DcTppHbsetVmn0xVBcU5nNs45JAFlsfERplptnyfVCCLqaqjsevIJmgaszC0xd6zp1h85NJoSZv CD2iIV7rkchvcUHArehu+r037ix2RoRkXmXRelGRpEU6JmLCekc0yHqjvbkWxBO42fhhNI7vJ4yiOr rWrzg2siEhd2cLoo2nSxKe77+dm6S/BkmAHxipZcx62+dsJPF1U7YiQmcMF1E9DlD2rzqns+dYiAK/ brW8ciuN9vRmVxUFXrnCvdInh9vB26liD4QaJ8NVkptYfEmcigwrxXHfRi5rQJLzn7Igcn97xz0GXv 63SgGXOHiwZLIIUcifSK+m7B1KHamhfuua3IwoaULHLw5nNTjKB6ylPunzig== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Make updates in preparation for adding more test cases to this selftest: - Convert from CHECK_ to ASSERT macros. - Use BPF skeleton - Fix typo sping -> spin - Rename spinlock.c -> spin_lock.c Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/spin_lock.c | 49 +++++++++++++++++++ .../selftests/bpf/prog_tests/spinlock.c | 45 ----------------- .../selftests/bpf/progs/test_spin_lock.c | 4 +- 3 files changed, 51 insertions(+), 47 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/spin_lock.c delete mode 100644 tools/testing/selftests/bpf/prog_tests/spinlock.c diff --git a/tools/testing/selftests/bpf/prog_tests/spin_lock.c b/tools/testing/selftests/bpf/prog_tests/spin_lock.c new file mode 100644 index 000000000000..fab061e9d77c --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/spin_lock.c @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#include "test_spin_lock.skel.h" + +static void *spin_lock_thread(void *arg) +{ + int err, prog_fd = *(u32 *) arg; + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 10000, + ); + + err = bpf_prog_test_run_opts(prog_fd, &topts); + ASSERT_OK(err, "test_run"); + ASSERT_OK(topts.retval, "test_run retval"); + pthread_exit(arg); +} + +void test_spinlock(void) +{ + struct test_spin_lock *skel; + pthread_t thread_id[4]; + int prog_fd, i; + void *ret; + + skel = test_spin_lock__open_and_load(); + if (!ASSERT_OK_PTR(skel, "test_spin_lock__open_and_load")) + return; + prog_fd = bpf_program__fd(skel->progs.bpf_spin_lock_test); + for (i = 0; i < 4; i++) { + int err; + + err = pthread_create(&thread_id[i], NULL, &spin_lock_thread, &prog_fd); + if (!ASSERT_OK(err, "pthread_create")) + goto end; + } + + for (i = 0; i < 4; i++) { + if (!ASSERT_OK(pthread_join(thread_id[i], &ret), "pthread_join")) + goto end; + if (!ASSERT_EQ(ret, &prog_fd, "ret == prog_fd")) + goto end; + } +end: + test_spin_lock__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/prog_tests/spinlock.c b/tools/testing/selftests/bpf/prog_tests/spinlock.c deleted file mode 100644 index 15eb1372d771..000000000000 --- a/tools/testing/selftests/bpf/prog_tests/spinlock.c +++ /dev/null @@ -1,45 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -#include -#include - -static void *spin_lock_thread(void *arg) -{ - int err, prog_fd = *(u32 *) arg; - LIBBPF_OPTS(bpf_test_run_opts, topts, - .data_in = &pkt_v4, - .data_size_in = sizeof(pkt_v4), - .repeat = 10000, - ); - - err = bpf_prog_test_run_opts(prog_fd, &topts); - ASSERT_OK(err, "test_run"); - ASSERT_OK(topts.retval, "test_run retval"); - pthread_exit(arg); -} - -void test_spinlock(void) -{ - const char *file = "./test_spin_lock.bpf.o"; - pthread_t thread_id[4]; - struct bpf_object *obj = NULL; - int prog_fd; - int err = 0, i; - void *ret; - - err = bpf_prog_test_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd); - if (CHECK_FAIL(err)) { - printf("test_spin_lock:bpf_prog_test_load errno %d\n", errno); - goto close_prog; - } - for (i = 0; i < 4; i++) - if (CHECK_FAIL(pthread_create(&thread_id[i], NULL, - &spin_lock_thread, &prog_fd))) - goto close_prog; - - for (i = 0; i < 4; i++) - if (CHECK_FAIL(pthread_join(thread_id[i], &ret) || - ret != (void *)&prog_fd)) - goto close_prog; -close_prog: - bpf_object__close(obj); -} diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock.c b/tools/testing/selftests/bpf/progs/test_spin_lock.c index 7e88309d3229..5bd10409285b 100644 --- a/tools/testing/selftests/bpf/progs/test_spin_lock.c +++ b/tools/testing/selftests/bpf/progs/test_spin_lock.c @@ -45,8 +45,8 @@ struct { #define CREDIT_PER_NS(delta, rate) (((delta) * rate) >> 20) -SEC("tc") -int bpf_sping_lock_test(struct __sk_buff *skb) +SEC("cgroup_skb/ingress") +int bpf_spin_lock_test(struct __sk_buff *skb) { volatile int credit = 0, max_credit = 100, pkt_len = 64; struct hmap_elem zero = {}, *val; From patchwork Fri Nov 11 19:32:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040753 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3D57C433FE for ; Fri, 11 Nov 2022 19:34:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234107AbiKKTen (ORCPT ); Fri, 11 Nov 2022 14:34:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234042AbiKKTei (ORCPT ); Fri, 11 Nov 2022 14:34:38 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F42267F4E for ; Fri, 11 Nov 2022 11:34:37 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id e129so5118076pgc.9 for ; Fri, 11 Nov 2022 11:34:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D5mhxE7t9/w1waItLCLixY08yGr4GoIXj2ByOScU2oQ=; b=mlxr0yb/flqO7XtJbc20K1zEjY1XDrJPPTs+3gs8xgceGzo/IWYYXfGf0IS04PtWqH vtU5T9g72POjD+VWWgVMQhiANoj2brCozSlWxkNiPzKd0zacidYyhWIkeYq/75DdotaB SwRtjRWkM564DMDbwPIZzcJsk0pfiKoLK63RzL/2ffomWzthvKwRsmoPHDUXUkJ5watx DEr6L4WuCfeE8RPNaiXvmf8D5nt34sTa1O2bV2lCJWbBJFJK42FtbMUNl0HxZPB6RQYm /ceVfT63Y5ZbLmL4S71QBJSO8Rc4NSrmVeMmP7HCKSXJCwFcRUpef16gZK2KGBvcH6yZ o76w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D5mhxE7t9/w1waItLCLixY08yGr4GoIXj2ByOScU2oQ=; b=jziAgYo5NgtIPpkT6U5vxEwFOwMQcOIRK11TlPQxk1/av5R+efqiVM8bRNGP8P6ZFd MHXrDkk6J0IO3oayWq/DLxgARObOSKxhhQU9giHhjpoaSYUv/zlxREx150W9wShe3TRm 0KuhV9c3eONrG81MQRQSH9VWDa46/hMW1dkutQeVaOTu+gEHf0ihPZUieHCmG0iA/2fM evmeMeQ1TqC7ed6tK+Q75Ah2H7mX8sUe814k3kSRQlbk4SK4F5FYMvRuFkHUuHFQ8s4N GnCapEABF75LlOFvX1khEYwOVTps+9xxKUJVojxGXARYtBx/kgxKXxXG8CL9pgHZcvLy CYUA== X-Gm-Message-State: ANoB5plhWpnGZHevP/MF2ECZXeuU89/VOCcbsLnWIVb0FwP6G0x9b89+ J9skyD5xMLHJDCxia8DDGCCded3XGMvePA== X-Google-Smtp-Source: AA0mqf5ocPTvNbcBhkPAXX4isd57IOKgOqfd7GSsYVuViywWkGjr8ZEtxsVYrcqJMJZwaJef/XwO3w== X-Received: by 2002:a63:1801:0:b0:470:7886:e200 with SMTP id y1-20020a631801000000b004707886e200mr2986758pgl.111.1668195276450; Fri, 11 Nov 2022 11:34:36 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id l4-20020a622504000000b00560a25fae1fsm1914896pfl.206.2022.11.11.11.34.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:36 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 24/26] selftests/bpf: Add failure test cases for spin lock pairing Date: Sat, 12 Nov 2022 01:02:22 +0530 Message-Id: <20221111193224.876706-25-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10926; i=memxor@gmail.com; h=from:subject; bh=EgO226hLDLT8uUjs61C2Qe2aCLhsrlTx0+w4ds2Mqpc=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIpgMT9Abs9cB+op+OrrkFyBfHhPTKvNDwLe6tH HY3gwCyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8Ryg/xD/ 9J6GEFqii3Q8NTgDMvketEFmLLbE1042/M+SEIR9JMF71xBmHQXH2yr+vxf0potLnKTX3OVduOAwAL I1vhRyDmzwrP4LsQvDPudgCx0ZImrQaUnJHRJtmkfmASXY4rBUxaGLgjmmrc23bTYOoCFD0E2jRJXB Ck+o7aBH5mATCB+3V4/TBeCrXnOBSdg6xDpjH807URexeORoMhnKJBISd9bLNRH0/hSkgLqG1CG8MN 7Ckv4UN+mapaHXAI2QllEj8c1mr6FjJ4DQu8utmrDSFN2WCp3yxnJUSCROJgyajFk9r0Zo/N4a/0vw fUH6d8swtGpg8mKjZND7IDVZ1DprTP8V32mRqYqCfKdrKX22C6WLIUdhFZ7Gf+w22MLL+7PBtcqd6E CnxJoyP9Hulsy5xVBrmKAJXujb3nepg6GNgg5hDlh0SiyCCCxkD7wa6dH9vh3jjLvbbRyoBxJcXNMH yVzMgCre7GugdLzITYuQ6vBC9UPSppEOX7ouqFGFAppwyoek6UJm+ExPh/VnJo4iSIJiFanlULaxxI gczsXC8gaZtiX/ToRc+7QnbzbZVD/elv432O1Te6rpQvy5L6IzJp+WebqdDueWnTF1kgOWhEOhZYIb HAOw5anZRzgEemKA8GMX0SgKD12gTzZgEIo62lv5VLFFUdYbiAB9aGdk1cBQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net First, ensure that whenever a bpf_spin_lock is present in an allocation, the reg->id is preserved. This won't be true for global variables however, since they have a single map value per map, hence the verifier harcodes it to 0 (so that multiple pseudo ldimm64 insns can yield the same lock object per map at a given offset). Next, add test cases for all possible combinations (kptr, global, map value, inner map value). Since we lifted restriction on locking in inner maps, also add test cases for them. Currently, each lookup into an inner map gets a fresh reg->id, so even if the reg->map_ptr is same, they will be treated as separate allocations and the incorrect unlock pairing will be rejected. Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/spin_lock.c | 89 +++++++- .../selftests/bpf/progs/test_spin_lock_fail.c | 204 ++++++++++++++++++ 2 files changed, 292 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/test_spin_lock_fail.c diff --git a/tools/testing/selftests/bpf/prog_tests/spin_lock.c b/tools/testing/selftests/bpf/prog_tests/spin_lock.c index fab061e9d77c..72282e92a78a 100644 --- a/tools/testing/selftests/bpf/prog_tests/spin_lock.c +++ b/tools/testing/selftests/bpf/prog_tests/spin_lock.c @@ -3,6 +3,79 @@ #include #include "test_spin_lock.skel.h" +#include "test_spin_lock_fail.skel.h" + +static char log_buf[1024 * 1024]; + +static struct { + const char *prog_name; + const char *err_msg; +} spin_lock_fail_tests[] = { + { "lock_id_kptr_preserve", + "5: (bf) r1 = r0 ; R0_w=ptr_foo(id=2,ref_obj_id=2,off=0,imm=0) " + "R1_w=ptr_foo(id=2,ref_obj_id=2,off=0,imm=0) refs=2\n6: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=ptr_ expected=percpu_ptr_" }, + { "lock_id_global_zero", + "; R1_w=map_value(off=0,ks=4,vs=4,imm=0)\n2: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=map_value expected=percpu_ptr_" }, + { "lock_id_mapval_preserve", + "8: (bf) r1 = r0 ; R0_w=map_value(id=1,off=0,ks=4,vs=8,imm=0) " + "R1_w=map_value(id=1,off=0,ks=4,vs=8,imm=0)\n9: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=map_value expected=percpu_ptr_" }, + { "lock_id_innermapval_preserve", + "13: (bf) r1 = r0 ; R0=map_value(id=2,off=0,ks=4,vs=8,imm=0) " + "R1_w=map_value(id=2,off=0,ks=4,vs=8,imm=0)\n14: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=map_value expected=percpu_ptr_" }, + { "lock_id_mismatch_kptr_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_kptr_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_kptr_mapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_kptr_innermapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_mapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_innermapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_mapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_innermapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_innermapval1", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_innermapval2", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_mapval", "bpf_spin_unlock of different lock" }, +}; + +static void test_spin_lock_fail_prog(const char *prog_name, const char *err_msg) +{ + LIBBPF_OPTS(bpf_object_open_opts, opts, .kernel_log_buf = log_buf, + .kernel_log_size = sizeof(log_buf), + .kernel_log_level = 1); + struct test_spin_lock_fail *skel; + struct bpf_program *prog; + int ret; + + skel = test_spin_lock_fail__open_opts(&opts); + if (!ASSERT_OK_PTR(skel, "test_spin_lock_fail__open_opts")) + return; + + prog = bpf_object__find_program_by_name(skel->obj, prog_name); + if (!ASSERT_OK_PTR(prog, "bpf_object__find_program_by_name")) + goto end; + + bpf_program__set_autoload(prog, true); + + ret = test_spin_lock_fail__load(skel); + if (!ASSERT_ERR(ret, "test_spin_lock_fail__load must fail")) + goto end; + + if (!ASSERT_OK_PTR(strstr(log_buf, err_msg), "expected error message")) { + fprintf(stderr, "Expected: %s\n", err_msg); + fprintf(stderr, "Verifier: %s\n", log_buf); + } + +end: + test_spin_lock_fail__destroy(skel); +} static void *spin_lock_thread(void *arg) { @@ -19,7 +92,7 @@ static void *spin_lock_thread(void *arg) pthread_exit(arg); } -void test_spinlock(void) +void test_spin_lock_success(void) { struct test_spin_lock *skel; pthread_t thread_id[4]; @@ -47,3 +120,17 @@ void test_spinlock(void) end: test_spin_lock__destroy(skel); } + +void test_spin_lock(void) +{ + int i; + + test_spin_lock_success(); + + for (i = 0; i < ARRAY_SIZE(spin_lock_fail_tests); i++) { + if (!test__start_subtest(spin_lock_fail_tests[i].prog_name)) + continue; + test_spin_lock_fail_prog(spin_lock_fail_tests[i].prog_name, + spin_lock_fail_tests[i].err_msg); + } +} diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock_fail.c b/tools/testing/selftests/bpf/progs/test_spin_lock_fail.c new file mode 100644 index 000000000000..86cd183ef6dc --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_spin_lock_fail.c @@ -0,0 +1,204 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include "bpf_experimental.h" + +struct foo { + struct bpf_spin_lock lock; + int data; +}; + +struct array_map { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct foo); + __uint(max_entries, 1); +} array_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); + __uint(max_entries, 1); + __type(key, int); + __type(value, int); + __array(values, struct array_map); +} map_of_maps SEC(".maps") = { + .values = { + [0] = &array_map, + }, +}; + +SEC(".data.A") struct bpf_spin_lock lockA; +SEC(".data.B") struct bpf_spin_lock lockB; + +SEC("?tc") +int lock_id_kptr_preserve(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_this_cpu_ptr(f); + return 0; +} + +SEC("?tc") +int lock_id_global_zero(void *ctx) +{ + bpf_this_cpu_ptr(&lockA); + return 0; +} + +SEC("?tc") +int lock_id_mapval_preserve(void *ctx) +{ + struct foo *f; + int key = 0; + + f = bpf_map_lookup_elem(&array_map, &key); + if (!f) + return 0; + bpf_this_cpu_ptr(f); + return 0; +} + +SEC("?tc") +int lock_id_innermapval_preserve(void *ctx) +{ + struct foo *f; + int key = 0; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f = bpf_map_lookup_elem(map, &key); + if (!f) + return 0; + bpf_this_cpu_ptr(f); + return 0; +} + +#define CHECK(test, A, B) \ + SEC("?tc") \ + int lock_id_mismatch_##test(void *ctx) \ + { \ + struct foo *f1, *f2, *v, *iv; \ + int key = 0; \ + void *map; \ + \ + map = bpf_map_lookup_elem(&map_of_maps, &key); \ + if (!map) \ + return 0; \ + iv = bpf_map_lookup_elem(map, &key); \ + if (!iv) \ + return 0; \ + v = bpf_map_lookup_elem(&array_map, &key); \ + if (!v) \ + return 0; \ + f1 = bpf_obj_new(typeof(*f1)); \ + if (!f1) \ + return 0; \ + f2 = bpf_obj_new(typeof(*f2)); \ + if (!f2) { \ + bpf_obj_drop(f1); \ + return 0; \ + } \ + bpf_spin_lock(A); \ + bpf_spin_unlock(B); \ + return 0; \ + } + +CHECK(kptr_kptr, &f1->lock, &f2->lock); +CHECK(kptr_global, &f1->lock, &lockA); +CHECK(kptr_mapval, &f1->lock, &v->lock); +CHECK(kptr_innermapval, &f1->lock, &iv->lock); + +CHECK(global_global, &lockA, &lockB); +CHECK(global_kptr, &lockA, &f1->lock); +CHECK(global_mapval, &lockA, &v->lock); +CHECK(global_innermapval, &lockA, &iv->lock); + +SEC("?tc") +int lock_id_mismatch_mapval_mapval(void *ctx) +{ + struct foo *f1, *f2; + int key = 0; + + f1 = bpf_map_lookup_elem(&array_map, &key); + if (!f1) + return 0; + f2 = bpf_map_lookup_elem(&array_map, &key); + if (!f2) + return 0; + + bpf_spin_lock(&f1->lock); + f1->data = 42; + bpf_spin_unlock(&f2->lock); + + return 0; +} + +CHECK(mapval_kptr, &v->lock, &f1->lock); +CHECK(mapval_global, &v->lock, &lockB); +CHECK(mapval_innermapval, &v->lock, &iv->lock); + +SEC("?tc") +int lock_id_mismatch_innermapval_innermapval1(void *ctx) +{ + struct foo *f1, *f2; + int key = 0; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f1 = bpf_map_lookup_elem(map, &key); + if (!f1) + return 0; + f2 = bpf_map_lookup_elem(map, &key); + if (!f2) + return 0; + + bpf_spin_lock(&f1->lock); + f1->data = 42; + bpf_spin_unlock(&f2->lock); + + return 0; +} + +SEC("?tc") +int lock_id_mismatch_innermapval_innermapval2(void *ctx) +{ + struct foo *f1, *f2; + int key = 0; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f1 = bpf_map_lookup_elem(map, &key); + if (!f1) + return 0; + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f2 = bpf_map_lookup_elem(map, &key); + if (!f2) + return 0; + + bpf_spin_lock(&f1->lock); + f1->data = 42; + bpf_spin_unlock(&f2->lock); + + return 0; +} + +CHECK(innermapval_kptr, &iv->lock, &f1->lock); +CHECK(innermapval_global, &iv->lock, &lockA); +CHECK(innermapval_mapval, &iv->lock, &v->lock); + +#undef CHECK + +char _license[] SEC("license") = "GPL"; From patchwork Fri Nov 11 19:32:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040754 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C54BAC4332F for ; Fri, 11 Nov 2022 19:34:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234121AbiKKTet (ORCPT ); Fri, 11 Nov 2022 14:34:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36268 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234101AbiKKTem (ORCPT ); Fri, 11 Nov 2022 14:34:42 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7A2C787BD for ; Fri, 11 Nov 2022 11:34:40 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id m6-20020a17090a5a4600b00212f8dffec9so5555328pji.0 for ; Fri, 11 Nov 2022 11:34:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oqNJljVDIsXhhzjw1efhRO0V8DNidbAkWSwgvIaFExY=; b=dY9OgrxjSlFWxbj08dQOCaArL53A5lFxtilth8+uau8vC050IWJ2CnRqmFuFu6rywI FvxUhjg0G2UdWY1WorvDBCwO5FF/qzDjIesaV935we01aUv5hUqhE9yCr90NljhHI91T PZLgAB5w2opAUfaotBFxZ5F/m3n7Mk20fjE0bSukmhcnnD2Cw9ViwwMYS1+hGA/rEkMM tGDm9WmrHNERlfT5agHIU6hlDgz51rp6LOrIrRjJk9wINWWmJHkFyNdCT0syK2MEVfgx ViC3E2iS/tDq74/zSbJKYrRVB16s6icY/7mtjwTA3Bm4Rj7szjOQAO7DTUHn3dmQ1+w6 FrAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oqNJljVDIsXhhzjw1efhRO0V8DNidbAkWSwgvIaFExY=; b=HuiSesG8F0GwAkEb3nBD0i5HamuZBfB8eFAJx0VtYzVks+Kpj/ZHf2iwSXOmLnxGl3 n0L/wHZVCZ3LVyNXC8gzb2ICsWkOXU0JWXzpcBo/JKCzxdhTaRAxOvjh9joI0G2dseQW nyRrPXvHkZeoMqvoorwCOZd1bs26uoeUvmkRlp03ZDQnVLTT1w1eBrZhVCs9MKl5A2eq cEYz2s9fPnUXmGgGEqK3lOLReIbDwcUsnypouSDaXrmSyByVXD9WAAy1Cb3JSE97N+kv 9t7CR65zx5yisHtlIftDIy30qQ3LmwNYvPxbEmMdgDI6Tnqc+fs1aIH85LyBMW/ICiJf fApQ== X-Gm-Message-State: ANoB5plzq9zIOeu4k9bwaVI1lBRcjNyY2EWEtBzf1FMCLE2s4ArKOdgM 3OQ30737M34hOzGUSw0JqhkaxWRpaQjM1w== X-Google-Smtp-Source: AA0mqf4/ESAsCWz2HVUx0yjVBes9fZPET6JVp6HpsWM/kE7csiP/JdohCjRpcHrEMvU93+tBzDPevw== X-Received: by 2002:a17:902:a508:b0:188:9542:515 with SMTP id s8-20020a170902a50800b0018895420515mr3770721plq.102.1668195279489; Fri, 11 Nov 2022 11:34:39 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id ch16-20020a17090af41000b00210125b789dsm1932341pjb.54.2022.11.11.11.34.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:39 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 25/26] selftests/bpf: Add BPF linked list API tests Date: Sat, 12 Nov 2022 01:02:23 +0530 Message-Id: <20221111193224.876706-26-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=35474; i=memxor@gmail.com; h=from:subject; bh=ym5zGZElr4FjpGXZRLgvXVHCu29Kyfq3u+6nwwTOb9k=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIpeXj1Bit6Mr1FSEmhpj7QFg77VoSpOKZdIJK2 mErCvZ2JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8Ryq5hD/ 9R2rq8Qulr9NZ0bcOhk0O7RPAGMqjDoSelbSuXk2sJ15NbL4zCwBq+MXf7pgOREL3EU5AOwEKAjuKF Gk+Eok5z4PwPy5LWmtRiwPX5YK6VPqHpbWTU1NJlW/o8BADOCZv4Foj5CbifTCM4d0rpoC1sYObN/R 6SRj2pDcgylS4XIOzhQZIINuSucLmhRFK44z99pKbqA1ptOR6e8Lp74L1DSuP6/ZsFBFNYJFMFiY4U OQyOs9pwKKLn50gxxS03QNu0XpgofbPFv/WT7q9/jgGWoyP3PvDrUcTRqClNomasivkpZQRo2B9uKF M2OegPVkI7K5j9k/zBAOijbzHqv4vRoImTo0oxNBH83Bs20HwYmsyVMwozKZtQvqCPa19MaaH1wSi1 /0/4tF+lcQXEXd1vk7napohdc8dg+gOoxB7i5Gvh1jvG0E3aWn8sPkjmLpswfqfJ5xcOfCzGPLfbnw p61WUMhmLBl011QhAbJc9D0rccmf48mxM5ocSQGR2+J9Y0E/lqYKYUAqoDhQNYHQg1+omZjjDeUQDe I8fdU9RJf5uuN9XcTf11koJV/g44QRIrnA6bk48w7sWI1h/1LHr8RAlUTmxj+GiAiQ3a1WnoQQiXBh 9geWbMsx4AQguqWv2igGsoWxon+2hwsDI1wC/9ZEC1rO/7A/QtA9xPnFKFxQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Include various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 253 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 5 files changed, 1261 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.c diff --git a/tools/testing/selftests/bpf/DENYLIST.s390x b/tools/testing/selftests/bpf/DENYLIST.s390x index be4e3d47ea3e..072243af93b0 100644 --- a/tools/testing/selftests/bpf/DENYLIST.s390x +++ b/tools/testing/selftests/bpf/DENYLIST.s390x @@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline) diff --git a/tools/testing/selftests/bpf/prog_tests/linked_list.c b/tools/testing/selftests/bpf/prog_tests/linked_list.c new file mode 100644 index 000000000000..e8569db2f3bc --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/linked_list.c @@ -0,0 +1,253 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#include "linked_list.skel.h" +#include "linked_list_fail.skel.h" + +static char log_buf[1024 * 1024]; + +static struct { + const char *prog_name; + const char *err_msg; +} linked_list_fail_tests[] = { +#define TEST(test, off) \ + { #test "_missing_lock_push_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_push_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, + TEST(kptr, 32) + TEST(global, 16) + TEST(map, 0) + TEST(inner_map, 0) +#undef TEST +#define TEST(test, op) \ + { #test "_kptr_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ + { #test "_global_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ + { #test "_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \ + { #test "_inner_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, + TEST(kptr, push_front) + TEST(kptr, push_back) + TEST(kptr, pop_front) + TEST(kptr, pop_back) + TEST(global, push_front) + TEST(global, push_back) + TEST(global, pop_front) + TEST(global, pop_back) + TEST(map, push_front) + TEST(map, push_back) + TEST(map, pop_front) + TEST(map, pop_back) + TEST(inner_map, push_front) + TEST(inner_map, push_back) + TEST(inner_map, pop_front) + TEST(inner_map, pop_back) +#undef TEST + { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, + { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, + { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_drop_non_zero_off", "R1 must have zero offset when passed to release func" }, + { "new_null_ret", "R0 invalid mem access 'ptr_or_null_'" }, + { "obj_new_acq", "Unreleased reference id=" }, + { "use_after_drop", "invalid mem access 'scalar'" }, + { "ptr_walk_scalar", "type=scalar expected=percpu_ptr_" }, + { "direct_read_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_write_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_read_head", "direct access to bpf_list_head is disallowed" }, + { "direct_write_head", "direct access to bpf_list_head is disallowed" }, + { "direct_read_node", "direct access to bpf_list_node is disallowed" }, + { "direct_write_node", "direct access to bpf_list_node is disallowed" }, + { "write_after_push_front", "only read is supported" }, + { "write_after_push_back", "only read is supported" }, + { "use_after_unlock_push_front", "invalid mem access 'scalar'" }, + { "use_after_unlock_push_back", "invalid mem access 'scalar'" }, + { "double_push_front", "arg#1 expected pointer to allocated object" }, + { "double_push_back", "arg#1 expected pointer to allocated object" }, + { "no_node_value_type", "bpf_list_node not found for allocated object\n" }, + { "incorrect_value_type", "bpf_list_head value type does not match arg#1" }, + { "incorrect_node_var_off", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_node_off1", "bpf_list_node not found at offset=1" }, + { "incorrect_node_off2", "arg#1 offset must be for bpf_list_node at off=0" }, + { "no_head_type", "bpf_list_head not found for allocated object" }, + { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, + { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, + { "incorrect_head_off2", "bpf_list_head not found at offset=1" }, + { "pop_front_off", + "15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " + "R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, + { "pop_back_off", + "15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " + "R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, +}; + +static void test_linked_list_fail_prog(const char *prog_name, const char *err_msg) +{ + LIBBPF_OPTS(bpf_object_open_opts, opts, .kernel_log_buf = log_buf, + .kernel_log_size = sizeof(log_buf), + .kernel_log_level = 1); + struct linked_list_fail *skel; + struct bpf_program *prog; + int ret; + + skel = linked_list_fail__open_opts(&opts); + if (!ASSERT_OK_PTR(skel, "linked_list_fail__open_opts")) + return; + + prog = bpf_object__find_program_by_name(skel->obj, prog_name); + if (!ASSERT_OK_PTR(prog, "bpf_object__find_program_by_name")) + goto end; + + bpf_program__set_autoload(prog, true); + + ret = linked_list_fail__load(skel); + if (!ASSERT_ERR(ret, "linked_list_fail__load must fail")) + goto end; + + if (!ASSERT_OK_PTR(strstr(log_buf, err_msg), "expected error message")) { + fprintf(stderr, "Expected: %s\n", err_msg); + fprintf(stderr, "Verifier: %s\n", log_buf); + } + +end: + linked_list_fail__destroy(skel); +} + +static void clear_fields(struct bpf_map *map) +{ + char buf[24]; + int key = 0; + + memset(buf, 0xff, sizeof(buf)); + ASSERT_OK(bpf_map__update_elem(map, &key, sizeof(key), buf, sizeof(buf), 0), "check_and_free_fields"); +} + +enum { + TEST_ALL, + PUSH_POP, + PUSH_POP_MULT, + LIST_IN_LIST, +}; + +static void test_linked_list_success(int mode, bool leave_in_map) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 1, + ); + struct linked_list *skel; + int ret; + + skel = linked_list__open_and_load(); + if (!ASSERT_OK_PTR(skel, "linked_list__open_and_load")) + return; + + if (mode == LIST_IN_LIST) + goto lil; + if (mode == PUSH_POP_MULT) + goto ppm; + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.map_list_push_pop), &opts); + ASSERT_OK(ret, "map_list_push_pop"); + ASSERT_OK(opts.retval, "map_list_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.array_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.inner_map_list_push_pop), &opts); + ASSERT_OK(ret, "inner_map_list_push_pop"); + ASSERT_OK(opts.retval, "inner_map_list_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.inner_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_push_pop), &opts); + ASSERT_OK(ret, "global_list_push_pop"); + ASSERT_OK(opts.retval, "global_list_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.data_A); + + if (mode == PUSH_POP) + goto end; + +ppm: + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.map_list_push_pop_multiple), &opts); + ASSERT_OK(ret, "map_list_push_pop_multiple"); + ASSERT_OK(opts.retval, "map_list_push_pop_multiple retval"); + if (!leave_in_map) + clear_fields(skel->maps.array_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.inner_map_list_push_pop_multiple), &opts); + ASSERT_OK(ret, "inner_map_list_push_pop_multiple"); + ASSERT_OK(opts.retval, "inner_map_list_push_pop_multiple retval"); + if (!leave_in_map) + clear_fields(skel->maps.inner_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_push_pop_multiple), &opts); + ASSERT_OK(ret, "global_list_push_pop_multiple"); + ASSERT_OK(opts.retval, "global_list_push_pop_multiple retval"); + if (!leave_in_map) + clear_fields(skel->maps.data_A); + + if (mode == PUSH_POP_MULT) + goto end; + +lil: + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.map_list_in_list), &opts); + ASSERT_OK(ret, "map_list_in_list"); + ASSERT_OK(opts.retval, "map_list_in_list retval"); + if (!leave_in_map) + clear_fields(skel->maps.array_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.inner_map_list_in_list), &opts); + ASSERT_OK(ret, "inner_map_list_in_list"); + ASSERT_OK(opts.retval, "inner_map_list_in_list retval"); + if (!leave_in_map) + clear_fields(skel->maps.inner_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_in_list), &opts); + ASSERT_OK(ret, "global_list_in_list"); + ASSERT_OK(opts.retval, "global_list_in_list retval"); + if (!leave_in_map) + clear_fields(skel->maps.data_A); +end: + linked_list__destroy(skel); +} + +void test_linked_list(void) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(linked_list_fail_tests); i++) { + if (!test__start_subtest(linked_list_fail_tests[i].prog_name)) + continue; + test_linked_list_fail_prog(linked_list_fail_tests[i].prog_name, + linked_list_fail_tests[i].err_msg); + } + test_linked_list_success(PUSH_POP, false); + test_linked_list_success(PUSH_POP, true); + test_linked_list_success(PUSH_POP_MULT, false); + test_linked_list_success(PUSH_POP_MULT, true); + test_linked_list_success(LIST_IN_LIST, false); + test_linked_list_success(LIST_IN_LIST, true); + test_linked_list_success(TEST_ALL, false); +} diff --git a/tools/testing/selftests/bpf/progs/linked_list.c b/tools/testing/selftests/bpf/progs/linked_list.c new file mode 100644 index 000000000000..2c7b615c6d41 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/linked_list.c @@ -0,0 +1,370 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_experimental.h" + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) +#endif + +#include "linked_list.h" + +static __always_inline +int list_push_pop(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) +{ + struct bpf_list_node *n; + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 2; + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + bpf_obj_drop(f); + return 3; + } + + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + bpf_obj_drop(f); + return 4; + } + + + bpf_spin_lock(lock); + f->data = 42; + bpf_list_push_front(head, &f->node); + bpf_spin_unlock(lock); + if (leave_in_map) + return 0; + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (!n) + return 5; + f = container_of(n, struct foo, node); + if (f->data != 42) { + bpf_obj_drop(f); + return 6; + } + + bpf_spin_lock(lock); + f->data = 13; + bpf_list_push_front(head, &f->node); + bpf_spin_unlock(lock); + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (!n) + return 7; + f = container_of(n, struct foo, node); + if (f->data != 13) { + bpf_obj_drop(f); + return 8; + } + bpf_obj_drop(f); + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 9; + } + + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 10; + } + return 0; +} + + +static __always_inline +int list_push_pop_multiple(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) +{ + struct bpf_list_node *n; + struct foo *f[8], *pf; + int i; + + for (i = 0; i < ARRAY_SIZE(f); i++) { + f[i] = bpf_obj_new(typeof(**f)); + if (!f[i]) + return 2; + f[i]->data = i; + bpf_spin_lock(lock); + bpf_list_push_front(head, &f[i]->node); + bpf_spin_unlock(lock); + } + + for (i = 0; i < ARRAY_SIZE(f); i++) { + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (!n) + return 3; + pf = container_of(n, struct foo, node); + if (pf->data != (ARRAY_SIZE(f) - i - 1)) { + bpf_obj_drop(pf); + return 4; + } + bpf_spin_lock(lock); + bpf_list_push_back(head, &pf->node); + bpf_spin_unlock(lock); + } + + if (leave_in_map) + return 0; + + for (i = 0; i < ARRAY_SIZE(f); i++) { + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (!n) + return 5; + pf = container_of(n, struct foo, node); + if (pf->data != i) { + bpf_obj_drop(pf); + return 6; + } + bpf_obj_drop(pf); + } + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 7; + } + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 8; + } + return 0; +} + +static __always_inline +int list_in_list(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) +{ + struct bpf_list_node *n; + struct bar *ba[8], *b; + struct foo *f; + int i; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 2; + for (i = 0; i < ARRAY_SIZE(ba); i++) { + b = bpf_obj_new(typeof(*b)); + if (!b) { + bpf_obj_drop(f); + return 3; + } + b->data = i; + bpf_spin_lock(&f->lock); + bpf_list_push_back(&f->head, &b->node); + bpf_spin_unlock(&f->lock); + } + + bpf_spin_lock(lock); + f->data = 42; + bpf_list_push_front(head, &f->node); + bpf_spin_unlock(lock); + + if (leave_in_map) + return 0; + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (!n) + return 4; + f = container_of(n, struct foo, node); + if (f->data != 42) { + bpf_obj_drop(f); + return 5; + } + + for (i = 0; i < ARRAY_SIZE(ba); i++) { + bpf_spin_lock(&f->lock); + n = bpf_list_pop_front(&f->head); + bpf_spin_unlock(&f->lock); + if (!n) { + bpf_obj_drop(f); + return 6; + } + b = container_of(n, struct bar, node); + if (b->data != i) { + bpf_obj_drop(f); + bpf_obj_drop(b); + return 7; + } + bpf_obj_drop(b); + } + bpf_spin_lock(&f->lock); + n = bpf_list_pop_front(&f->head); + bpf_spin_unlock(&f->lock); + if (n) { + bpf_obj_drop(f); + bpf_obj_drop(container_of(n, struct bar, node)); + return 8; + } + bpf_obj_drop(f); + return 0; +} + +static __always_inline +int test_list_push_pop(struct bpf_spin_lock *lock, struct bpf_list_head *head) +{ + int ret; + + ret = list_push_pop(lock, head, false); + if (ret) + return ret; + return list_push_pop(lock, head, true); +} + +static __always_inline +int test_list_push_pop_multiple(struct bpf_spin_lock *lock, struct bpf_list_head *head) +{ + int ret; + + ret = list_push_pop_multiple(lock ,head, false); + if (ret) + return ret; + return list_push_pop_multiple(lock, head, true); +} + +static __always_inline +int test_list_in_list(struct bpf_spin_lock *lock, struct bpf_list_head *head) +{ + int ret; + + ret = list_in_list(lock, head, false); + if (ret) + return ret; + return list_in_list(lock, head, true); +} + +SEC("tc") +int map_list_push_pop(void *ctx) +{ + struct map_value *v; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop(&v->lock, &v->head); +} + +SEC("tc") +int inner_map_list_push_pop(void *ctx) +{ + struct map_value *v; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &(int){0}); + if (!map) + return 1; + v = bpf_map_lookup_elem(map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop(&v->lock, &v->head); +} + +SEC("tc") +int global_list_push_pop(void *ctx) +{ + return test_list_push_pop(&glock, &ghead); +} + +SEC("tc") +int map_list_push_pop_multiple(void *ctx) +{ + struct map_value *v; + int ret; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop_multiple(&v->lock, &v->head); +} + +SEC("tc") +int inner_map_list_push_pop_multiple(void *ctx) +{ + struct map_value *v; + void *map; + int ret; + + map = bpf_map_lookup_elem(&map_of_maps, &(int){0}); + if (!map) + return 1; + v = bpf_map_lookup_elem(map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop_multiple(&v->lock, &v->head); +} + +SEC("tc") +int global_list_push_pop_multiple(void *ctx) +{ + int ret; + + ret = list_push_pop_multiple(&glock, &ghead, false); + if (ret) + return ret; + return list_push_pop_multiple(&glock, &ghead, true); +} + +SEC("tc") +int map_list_in_list(void *ctx) +{ + struct map_value *v; + int ret; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 1; + return test_list_in_list(&v->lock, &v->head); +} + +SEC("tc") +int inner_map_list_in_list(void *ctx) +{ + struct map_value *v; + void *map; + int ret; + + map = bpf_map_lookup_elem(&map_of_maps, &(int){0}); + if (!map) + return 1; + v = bpf_map_lookup_elem(map, &(int){0}); + if (!v) + return 1; + return test_list_in_list(&v->lock, &v->head); +} + +SEC("tc") +int global_list_in_list(void *ctx) +{ + return test_list_in_list(&glock, &ghead); +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/linked_list.h b/tools/testing/selftests/bpf/progs/linked_list.h new file mode 100644 index 000000000000..8db80ed64db1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/linked_list.h @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef LINKED_LIST_H +#define LINKED_LIST_H + +#include +#include +#include "bpf_experimental.h" + +struct bar { + struct bpf_list_node node; + int data; +}; + +struct foo { + struct bpf_list_node node; + struct bpf_list_head head __contains(bar, node); + struct bpf_spin_lock lock; + int data; + struct bpf_list_node node2; +}; + +struct map_value { + struct bpf_spin_lock lock; + int data; + struct bpf_list_head head __contains(foo, node); +}; + +struct array_map { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct map_value); + __uint(max_entries, 1); +}; + +struct array_map array_map SEC(".maps"); +struct array_map inner_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); + __uint(max_entries, 1); + __type(key, int); + __type(value, int); + __array(values, struct array_map); +} map_of_maps SEC(".maps") = { + .values = { + [0] = &inner_map, + }, +}; + +#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(A) struct bpf_spin_lock glock; +private(A) struct bpf_list_head ghead __contains(foo, node); +private(B) struct bpf_spin_lock glock2; + +#endif diff --git a/tools/testing/selftests/bpf/progs/linked_list_fail.c b/tools/testing/selftests/bpf/progs/linked_list_fail.c new file mode 100644 index 000000000000..1d9017240e19 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/linked_list_fail.c @@ -0,0 +1,581 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_experimental.h" + +#include "linked_list.h" + +#define INIT \ + struct map_value *v, *v2, *iv, *iv2; \ + struct foo *f, *f1, *f2; \ + struct bar *b; \ + void *map; \ + \ + map = bpf_map_lookup_elem(&map_of_maps, &(int){ 0 }); \ + if (!map) \ + return 0; \ + v = bpf_map_lookup_elem(&array_map, &(int){ 0 }); \ + if (!v) \ + return 0; \ + v2 = bpf_map_lookup_elem(&array_map, &(int){ 0 }); \ + if (!v2) \ + return 0; \ + iv = bpf_map_lookup_elem(map, &(int){ 0 }); \ + if (!iv) \ + return 0; \ + iv2 = bpf_map_lookup_elem(map, &(int){ 0 }); \ + if (!iv2) \ + return 0; \ + f = bpf_obj_new(typeof(*f)); \ + if (!f) \ + return 0; \ + f1 = f; \ + f2 = bpf_obj_new(typeof(*f2)); \ + if (!f2) { \ + bpf_obj_drop(f1); \ + return 0; \ + } \ + b = bpf_obj_new(typeof(*b)); \ + if (!b) { \ + bpf_obj_drop(f2); \ + bpf_obj_drop(f1); \ + return 0; \ + } + +#define CHECK(test, op, hexpr) \ + SEC("?tc") \ + int test##_missing_lock_##op(void *ctx) \ + { \ + INIT; \ + void (*p)(void *) = (void *)&bpf_list_##op; \ + p(hexpr); \ + return 0; \ + } + +CHECK(kptr, push_front, &f->head); +CHECK(kptr, push_back, &f->head); +CHECK(kptr, pop_front, &f->head); +CHECK(kptr, pop_back, &f->head); + +CHECK(global, push_front, &ghead); +CHECK(global, push_back, &ghead); +CHECK(global, pop_front, &ghead); +CHECK(global, pop_back, &ghead); + +CHECK(map, push_front, &v->head); +CHECK(map, push_back, &v->head); +CHECK(map, pop_front, &v->head); +CHECK(map, pop_back, &v->head); + +CHECK(inner_map, push_front, &iv->head); +CHECK(inner_map, push_back, &iv->head); +CHECK(inner_map, pop_front, &iv->head); +CHECK(inner_map, pop_back, &iv->head); + +#undef CHECK + +#define CHECK(test, op, lexpr, hexpr) \ + SEC("?tc") \ + int test##_incorrect_lock_##op(void *ctx) \ + { \ + INIT; \ + void (*p)(void *) = (void *)&bpf_list_##op; \ + bpf_spin_lock(lexpr); \ + p(hexpr); \ + return 0; \ + } + +#define CHECK_OP(op) \ + CHECK(kptr_kptr, op, &f1->lock, &f2->head); \ + CHECK(kptr_global, op, &f1->lock, &ghead); \ + CHECK(kptr_map, op, &f1->lock, &v->head); \ + CHECK(kptr_inner_map, op, &f1->lock, &iv->head); \ + \ + CHECK(global_global, op, &glock2, &ghead); \ + CHECK(global_kptr, op, &glock, &f1->head); \ + CHECK(global_map, op, &glock, &v->head); \ + CHECK(global_inner_map, op, &glock, &iv->head); \ + \ + CHECK(map_map, op, &v->lock, &v2->head); \ + CHECK(map_kptr, op, &v->lock, &f2->head); \ + CHECK(map_global, op, &v->lock, &ghead); \ + CHECK(map_inner_map, op, &v->lock, &iv->head); \ + \ + CHECK(inner_map_inner_map, op, &iv->lock, &iv2->head); \ + CHECK(inner_map_kptr, op, &iv->lock, &f2->head); \ + CHECK(inner_map_global, op, &iv->lock, &ghead); \ + CHECK(inner_map_map, op, &iv->lock, &v->head); + +CHECK_OP(push_front); +CHECK_OP(push_back); +CHECK_OP(pop_front); +CHECK_OP(pop_back); + +#undef CHECK +#undef CHECK_OP +#undef INIT + +SEC("?kprobe/xyz") +int map_compat_kprobe(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?kretprobe/xyz") +int map_compat_kretprobe(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?tracepoint/xyz") +int map_compat_tp(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?perf_event") +int map_compat_perf(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?raw_tp/xyz") +int map_compat_raw_tp(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?raw_tp.w/xyz") +int map_compat_raw_tp_w(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?tc") +int obj_type_id_oor(void *ctx) +{ + bpf_obj_new_impl(~0UL, NULL); + return 0; +} + +SEC("?tc") +int obj_new_no_composite(void *ctx) +{ + bpf_obj_new_impl(bpf_core_type_id_local(int), (void *)42); + return 0; +} + +SEC("?tc") +int obj_new_no_struct(void *ctx) +{ + + bpf_obj_new(union { int data; unsigned udata; }); + return 0; +} + +SEC("?tc") +int obj_drop_non_zero_off(void *ctx) +{ + void *f; + + f = bpf_obj_new(struct foo); + if (!f) + return 0; + bpf_obj_drop(f+1); + return 0; +} + +SEC("?tc") +int new_null_ret(void *ctx) +{ + return bpf_obj_new(struct foo)->data; +} + +SEC("?tc") +int obj_new_acq(void *ctx) +{ + bpf_obj_new(struct foo); + return 0; +} + +SEC("?tc") +int use_after_drop(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_obj_drop(f); + return f->data; +} + +SEC("?tc") +int ptr_walk_scalar(void *ctx) +{ + struct test1 { + struct test2 { + struct test2 *next; + } *ptr; + } *p; + + p = bpf_obj_new(typeof(*p)); + if (!p) + return 0; + bpf_this_cpu_ptr(p->ptr); + return 0; +} + +SEC("?tc") +int direct_read_lock(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + return *(int *)&f->lock; +} + +SEC("?tc") +int direct_write_lock(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + *(int *)&f->lock = 0; + return 0; +} + +SEC("?tc") +int direct_read_head(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + return *(int *)&f->head; +} + +SEC("?tc") +int direct_write_head(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + *(int *)&f->head = 0; + return 0; +} + +SEC("?tc") +int direct_read_node(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + return *(int *)&f->node; +} + +SEC("?tc") +int direct_write_node(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + *(int *)&f->node = 0; + return 0; +} + +static __always_inline +int write_after_op(void (*push_op)(void *head, void *node)) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + push_op(&ghead, &f->node); + f->data = 42; + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int write_after_push_front(void *ctx) +{ + return write_after_op((void *)bpf_list_push_front); +} + +SEC("?tc") +int write_after_push_back(void *ctx) +{ + return write_after_op((void *)bpf_list_push_back); +} + +static __always_inline +int use_after_unlock(void (*op)(void *head, void *node)) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + f->data = 42; + op(&ghead, &f->node); + bpf_spin_unlock(&glock); + + return f->data; +} + +SEC("?tc") +int use_after_unlock_push_front(void *ctx) +{ + return use_after_unlock((void *)bpf_list_push_front); +} + +SEC("?tc") +int use_after_unlock_push_back(void *ctx) +{ + return use_after_unlock((void *)bpf_list_push_back); +} + +static __always_inline +int list_double_add(void (*op)(void *head, void *node)) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + op(&ghead, &f->node); + op(&ghead, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int double_push_front(void *ctx) +{ + return list_double_add((void *)bpf_list_push_front); +} + +SEC("?tc") +int double_push_back(void *ctx) +{ + return list_double_add((void *)bpf_list_push_back); +} + +SEC("?tc") +int no_node_value_type(void *ctx) +{ + void *p; + + p = bpf_obj_new(struct { int data; }); + if (!p) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, p); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_value_type(void *ctx) +{ + struct bar *b; + + b = bpf_obj_new(typeof(*b)); + if (!b) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, &b->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_node_var_off(struct __sk_buff *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, (void *)&f->node + ctx->protocol); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_node_off1(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, (void *)&f->node + 1); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_node_off2(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, &f->node2); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int no_head_type(void *ctx) +{ + void *p; + + p = bpf_obj_new(typeof(struct { int data; })); + if (!p) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(p, NULL); + bpf_spin_lock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_head_var_off1(struct __sk_buff *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front((void *)&ghead + ctx->protocol, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_head_var_off2(struct __sk_buff *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front((void *)&f->head + ctx->protocol, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_head_off1(void *ctx) +{ + struct foo *f; + struct bar *b; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + b = bpf_obj_new(typeof(*b)); + if (!b) { + bpf_obj_drop(f); + return 0; + } + + bpf_spin_lock(&f->lock); + bpf_list_push_front((void *)&f->head + 1, &b->node); + bpf_spin_unlock(&f->lock); + + return 0; +} + +SEC("?tc") +int incorrect_head_off2(void *ctx) +{ + struct foo *f; + struct bar *b; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + + bpf_spin_lock(&glock); + bpf_list_push_front((void *)&ghead + 1, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +static __always_inline +int pop_ptr_off(void *(*op)(void *head)) +{ + struct { + struct bpf_list_head head __contains(foo, node2); + struct bpf_spin_lock lock; + } *p; + struct bpf_list_node *n; + + p = bpf_obj_new(typeof(*p)); + if (!p) + return 0; + bpf_spin_lock(&p->lock); + n = op(&p->head); + bpf_spin_unlock(&p->lock); + + bpf_this_cpu_ptr(n); + return 0; +} + +SEC("?tc") +int pop_front_off(void *ctx) +{ + return pop_ptr_off((void *)bpf_list_pop_front); +} + +SEC("?tc") +int pop_back_off(void *ctx) +{ + return pop_ptr_off((void *)bpf_list_pop_back); +} + +char _license[] SEC("license") = "GPL"; From patchwork Fri Nov 11 19:32:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13040755 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3A2DC43219 for ; Fri, 11 Nov 2022 19:34:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234067AbiKKTeu (ORCPT ); Fri, 11 Nov 2022 14:34:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234042AbiKKTep (ORCPT ); Fri, 11 Nov 2022 14:34:45 -0500 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E753079D21 for ; Fri, 11 Nov 2022 11:34:43 -0800 (PST) Received: by mail-pg1-x543.google.com with SMTP id q71so5124644pgq.8 for ; Fri, 11 Nov 2022 11:34:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ikZwvFVx43nJH39b53Lifsz5EBKM+wB7iSYfgKZ+Cw0=; b=CtWPSgnFu0fcJi1zMBYcdFqcRN0qTbgR0r37AVQq47lthBbK0z4HDkVGwDiyqXFk4s Xzpi+dXKz8LJsALYYZuVnrVqBch4l+c0Y8S7XhGaZcADLBhZCDkyvxSKrOubqg96Ws9i vpHAP+iNraNu+cuWBvKsKykyTAKp42+Pq1DR04MOw+ltlWjpLjLQBQkqeKbA8dQiyZ5K WURimCLVNR4S7gTiS1eCpTSFsbzThu1AuKFBPSpPiJIP2IOel7sBSBYw4u8ywHweYVCj oGCvgW2oZ9QsIgT9M8E+VHTKyljFKI9l5oLH3+FTrQ1pRYaMX/jqYA/qARF+JKs+BRx6 my8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ikZwvFVx43nJH39b53Lifsz5EBKM+wB7iSYfgKZ+Cw0=; b=BwINidYnip+zn+DL/guyosnImxG95y0nx+gzCTgCFG5R3GkvYWAP5UpVSV3p5j97OR LOXc4UkkKIh9KcEL12K34pKJ2bOlebuDuFaDcMP68RQkvFEF06Xl0Vc1OSvuh9HlTaud GRg+3/9hL8iCJcPjiQkT/m54TizpsuJzqAOxKMtAJ+CNgjSnwNrpWZ0+nSpJD/7KxpKv R2K+VrSELTbM7TbKjLoTMeaqoOTRsMjwuXL80yy5fmyr4PG90SojanKPW54Wm6yNXh9b mKVo9X16+x067cCvIKrf/7HWZyYO6KO9Gi4lPXIyfgnsuxsD50SFzGULlunbga563cao ovAg== X-Gm-Message-State: ANoB5pkUYOoL/uTEoY6Ljz86IlPXJ1GsgA8GtJV9HQB03mq8tqM8sw2O EjrOhzDuzEgkQswDBGDbNChrO5tM0xaStw== X-Google-Smtp-Source: AA0mqf43Yl2gcde3XHGHed7LGnACoziIcQJf7cU3snz53p+RMHeznkeyPPYUj+DDJtF4yhc/wwpBFg== X-Received: by 2002:a63:d642:0:b0:46f:d2d4:bac4 with SMTP id d2-20020a63d642000000b0046fd2d4bac4mr2888369pgj.178.1668195283156; Fri, 11 Nov 2022 11:34:43 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id oj17-20020a17090b4d9100b00212d9a06edcsm5213024pjb.42.2022.11.11.11.34.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Nov 2022 11:34:42 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v6 26/26] selftests/bpf: Add BTF sanity tests Date: Sat, 12 Nov 2022 01:02:24 +0530 Message-Id: <20221111193224.876706-27-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221111193224.876706-1-memxor@gmail.com> References: <20221111193224.876706-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17214; i=memxor@gmail.com; h=from:subject; bh=v1LI+fTgGiLUO8irIiQN9nGTkKTb9ImcS7iWlL1GYas=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjbqIpCJRIpIn4JAqKr4qE23QDsjdcoPPPSP3mg+KN HkCcjVGJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY26iKQAKCRBM4MiGSL8Ryqq0D/ 435TAP3YiWBnh35HVPeXIygx3L7/qwrv0A+BZIrUC0KuBfKY+S1Hfnnk13jrvxb9+JzboiTBLkEhk6 bG4GM7C4Ui5F8N/VbELdJ60Qyb+5JGilopKqNSOglt/AeycpaG7ZUnzleDCQ0FJWmdLsezbfzM4yE0 GKg/6PHgxLhZwT3DjxR8GE1jgxsZ6/0NBXoBu3+8jbQOGju5axg8sv+UmhUmLbBMPBOhsEkZtJAF9A IN8c0mgKcNA0FRFRMoBpHSUp4WeKLaBNewTocNHRDcEaZ1GVI5XV3yOLyA95Hyk3QE+DKlbHwDt04M r05flQQ50P/fM7Vr4oeZgKimvpJUFeK5U22IKlXNt+KRIyUj87pxAAN8cTV/zQHq0vz2kvC/ZngbMi KHT2WBMyGjTt1XQxvlUqOHs+9THYEF9h8IWQqxFjaiQQmLn2eN+FkjNYaaVBcm6I8Lkl6Tf6qpM7Wk nAF8SonoBiydS/Nt6Jrmi7QLC9WW6OJdvQzSGj6Ou/jkXEvpiTUH/x/UVExo4fU+RG0M2rfFu2yloP CQvuSwzdjgDuFi5v7knGfuhvvhCStzDT6/yP7StlUGaaEhnZb4i46R22Mjvfx6lxkIjUijOAih7kMv Ov96ZCiPjq0LuPxMUpSlQ4NgZd63b4ZjTezSxvlHYioK5JnqbZL3L+8KKafA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Preparing the metadata for bpf_list_head involves a complicated parsing step and type resolution for the contained value. Ensure that corner cases are tested against and invalid specifications in source are duly rejected. Also include tests for incorrect ownership relationships in the BTF. Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/linked_list.c | 485 ++++++++++++++++++ 1 file changed, 485 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/linked_list.c b/tools/testing/selftests/bpf/prog_tests/linked_list.c index e8569db2f3bc..bdc5a4f82e79 100644 --- a/tools/testing/selftests/bpf/prog_tests/linked_list.c +++ b/tools/testing/selftests/bpf/prog_tests/linked_list.c @@ -1,4 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 +#include +#include +#include #include #include @@ -233,6 +236,487 @@ static void test_linked_list_success(int mode, bool leave_in_map) linked_list__destroy(skel); } +#define SPIN_LOCK 2 +#define LIST_HEAD 3 +#define LIST_NODE 4 + +static struct btf *init_btf(void) +{ + int id, lid, hid, nid; + struct btf *btf; + + btf = btf__new_empty(); + if (!ASSERT_OK_PTR(btf, "btf__new_empty")) + return NULL; + id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED); + if (!ASSERT_EQ(id, 1, "btf__add_int")) + goto end; + lid = btf__add_struct(btf, "bpf_spin_lock", 4); + if (!ASSERT_EQ(lid, SPIN_LOCK, "btf__add_struct bpf_spin_lock")) + goto end; + hid = btf__add_struct(btf, "bpf_list_head", 16); + if (!ASSERT_EQ(hid, LIST_HEAD, "btf__add_struct bpf_list_head")) + goto end; + nid = btf__add_struct(btf, "bpf_list_node", 16); + if (!ASSERT_EQ(nid, LIST_NODE, "btf__add_struct bpf_list_node")) + goto end; + return btf; +end: + btf__free(btf); + return NULL; +} + +static void test_btf(void) +{ + struct btf *btf = NULL; + int id, err; + + while (test__start_subtest("btf: too many locks")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 24); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", SPIN_LOCK, 0, 0); + if (!ASSERT_OK(err, "btf__add_struct foo::a")) + break; + err = btf__add_field(btf, "b", SPIN_LOCK, 32, 0); + if (!ASSERT_OK(err, "btf__add_struct foo::a")) + break; + err = btf__add_field(btf, "c", LIST_HEAD, 64, 0); + if (!ASSERT_OK(err, "btf__add_struct foo::a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -E2BIG, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: missing lock")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 16); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_struct foo::a")) + break; + id = btf__add_decl_tag(btf, "contains:baz:a", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:baz:a")) + break; + id = btf__add_struct(btf, "baz", 16); + if (!ASSERT_EQ(id, 7, "btf__add_struct baz")) + break; + err = btf__add_field(btf, "a", LIST_NODE, 0, 0); + if (!ASSERT_OK(err, "btf__add_field baz::a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -EINVAL, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: bad offset")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 36); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::c")) + break; + id = btf__add_decl_tag(btf, "contains:foo:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:foo:b")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -EEXIST, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: missing contains:")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 24); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", SPIN_LOCK, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_HEAD, 64, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -EINVAL, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: missing struct")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 24); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", SPIN_LOCK, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_HEAD, 64, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + id = btf__add_decl_tag(btf, "contains:bar:bar", 5, 1); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:bar")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -ENOENT, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: missing node")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 24); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", SPIN_LOCK, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_HEAD, 64, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + id = btf__add_decl_tag(btf, "contains:foo:c", 5, 1); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:foo:c")) + break; + + err = btf__load_into_kernel(btf); + btf__free(btf); + ASSERT_EQ(err, -ENOENT, "check btf"); + break; + } + + while (test__start_subtest("btf: node incorrect type")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 20); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", SPIN_LOCK, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + id = btf__add_decl_tag(btf, "contains:bar:a", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:a")) + break; + id = btf__add_struct(btf, "bar", 4); + if (!ASSERT_EQ(id, 7, "btf__add_struct bar")) + break; + err = btf__add_field(btf, "a", SPIN_LOCK, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar::a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -EINVAL, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: multiple bpf_list_node with name b")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 52); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 256, 0); + if (!ASSERT_OK(err, "btf__add_field foo::c")) + break; + err = btf__add_field(btf, "d", SPIN_LOCK, 384, 0); + if (!ASSERT_OK(err, "btf__add_field foo::d")) + break; + id = btf__add_decl_tag(btf, "contains:foo:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:foo:b")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -EINVAL, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: owning | owned AA cycle")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 36); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field foo::c")) + break; + id = btf__add_decl_tag(btf, "contains:foo:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:foo:b")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -ELOOP, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: owning | owned ABA cycle")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 36); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field foo::c")) + break; + id = btf__add_decl_tag(btf, "contains:bar:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:b")) + break; + id = btf__add_struct(btf, "bar", 36); + if (!ASSERT_EQ(id, 7, "btf__add_struct bar")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field bar::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field bar::c")) + break; + id = btf__add_decl_tag(btf, "contains:foo:b", 7, 0); + if (!ASSERT_EQ(id, 8, "btf__add_decl_tag contains:foo:b")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -ELOOP, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: owning -> owned")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 20); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", SPIN_LOCK, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + id = btf__add_decl_tag(btf, "contains:bar:a", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:a")) + break; + id = btf__add_struct(btf, "bar", 16); + if (!ASSERT_EQ(id, 7, "btf__add_struct bar")) + break; + err = btf__add_field(btf, "a", LIST_NODE, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar::a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, 0, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: owning -> owning | owned -> owned")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 20); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", SPIN_LOCK, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + id = btf__add_decl_tag(btf, "contains:bar:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:b")) + break; + id = btf__add_struct(btf, "bar", 36); + if (!ASSERT_EQ(id, 7, "btf__add_struct bar")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field bar::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field bar::c")) + break; + id = btf__add_decl_tag(btf, "contains:baz:a", 7, 0); + if (!ASSERT_EQ(id, 8, "btf__add_decl_tag contains:baz:a")) + break; + id = btf__add_struct(btf, "baz", 16); + if (!ASSERT_EQ(id, 9, "btf__add_struct baz")) + break; + err = btf__add_field(btf, "a", LIST_NODE, 0, 0); + if (!ASSERT_OK(err, "btf__add_field baz:a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, 0, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: owning | owned -> owning | owned -> owned")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 36); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field foo::c")) + break; + id = btf__add_decl_tag(btf, "contains:bar:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:b")) + break; + id = btf__add_struct(btf, "bar", 36); + if (!ASSERT_EQ(id, 7, "btf__add_struct bar")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar:a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field bar:b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field bar:c")) + break; + id = btf__add_decl_tag(btf, "contains:baz:a", 7, 0); + if (!ASSERT_EQ(id, 8, "btf__add_decl_tag contains:baz:a")) + break; + id = btf__add_struct(btf, "baz", 16); + if (!ASSERT_EQ(id, 9, "btf__add_struct baz")) + break; + err = btf__add_field(btf, "a", LIST_NODE, 0, 0); + if (!ASSERT_OK(err, "btf__add_field baz:a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -ELOOP, "check btf"); + btf__free(btf); + break; + } + + while (test__start_subtest("btf: owning -> owning | owned -> owning | owned -> owned")) { + btf = init_btf(); + if (!ASSERT_OK_PTR(btf, "init_btf")) + break; + id = btf__add_struct(btf, "foo", 20); + if (!ASSERT_EQ(id, 5, "btf__add_struct foo")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field foo::a")) + break; + err = btf__add_field(btf, "b", SPIN_LOCK, 128, 0); + if (!ASSERT_OK(err, "btf__add_field foo::b")) + break; + id = btf__add_decl_tag(btf, "contains:bar:b", 5, 0); + if (!ASSERT_EQ(id, 6, "btf__add_decl_tag contains:bar:b")) + break; + id = btf__add_struct(btf, "bar", 36); + if (!ASSERT_EQ(id, 7, "btf__add_struct bar")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field bar::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field bar::c")) + break; + id = btf__add_decl_tag(btf, "contains:baz:b", 7, 0); + if (!ASSERT_EQ(id, 8, "btf__add_decl_tag")) + break; + id = btf__add_struct(btf, "baz", 36); + if (!ASSERT_EQ(id, 9, "btf__add_struct baz")) + break; + err = btf__add_field(btf, "a", LIST_HEAD, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bar::a")) + break; + err = btf__add_field(btf, "b", LIST_NODE, 128, 0); + if (!ASSERT_OK(err, "btf__add_field bar::b")) + break; + err = btf__add_field(btf, "c", SPIN_LOCK, 256, 0); + if (!ASSERT_OK(err, "btf__add_field bar::c")) + break; + id = btf__add_decl_tag(btf, "contains:bam:a", 9, 0); + if (!ASSERT_EQ(id, 10, "btf__add_decl_tag contains:bam:a")) + break; + id = btf__add_struct(btf, "bam", 16); + if (!ASSERT_EQ(id, 11, "btf__add_struct bam")) + break; + err = btf__add_field(btf, "a", LIST_NODE, 0, 0); + if (!ASSERT_OK(err, "btf__add_field bam::a")) + break; + + err = btf__load_into_kernel(btf); + ASSERT_EQ(err, -ELOOP, "check btf"); + btf__free(btf); + break; + } +} + void test_linked_list(void) { int i; @@ -243,6 +727,7 @@ void test_linked_list(void) test_linked_list_fail_prog(linked_list_fail_tests[i].prog_name, linked_list_fail_tests[i].err_msg); } + test_btf(); test_linked_list_success(PUSH_POP, false); test_linked_list_success(PUSH_POP, true); test_linked_list_success(PUSH_POP_MULT, false);